XML Generator Sitemap Generator
  • EN English
  • RU
Contact Customer Service
Contact us if you need help!
Indexing Cyrillic Domains
Plans and Pricing Buy Access PIN Webmaster Help Center Customer Support

How will the web pages that use robots meta or X-Robots-Tag headers be processed?

If the corresponding options are active, the pages will be processed according to the set values of robots meta (noindex, nofollow).
If necessary, you can always apply the processing of only noindex or only nofollow independently of each other.
The tags intended for certain search robots (for example, Googlebot) are considered when choosing the appropriate crawler identification option as a search robot.
Also, you can use meta tags on webpages for our robot, which will be taken into account when choosing the direct identification of our robots.
Examples of using robots meta tag:
    <meta name="robots" content="noindex" />
    <meta name="robots" content="nofollow" />
    <meta name="robots" content="noindex,nofollow" />

Examples of using X-Robots-Tag HTTP headers:
    X-Robots-Tag: noindex
    X-Robots-Tag: nofollow
    X-Robots-Tag: noindex, nofollow
Follow us to keep up with all the news!
Share Service!
©Sitemap Generator - Online Generator
VisaMasterCard Secure
This website uses cookies. Using this website means you're agreeing with this. More info Accept