What are hidden pages (Deep Web) and how to include them in a Sitemap?


Deep Web (also called the Deepnet, the Invisible Web, the Undernet or the hidden Web) web-pages, which are not indexed by search engines, because such pages do not have hyperlinks from accessible pages. For example these are the pages, generated through HTML-forms interface or frame content.
If you wish to discover and include such pages into Sitemap, please tick the appropriate options:
  • "Crawl html forms" (submit occurs without filling the form);
  • "Crawl framed content" (contents of <frameset> <iframe>).