Choosing optimal crawl speed and load capacity on your web server


In the options of the crawler there are three levels of crawl speed, creating appropriate load capacities on the server being indexed:
  • Maximum - this load capacity is used by default. If you have a quality paid hosting, most likely you do not need to worry about creating a load while crawling your site. We recommend using this load value, which allows the crawler to index your website at top speed.
  • Average – choose this load capacity, if your server requires a gentle mode of indexation.
  • Low – level of load capacity, which allows crawl your site, creating a minimum load on the server. This load level is recommended for websites, located on a free hosting or for sites that require limited flow of traffic.
    We recommend that you select this mode when crawling sites located on free hosting servers.
    However, note that this level slows down the process of crawl your site.