Choosing the optimal crawl speed depending on your web server


The crawler settings provide three crawling speed levels, which create a corresponding load on the web server:
  • Maximum - this load capacity is used by default. If you have a quality paid hosting, most likely you do not need to worry about creating a load while crawling your website. We recommend using this load value, which allows the crawler to process your website at top speed.
  • Average – choose this load capacity, if your web server requires a gentle mode of crawling.
  • Low – level of load capacity, which allows crawl your website, creating a minimum load on the web server. This load level is recommended for websites hosted on free hosting, or for websites that have significantly limited bandwidth.
    However, please note that this level slows down the crawling process of your website.