VercatusGet started
Grade AR-168Technical

Crawl rate server speed

Google raises a site's crawl capacity limit when responses, including latency and time to first byte, stay stable or improve, and lowers it when the site slows down or returns 5xx or 429. Google's crawl-budget guide is aimed at sites with 1 million+ pages changing weekly or 10,000+ pages changing daily. web.dev rates a TTFB of 0.8 seconds or less as good and above 1.8 seconds as poor.

Sources

  1. 1
    Google Crawling Infrastructure · 2026Crawl budget managementdevelopers.google.com
  2. 2
    web.dev · 2025Time to First Byte (TTFB)web.dev

AGrade A: official documentation or a standard. A grade describes the provenance of the sources above; it is not a probability that a change will work. The effect basis says how the size of the effect is known: measured by a study, taken from an industry report, only documented without a size, or not quantified.

Put this evidence to work

Related entries