Crawl Budget
Crawl budget is the number of URLs a search engine crawler will fetch on a site within a given period, shaped by crawl rate limit and crawl demand. It matters most for large sites where server capacity, URL volume, and content freshness determine how much gets crawled.
Why It Matters
If crawlers waste requests on duplicate, low-value, or parameter URLs, important pages get crawled and refreshed less often. That delays indexing of new content and updates, which can suppress visibility for pages that should be ranking.
Common Mistake
Small sites obsessing over crawl budget when it rarely constrains them. The real error on large sites is letting faceted navigation, session IDs, and infinite parameter combinations generate endless crawlable URLs instead of blocking or canonicalizing them.