How Does Google’s Crawl Budget Work?

Share on facebook
Share on twitter

Hi Lunch Buddies!

Last week my family and I took a mini-vacation after being offered an opportunity to rent a friend’s lake cabin. It was a great getaway given how COVID-19 has impacted the safety of traditional travel.

I’m not much of a fisherman (despite trying my hardest) but I caught the biggest crappie I’ve ever seen in person.

Back to the SEO world… I wrote another article. This time on the topic of scaling product redirects for your SEO migrations over on Please check it out, share, and let me know your thoughts.

Let’s jump into this week’s #SEOForLunch issue. 
– Nick

Get the #SEOForLunch Sent Directly To Your Inbox

Never miss a Google algorithm update again.

What you need to know

Article: Google’s Martin Spitt and Merkle’s Alexis Sanders Discuss Googlebot Crawl Budget

What’s Important:
Ever heard of Google’s elusive “crawl budget”? Alexis asks Martin a handful of fantastically crafted questions to unlock a ton of valuable information.

Nick’s Take:
Martin said that you don’t typically have to worry about crawl budget if your site is under 1M pages. However, he and Alexis talk about so much more that I wouldn’t dare ignore crawl budget just because my site falls under that threshold.

Interesting Reads in Your Spare Time

Does Google Respect the Search Console URL Parameter Tool?
Written by Chris Long, Search Engine Land
This post is beyond timely. I decided to try out the URL parameter tool in Google Search Console after reading that its Google’s preferred method for controlling crawling (vs robots.txt). After reviewing server logs I found that Google was NOT honoring my request so I pinged Google’s John Mueller on Twitter. John validated this tool is indeed a hint and not to be compared to a robots.txt directive.

What You Know About Google’s Organic CTR Is Wrong
Written by Johannes Beus, Sistrix
Back in the day, it was believed that a #1 organic search listing equaled roughly 30% click-through for all searches done for a given query. Johannes walks us through why calculating an organic CTR percentage isn’t nearly as easy today.

How To Generate Leads With SEO
Written by Jonas Sickler, Terakeet
Lead generation doesn’t start or end with SEO. However, SEO can play an amazing role in the larger acquisition strategy. Jonas writes about the value SEO plays in generating leads for all businesses.

OpenAI’s GPT-3 may be the biggest thing since bitcoin
Written by Manuel Araoz, maraoz
Holy Crap. We know about machine learning and we’ve even seen it do some pretty impressive work. GPT-3 can be an absolute gamechanger as it pertains to writing content. I don’t for a second want to give anything more away. If you read one article this week from this newsletter let it be this one.

Image SEO and Alt Tags: 7 Advanced Image Optimization Tips
Written by Melissa Fach, SEMRush
I have a client who came to me with significant issues with their images being found in Google. They simply weren’t. I followed a very similar process to Melissa’s guide here and they are starting to see some really good results. Don’t sleep on this underrated SEO strategy – Image SEO can be incredibly valuable.

The Complete Guide to SEO Testing
Written by Nick Swan, SEO Testing
Did you know that you can run SEO tests on your website? Not enough folks do it. Sometimes it’s due to technical limitations but other times it’s simply that they don’t know how to implement and measure it appropriately. Oh, you might just see a familiar face quoted. Click to read my contribution to the post.

Nick LeRoy
Nick LeRoy
Nick has 10+ years of SEO consulting experience building and executing strategies for clients ranging from multi-billion dollar consumer packaged goods to fashion entities with hundreds of physical store locations. He also writes the weekly SEO newsletter #SEOForLunch that allows you to stay up to date with industry updates in the amount of time it takes you to eat your lunch.

Make SEO Business Decisions?

The #SEOForLunch newsletter keeps you up to date with SEO industry updates over your lunch break once a week.

Recent Publications