Crawling Enterprise Level Sites for SEO – SEOForLunch Issue #56

Share on facebook
Share on twitter

Issue #56
November 21st, 2017

Hey, lunch buddies.  Wow! Did the weekend go quick!?  My wife and I enjoyed the long weekend in sunny California enjoying the 70+ degree weather, the ocean and of course, the awesome BMW 428i convertible we rented.   It was a much-needed getaway and definitely helped recharge us before the holidays start up.  Below is a fun picture of me enjoying our ride for the weekend!

This is it for vacations for the LeRoy family until the big one in February where we take the 3 kiddos to Florida to enjoy the beach and a few days at Disney World.

What were you all up to this past week/end?

What You Must Know

Article: How Topix Reversed Being Crushed by Google

What’s important?
Topix was pulling in 200 million page views per month before Google algorithms nearly bankrupted the site.

Quickly identifying the core issue, Topix CEO Chris Tolles changed the sites editorial calendar to Google compliant and once again started cashing in “free” organic traffic leading to all-time records for both page views and revenue.

Nick’s take:
I don’t typically post stories in this section of the newsletter but this was a great “feel good story” showing that site highly dependent on organic traffic can recover from Google algorithms.

This story also reiterates why it’s not worth knowingly taking shortcuts when depending on “free” organic traffic.  It’s a good read.  Hopefully, it inspires anyone currently being impacted by algorithm updates to taking a step back and addressing the quality concerns impacting their site.

SEO Tips & Advice

Are you in charge of implementing SEO on large scale websites?  If so, you know the value of running crawls on the site before and after your changes take effect.

Glenn Glabe wrote up a great piece on covering crawling enterprise sites and tracking technical SEO improvements over time.

In addition to showing crawl reports from Deepcrawl, ScreamingFrog and Sitebulb Glenn also explains exactly why it’s critical to run concurrent crawls before and after releases to production.   With extremely large websites it’s possible (and very likely) that while fixing a handful of issues that several more will pop up.

If you manage an enterprise level site this is a must-read.

Interesting Reads in Your Spare Time

That’s everything for this week folks.  If you want to read previous issues of the #SEOForLunch I publish them with only a week or two delay on

If you were forwarded this email you can sign up for the weekly email over at


Nick LeRoy
Nick LeRoy
Nick has 10+ years of SEO consulting experience building and executing strategies for clients ranging from multi-billion dollar consumer packaged goods to fashion entities with hundreds of physical store locations. He also writes the weekly SEO newsletter #SEOForLunch that allows you to stay up to date with industry updates in the amount of time it takes you to eat your lunch.

Make SEO Business Decisions?

The #SEOForLunch newsletter keeps you up to date with SEO industry updates over your lunch break once a week.

Recent Publications