Is It To Your Advantage To Block SEOmoz, Raven & Majestic Link Analysis Tools?
This weeks discussion is one based off the knowledge that no free back link analysis programs exist anymore. The ability to freely dissect and replicate competitor link profiles ended when Yahoo Site Explorer (YSE) died last month. The alternative to YSE are paid solutions such as SEOmoz, Raven or MajesticSEO. With that said is it in your best interest to block these link analysis tools from spidering your website? Each one of these tools are not affiliated with a search engine so it won’t have a direct effect on your search results. Allowing these spider bots into your website could give up your link building tactics to your competitors.
The obvious draw back to this is that you don’t get the value that SEOmoz, Raven, or MajesticSEO offers you. But I would argue that SEOs should know what links they are building to their own sites and they should have access to their link profile through Google Webmaster Tools and Bing Webmaster Tools. This might be easier for brand new websites then established sites so that’s why I ask you this week is it to your advantage to block SEOmoz, Raven & Majestic link analysis tools from spidering your sites?
Nick LeRoy
Nick LeRoy is a Minneapolis SEO consultant offering custom SEO audits & monthly SEO services to clients in all niches. He also regularly blogs about SEO at NickLeRoy.com
RSS Feed
E-mail Feed
Follow Nick on Twitter!
I will also note that I do not block these link analysis tools for NickLeRoy.com. I would say 90% of the links to this site are not built but naturally acquired through the value of the content. It would be difficult to replicate most of these links.
However, I have one site that I started up that is in its infancy and is having links manually built to it. I blocked these spiders from accessing it as I don’t want to give my competitors a list of sites in my niche that are obviously open to linking out if the right resources are given. I may open it up to them once more naturally acquired links are built based off the content of the site.
You should add ‘how’ you’re blocking them: Tricky PHP, htaccess, robots.txt, etc.
Nothing crazy – Robots.txt . I suppose I could post the userbot IDs huh?
Yep
Majestic-12:
Rogerbot
having difficulties with the raven tools one at the moment.
Blocking the link tool spiders from your site won’t stop them reporting what backlinks that site has, it will only stop them from discovering who you are linking to.
You need to pursuade those linking to you to block them if you don’t want your own backlinks discovered.
Oh, and Rogerbot blocking wont keep anything out of Linkscape and related tools you need to block:
dotbot
gigabot
exabot
Seomoz seem to get most of their data from dotbot but the other two are suspicious, despite their talk of having their own crawler they still won’t name it so it can be excluded via robots.txt, but if you block those 3 it usually stops stuff from Linkscape so that’s the smoking gun.
The safest thing to do is only allow robots and hosts you want to appear in the index for and block everyone else.
Good points! Thanks for listing the 3 bots – I was unaware of them. As I mentioned in this discussion it was just a random hypothetical. Glad others like yourself chimed in!
Interesting thought, Nick. Obviously I’d be against it as I work at Raven
One thing I did want to clarify is that you can’t specifically block Raven because we don’t actually spider anything. Instead, we pull in data from SEOmoz and MajesticSEO so all you would need to do is block them in order to block us.
Just didn’t want people thinking we were crawling their site.
Hey Taylor, thanks for dropping by and for clarifying something I failed to do in the original post. I grouped Raven into site crawlers when I really just meant avenues in which to receive competitive link reports. My apologies for the confusion.
Like channel5 mentioned, I don’t think it’d have any effect since the crawlers will see what sites are linking to you, which is WAY more interesting then to see who you are linking to.
I guess if you have access to link hubs, you could make a case about blocking access there, since they’d be the ones providing your (or your client’s) sites with real link value.