4 Quick Facts About Googlebot and What It Does To Your Blog

• Buffer
• Share

What is a Googlebot?

Have you ever wondered what is a Googlebot? I’ve found it mind boggling sometimes to think how the web really works behind the scenes. If you have too, this information and infographic, may make it easier and understand why SEO is still important today.

A googlebot crawls the web looking for new pages to crawl and index. Google uses a huge set of computers to crawl the web constantly. They use an algorithmic process via computer programs determine which sites to crawl, how often, and how many pages to fetch from each website.

Google’s crawl process begins with a list of website page URLs, generated from previous crawl processes. It is then increased with Sitemap data provided by webmasters. (See why sitemaps are important now?)  As a Googlebot visits each of these websites it detects links on each page and adds them to its list of pages to crawl. New websites, changes to existing websites, and dead links are noted and used to update the Google index. (See why updating with fresh content is so important?) 

Google doesn’t accept payment to crawl a site more frequently and they claim they don’t go by who spends what on Google AdWords.

After a Google does a crawl they index the web pages by compiling an index of words it sees and their location on a page.  Title tags and alternative attributes are definitely a plus in this crawl process. Many rich media files do not contain these and may not be indexed.

The results you see when you do a Google Search are then based on the Googlebot’s crawling of the websites that are relevant to your search keywords. The relevancy is based on 200 plus ranking factors. To rank well in this you must be sure that Google can crawl your website and index it properly. You can always check your Google Webmaster tools for hints on this!

The Nasty Googlebots

Did you note that for every 24 Googlebots that visit your site 1 is a fake? Those are those nasty spammers and scrapers!  According to Search Engine Watch these nasty Googlebots are up 61% this year. If you know for sure you have them you can block them via your Google Webmaster tools but be sure not to block Google or the other major search engines as well.

Speaking of nasty Googlebots and spammers – there are some now impersonating bloggers. I’ve had this happen several times now and want to share so you can be on the lookout. If you had not read Adrienne’s column on these imposters do check it out today.

@Lisapatb Hey Lisa, someone left a comment on two of my blogs as you, but I know it’s not you. So weird..

— Mitch Mitchell (@Mitch_M) September 1, 2014

So be extra careful approving blog comments out there! Thanks to Mitch Mitchell for pointing this out as well as Brian Hawkins at Hot Blog Tips.

Does your blog or website have a sitemap? How often do you update your blog?