“RSS/Atom feeds have been very popular in recent years as a mechanism for content publication. They allow readers to check for new content from publishers. Using feeds for discovery allows us to get these new pages into our index more quickly than traditional crawling methods. We may use many potential sources to access updates from feeds including Reader, notification services, or direct crawls of feeds. Going forward, we might also explore mechanisms such as PubSubHubbub to identify updated items.”

With services like Google Reader and FeedBurner under their control, it wouldn’t be that hard for Google to index content in real-time as and when they are published. We don’t know whether Google uses the feed just to discover pages or if they use the content in the feed as well.

If Google is using the content in feeds for indexing, then I guess they must come out and talk about it openly because some of us publish partial feeds, whereas other publish entire content. Those publishing partial feeds would be losing the juice!

Googlebot needs to access your feeds inorder to get your fresh content indexed, so if you are blocking your feeds using Robots.txt, it may be wise to modify and grant access to Googlebot. To know more about Robots.txt, please refer Google Webmaster Central.

Stories you may like:

1. Full Feeds vs Partial Feeds – Which one do you Prefer ?
2. Google’s Secret Evaluation Site
3. Get Perfect Scrap Notifications with Orkut Feeds

This article, Google Uses RSS Feeds to Index Your Site, was published at TechBuzz. Please don't violate our copyright