Getting Google to crawl on-demand

When site owners add new content to their sites, it can be frustrating to wait hours and even days for the changes to show up in a typical Google search

. This is because the new pages are virtually invisible to the public until they’ve been crawled and indexed by Google. Well, users of Google’s Site Search search-as-a-service offering need wait no more. Google just added a crawl-on-demand feature to Site Search that lets users get their sites crawled and indexed at the push of a button. It’s like having Google at your beck and call.

Information Week reports that the feature will be available for most Site Search users today, and it quotes Google’s Nitin Mangtani explaining the new feature this way:

“We re-index the Web sites when they change but we don’t give the end-user any control over when they want them re-indexed,” said Mangtani. “With this release, that’s what we’re offering them.”

Mangtani says the change may not matter to high-volume content producers, such as typical news sites, because they tend to be crawled more often by Google. But other sites, such as business or government Web sites that typically aren’t indexed very frequently, should welcome the new feature.