Discussing several posts on Google Crawling HTML Forms, plus more on the ongoing discussion of Dynamic Urls versus Static URLs that were posted on the Google Webmaster Central Blog.
Jeremiah Andrick of MSN Live Webmaster Tools gives some advice on how to best use MSN Live when it comes to pagerank, results, plus he answers question on live results relevance compared to Google and Video submissions.
Vanessa Fox discuses the Robots Exclusion Protocol and Managing Robots Access To Your Website, including managing capabilities and implementing the REP, plus she answers a listener question about page redirects that are disallowed.
We discuss the Google Webmaster Central blog post that that they are now crawling through HTML forms to try to discover new web pages and URLs that otherwise couldn’t be found and index those pages on Google.
We discuss how spider traps create duplicate content and what to look for now when doing directory submissions.
As seen on the forums recently and over the past years, many people would like to know how often does Google index a site with a PR5 (or 4 or 6, etc.)? The related question is always if Google doesn’t index my site every day, can you tell me how to make them come back more often?
Example threads:
What is a robot or a spider? We’ll introduce you to them and explain a little how they work.
Question asked by StatMan with his Organizational Tips.
Fictional renditions of the bots. If you want to read the full mockery of a post, you can find it here
