Now I’m seeing the Googlebot request /about/ pages relative to known blogs that don’t have any links to any /about/ URI. The last time the Googlebot flailed around like this it was fun to watch for a little bit and wonder what they had cooking in the labs, but then it got annoying. I don’t know if there are rules of bot etiquette, but requesting imagined unlinked resources while spidering can’t be a best practice.
There is of course one blog vendor who consistently has about pages at Now my question Google is: what should the rest of us do if we want our about pages indexed by this new system? Mine happens to be in the /about/ URIs, and that’s Typepad.about subdirectory of my blog, but what about people who have about.html or about-me.php? Should I set up a permanent redirect for every blog I have redirecting to the real about page?
(Note: That’s faux indignation. I don’t have any juicy conspiracy theories, and I’m not really that peeved, mostly I’m just curious what they’re up to. However juicy conspiracy theories are welcome in the comments. [As long as they don’t make fun of me for noticing these things.] )
UPDATE: It just requested a non-existent non-linked /contact/ URI.
UPDATE: It just requested a non-existent non-linked /stats/ URI.
DEVELOPING . . .
