Author: freemontfreemont
Date: May 15, 2008 04:57
Almost a month ago I posted here asking for ideas as to why a new site
wasn't getting indexed by the search engines.
Well earlier this week I caught Google in the log, and I saw that the bot
had come to the site looking for robots.txt, didn't find it (500), and
pissed off. Apparently now Google REQUIRES robots.txt. (?) Who knew.
I placed a robots.txt in the site and now it's getting spidered normally.
--
"Because all you of Earth are idiots!"
¯`·.¸¸.·´¯`·-> freemont© <-·´¯`·.¸¸.·´¯
|