More robot woes
September 16th, 2003
The Google robots are back and wreaking havoc on my referer log. I have not checked my referer stats in some time so I do not know if this started before or after I rearranged my blogroll and other sidebar information. My guess is that the link that I added for my aggregator was picked up by a robot and this started my aggregator indexing woes, again. Unfortunately, until this is sorted out, I have removed my blogroll. I have also, sadly, decided to no longer use the built-in aggregator.As strange as it may seem, I hate a referer log inundated with graphic search requests more than I love the convenience of my aggregator. I could probably figure out how to keep Google from indexing my aggregator and avoid the very troubling and offensive search requests ending up in my stats, but with two classes, a full time job, and a thesis to write, I am a little short on time and patience to play around with this blog. To say the least, after spending four hours on Sunday trying to solve my weblog formatting and indexing woes, I am very unhappy and cranky. *sigh*
I may have to revert to the awful red (…she says with a shudder) default blog or permanently move to my “secret” Typepad weblog. All I want to do is spend time on creating content for this site with the option to slightly (and easily) customize my blog (e.g., choose background and text colors and the location, content, and number of blogroll/sidebar items) to match my content. Am I asking for too much? *sigh, again*
Even if I move my blog, I do want the Harvard weblog book project (for lack of a better title) to happen. As frustrated (and thwarted) as I feel by technology and my abilities to deal with it right now, I still really like that idea. And I am glad to see that other people like the idea, too.
Entry Filed under: Administrative Matters
2 Comments Add your own
Leave a Comment
You must be logged in to post a comment.
Trackback this post | Subscribe to the comments via RSS Feed
1. Nate | September 16th, 2003 at 7:25 pm
You can set up your site to keep the robots away from your aggregator.
Go to Prefs -> Advanced -> Robots.txt. Then type “/aggregator/” in the box and add the path. This keeps the bots from compiling stuff from your aggregator.
2. Vernica | September 17th, 2003 at 10:44 am
Thanks for the suggestion, Nate. However, I already did that a while back when I first had trouble with this. (I just realized that I forgot to link to the original post on this problem and its resolution–see the comments section for that post). Unfortunately, even with my aggregator and several other pages on the Robots.txt list, they are being indexed in Google again. As I said, I think the link to my aggregator was picked up and that is the source of the problem.
Grrr…I really want those Google bots to leave my site (or, at least, my aggregator) alone.