Utility Dilution

The HTTP "referer[sic]" header is pretty cool. I can use it to see where people have linked to lemonodor. I've found some good sites this way.

Here are a few random referers from my logs. If you follow these links, you'll find a link to lemonodor somewhere.

• http://radio.weblogs.com/0109158/
• http://www.google.com/search?q=evolution+video+demo+er1
• http://oconnor.cx/
• http://www.winterdom.com/weblog/

Unfortunately, the referer header seems to be becoming less useful by the day. There is a trend for RSS readers and aggregator applications to use the application's homepage as the referer url.

For example, here are some referers from my logs that will take you to pages that have nothing to do with lemonodor:

• http://radio.userland.com/newsAggregator
• http://bitworking.org/AggieReferrers.html
• http://disobey.com/amphetadesk/
• http://www.iki.fi/juri/straw.html

It's all just fodder for grep -v. And while it's not a big deal, it is annoying. The user-agent header is right there for the very purpose of expressing information about the program fetching the url. I would imagine it makes it harder for people to use log file analyzers and things and get meaningful results, when suddenly it looks like several hundred hits are coming from a single url--a url that has nothing to do with your site

I think this behavior was pioneered by the Radio aggregator. When Dave Winer added the feature, though, I thought it was pretty cool because Radio actually sent a referer header containing the url of the radio weblog of the person running the aggregator. It wasn't strictly by the RFCs or anything, but it did give me information about people that were reading this site.

But just sticking your reader's homepage url in there... well, it's only slightly more interesting to me than the person that spams my logs with a referer to

http://porn-homepage.info/

Posted by jjwiseman at September 03, 2002 12:02 AM