Beware of Referer Spam
October 11th, 2008 Aizat Wordpress
One of the peculiar aspects of having a blog is that you’re not only a writer, but also a website administrator. If you have your own domain, you pay for bandwidth, and you allow people to come on and write whatever pops into their heads. As a result, there are things you have to watch.
Do you usually check your referrer site on your blog ? You should because there might be referer spam on your blog that you sometime didn’t notice. What the consequence ? You will lost a massive amount of traffic and the most important thing is it will effect your adsense earning. Your adsense earning will decrease with the lost of traffic.
What is Referer Spam?
According to wikipedia:
Referer spam is a kind of spamdexing (spamming aimed at search engines). The technique involves making repeated web site requests using a fake referer url that points to the site the spammer wishes to advertise. Sites that publicize their access logs, including referer statistics, will then end up linking to the spammer’s site, which will in turn be indexed by the search engines as they crawl the access logs.
This benefits the spammer because of the free link, and also gives the spammer’s site improved search engine placement due to link-counting algorithms that search engines use.
What’s the point?
Why go to the effort of leaving a web site address in someone else’s log files?
Most web servers have the ability to log an extensive amount of information about web sites visitors. Many webmasters and bloggers use web-based software to parse those log files automatically. The result is one or more pages breaking the information down into very detailed statistics. These statistics include the referer information and often those referers are displayed as hyperlinks.
Bloggers quite often will display a link to the most frequent or most recent web site found in their referer logs using scripts. Some will even put those links right on the front page of their site in a sidebar area.
Unscrupulous web site owners are spamming the log files in order to have their web sites listed on those referer links. This creates an artificial boost in that site’s popularity among those search engines that measure the number of links to a site. It also generates traffic when curious visitors of a victim site clicks the links displayed in the referer listing.
Simply put, these people are running advertisements on your web site and using it to boost their search engine rankings. They do this without your knowledge, without your permission, and without compensating you in any way for the use of your network.
How to stop referer spam?
Log into your site’s FTP server. Make sure it is set to display hidden files on the server. Check your FTP client’s documentation for help with that. If there is a file named .htaccess, download it and open it in a text editor. If there isn’t one, create one on your hard drive and upload it when you are finished.
Put this in the .htaccess file, changing “spammersite” to the name of sites found spamming your logs. The first site listed should not have “NC”, and the last one should not have “OR”.
If your site starts generating errors after you upload this file, remove the # from the first line. If you use Microsoft Frontpage on your site, do not do this. Changing the .htaccess file could interfere with Frontpage.
# Options +FollowSymlinks
RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://(www\.)?spammersite1.com.*$ [OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?spammersite2.com.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?spammersite3.com.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?spammersite4.com.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?spammersite5.com.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?spammersite6.com.*$ [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://(www\.)?spammersite7.com.*$ [NC]
RewriteRule \.*$ http://www.some-other-website.com [R,L]
This will redirect any request with a spammer site in the refer to any other site you wish. If you prefer, you can rewrite that last line as “RewriteRule .* - [F,L]” to give them a “Forbidden” error. Either way, it keeps them out of your referer logs and there is no way to defeat it.
Solution by: SpywareInfo
A new tactic - using SetEnvIfNoCase instead of RewriteCond - seems to be quite effective (esp for referrers). Original version found at koehntopp. Also you can have a look at my .htaccess for some clue.
No comments yet.