Blocking yandex.ru bot

robots.txtsearch-engine

I want to block all request from yandex.ru search bot. It is very traffic intensive (2GB/day).
I first blocked one C class IP range, but it seems this bot appear from different IP ranges.

For example:

spider31.yandex.ru -> 77.88.26.27
spider79.yandex.ru -> 95.108.155.251 etc..

I can put some deny in robots.txt but not sure if it respect this.
I am thinking of blocking a list of IP ranges.

Can somebody suggest some general solution.

Best Answer

Don't believe what you read on forums about this! Trust what your server logs tell you. If Yandex obeyed robots.txt, you would see the evidence in your logs. I have seen for myself that Yandex robots do not even READ the robots.txt file!

Quit wasting time with long IP lists that only serve to slow down your site drastically.

Enter the following lines in .htaccess (in the root folder of each of your sites):

SetEnvIfNoCase User-Agent "^Yandex*" bad_bot
Order Deny,Allow
Deny from env=bad_bot

I did, and all Yandex gets now are 403 Access denied errors.

Good bye Yandex!