Over the last few days I have noticed that my WordPress website had been running quite slowly, so I decided to investigate. After checking my database I saw that a table which was responsible for tracking 404 errors was over 1GB is size. At this point it was evident I was being targeted by bots.
After checking my access log I could see that there was a pattern of sorts, the bot seemed to land on a legitimate page which listed my categories and then move into a category page and at this point they request seemingly random page numbers, many of which are non-existent pages causing the issue.
/watch-online/ - Landing Page /category/evolution/page/7 - 404 /category/evolution/page/1 /category/evolution/page/3 /category/evolution/page/5 - 404 /category/evolution/page/8 - 404 /category/evolution/page/4 - 404 /category/evolution/page/2 /category/evolution/page/6 - 404 /category/evolution/page/9 - 404 /category/evolution/page/10 - 404
This is the actual order of requests and they all happen within a second, at this point the IP becomes blocked as too many 404’s have been thrown but this seems to have no affect due to the sheer number of bots all doing the same thing.
Also the category changes with each bot so they are all attacking random categories and generating 404 pages.
At the moment there are 2037 unique ip’s which have thrown similar 404s in the last 24 hours.
I also use Cloudflare and have manually blocked many ip’s from ever reaching my box but this attack is relentless and it seems as though they keep generating new ip’s. Here is a list of some offending ip’s:
126.96.36.199 188.8.131.52 184.108.40.206 220.127.116.11 18.104.22.168 22.214.171.124 126.96.36.199 188.8.131.52 184.108.40.206 220.127.116.11 18.104.22.168 22.214.171.124 126.96.36.199 188.8.131.52 184.108.40.206 220.127.116.11 18.104.22.168 22.214.171.124 126.96.36.199
Other than automatically blocking ip’s for too many 404 errors I can think of no other real solution and this in itself is quite ineffective due to the sheer number of ip’s.
Any suggestions on how to deal with this would be greatly appreciated as there appears to be no end to this attack and my websites performance really is taking a hit.
Read more here: 404 Bot Attack on My Website (DDoS of Sorts)