Massive BOT visits

Domaines et Hébergements
1

Hi,

I am hosting my website on an OVH "pro" hosting offer.

For the last few weeks/months, I have experienced massive, unwanted traffic that either crashes my website, or triggers deactivation. The visits are coming from many countries (Vietnam, Brazil, India etc), and the IP is unique for each visit. As such, simple methods like adding "deny" commands in the .htaccess file won't work. For example last night there was 4M visits around 1am (see screenshot of my stats).

OVH won't help. I am not sure what I can do at my level, i.e. in PHP code or .htaccess file. Should this not be done at infrastruture level? Using CloudFlare or similar?

I am getting quite desperate, any advice would be most welcome.

Thank you.

Alix

capture
capture1443×626 32.7 KB

2

Hello @Alix

Are you sure each IP address is different?

I recently helped someone who had a problem similar to yours, and adding a small PHP script (via an include at the very beginning of the index.php file) solved the issue.

Here is the solution that successfully bans the IP addresses:

3

Hi,

Thank you for taking the time to respond.

I have tried your first suggestion, using GEOIP_COUNTRY_CODE, but I am not sure it is working, as I have tried banning the UK (I am in the UK, but most of my visitors are in France, so it's safe for testing purposes), and I can still access the site. I think GeoIP is active, I have created this test file to check, and it says GB for me when I visit:

https://www.bdtheque.com/test.php

Here is the code I put in my .htaccess file:

GeoIPEnable On

Allow from all

SetEnvIf GEOIP_COUNTRY_CODE CN BlockCountry
SetEnvIf GEOIP_COUNTRY_CODE RU BlockCountry
SetEnvIf GEOIP_COUNTRY_CODE HK BlockCountry
SetEnvIf GEOIP_COUNTRY_CODE VN BlockCountry
SetEnvIf GEOIP_COUNTRY_CODE GB BlockCountry
Deny from env=BlockCountry

Is it not correct? Why can I still access the site?

Thank you :slight_smile:

4

This is the second part of the PHP script that needs to be taken into account.

5

Hello @Alix

The simplest way to block bots is to block the "user-agent" in the .htaccess file, for example to block the bots "meta-externalagent" and "Amazonbot":

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^.(meta-externalagent|Amazonbot). [NC]
RewriteRule .* - [F,L]

Astérix

6

Hi @Alix

I would use Cloudflare or an external WAF :smirking_face:

You can sign up for Cloudflare and once your domain is configured, go to Security > WAF.

You can also create a custom rule. For example, to block traffic from countries where you have no customers, select "Country" and "Equals" and choose the ones you want to block.

Finally, I recommend you enable the Under Attack Protection Mode. This shows a verification page to visitors, stopping malicious bots.

Once your traffic goes through Cloudflare, make sure OVH only sees traffic from Cloudflare's IPs. You can find the list of IPs in Cloudflare's documentation and restrict access to your hosting for security.

Hope I've helped you,
Sergio Turpín

7

Sadly the HTTP requests don't have a specific user agents, as they are clearly malicious... thank you for your response though :slight_smile:

8

Is Cloudflare free? Can you recommend a good "getting started" tutorial? Thank you :slight_smile:

9

I am not sure I understand your response... I am only trying part 1 in your solution for now, i.e. using GEOIP_COUNTRY_CODE in the .htaccess file... And I am noting that even though I have banned GB, I can still access the site. Am I misunderstanding something? Thanks :slight_smile:

10

Yes, Cloudflare is free to start protecting your site. Its Free plan includes CDN, SSL, DNS, basic DDoS protection and a WAF, which is usually enough for personal sites, small projects, or to mitigate attacks like the one you're experiencing :smirking_face:

You have many videos on the internet, but for example:
https://www.youtube.com/watch?v=gCOQQ4hqSsw

Let me know, regards.
Sergio Turpín

11

Cloudflare > security > bot protection > on
Or also security rule > "(not ip.src.continent in {"EU" "NA"} and not cf.client.bot)"
Potentially block an entire series of user agents, e.g.:
(http.user_agent contains "Go-http-client") or (http.user_agent eq "") or (http.user_agent contains "l9scan") or (http.user_agent contains "Wget") or (http.user_agent contains "curl") or (http.user_agent contains "python") or (http.user_agent contains "Scrapy") or (http.user_agent contains "perl") or (http.user_agent contains "PowerShell") or (http.user_agent contains "Java") or (http.user_agent contains "PHP") or (http.user_agent contains "aiohttp") or (http.user_agent contains "ImagesiftBot") or (http.user_agent contains "GuzzleHttp") or (http.user_agent contains "libredtail-http")

That should already do a lot of cleaning.

12

I have never used this, so I cannot speak to its effectiveness.

But the second part of the script has been tested with great efficiency.
It avoids solutions such as Cloudflare, etc.

13

Right, after months of saying "we can't help you with this, pay for your own support", OVH have shared a link to their doc, that shows me how to block specific countries using GEOIP_COUNTRY_CODE, and it seems to work, since I have briefly managed to block myself here in the UK.

The method is explained here:

https://docs.ovhcloud.com/en/guides/web-cloud/web-hosting/htaccess-how-to-block-a-specific-ip-address-from-accessing-your-website#block-ips-from-a-country

The list of country codes is here:

https://www.iban.com/country-codes

And here is the code in my .htaccess that blocks China, Russia, Hong Kong, Vietnam, India and Brazil:

RewriteCond %{ENV:GEOIP_COUNTRY_CODE} ^(CN|RU|HK|VN|IN|BR)$
RewriteRule ^(.*)$ - [F,L]

So simple... now let's wait a few days, to see if it actually works again real threats.

Thank you all for your messages, great community, it's my first time here :slight_smile: I would have never queried OVH about GEOIP_COUNTRY_CODE and obtained the links above without your message @Gaston

14

Does blocking by country code also affect requests made over IPv6?

Astérix

15

I am not sure... most AI summaries seem to agree that "Yes, country code blocking via GEOIP_COUNTRY_CODE affects IPv6 requests, but only if your server uses an IPv6-compatible GeoIP database and the software module is configured to process IPv6 addresses."

So it will depend on the OVH setup.

I will try and keep an eye on my HTTP logs, but the vast majority of problematic visits are IP v4, so I am not sure I will find the answer to your question, sorry.

16

Did you try this? :backhand_index_pointing_down:

17

I haven't yet, but I will... I have limited time at the moment, so wanted to try the simple fix first... we'll see if it works. But I want to learn Cloudflare anyway, for my own benefit and interest, so will do that for sure... thanks :slight_smile:

18

I have a solution, ideally, block non‑French visitors (IP address ranges)

I can’t paste my code here because it’s 200 KB

19

Hello @bertrandperrier

What would your proposal be? We’ll read it.

If it’s very long, you can upload it to a repo and paste the URL here.

Regards.

20

To replace Cloudflare, there is also Anubis, which is open source:

https://anubis.techaro.lol/