From d52e09a4c1ff99546943ed0232c23acafc4c9ff0 Mon Sep 17 00:00:00 2001 From: Wirlaburla Date: Fri, 9 May 2025 22:13:58 -0500 Subject: [PATCH] fix http.md and add a few more --- http.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/http.md b/http.md index 7a1a6f1..865f4b8 100644 --- a/http.md +++ b/http.md @@ -8,8 +8,8 @@ Clients blocked by user agent are given a 471 response code. | `~libwww-perl` | ? | Library often used with scrapers that don't respect robots.txt. | Never | | `~Go-http-client` | ? | Most likely a Matrix server (DDoS). | Never | | `~Synapse` | ? | Matrix server AKA DDoSer. | Never | -| `~(Condu(wu)?it|Tuwunnel)` | 2025-05-09 | Matrix server AKA DDoSer. | Never | -| `~Byte(spider|dance)~` | ? | Chinese. Doesn't respect robots.txt. DDoSer. | Never | +| `~(Condu(wu)?it\|Tuwunnel)` | 2025-05-09 | Matrix server AKA DDoSer. | Never | +| `~Byte(spider\|dance)` | ? | Chinese. Doesn't respect robots.txt. DDoSer. | Never | | `~AhrefsBot` | ? | Doesn't respect robots.txt. | Never | | `~PetalBot` | ? | Chinese. Doesn't respect robots.txt. | Never | | `~Kinza` | ? | Doesn't respect robots.txt. | Never | @@ -19,16 +19,18 @@ Clients blocked by user agent are given a 471 response code. | `~serpstatbot` | ? | Webstat bot. Abusive. | Never | | `~ClaudeBot` | 2025-05-09 | AI Scraper. | Never | | `~AmazonBot` | 2025-05-09 | AI Scraper. | Never | -| `~GPTBot` | 2025-05-09 | AI Scraper that doesn't respect robots.txt. | Never | +| `~GPT` | 2025-05-09 | AI Scraper that doesn't respect robots.txt. | Never | | `~OpenAI` | 2025-05-09 | AI Scraper. | Never | | `~DotBot` | 2025-05-09 | Chinese scraper. | Never | | `~AsyncHttpClient` | 2025-05-09 | Scraper that doesn't respect robots.txt. | Never | | `~Expanse` | 2025-05-09 | Scraper with a really bad user agent. | Never | +| `~CheckHost` | 2025-04-02 | Webstat bot. Abusive. | Never | +| `~MJ12` | 2025-04-02 | Scraper. Does not respect robots.txt | Never | ## Referals Clients blocked by referer are given a 472 response code. | Match | Added | Reason | Expires | | - | - | - | - | -| `~(soyjak.(st|party))` | ? | Attempted Brigading with CP material. | Never | +| `~(soyjak.(st\|party))` | ? | Attempted Brigading with CP material. | Never | | `~gusfckschulz(.wixsite.com)?` | ? | Attempted Brigading with CP material. | Never | | `~onionfarms.com` | ? | No reason given. | Never | \ No newline at end of file