diff --git a/http.md b/http.md index 7a1a6f1..865f4b8 100644 --- a/http.md +++ b/http.md @@ -8,8 +8,8 @@ Clients blocked by user agent are given a 471 response code. | `~libwww-perl` | ? | Library often used with scrapers that don't respect robots.txt. | Never | | `~Go-http-client` | ? | Most likely a Matrix server (DDoS). | Never | | `~Synapse` | ? | Matrix server AKA DDoSer. | Never | -| `~(Condu(wu)?it|Tuwunnel)` | 2025-05-09 | Matrix server AKA DDoSer. | Never | -| `~Byte(spider|dance)~` | ? | Chinese. Doesn't respect robots.txt. DDoSer. | Never | +| `~(Condu(wu)?it\|Tuwunnel)` | 2025-05-09 | Matrix server AKA DDoSer. | Never | +| `~Byte(spider\|dance)` | ? | Chinese. Doesn't respect robots.txt. DDoSer. | Never | | `~AhrefsBot` | ? | Doesn't respect robots.txt. | Never | | `~PetalBot` | ? | Chinese. Doesn't respect robots.txt. | Never | | `~Kinza` | ? | Doesn't respect robots.txt. | Never | @@ -19,16 +19,18 @@ Clients blocked by user agent are given a 471 response code. | `~serpstatbot` | ? | Webstat bot. Abusive. | Never | | `~ClaudeBot` | 2025-05-09 | AI Scraper. | Never | | `~AmazonBot` | 2025-05-09 | AI Scraper. | Never | -| `~GPTBot` | 2025-05-09 | AI Scraper that doesn't respect robots.txt. | Never | +| `~GPT` | 2025-05-09 | AI Scraper that doesn't respect robots.txt. | Never | | `~OpenAI` | 2025-05-09 | AI Scraper. | Never | | `~DotBot` | 2025-05-09 | Chinese scraper. | Never | | `~AsyncHttpClient` | 2025-05-09 | Scraper that doesn't respect robots.txt. | Never | | `~Expanse` | 2025-05-09 | Scraper with a really bad user agent. | Never | +| `~CheckHost` | 2025-04-02 | Webstat bot. Abusive. | Never | +| `~MJ12` | 2025-04-02 | Scraper. Does not respect robots.txt | Never | ## Referals Clients blocked by referer are given a 472 response code. | Match | Added | Reason | Expires | | - | - | - | - | -| `~(soyjak.(st|party))` | ? | Attempted Brigading with CP material. | Never | +| `~(soyjak.(st\|party))` | ? | Attempted Brigading with CP material. | Never | | `~gusfckschulz(.wixsite.com)?` | ? | Attempted Brigading with CP material. | Never | | `~onionfarms.com` | ? | No reason given. | Never | \ No newline at end of file