Anthropic Intensifies Web Crawling Affecting Websites

Anthropic has dramatically increased web crawling activity in recent months, prompting complaints from website operators, SEO professionals, and publishers who say high request volumes strain servers and ignore robots.txt directives. Reports identify ClaudeBot and anthropic-ai user-agents making tens to hundreds of thousands of requests, leading site owners to use IP blocks, CAPTCHAs, legal action, or EU opt-outs; Anthropic says it provides blocking guidance and is reviewing practices.