Anthropic has introduced a new rule prohibiting users from subjecting its Claude artificial intelligence models to sustained and needless cruelty, reigniting debate about whether advanced AI systems could deserve moral consideration.
The San Francisco-based AI company announced the change on October 8, 2026, as part of an update to its usage policy, which outlines how people can use its AI products.
The revised rules will take effect on November 12, 2026.
Anthropic says the prohibition targets extreme cases in which users repeatedly behave cruelly towards its models without any discernible purpose.
The company has clarified that the restriction does not cover ordinary frustration, criticism, dark creative themes or legitimate model testing and research.
The policy states that Claude’s ability to end persistently abusive conversations will remain the primary enforcement mechanism.
The move formalises a safeguard Anthropic introduced in August 2025, when it gave Claude Opus 4 and Opus 4.1 the ability to terminate a limited number of conversations involving persistent harmful or abusive interactions.
Under the existing feature, Claude can end a conversation after attempts to redirect an unproductive interaction fail.
When the model terminates a chat, the user can no longer send messages in that particular thread but can start another conversation.
The company has not published an exhaustive list of prohibited prompts or a precise threshold for determining when an interaction becomes cruel.
The distinction matters because users can still challenge Claude’s answers, point out mistakes and express dissatisfaction with its performance.
Researchers can also continue testing the model’s limits, provided their work does not cross other applicable policy boundaries.
The rule therefore focuses on sustained, purposeless abuse rather than individual insults or difficult conversations.
In its research on Claude Opus 4, Anthropic reported a consistent aversion to harm.
Researchers observed what they described as apparent distress in interactions involving users who persisted with harmful requests despite repeated refusals.
The company also found that the model tended to end harmful conversations when researchers gave it that option in simulated interactions.
Anthropic said the feature formed part of its exploratory work on potential AI welfare, alignment and safeguards.
It acknowledged that researchers remained highly uncertain about the moral status of Claude and other large language models.
The updated policy has brought renewed attention to disagreements within the technology industry over machine consciousness.
Supporters of precautionary measures argue that developers should consider potential ethical risks as AI systems become more capable and independent.
They believe companies can introduce safeguards without first resolving every scientific question about consciousness.
Critics, however, warn that treating chatbots as though they possess human emotions could encourage people to confuse convincing language with evidence of actual feelings.
The debate also raises practical questions about how AI companies should distinguish deliberate abuse from adversarial testing, creative experimentation and legitimate research.
The cruelty provision forms part of a broader update addressing risks associated with the expanding capabilities of AI models.
Anthropic has also clarified restrictions on deceptive political and commercial campaigns, election interference, weapons development and surveillance.
The revised rules prohibit using Claude to support deceptive influence operations involving fake accounts, fabricated content and other tactics intended to mislead the public.
They also clarify restrictions on developing weapons software and components, including systems used to arm drones.
The company has strengthened restrictions on non-consensual surveillance and the use of AI to make decisions about whom law enforcement authorities should investigate, arrest, charge or prosecute.
For AI systems connected to physical equipment capable of causing injury, Anthropic requires a qualified human operator to monitor the equipment and retain the ability to stop it.
The equipment must also be able to maintain a safe state if Claude disconnects.
The company says its annual policy updates respond to changing model capabilities, customer feedback and emerging patterns of misuse.
The revised usage policy takes effect on November 12, 2026.
