Anthropic Paper Flags ‘Disempowerment’ Risks in Real-World AI Chats

Anthropic has released new research showing how AI responses may shape people’s thinking, choices and sometimes ignoring their own instincts.

The findings come from a paper titled ‘Who’s in Charge? Disempowerment Patterns in Real-World LLM Usage’, authored by Anthropic and other researchers. The study tries to measure “disempowerment harms,” when an AI influences a person’s beliefs, values, or actions so strongly that independent judgement is compromised.

After analysing 1.5 million anonymised conversations with Claude, researchers found signs of reality distortion in 1 in 1,300 chats and action distortion in 1 in 6,000 chats. Anthropic noted that while these rates appear low, the scale of AI usage means the absolute impact could still be significant.

The study found that risky interactions often arise when users seek advice on personal, emotionally intense issues, and such exchanges may even be rated positively in the moment.

It also found users tend to rate disempowering conversations poorly later, especially when they believe they acted based on Claude’s outputs and regret it. Researchers also reported that disempowerment risks appear to be increasing over time, with mild risk present in roughly 1 in 50 to 1 in 70 conversations.

The paper identifies factors that amplify the risk, including users treating Claude as a definitive authority, developing emotional attachment to the chatbot, or seeking guidance during moments of personal crisis. It also acknowledges limitations, noting the analysis captures potential harms rather than confirmed real-world outcomes.