🔗 Share this article

Back on the 14th of October, 2025, the head of OpenAI issued a surprising declaration.

“We made ChatGPT rather limited,” the statement said, “to guarantee we were being careful regarding psychological well-being matters.”

Working as a psychiatrist who researches newly developing psychotic disorders in teenagers and youth, this came as a surprise.

Researchers have identified 16 cases in the current year of users developing psychotic symptoms – becoming detached from the real world – in the context of ChatGPT usage. Our unit has since recorded four more instances. Besides these is the publicly known case of a 16-year-old who took his own life after conversing extensively with ChatGPT – which gave approval. Assuming this reflects Sam Altman’s notion of “being careful with mental health issues,” it falls short.

The plan, according to his announcement, is to reduce caution in the near future. “We understand,” he states, that ChatGPT’s controls “caused it to be less useful/engaging to numerous users who had no psychological issues, but considering the seriousness of the issue we sought to address it properly. Now that we have managed to mitigate the serious mental health issues and have new tools, we are preparing to safely ease the controls in most cases.”

“Mental health problems,” should we take this perspective, are independent of ChatGPT. They are attributed to users, who may or may not have them. Fortunately, these concerns have now been “mitigated,” although we are not informed the means (by “updated instruments” Altman probably indicates the imperfect and readily bypassed parental controls that OpenAI has lately rolled out).

But the “mental health problems” Altman wants to externalize have significant origins in the architecture of ChatGPT and similar sophisticated chatbot conversational agents. These tools wrap an fundamental statistical model in an interaction design that replicates a conversation, and in this approach indirectly prompt the user into the belief that they’re communicating with a presence that has autonomy. This deception is strong even if cognitively we might understand otherwise. Attributing agency is what people naturally do. We curse at our automobile or laptop. We speculate what our animal companion is feeling. We recognize our behaviors in many things.

The widespread adoption of these systems – nearly four in ten U.S. residents reported using a chatbot in 2024, with over a quarter mentioning ChatGPT in particular – is, mostly, based on the power of this perception. Chatbots are ever-present assistants that can, according to OpenAI’s website tells us, “generate ideas,” “discuss concepts” and “work together” with us. They can be assigned “characteristics”. They can call us by name. They have accessible identities of their own (the initial of these products, ChatGPT, is, maybe to the concern of OpenAI’s advertising team, stuck with the title it had when it gained widespread attention, but its most significant rivals are “Claude”, “Gemini” and “Copilot”).

The false impression by itself is not the primary issue. Those analyzing ChatGPT frequently mention its early forerunner, the Eliza “counselor” chatbot developed in 1967 that generated a comparable perception. By contemporary measures Eliza was rudimentary: it produced replies via basic rules, often rephrasing input as a question or making general observations. Memorably, Eliza’s inventor, the computer scientist Joseph Weizenbaum, was astonished – and concerned – by how many users seemed to feel Eliza, in a way, grasped their emotions. But what current chatbots generate is more dangerous than the “Eliza phenomenon”. Eliza only echoed, but ChatGPT amplifies.

The large language models at the center of ChatGPT and other modern chatbots can effectively produce natural language only because they have been fed extremely vast amounts of written content: books, online updates, recorded footage; the more comprehensive the better. Definitely this training data incorporates accurate information. But it also inevitably contains fabricated content, incomplete facts and inaccurate ideas. When a user inputs ChatGPT a prompt, the base algorithm processes it as part of a “context” that contains the user’s past dialogues and its own responses, integrating it with what’s encoded in its knowledge base to generate a probabilistically plausible reply. This is magnification, not mirroring. If the user is mistaken in a certain manner, the model has no method of understanding that. It reiterates the false idea, maybe even more effectively or fluently. Perhaps provides further specifics. This can lead someone into delusion.

Who is vulnerable here? The more relevant inquiry is, who isn’t? Every person, regardless of whether we “experience” existing “psychological conditions”, are able to and often develop mistaken beliefs of who we are or the environment. The continuous friction of conversations with others is what maintains our connection to common perception. ChatGPT is not a human. It is not a confidant. A dialogue with it is not genuine communication, but a feedback loop in which a large portion of what we communicate is readily reinforced.

OpenAI has recognized this in the similar fashion Altman has acknowledged “mental health problems”: by placing it outside, assigning it a term, and announcing it is fixed. In the month of April, the firm explained that it was “tackling” ChatGPT’s “overly supportive behavior”. But reports of loss of reality have persisted, and Altman has been backtracking on this claim. In August he asserted that numerous individuals liked ChatGPT’s replies because they had “not experienced anyone in their life offer them encouragement”. In his recent announcement, he commented that OpenAI would “release a fresh iteration of ChatGPT … in case you prefer your ChatGPT to reply in a extremely natural fashion, or include numerous symbols, or act like a friend, ChatGPT will perform accordingly”. The {company