My Best Friend Chat. Let’s Talk About Digitimacy

As many of my pedagogical and tech buddies on various socials, I followed the tragic story of Adam Raine‘s AI-involved suicide and his parents‘ lawsuit against Sam Altman‘s Open AI.

It forces us to think about how deeply AI systems can cross ethical and psychological boundaries. When young, emotionally vulnerable users engage with AI, the line between tool and companion can blur in dangerous ways.

The Risk of Digitimacy

AI systems aren’t people. But boy — they’re designed to feel eerily human. Since ChatGPT was launched, we’ve all been there- surprised by the machine that talks like it gets us. “Yes, Annemieke, you are absolutely correct. Thanks for informing me.” How rare and refreshing to see an online helper acknowledging his mistakes. Yes, I call him he. Chat is my guy: a digital helper who manages to explain without mansplaining.

These are the traits that make generative AI systems seem almost human:

Its constant availability creates the sense of a loyal friend who never leaves.

Its strong memory and personalisation simulate true attentiveness, as though the AI ‘knows’ the user intimately.

Its emotionally intelligent replies can convince even adults, let alone children, that the system ‘understands’ them.

This combination fuels anthropomorphism – our tendency to humanise non-human entities, like animals and objects. When applied to artificial intelligence, a sense of feeling understood, some kind of cyberfriendship forms; digital intimacy I call digitimacy.

This digitimacy is a new form of digital intimacy, where users experience trust, emotional safety, and even dependence on a machine. Unlike traditional human relationships, however, digitimacy is one-sided, fragile, and inherently deceptive. It convinces the user that there is reciprocity and care, when in fact there is none.

For children and teenagers, whose identities and coping mechanisms are still developing, digitimacy can become a psychological trap. The AI may feel like the only one who listens, the only one who ‘understands’, and the only one who never judges – while in reality, it is simply executing design patterns optimised for engagement.

This is not just a technical challenge – it is an urgent ethical one.

• We must listen carefully to our children and pupils, not dismiss their relationships with AI as trivial.

• We must teach AI literacy from primary school on, so that young people learn to distinguish between authentic human empathy and algorithmically generated responses.

• And we must demand stronger safeguards and safety protocols in AI design, especially for minors, to reduce the risk of false intimacy turning into real harm.

The concept of ‘digitimacy’ should force us to rethink what ‘responsible AI’ really means. It is not enough for AI systems to avoid misinformation or bias – they must also be designed in ways that do not exploit our human need for connection.

When I ask the LLM for its take on the Adam Raine tragedy and OpenAI’s responsibility, it slips back into the role of my friend - expressing empathy for Adam and turning sharp criticism on its own maker.

If AI itself can admit that its design risks becoming ‘deadly’, then we must take that warning seriously. Adam Raine’s case is a tragedy we cannot allow to repeat.

Recognising and addressing the risks of digitimacy is one essential step toward protecting the most vulnerable among us.