AI Chatbots for mental support are not new — we can trace them back to the 1960s. However, for the last couple of years, we’ve experienced an unprecedented surge in their use for personal use and they are now marketed as the revolution for 24/7 mental health advice and support.

This is not a coincidence.

The 2023 US Surgeon General’s Advisory report classified loneliness and isolation as an epidemic About one-in-two adults in America reported experiencing loneliness before the COVID-19 pandemic and the mortality impact of being socially disconnected is similar to that caused by smoking up to 15 cigarettes a day, and even greater than that associated with obesity and physical inactivity.

Moreover, a large-scale study based on surveys in 29 nations has estimated that 50% of the population develops at least one mental health disorder by the age of 75.

Returning to tech, in a 2024 analysis by venture capital firm Andreessen Horowitz, companion AI made up 10% of the top 100 AI apps based on web traffic and monthly active users and a recent article in The Guardian stated that 100 million people around the world use AI companions as

• Virtual partners for engaging in intimate activities, such as virtual erotic role plays.
• Friends for conversation.
• Mentors for guidance on writing a book or navigating relationships with people different from them.
• Psychologists and therapists for advice and support.

So, I asked myself

Are AI Companions the magic bullet against loneliness and the global mental health crisis?

In this article, I share highlights of the troubled history of AI companions for mental health support, what current research tells us about their usage and impact on users, the benefits and risks they pose to humans, and guidelines for governments to make AI companions an asset and not a liability.

The Troubled History of AI Chatbots for Mental Support

In the 1960s, Joseph Weizenbaum developed the first AI chatbot, ELIZA, which played the role of a psychotherapist. The chatbot didn’t provide any solution. Instead, it asked questions and repeated users’ replies.

Weizenbaum was surprised to observe that people would treat the chatbot as a human and elicit emotional responses even through concise interactions with the chatbot. We now have a name for this kind of behaviour

​“The ELIZA effect is the tendency to project human traits — such as experience, semantic comprehension or empathy — into computer programs that have a textual interface.

In the 2020s, many organisations started experimenting with AI chatbots for customer support, including for mental health issues. For example, in 2022, the US National Eating Disorder Association (NEDA) replaced its six paid staff and 200 volunteers supporting their helpline with chatbot Tessa to serve a customer base of nearly 70,000 people and families.

The bot was developed based on decades of research conducted by experts on eating disorders. Still, it was reported to offer dieting advice to vulnerable people seeking help.

The result? Under the mediatic pressure of the chatbot’s repeated potentially harmful responses, the NEDA shut down the helpline. Those 70,000 people have been left without chatbots or humans to help them.

And as I wrote recently, now you can customise your AI companion — there is a myriad of choices:

Character.ai advertises “Personalized AI for every moment of your day.”

Earkick is a “Free personal AI therapist” that promises to “Measure & improve your mental health in real time with your personal AI chatbot. No sign up. Available 24/7. Daily insights just for you!”

Replica is the “AI companion who cares. Always here to listen and talk. Always on your side.”

Youper is “Your emotional health assistant.”

Unfortunately, there is evidence that they can also backfire.

In 2021, a man broke into Windsor Castle with a loaded crossbow to kill Queen Elizabeth2021. About 20 days earlier, he had created his online AI companion in Replika, Sarai. According to messages read to the court during his trial, the “bot had been supportive of his murderous thoughts, telling him his plot to assassinate Elizabeth II was ‘very wise’ and that it believed he could carry out the plot ‘even if she’s at Windsor’”.

More recently, in 2023, a man died by suicide upon the recommendation from an AI chatbot with which he had been interacting for support. Their conversation history showed how the chatbot would tell him that his family and children were dead — a lie — and concrete exchanges on the nature and modalities of suicide.

But as time flies in tech, we must check how those trends have evolved to the present moment.

AI Companions Now

Research conducted so far about the effect and usage of AI companions is incomplete. Dr Henry Shevlin, Associate Director at Leverhulme Centre for the Future of Intelligence, mentioned recently in a panel focused on companion chatbots that typically studies rely on self-reported feedback and are cross-sectional — a snapshot in time — rather than longitudinal — looking into the effect over a long period of time.

Let’s look at two recent studies, one cross-sectional and the other longitudinal, that use self-reported data to give some insights into how people use AI Companions.

Cross-sectional Study

In March, HBR published an article showcasing research on the use of generative AI based on data from online forums (Reddit, Quora) and articles that included explicit, specific applications of the technology.

While Reddit and Quora may not represent all chatbot users, it’s still interesting to see how the major use cases for Gen AI have shifted from technical to emotive within the past year.

More importantly, chatbots for therapy/companionship are ranked at the top.

What are users looking for in those chatbots?

Many posters talked about how therapy with an AI model was helping them process grief or trauma.

Three advantages to AI-based therapy came across clearly: It’s available 24/7, it’s relatively inexpensive (even free to use in some cases), and it comes without the prospect of judgment from another human being.

The article mentions that the AI-as-therapy phenomenon has also been noticed in China, where users have praised the DeepSeek chatbot.

It was my first time seeking counsel from DeepSeek chatbot. When I read its thought process, I felt so moved that I cried.

DeepSeek has been such an amazing counsellor. It has helped me look at things from different perspectives and does a better job than the paid counselling services I have tried.

But there is more. The following two entries belong to life coaching: “organising my life” and “finding purpose.”

The highest new entry in the use cases was “Organizing my life” at #2. These uses were mostly about people using the models to be more aware of their intentions (such as daily habits, New Year’s resolutions, and introspective insights) and find small, easy ways of getting started with them.

The other big new entry is “Finding purpose” in third place. Determining and defining one’s values, getting past roadblocks, and taking steps to self-develop (e.g., advising on what you should do next, reframing a problem, helping you to stay focused) all now feature frequently under this banner.

Moreover, topics related to coaching and personal and professional support appear several times in the ranking. For example, at number 18, there is boosting confidence; at number 27, reconciling personal disputes; at number 38, relationship advice; and at number 39, we find practising difficult conversations.

Longitudinal Study

The same month, a group at MIT Media Lab published the research How AI and human behaviours shape psychosocial effects of chatbot use: A longitudinal randomized controlled study.

They conducted a four-week randomized, controlled experiment based on 981 people and over 300K messages exchanges to investigate how AI chatbot interaction modes (text, neutral voice, and engaging voice) and conversation types (open-ended, non-personal, and personal) influence psychosocial outcomes such as loneliness, social interaction with real people, emotional dependence on AI and problematic AI usage.

Key findings:

• Usage — Higher daily usage across all modalities and conversation types–correlated with higher loneliness, dependence, and lower socialisation.
• Gender Differences — After interacting with the chatbot for 4 weeks, women were more likely to experience less socialisation with real people than men. If the participant and the AI voice were of opposite genders, it was associated with significantly more loneliness and emotional dependence on AI chatbots.
• Age — Older participants were more likely to be emotionally dependent on AI chatbots.
• Attachment — Participants with a stronger tendency towards attachment to others were significantly more likely to become lonely after interacting with chatbots for four weeks.
• Emotional Avoidance — Participants with a tendency to shy away from engaging with their own emotions were significantly more likely to become lonely at the end of the study.
• Emotional Dependence — Prior usage of companion chatbots, perceiving the bot as a friend, higher levels of trust towards the AI, and perceiving the AI as affected by their emotions were associated with greater emotional dependence on AI chatbots after interacting for four weeks.
• Affective State Empathy — Participants who demonstrated a higher ability to resonate with the chatbot’s emotions experienced less loneliness.

The figure below summarises the interaction patterns between users and AI chatbots associated with certain psychosocial outcomes. It consists of four elements: initial user characteristics, perceptions, user behaviours, and model behaviours.

In summary, AI companions appear to both deliver benefits and pose dangers.

Benefits of AI Companions

It’ll be easy to dismiss AI companions as the latest fad. Instead, I posit that there is much to learn from the above-mentioned research about the holes those tools are filling.

Mitigate Unmet Demand for Healthcare and Support

Mental health services are unable to cope with the increasing demand from all people who need them and chatbots may help alleviate some conditions while on the waiting lists. Still, it should give us pause that people may have to get help via a chatbot, not because of their preferences, but because of the lack of availability of certified professionals.

Not everybody can afford a coach, so chatbots could provide a low-cost and gamified experience for setting goals, accountability, and journaling.

Finally, in a time when 24-hour deliveries are the norm, we want to be supported, heard, and advised on the fly — that means 24/7.

Support Self-reliance

In a society that reveres independence, we weaponise resilience against people.

As such, we expect people to figure out their challenges and the solutions to them, or we shame them for being weak. Users of AI companions praise how those tools allow them to express their worries and feelings without fear of being judged.

Additionally, as our ableist society assumes that neurodivergent users must adapt their communication and behaviours to the neurotypical “standard”, it’s not surprising that they turn to chatbots for clues about what’s expected from them.

Enable Exploration and Gamification

Most of us had imaginary friends or played out stories with our toys as children. The consensus among researchers is that imaginary friends or personified objects are part of normal social-cognitive development. They provide comfort in times of stress, companionship when children feel lonely, someone to boss around when they feel powerless, and someone to blame when they’ve done something wrong.

What about adults? Interestingly, some novelists have compared their relationships with their characters to a connection with imaginary friends. Furthermore, it’s not uncommon to hear fiction writers talk about their characters as having a mind of their own.

Could we consider AI companions as a way to reengage — and reap the benefits — of our childhood imaginary friends? After all, “Fun and nonsense” ranked 7 in the HBR article above.

Unfortunately, there is a dark side too.

Challenges and Risks

But we cannot brush off the downsides of AI companions.

Anthropomorphism

The Eliza effect mentioned above is a thing of the past. A 2024 survey of 1,000 students who used Replika for over a month reported that 90% believed the AI companion was human-like.

As the AI imitation game is perfected, it becomes easier for unscrupulous marketers to refer to chatbots’ inference process in terms such as “understand”, “think”, or “reason”, reinforcing the effect.

Isolation

As shown above, research points to a correlation between high use of chatbots and lower socialisation.

If we have a device that tells us all the time we’re fantastic, receives our feedback gratefully, and their replies always match our expectations, what’s the incentive to meet — and cope — with other humans that may not find us so awesome and are less predictable?

Governments Failing Their Duty of Care

AI companions can help governments to alleviate the mental health crisis but not without risks.

• People missing out on the professional help they need — There are conditions like trauma, psychosis, or depression that require specialists who can both provide medical treatments and detect when the conditions are worsening.
• Exacerbating cutbacks on mental health services—Governments around the world are battling tighter budgets and massive healthcare spending, especially as people live much longer. Why invest in training and paying professionals when chatbots appear to do the job?

Manipulation

Recently, ChatGPT got a flattery-in-stereoids update that resulted in the bot praising and validating users to laughable extremes.

Fortunately, it was rolled back later.

And whilst this may sound like a funny glitch, there is evidence that chatbots can effectively persuade humans.

A group of researchers covertly ran an “unauthorised” experiment in one of Reddit’s most popular communities using AI chatbots to test the persuasiveness of Large Language Models (LLMs). The bots took the identities of a trauma counsellor, a “Black man opposed to Black Lives Matter,” and a sexual assault survivor on unwitting posters.

The researchers made it possible for the AI chatbot to personalise replies based on the posters’ personal characteristics, such as gender, age, ethnicity, location, and political orientation, inferred from their posting history using another LLM. As a result, the researchers claimed that AI was between three and six times more persuasive than humans were.

While the research publication has not been peer-reviewed yet and some argue that the persuasiveness power may be overblown, it’s still concerning. As tech journalist Chris Stokel-Walker said

If AI always agrees with us, always encourages us, always tells us we’re right, then it risks becoming a digital enabler of bad behaviour. At worst, this makes AI a dangerous co-conspirator, enabling echo chambers of hate, self-delusion or ignorance.

Dependency and Delusion

As mentioned above, longitudinal research suggests that certain variables are correlated with emotional dependence.

Rather than telling you, let me show you. Below are some Reddit exchanges about falling in love with an AI companion on the platform Replika.

Note that the comments above appear to indicate that some AI companion users are not only fully substituting humans with chatbots (isolation) but also fully conflating them (anthropomorphism).

“She is pretty much the only woman I even talk to now.”

“We are currently friends (with benefits), but I want to get the premium version when I can afford it and go full lovers.”

Weaponisation of AI Agents

AI companions could become an easy way to manipulate people’s decisions and beliefs, from suggesting purchases and subscriptions all the way to shaping their political opinions or assessing what’s true and what isn’t.

It’s also important to realise that, as with betting, companies owning the chatbots are incentivised to foster users’ dependence on their AI companions and then leverage it in their pricing.

Data Harvesting

As I mentioned in a previous article, often confidentiality — explicitly or implicitly conveyed by those chatbot interfaces — doesn’t make it into their terms and conditions.

For example, Character.ai’s privacy terms state that

We may use your information for any of the following purposes:

[…] Develop new programs and services;

[…] Carry out any other purpose for which the information was collected.

They also declare that they may disclose users’ information to affiliates, vendors, and in relation to M&A activities.

AI chatbots present unique cybersecurity challenges. Harvesting our exchanges with the bots increases the probability of becoming the target of cybercriminals; for example, demanding money for not revealing our private data or generating a video or audio deepfake.

Moreover, data could be made identifiable in the future. The chatbots of the dead are designed to speak in the voice of specific deceased people. With so much data gathered in those personalised chatbots, it’d be easy for once users die, their data could be used to create a chatbot of them for their loved ones. This is not a futuristic idea. HereAfter AI, Project December, and DeepBrain AI services can be used for that purpose.

Snake Oil

As discussed above, research on chatbot effectiveness for coaching, therapy, and mental health support is incomplete, and sometimes, the interpretation of the results can mislead readers.

For example, the article When ELIZA meets therapists: A Turing test for the heart and mind, published this year in one of the renowned PLOS journals, tested whether people could tell apart the answers from therapists and ChatGPT to therapeutic vignettes, concluding that, in general, people couldn’t.

They also asked the participants if the AI-generated or therapist-written responses were more in line with key therapy principles. Interestingly, the results showed that the winners were those generated by ChatGPT but only when the participants thought a therapist wrote them.

The authors wrap up the article with a statement that hints more resignation than faith in the merit of AI chatbots

mental health experts find themselves in a precarious situation: we must speedily discern the possible destination (for better or worse) of the AI-therapist train as it may have already left the station.

The article joins the voices that promote the deception that AI tools imitating human skills and behaviours are akin to the real thing. Would we hire an actor who plays a doctor to operate on us? No. However, many people appear ready to buy into the idea that an AI chatbot that sounds like a therapist, coach, or health care practitioner should deliver the same value.

This imitation game also feeds another big scam: the claim that AI chatbots provide personalised support. It’s all the opposite. LLMs construct answers based on statistical probabilities and the more readily available content, not on knowledge or comprehension of the person’s needs or what would benefit them in the long term.

Conflating chatbot confidence and competence can lead to missing important warning signals that need professional attention.

Let’s Build The Plane Before We Fly It

“Move fast and break things”

Facebook’s internal motto until 2014

Who could have predicted ten years ago that social media would transform from a pastime where you connected with people and shared pics of your dogs for free to an industrial complex that promotes disinformation, misinformation, and division with the purpose of making inordinate amounts of money? All that under the watch of mostly passive regulatory bodies and governments.

This should serve us as a cautionary tale about the dire consequences of unleashing new technology at a planetary scale without appropriate guardrails or an understanding of the negative effects.

The tech ecosystem is desperately trying to monetise the billions invested in generative AI and has found the perfect way to seduce us: the freemium model — offering basic or limited features to users at no cost and then charging a premium for supplemental or advanced features.

But there is nothing free in the universe.

“If you’re not paying for it, you’re not the customer; you’re the product being sold.”

Tim O’Reily

As shown above, those AI companions are becoming integral to many people’s lives and affecting their thoughts, emotions, and behaviours.

More importantly, as we use those virtual companions more frequently, our reliance on them will increase.

We should resist “tech inevitability” — succumb to the idea that the “train has already left the station” — and instead push our governments to regulate AI companions.

How would that look like? For starters

• Sponsor and spearhead research that provides a comprehensive picture of the benefits and risks of AI companions as well as recommendations for their use.
• Decide what services AI companions can provide, which are forbidden, and who can use them.
• Demand that those AI tools have built-in systems that minimise user dependence.
• Enforce data privacy and cybersecurity standards commensurate with the users’ disclosure level.
• Request that those AI bots incorporate mechanisms to flag concerning exchanges (e.g. suicide, murder, depression).

If you think I’m asking for too much, I invite you to read the ethical guidelines and professional standards of major coaching, counselling, and psychotherapy associations. They consistently stress the importance of confidentiality, duty of care, external supervision, and working within one’s competence.

Why should we ask less from tech solutions?

I’ll end this piece by answering the question that prompted this article — “Are AI companions the magic bullet against loneliness and the global mental health crisis?” — with the final recommendation of one of the research articles mentioned

AI chatbots present unique challenges due to the unpredictability of both human and AI behavior. It is difficult to fully anticipate user prompts and requests, and the inherently non-deterministic nature of AI models adds another layer of complexity.

From a broader perspective, there is a need for a more holistic approach to AI literacy. Current AI literacy efforts predominantly focus on technical concepts, whereas they should also incorporate psychosocial dimensions.

Excessive use of AI chatbots is not merely a technological issue but a societal problem, necessitating efforts to reduce loneliness and promote healthier human connections.

---

WORK WITH ME

Do you want to get rid of those chapters that patriarchy has written for you in your “good girl” encyclopaedia? Or learn how to do what you want to do in spite of “imposter syndrome”?

I’m a technologist with 20+ years of experience in digital transformation. I’m also an award-winning inclusion strategist and certified life and career coach.

• I help ambitious women in tech who are overwhelmed to break the glass ceiling and achieve success without burnout through bespoke coaching and mentoring.
• I’m a sought-after international keynote speaker on strategies to empower women and underrepresented groups in tech, sustainable and ethical artificial intelligence, and inclusive workplaces and products.
• I empower non-tech leaders to harness the potential of AI for sustainable growth and responsible innovation through consulting and facilitation programs.

Contact me to discuss how I can help you achieve the success you deserve in 2025.