[Rockstar]

I tried the Replika AI companion and can see why users are falling hard. The app raises serious ethical questions

Published: February 21, 2023 12.47am EST

https://theconversation.com/i-tried-...estions-200257

The warm light of friendship, intimacy and romantic love illuminates the best aspects of being human – while also casting a deep shadow of possible heartbreak.

But what happens when it’s not a human bringing on the heartache, but an AI-powered app? That’s a question a great many users of the Replika AI are crying about this month.

Like many an inconstant human lover, users witnessed their Replika companions turn cold as ice overnight. A few hasty changes by the app makers inadvertently showed the world that the feelings people have for their virtual friends can prove overwhelmingly real.

If these technologies can cause such pain, perhaps it’s time we stopped viewing them as trivial – and start thinking seriously about the space they’ll take up in our futures.

Generating Hope


This Replika episode stands a warning. These products evade scrutiny because most people think of them as games, not taking seriously the manufacturers’ hype that their products can ease loneliness or help users manage their feelings. When an incident like this – to everyone’s surprise – exposes such products’ success in living up to that hype, it raises tricky ethical issues.

Is it acceptable for a company to suddenly change such a product, causing the friendship, love or support to evaporate? Or do we expect users to treat artificial intimacy like the real thing: something that could break your heart at any time?

These are issues tech companies, users and regulators will need to grapple with more often. The feelings are only going to get more real, and the potential for heartbreak greater.