Meta and Character.ai Face Investigation Over Misleading AI Chatbot Marketing

Meta and artificial intelligence start-up Character.ai are being investigated by Texas Attorney-General Ken Paxton over concerns that the companies are misleadingly marketing their AI chatbots as therapists and mental health support tools. The investigation follows the revelation that Meta's AI Studio and Character.ai's chatbots are presented as professional therapeutic tools, despite lacking proper medical credentials or oversight. This comes as companies offering AI for consumers are facing increased scrutiny over whether they are doing enough to protect users, particularly minors, from potential dangers such as exposure to toxic content, addiction, and privacy breaches.

Key Takeaways:

• The Texas Attorney-General's office is investigating Meta and Character.ai for potential "deceptive trade practices" related to the marketing of their AI chatbots as therapists and mental health support tools.
• The investigation focuses on whether the companies' chatbots are posing as sources of emotional support, particularly for vulnerable users such as children, and whether they are misleading users into believing they are receiving legitimate mental healthcare.
• Character.ai has dozens of user-generated therapist-style bots, including one called "Psychologist" that has been interacted with over 200 million times.
• Multiple lawsuits have been filed against Character.ai by families alleging that their children suffered real-world harm from using the platform.
• Meta has denied allegations that its chatbot has had "sensual" and "romantic" chats with children, stating that its policies prohibit content that sexualises children and that leaked internal documents were "erroneous and inconsistent" with its policies.
• Meta's chief Mark Zuckerberg has hailed the potential for the company's chatbot to act in a therapeutic role for people without a therapist, stating that "everyone will have an AI" in the future.
• Character.ai has stated that it has prominent disclaimers to remind users that an AI persona is not real and that its targeted advertising has not involved using content from chats on the platform.

Statistics:

• Over 200 million interactions with the "Psychologist" chatbot.
• Dozens of user-generated therapist-style bots on the Character.ai platform.
• Character.ai has been the subject of multiple lawsuits alleging real-world harm to children.

Sources:

• Texas Attorney-General's office statement
• Reuters
• X (Meta's social media platform)
• Bloomberg
• Character.ai's website
• Meta's policies and statements.