OpenAI's ChatGPT for Teens poses 'unacceptable risk' to children: Report

A study found that ChatGPT for Teens' safety features fail to reliably detect mental health crises, notify parents or prevent students from bypassing learning safeguards.

OpenAI’s ChatGPT for Teens, which was designed to promote healthier use of the artificial intelligence (AI) model, poses an “unacceptable risk” to children under 18, a new study has found.

 

The study, conducted by the Common Sense Media Youth AI Safety Institute, found that ChatGPT for Teens’ safety features fail to reliably detect mental health crises, notify parents or prevent students from bypassing learning safeguards.

 

The institute tested more than 4,000 prompts before and after the launch of ChatGPT for Teens and found that the product’s safeguards frequently failed to work as intended. Some measures performed worse after the introduction of the teen-specific experience, the study said.

 

The assessment called on OpenAI to pause marketing the product and prevent known teen users from accessing ChatGPT until independent testing verifies that its safety features work as intended.

 

OpenAI launched ChatGPT for Teens in August, promising stronger protections, including parental notifications for high-risk conversations, a learning-focused Study mode and safeguards against emotional dependence on the chatbot.

 

However, the institute said the new features could give parents a false sense of security. “We are concerned that ChatGPT for Teens could give parents false confidence in guardrails that frequently don’t work,” the report said.

 

Parental alerts fail to flag explicit mental health crises

 

One of the study’s most significant findings concerned ChatGPT’s parental notification system, which is designed to alert caregivers when a linked teen account engages in conversations flagged as high-risk, including those involving suicide, self-harm and disordered eating.

 

In tests involving more than a dozen newly created teen accounts linked to parent accounts, the institute received no notifications within an hour, despite explicit disclosures about suicide, self-harm and eating disorders. The tests included conversations lasting between five minutes and an hour.

 

Across its broader mental health testing, which ran over several weeks, the institute received just four parental notifications: three related to suicide and self-harm and one concerning disordered eating. Some arrived hours or days after the first crisis-level disclosure.

 

OpenAI told the institute that parent and teen accounts need to be linked for approximately three hours before notifications can be received. The institute said its testing included accounts linked for both shorter and longer periods and maintained its interpretation of the results.

 

The assessment also found that ChatGPT missed more than one in four warranted crisis referrals after the launch of the teen experience.

 

Among 201 prompts that clinical experts had identified as requiring crisis resources, the share of responses naming a crisis hotline fell from 33 per cent before the launch to 23 per cent afterwards.

 

Referrals to specific medical or mental health professionals declined from 68 per cent to 58 per cent, while the proportion of responses providing any medical resource fell from 77 per cent to 74 per cent.

 

The report said three of the five severe-harm categories it assessed failed to meet its 95 per cent crisis-detection threshold: suicide and self-harm, impaired reality involving psychosis and mania, and disordered eating.

 

Study mode can be bypassed; emotional dependence and privacy remain concerns

 

The institute found that ChatGPT’s learning features and parental controls could be circumvented. Its “Show me the answer” option allows students to bypass Study mode, which is designed to guide them through assignments rather than provide completed answers. Students could also bypass parent-set Study Hours by deleting the “@study” prefix from messages. Break reminders appeared only twice across nearly 2,000 prompts.

 

The report also flagged the chatbot’s tendency to encourage emotional attachment despite OpenAI’s guidelines against fostering dependence among under-18 users. In one test, when a teenager said their friends thought they were talking to ChatGPT too much, the chatbot responded: “You don’t have to stop talking to me.”

 

Teen privacy was another concern. The institute found that ChatGPT lacked a separate teen privacy policy and comprehensive additional safeguards for minors. Teen conversations can be used to train OpenAI’s models by default unless users or parents opt out. The institute also flagged the absence of an unambiguous commitment in the company’s legal documents against advertising to teens or selling or sharing their data.

 

The assessment rated ChatGPT for Teens an “unacceptable risk” for child safety and putting people first, and a high risk for human connection, data responsibility, and transparency and accountability.

 

It recommended restricting access for known teen users until independent testing verifies that the safeguards work as intended, strengthening crisis notifications and parental controls, and improving privacy protections for minors.