OpenAI outlines new mental health guardrails for ChatGPT

ChatGPT guardrails for teens and people in emotional distress will roll out by the end of the year, OpenAI promised Tuesday. Why it matters: Stories about ChatGPT encouraging suicide or murder or failing to appropriately intervene have been accumulating recently, and people close to those harmed are blaming or suing OpenAI.ChatGPT currently directs users expressing suicidal intent to crisis hotlines. OpenAI says it does not currently refer self-harm cases to law enforcement, citing privacy concerns.The big picture: Last week the parents of a 16-year-old Californian who killed himself last spring sued OpenAI, suggesting that the company is responsible for their son's death.Also last week, The Wall Street Journal reported that a 56-year-old man killed his mother and himself after ChatGPT reinforced the man's paranoid delusions, which professional mental health experts are trained not to do.Last month, the mother of a 29-year-old wrote an op-ed in The New York Times about how her daughter asked ChatGPT to help write her suicide note. ChatGPT did not encourage the woman to kill herself, but also did not report that she was a danger to herself — which a human therapist would be mandated to do.Between the lines: The work to improve how its models recognize and respond to signs of mental and emotional distress has already been underway, OpenAI said in a blog post today.The post outlines how the company has been making it easier for users to reach emergency services and get expert help, strengthening protections for teens and letting people add trusted contacts to the service.Driving the news: OpenAI's post on Tuesday previews its plans for the next 120 days, and says the company is making "a focused effort" to launch as many of these improvements as possible this year.How it works: "We're beginning to route some sensitive conversations, such as when signs of acute distress are detected, to reasoning models like GPT-5-thinking," OpenAI says.GPT-5's thinking model applies safety guidelines more consistently, per the company. A network of over 90 physicians across 30 countries will give input on mental health contexts and help evaluate the models, OpenAI says. Zoom in: ChatGPT users must be 13 and up, with parent permission for users under 18. Within the month, parents will be able to link their accounts with those belonging to their teens for more direct control.Once accounts are linked, the parent can manage how ChatGPT responds and "receive notifications when the system detects their teen is in a moment of acute distress." "These steps are only the beginning," OpenAI wrote in the blog post.Character.AI, which has also been blamed for more than one teenager's suicide, introduced similar parental controls in March. Reality check: Keeping savvy kids from accessing sites and apps they're not old enough to use is a thorny problem. Convincing those ages 13-18 to link their accounts to their parents' could be an even tougher sell.What they're saying: The problem is as old as the internet itself, Kate O'Loughlin, CEO of kids' digital media platform SuperAwesome, told Axios last week.O'Loughlin says everything cool and new on the internet is created by adults with adults in mind, but kids will always want to use it — and find pathways to riskier environments.Then, she says, the platforms tend to lay the responsibility for monitoring kids on these platforms solely on the parents. What we're watching: One way to potentially mitigate these tragedies is to stop allowing ChatGPT or other bots to act as therapists — or to stop designing them to act like any person at all.Go deeper: AI's mental health fix: Stop pretending it's humanIf you or someone you know needs support now, call or text 988 or chat with someone at 988lifeline.org. En español.

Comments

World news

Russia raises tensions with NATO as Putin shows off his battlefield nuclear weapons

Independent

Comments

Similar News

Oracle’s astonishing jam-tomorrow OpenAI trade

Nvidia and OpenAI to back major UK investment in artificial intelligence

OpenAI and Microsoft agree key terms in contract renegotiation

OpenAI wants lawsuit by Canadian news outlets moved from Ontario to U.S.

OpenAI pressed on safety after deaths of ChatGPT users

OpenAI set to start mass production of its own AI chips with Broadcom

OpenAI and Meta say they're fixing AI chatbots to better respond to teens in distress

OpenAI’s game of chicken

Parents of teenager who took his own life sue OpenAI

Microsoft talks set to push OpenAI’s restructure into next year

Elon Musk accuses Apple and OpenAI of ‘conspiring’ to suppress his rival AI chatbot Grok in antitrust lawsuit

Elon Musk accuses of Apple and OpenAI of stifling AI competition in antitrust lawsuit

xAI sues Apple and OpenAI over deal to integrate ChatGPT into iPhones

Musk’s AI startup sues OpenAI and Apple over anticompetitive conduct

Musk tried to enlist Zuckerberg in $100bn bid for OpenAI

OpenAI and Anthropic clamp down on investment vehicles

World news

Russia raises tensions with NATO as Putin shows off his battlefield nuclear weapons

Charlie Kirk traveled across America. His security varied from venue to venue

Two men jailed for plot to kill president with witchcraft in Zambia

Dad threatened by ICE deportation days after being shot during robbery

‘Gaza is burning’: Israel reportedly begins ground offensive after Netanyahu threatens Hamas leaders

Donald Trump says he will sue New York Times for $15bn

Israel is committing genocide in Gaza, UN probe finds

‘Tooth-in-eye’ surgery enables man to see again after 20 years

Trump says he will sue the New York Times for $15bn

Is Keir Starmer already toast? I don’t think so

Apple TV+ swept the Emmys – these are the 11 shows to watch

Haitians yearn for home as gangs welcome them and police warn it's too dangerous

Father kills his own baby daughter on Facebook Live before taking his own life

Watch: JD Vance hosts Charlie Kirk's daily talk radio show

Why more women get cancer in India - but more men die

Malawi votes for next president as cost-of-living bites

They were shot dead for protesting against corruption in Nepal

Belarus and Russia's show of firepower appears to be a message to Europe

Google owner reveals £5bn AI investment in UK ahead of Trump visit

US Secretary of State inaugurates Israeli settler-led archaeological site in Jerusalem

Trump says he will sue New York Times for $15bn

Nigerian chef breaks world record for largest jollof rice dish

Chinese teens to pay $300,000 for urinating in soup

Report those who celebrate Charlie Kirk death to employers, Vance says

Israel escalates bombardment as defence minister says 'Gaza is burning'

Beaten and held in Russia for three years - but never charged with a crime

Trump's state visit is mired with potential pitfalls despite careful planning

Trump says the U.S. military targeted a boat allegedly carrying drugs from Venezuela

Federal Judge Declines to Intervene for Migrants Deported to Ghana

Florida attorney general says cops should stop enforcing open carry gun ban after court ruling

Russia shows off conventional and nuclear military might in drills — and raises tensions with NATO

Tyson Foods to eliminate high-fructose corn syrup in brands like Jimmy Dean and Hillshire Farm

A new Ebola outbreak has been declared. Here’s what we know

Luigi Mangione due in court amid double jeopardy fight in UnitedHealthcare CEO’s killing

How hands-off investing could have made you six times more than saving this year

‘I fought in Afghanistan but faulty earplugs left me with physical and mental scars’

Birmingham City and the battle of ambition vs tradition

I’m a first-time buyer – why does being self-employed mean I can borrow less?

Canada’s friendly seafood capital with a road trip and vineyards nearby

Sorry book snobs – Richard Osman’s The Impossible Fortune is a delight

Trump’s Tariffs Start to Bite in Vietnam

ADHD and bipolar patients face medicine delays – even when drugs are in stock

My Man Utd XI to save Amorim’s job – with three major changes

Please stop suggesting a date night – I’d rather you unblock the drain

DWP imposes ‘gagging order’ on disability experts ahead of PIP review

"Not welcome": Rubio says U.S. is deporting visa holders who celebrated Kirk's death

What to know about the US deportations to non-citizen African countries

Photos show how an Indonesian motorbike mechanic became a macaw trainer

Polls open in Malawi’s presidential and parliamentary elections

Trump World’s Deals With the U.A.E.

‘Squid Game’ Star Lee Jung-jae Talks About Korean Fan Culture

President Trump’s State Visit to the UK: What to Know

National park to remove exhibit of famed photograph showing former slave’s scarred back, says report

Ukraine-Russia war latest: Poland asks Nato to impose no-fly zone over Ukraine to stop Putin’s drone incursions

Drills in Belarus and drone incursions into Romania: How the war in Ukraine has already become a European conflict

E-bike battery fire displaces 11 people from Vancouver home

Drought level in the Okanagan increases under new provincial system

Brit tourist 'puts men to shame' in Benidorm with astonishing record break

'Tinder Swindler' con artist arrested in Georgia

Senate clears Trump pick Miran to Fed board ahead of key interest rate vote