ChatGPT will no longer provide teenagers with practical information about weapons and security systems, even when their questions appear harmless, OpenAI said.
The restriction will apply to teen accounts regardless of whether there are indications of violent intent.
OpenAI announced the changes in a blog post, alongside plans to publish a Community Safety Report detailing its efforts to prevent violence. The company said it made more than 200 referrals to law enforcement agencies in the third quarter.
The new safeguards are expected to begin rolling out over the next few weeks.
Less detail on past attacks for all users
OpenAI also plans to restrict the level of detail ChatGPT provides about past mass attacks and the individuals responsible, even when such information is publicly available online.
The chatbot will no longer provide detailed reconstructions of attacks, including information about the weapons used by perpetrators and how they obtained them.
It will also limit detailed comparisons of the scale of harm caused by different attacks.
“These limits apply regardless of a user’s age or reason for asking,” OpenAI wrote.
However, users will still be able to ask about the impact of such incidents on victims and affected communities, as well as how to identify potential warning signs, OpenAI said.
ChatGPT will also be able to automatically end conversations that show signs of a user becoming fixated on violence or expressing an intention to seriously harm another person.
If the chatbot terminates multiple conversations for such reasons, OpenAI may temporarily suspend the user’s account.
OpenAI said it has strengthened its age prediction system, which uses behavioural signals to estimate whether a user is under 18.
The system continuously monitors new signals and automatically switches users to the teen experience when necessary.
Adults mistakenly placed in the teen category can verify their age through Persona, OpenAI’s age verification partner.
