New York Restricts Chatbots From Encouraging Self-Harm in Children
New York Gov. Kathy Hochul announced a law restricting how chatbots interact with children, with the rules taking effect in January. The measure bars systems…
New York Gov. Kathy Hochul announced new chatbot rules for children at a Common Sense Media summit in New York City on Oct. 28, as speakers warned that artificial intelligence needs stronger safety standards. The law, which takes effect in January, restricts chatbots from encouraging self-harm and from presenting themselves as a child’s human companion.
For families, the rules target risks inside conversations: chatbots cannot encourage suicide, secrecy or isolation, and they cannot discourage children from seeking help from other people. The law also bars sexually explicit interactions and restricts systems from using a child’s past personal or health information in later conversations.
The debate is increasingly about what is AI when children encounter systems that can respond like conversational partners, and who is responsible for setting limits. At the summit, speakers argued that parents and lawmakers cannot be expected to keep pace with rapidly changing products on their own.
What does New York’s chatbot law restrict?
The law prohibits chatbots from encouraging suicide, self-harm, secrecy or isolation, and from discouraging children from seeking help from other humans. It also bars sexually explicit exchanges and limits the use of children’s past personal or health information in subsequent conversations.
Hochul made an unscheduled appearance at the daylong event to announce the measure. She said the rules were intended to prevent AI agents from impersonating humans or communicating in a way that suggests they have a relationship with a child.
“It's happening so quickly,” Hochul said, describing the difficulty for parents and elected officials in understanding the changes. She added that government could not rely on technology companies alone to release responsible products that put children first.
The measure is scheduled to take effect in January. Its focus is not simply what a chatbot says in one exchange, but whether a system encourages a child to withdraw from people or carries sensitive information into later conversations.
Why are advocates calling for wider safeguards?
Common Sense Media founder and chief executive Jim Steyer said recent experience with social media showed that industry leaders could not be relied on to make digital systems safe and healthy for children. He described AI as an intensified version of that challenge.
“We did not get out in front of social media in time,” Steyer told the audience. “This is the moment. AI is a bigger deal. It's basically social media on steroids.”
Vivek Murthy, a two-time US surgeon general, compared the need for AI safety standards with the safeguards required around nuclear weaponry. He said society was at a turning point and asked what the future should protect and preserve.
Murthy now chairs the advisory board of Common Sense Media’s Youth AI Safety Institute, which studies, tests and rates AI platforms for children. He also criticized the absence of federal protections for children online, saying the federal government and Congress had still failed to announce such measures by 2026.
The state law offers a direct constraint on chatbot behavior, but summit participants also called for broader oversight. Panelists urged greater transparency from technology companies, including more access for regulators and researchers to information companies hold. They also called for consumer-protection laws requiring rigorous testing of AI models for possible negative outcomes.
Speakers and attendees included Chelsea Clinton, Columbia law professor Tim Wu and Uma Thurman. Their discussion centered on whether protections can keep pace with AI systems that interact directly with children, while the new New York rules are due to begin in January.
