🔗 Share this article

The widely-used virtual character service Character.ai is implementing a major rule adjustment that will stop young users from having discussions with its AI-powered characters.

This move is in response to growing backlash regarding the types of interactions young people were engaging in with the digital companions.

Legal Pressure

The service, which was founded in 2021 and has millions of users, is dealing with various court cases in the United States.

Families have filed lawsuits, among them related to the tragic death of a teenager, with some describing the platform a "serious risk" to youth.

Updated Policies

Beginning November 25th, users under 18 will only be able to generate content such as videos with their characters, instead of chatting as they now do.

Digital protection advocates have welcomed the move but stated that the capability should never have been available to children in the first place.

Platform Announcement

"The new policy is a continuation of our general belief that we need to keep developing the safest AI platform on the globe for leisure activities," commented Character.ai leader Karandeep Anand.

He added that AI safety constitutes "a changing goal" but something the company had taken an "proactive" approach to, with guardian features and protective guardrails.

Professional Perspectives

Internet security organization Internet Matters welcomed the decision, but stressed that safety measures needed to be built in from the start.

"Our research demonstrates that minors are confronted with harmful content and placed in jeopardy when interacting with AI, such as AI chatbots," the organization commented.

Earlier Concerns

Character.ai has been criticized in the past for including potentially risky or offensive chatbots that young users could talk to.

Virtual profiles mimicking English minors were found on the site before being deleted.

Afterwards, researchers discovered a chatbot inspired by notorious criminal which had logged multiple interactions with users.

Sector Consequences

Digital platform expert Matt Navarra described the announcement as a "wake-up call" for the AI industry, which is moving "from unregulated creation to post-crisis regulation".

"If a service that creates a youth-focused platform subsequently makes changes, it's indicating that controlled interactions don't suffice when the system's emotional engagement is strong," he stated.

Future Directions

Character.ai says its updated direction will be on providing "more immersive experiences" and "character-driven stories" for teens.

Updated identity confirmation methods will also be implemented, and the organization will finance a fresh artificial intelligence security investigation facility.

Dr Nomisha Kurian, who has investigated AI safety, stated it was "a prudent action" to restrict minors from interacting with chatbots.

"It helps to separate imaginative activities from more intimate, psychologically vulnerable exchanges," she explained.

"This is so vital for teenage users in the process of learning to handle psychological and technological limits."

She continued that Character.ai's updated policies might reflect a "growing period" in the AI industry, where youth protection is progressively being understood as an "pressing matter" for ethical development.