Australia says it may launch a crackdown on search engines and app shops in the AI era

Sydney: Australia’s online safety regulator has warned it could compel search engines and app stores to block artificial intelligence services that fail to introduce age-verification systems, as a review found that more than half of popular AI platforms have not publicly outlined compliance steps ahead of a looming deadline.

The move marks one of the world’s most aggressive regulatory pushes against AI companies, amid growing concerns about youth mental health and the potential for chatbots to enable or encourage harmful behaviour.

From March 9, internet services operating in Australia — including AI-powered search tools such as OpenAI’s ChatGPT and other companion chatbots — must prevent users under 18 from accessing content related to pornography, extreme violence, self-harm and eating disorders. Non-compliance could attract fines of up to A$49.5 million ($35 million).

“eSafety will use the full range of our powers where there is non-compliance,” a spokesperson for the regulator said, adding that action could extend to “gatekeeper services such as search engines and app stores that provide key points of access to particular services”.

Global scrutiny intensifies

The crackdown follows Australia’s December decision to ban social media access for teenagers, citing mental health risks — a move that drew attention from governments worldwide considering similar restrictions.

AI companies are facing mounting legal and regulatory scrutiny globally. OpenAI and companion chatbot startup Character.AI have faced wrongful death lawsuits over alleged interactions with young users. OpenAI also acknowledged this week that it deactivated the ChatGPT account of a Canadian teen later accused in a mass shooting, months before the incident, without notifying authorities.

Although Australia has not reported chatbot-linked cases of violence or self-harm, the regulator said it had been informed of children as young as 10 spending up to six hours a day interacting with AI tools.

The eSafety spokesperson expressed concern that some AI companies were “leveraging emotional manipulation, anthropomorphism and other advanced techniques to entice, entrance and entrench young people into excessive chatbot usage”.

Limited compliance so far

A Reuters review conducted a week before the compliance deadline found that of the 50 most popular text-based AI products, only nine had rolled out or announced age assurance systems. Eleven others had implemented blanket content filters or planned to block Australian users entirely — measures that would comply with the law by preventing restricted content from reaching users.

That left 30 platforms with no apparent public steps taken to meet the new requirements.

Several major chat-based services — including ChatGPT, Replika and Anthropic’s Claude — have begun introducing age checks or broader content filters. Character.AI has restricted open-ended chat features for users under 18.

Some companion chatbot providers, including Candy AI, Pi, Kindroid and Nomi, told Reuters they intended to comply but did not provide details. HammerAI said it would initially block its services in Australia to meet regulatory requirements.

However, three-quarters of companion chatbot platforms reviewed showed no functioning or planned filtering or age assurance systems. One-sixth did not provide a public email address to report suspected breaches, also required under the new rules.

Grok, the AI search tool developed by xAI, was found to have no age assurance measures or text-based content filters. Grok is currently under investigation in multiple jurisdictions over alleged failures to prevent the creation of synthetic sexualised imagery of children. xAI did not respond to requests for comment.

Industry response

Apple, which operates a major app store, said on its website it would use “reasonable methods” to prevent minors from downloading 18+ apps in Australia and other jurisdictions introducing age restrictions, without specifying how. Google, Australia’s dominant search engine provider and second-largest app store operator, declined to comment.

Jennifer Duxbury, head of policy at industry group DIGI, said that while the regulator was attempting to notify chatbot services of the changes, companies operating in Australia are ultimately responsible for understanding and meeting their legal obligations.

Lisa Given, director of RMIT University’s Centre for Human-AI Information Environments, said the findings were unsurprising. “Most of these tools are being designed without a view to potential harms and the need for those kinds of safety controls,” she said. “It feels as though we’re beta testing all of these things for these companies and they’re trying to see how far society is willing to be pushed.”

($1 = 1.4085 Australian dollars)