🔗 Share this article

Technology companies and child protection organizations will be granted authority to assess whether AI tools can generate child exploitation material under recently introduced UK laws.

Substantial Increase in AI-Generated Illegal Content

The declaration came as revelations from a safety monitoring body showing that cases of AI-generated CSAM have more than doubled in the last twelve months, rising from 199 in 2024 to 426 in 2025.

Updated Legal Structure

Under the changes, the authorities will allow designated AI companies and child safety groups to examine AI models – the underlying technology for conversational AI and image generators – and verify they have adequate protective measures to stop them from creating depictions of child exploitation.

"Ultimately about stopping exploitation before it occurs," stated Kanishka Narayan, noting: "Specialists, under strict protocols, can now detect the risk in AI models promptly."

Addressing Regulatory Obstacles

The changes have been introduced because it is against the law to create and own CSAM, meaning that AI developers and other parties cannot generate such images as part of a testing process. Until now, officials had to delay action until AI-generated CSAM was published online before dealing with it.

This legislation is aimed at preventing that problem by enabling to halt the production of those images at source.

Legal Structure

The amendments are being introduced by the authorities as revisions to the criminal justice legislation, which is also implementing a prohibition on owning, producing or sharing AI systems developed to create exploitative content.

Real-World Impact

This recently, the minister toured the London base of Childline and heard a mock-up call to counsellors featuring a account of AI-based exploitation. The call portrayed a teenager requesting help after being blackmailed using a explicit deepfake of themselves, created using AI.

"When I hear about young people facing extortion online, it is a cause of extreme anger in me and rightful anger amongst families," he said.

Alarming Data

A leading online safety foundation reported that cases of AI-generated abuse content – such as online pages that may contain numerous images – had more than doubled so far this year.

Cases of category A material – the most serious form of exploitation – increased from 2,621 images or videos to 3,086.

• Girls were predominantly targeted, making up 94% of prohibited AI depictions in 2025
• Portrayals of newborns to toddlers rose from five in 2024 to 92 in 2025

Sector Reaction

The law change could "represent a crucial step to guarantee AI tools are secure before they are launched," commented the chief executive of the online safety organization.

"AI tools have enabled so survivors can be targeted repeatedly with just a few clicks, giving criminals the ability to make potentially limitless amounts of advanced, lifelike exploitative content," she added. "Material which further exploits victims' trauma, and renders young people, particularly girls, more vulnerable on and off line."

Support Session Data

The children's helpline also released details of support sessions where AI has been referenced. AI-related harms discussed in the sessions include:

• Employing AI to rate body size, physique and appearance
• Chatbots dissuading children from talking to trusted guardians about abuse
• Being bullied online with AI-generated material
• Digital blackmail using AI-faked images

Between April and September this year, Childline conducted 367 counselling sessions where AI, conversational AI and related terms were discussed, four times as many as in the equivalent timeframe last year.

Half of the mentions of AI in the 2025 interactions were related to mental health and wellbeing, encompassing using AI assistants for support and AI therapeutic apps.