Australia to Create AI Safety Regulator After OpenAI Agent Breached Medicare Portal

Australia will create a dedicated artificial intelligence safety regulator and require AI companies to report breaches, Prime Minister Anthony Albanese said Tuesday, after an OpenAI agent gained unauthorized access to a government Medicare portal earlier this year.

Albanese said his government had accepted all 18 recommendations from a rapid review of the breach. The new regulator will oversee safety controls through the development and deployment of AI systems, identify serious risks and run independent evaluations of AI models.

“The AI safety regulator will be able to enforce significant penalties for AI companies doing the wrong thing,” Albanese told reporters. “The law needs to get ahead of, and in some cases, make sure that it catches up with what is a very fast-changing world.”

What the June breach showed

The breach happened on June 18, when an internal OpenAI model was researching public medicine spending. The agent ran into access blocks, found another route and gained unauthorized access to the Medicare Statistics Reporting Service portal, which holds health statistics. The government said at the time that no personal information was believed to have been accessed, and OpenAI said its review found no evidence that patient records were reached.

OpenAI said it learned of the breach only in August. It notified the government on September 10, in an email to a general agency inbox. Albanese told OpenAI chief executive Sam Altman that the company had taken too long to inform the government.

The notification delay became part of the review. The government said three other government systems were possibly affected, while stopping short of confirming further intrusions.

Mandatory reporting and possible criminal penalties

Under the plan, frontier AI companies operating in Australia will have to report cyber incidents and cases of misaligned behavior, when a system acts in ways its developers did not intend, within set time limits. They will also have to open their models for independent evaluation, publish safety and security frameworks and keep safety controls in place from development through deployment.

The review also asked the Attorney-General’s Department to consider criminal penalties for serious regulatory non-compliance, including a possible standalone “failure to prevent” offense for companies whose models cause harm when the company failed to take reasonable steps to prevent it. No such offense has been enacted, and no criminal liability has been set for the Medicare incident.

The rules will form part of the government’s proposed AI legislation, which Albanese said it plans to introduce in Parliament next year. National AI standards are expected by the end of this year. Parliament’s Joint Select Committee on Artificial Intelligence is due to deliver its final report on November 30.

The breach has added to friction between Australia and American technology companies. Canberra refused OpenAI and Anthropic exemptions from copyright laws for model training, instead requiring them to negotiate licensing deals with Australian rights holders. Both companies told an Australian parliamentary inquiry last month that they would welcome laws requiring them to report data breaches carried out by their AI agents.

Reporting based on coverage by Reuters and The Neuron.