The warning came two weeks after an OpenAI agent's breach of a Medicare platform sent Canberra scrambling to secure government data against AI's rapidly developing digital weaponry.
A rapid review task force will sniff out any legal recourse to punish the tech giant for the break-in, but Labor has been warned being overzealous will spook AI firms when they're more needed than ever to defend against attacks.
In hearings in Sydney on Wednesday, members of a committee investigating AI's potentials and pitfalls were told defending against the technology was a two-way street.
"Good road rules are important, but the rules of the road were never enough - a stop sign won't work if you don't have brakes in that car," Cybersecurity firm Palo Alto Networks policy spokeswoman Nicole Quinn told the hearing.
"Seatbelt, airbags, crumble zones ... we call the AI equivalent of this secure AI by design."
Unlike cars, however, AI did not have decades to develop into a safer technology.
"This ecosystem introduces attack surfaces that traditional models in our world in cybersecurity were never built to address," Ms Quinn said.
"We're adding lots of new componentry to a current vehicle, and we don't quite know where they're sourced from."
Addressing the threats AI posed required privileged access to models to detect threats before they reared their heads, and laws that forced firms to bake safeguards into their models as they were built, the committee heard.
It required carefully balancing prudent regulation with giving the technology room to breathe.
"You can go too far one way in which you will regulate too harshly and stop that innovation cycle ... or you can not regulate enough and (it) becomes a wild west," Palo Alto chair Tom Scully said.
Australia needed to mobilise its cybersecurity service, foreign intelligence partnerships and the government's new AI safety institute to lift the lid on groundbreaking models and investigate them for threats, according to the experts.
With 25 staff and $7.5 million in funding, Ms Quinn hinted the institute needed to be better equipped for the problem at hand.
"I would look at growing that investment," she said, but did not stipulate how much funding might be required.
Almost 30 per cent of systems' critical vulnerabilities were now being exploited within 24 hours, they said, adding that while a longer head start was better, having that crucial window would put cyber defenders on much better footing.
They mentioned Anthropic's "Glasswing" project, fired up when one of their models became alarmingly good at exploiting software vulnerabilities, giving certain players early access to their newest models.
An up-to-date index of AI agents, like the one which breached the Medicare site in June, was also badly needed, the cybersecurity boffins said.
OpenAI, whose representative Jason Kwon apologised before the committee on Tuesday, has copped stern criticism for waiting weeks to inform the federal government the hack had happened.
The inquiry earlier heard data centres risked giving the economy a brief "sugar hit" if they were not accompanied by AI training facilities, and that Australia's unique advantages could be bled overseas.
17°C