The AI Safety Institute began operations in June, tasked with co-ordinating the federal government's response to rapid developments in artificial intelligence.
Labor committed nearly $30 million to establish the AI Safety Institute with about $8 million annually from the end of the decade.
Independent MP Kate Chaney isn't convinced the institute has the capacity to confront the sector's significant challenges.
"We're investing less than $8 million a year in our frontline defence against a technology that has a non-trivial risk of human extinction," she said on Thursday.
"We invest 200 times more than that in biosecurity because we know how important it is, and AI is more powerful, more expensive to remediate, and faster than any pest, and its impact could be felt across the whole economy."
Nine independent MPs are calling for an increase in funding to $100 million a year for an immediate response, while floating the idea of setting up an AI department for the long term.
The government is working through possible concessions with big tech companies on copyright to open the door to AI training centres being built in Australia.
Ms Chaney said Australia was in a uniquely terrible position.
"We have a lose-lose-lose situation," she said, noting artists don't have the incentive to create and aren't receiving compensation, and the government has no leverage if tech companies ditch investment in emerging technologies.
Stella Eve, a Canberra-based singer-songwriter, said allowing an AI model to scrape creative content to train a model to generate music was inherently wrong.
"The idea that a service or company can take my works and use them without credit or compensation makes me sick," she told AAP.
"I feel like my whole life, everything that I've ever been good at has been based on my ability to create something from nothing. ...Music is something that is made by people, not generated."
A parliamentary committee into AI will begin on Friday, hearing evidence from government department representatives.
Lifeline 13 11 14
Kids Helpline 1800 55 1800 (for people aged 5 to 25)
