Australia led South Africa 42-17 in the 64th minute of Sunday's match in Perth, but the game was turned on its head when the Wallabies had a remarkable four players sin binned during a seven-minute stretch.
Captain Harry Wilson was the first to go in the 70th minute, with scrumhalf Ryan Lonergan, star back Joseph-Aukuso Suaalii and centre Hunter Paisami following suit shortly after.
It meant that from the 76.16 mark right through to the 80th minute, the Wallabies had just 11 players on the field, with Wilson and Lonergan only able to return for the last play after the final siren.
South Africa used the numerical advantage to close the gap to four points, but the Wallabies' intense defence had chewed up so much time that the Springboks were left with just one last-gasp chance to snare the win.
Australia's defence stood firm after the final siren, with an eventual obstruction call going against South Africa to signal the end of the pulsating contest, allowing the Wallabies to hang on for a famous 42-38 win.
Williams, who skippers the Western Force, stepped into the Wallabies captaincy role when Wilson was sent off in the 70th minute, and his calmness proved crucial in guiding Australia to victory.
The towering lock even had to deal with the bizarre situation of referee Andrea Piardi using a mobile phone during key reviews because of a breakdown in communication equipment.
Wallabies fullback Tom Wright hailed Williams as a "leader of men".
"The boys have enormous respect for Jez because of the way that he plays and the way he carries himself," Wright said.
"He's no nonsense. He gets along with number one to number 40 in the squad.
"The boys would do anything for him, and for him to step into the furnace, arguably when it's at its hottest with the least amount of players he can probably play with, the guys got right behind him."
Wilson's captaincy reign doesn't look set to end any time soon, with his heroic on-field displays and calmness off the pitch proving to be a huge asset.
But the assured display of Williams will give Wallabies coach Les Kiss plenty of heart knowing there will be a reliable option to turn to if Wilson is absent for any reason.
Williams played down the leadership role he played in Sunday's win.
"It's not something I've experienced before, but there were lots of guys out there that I could lean on," Williams said.
"You've just got to problem solve on the run, and guys like Donno (Ben Donaldson) and Wright were really leading in that space."
WALLABIES TO THE BIN
69.29: Harry Wilson yellow-carded for a dangerous tackle
70.24: Ryan Lonergan yellow-carded for offside
72.46: Joseph-Aukuso Suaalii yellow-carded for offside
76.16: Hunter Paisami yellow-carded for a dangerous tackle
