Emergency crews were dispatched to Long Reef Beach on Sydney's northern beaches shortly after 10am on Saturday, responding to reports a man had suffered critical injuries, police said.
He was retrieved from the surf and brought to shore but died at the scene.
Police have closed the beach and are liaising with wildlife experts to determine the species of shark involved.
Two sections of a surfboard have been recovered and taken for examination.
Beaches from Manly to Narrabeen have been closed pending further advice.
The identity of the man is yet to be confirmed.
Witnesses say the victim was brought ashore by two fellow surfers and that distraught family members rushed to the scene.
AAP has been told injuries to the man's limbs were extensive.
Local surfer Bill Sakula told reporters at the beach he heard about the attack while preparing to leave for a morning surf.
"It's going to send shockwaves through the community," he said.
"Everyone is going to be a little bit nervous for a while."
A group believed to be friends and family of the victim could be seen gathered on the beach on Saturday afternoon.
Surf Life Saving NSW has deployed a drone to search for further shark activity.
"Our deepest condolences go to the family of the man involved in this terrible tragedy," chief executive Steve Pearce said in a statement.
"Please remain clear of the water at beaches in the vicinity and follow the direction of lifeguards and lifesavers," he said.
A Department of Primary Industries spokesperson said the department is assisting the response.
"NSW government shark biologists will assess photographs of the incident to determine the species of shark involved," they said.
Northern Beaches Council issued a statement thanking the "many involved ... for their swift and professional response including members of the public ... first responders, Lifeguards, SLS volunteers, police and paramedics".
"The beaches from Narrabeen to Manly will remain closed today with Lifeguards monitoring on jetskis. Beach goers are advised to check the website for ongoing closures in the coming days."
Police at Dee Why were scheduled address media on the tragedy at 1.30pm.
A junior surf competition was taking place at the beach on Saturday morning.
The attack is thought to be the first in NSW this year.
The last fatal attack in Sydney occurred in February 2022, when British diving instructor Simon Nellist was taken by a great white off Little Bay in the city's east.
On the northern beaches, the last known fatal attack was at North Narrabeen, 90 years ago.
Shark nets were installed at 51 beaches between Newcastle and Wollongong on Monday as part of a seasonal program.
The NSW government has announced a trial to reduce the number of shark nets, asking three councils including Northern Beaches Council to nominate a beach where nets could be removed.
Long Reef Beach does not have a shark net but nearby Dee Why beach does.
Additional shark alert drumlines were deployed in the area on Saturday between Dee Why and Long Reef beaches.
15°C