Officers searched a derelict property in the semi-rural suburb of Dural on Sydney's north-west outskirts on Thursday night after identifying it as a makeshift stronghold for the kidnappers.
But there was no sign of the hostage, and police on Friday admitted things didn't look good for his safe recovery.
"It's been seven days now, so hope is fading," Detective Acting Superintendent Andrew Marks said.
"We have grave concerns for his safety, but we won't give up trying to find him."
No one lives at the Dural property, described by police as derelict and out of view from the street.
Police believe the property was seen in videos that emerged of the grandfather being restrained in a previously unknown location.
"Good old fashioned policing" lead authorities to Dural, Det Act Supt Marks said, before they were drawn to the house.
Evidence suggesting Mr Baghsarian had been at the property in the past seven days was seized, and police have established a crime scene.
Police on Friday were seen outside the ageing red-brick home, which is set over two levels and sits on a two-hectare property surrounded by large trees.
Mr Baghsarian's family spoke publicly for the first time earlier this week, describing the kidnapping as a surreal and harrowing ordeal.
"Our family is living through a nightmare we never thought possible," they said in a brief statement released via police.
Mr Baghsarian's lack of personal links to any criminal or underworld activity is making the case difficult for investigators, along with his family, according to Det Act Supt Marks.
"This is not the world that they live in, it's taken them by shock," he said.
"They're anxious, they're devastated, and they want some answers, and that's what we're here today to try and (do)."
No ransom has been demanded, leaving police to appeal directly to the kidnappers to return the grandfather to a safe location.
Police are continuing to appeal for public assistance to help identify and arrest those responsible for his kidnapping.
They have asked anyone who saw suspicious attendees at the abandoned property, or in Dural more generally, to come forward.
"Even if it is the most insignificant information, let us be the judge of that," Det Act Supt Marks said.