Starmer, in the top job for less than two years, had vowed on Monday to stay the course after large swathes of his parliamentary party turned on him following one of the worst defeats for Labour in local elections on May 7.
British media reports said several cabinet ministers, including interior minister Shabana Mahmood and foreign minister Yvette Cooper, have told Starmer he needs to set out a date at which he will step down.
Almost 80 MPs, from different ideological wings of the party, have called for Starmer to set out a timetable for his departure in order to trigger a leadership competition.
Senior minister Darren Jones told Times Radio on Tuesday that Starmer was listening to colleagues and it was up to him to make a decision.
"He's listening to colleagues, and he's talking to colleagues. I can't get ahead of any decision he may or may not take," Jones said.
Starmer had sought to shore up his position on Monday when he promised in a speech to act more boldly and with more urgency to tackle Britain's many problems.
But almost immediately his MPs started to publicly call for him to quit or signal that he would go, pushing borrowing costs higher.
Starmer had said the country would never forgive the centre-left Labour Party if it embarked on a leadership challenge, just two years after its huge parliamentary majority was supposed to bring an end to the political chaos that had gripped the country since Britain voted to leave the European Union 10 years ago.
Starmer is Britain's fourth prime minister in five years.
The move against him comes a day before King Charles is due to set out the government's legislative agenda in a parliamentary ceremony of pomp and splendour on Wednesday.
