The prime minister and chancellor will now self-isolate as normal after contact with Health Secretary Sajid Javid, who tested positive for coronavirus.
The reversal comes just hours after they said they would take part in a pilot scheme involving daily testing.
Opposition parties said it suggested there was “one rule for them and another for the rest of us”.
Downing Street said Boris Johnson will conduct meetings remotely at Chequers.
Chancellor Rishi Sunak said on Twitter: “Whilst the test and trace pilot is fairly restrictive, allowing only essential government business, I recognise that even the sense that the rules aren’t the same for everyone is wrong.”
Labour leader Sir Keir Starmer said the prime minister and chancellor had been “busted yet again for thinking the rules that we are all following don’t apply to them”.
“The public have done so much to stick to the rules. At a time when we need to maintain confidence in self-isolation, parents, workers and businesses will be wondering what on earth is going on in Downing Street,” he said.
“The way the prime minister conducts himself creates chaos, makes for bad government and has deadly consequences for the British public.”
Labour’s Jonathan Ashworth had earlier told the BBC’s Andrew Marr Show that it was unfair that politicians appeared to have access to “VIP testing” to avoid self-isolation, while Liberal Democrat Leader Ed Davey asked if it was only available to “the privileged few”.
The managing director of the Iceland supermarket, Richard Walker, also criticised the proposal for Mr Johnson and Mr Sunak to avoid self-isolation.
“Shame the hundreds of Iceland staff who’ve been pinged can’t avoid self-isolation. We can all do a daily lateral flow test,” he said.
Mr Javid tested positive on Saturday morning after a meeting at Downing Street the day before, and both Boris Johnson and Rishi Sunak were contacted by NHS Test and Trace.
No 10 had said a workplace pilot scheme would allow the prime minister and chancellor to keep working from Downing Street by taking daily tests.
A spokesman said they would only be able to carry out essential government business and would self-isolate at all other times.
Stepping in for Mr Javid, who is self-isolating after his positive test, Housing Secretary Robert Jenrick had defended the plan on the Andrew Marr Show.
“The scheme is a well known and long-standing one, it’s not just available for politicians,” he said. But he added that his own department was not part of the scheme.
Case numbers are continuing to rise ahead of the lifting of legal rules on social contact in England on Monday, but businesses have raised concerns about staff shortages due to the numbers of people being told to self-isolate.
More than half a million alerts telling people to self-isolate were sent by the NHS Covid-19 app in England and Wales in the first week of July, with unions saying it was causing “havoc” on production lines.
But Mr Jenrick said self-isolation was “an important part of keeping the virus under control”, adding that a study had showed the NHS Covid app had prevented 600,000 infections and 8,000 deaths.
