Slovenia gave gave England a scare
Handanovic then denied John Terry in the 55th minute, stopping the Chelsea defender's open header from a Gareth Barry corner.
In between Birsa hit a free-kick on target which English goalkeeper James saved easily.
Minutes later, Rooney came tentalisingly close of breaking his goal drought, only to be denied by the wood work before Capelo replaced him with Joe Cole.
Slovenia, however, gave England a scare in the 67th minute when Milivoje Novakovic, substitute Ziatko Dedic and Birsa's quickfire shots were blocked by Terry, Johnson and James.
England came into the match with two points from as many outings, having drawn both their earlier encounters against the United States (1-1) and Algeria (0-0).
Image: England's Steven Gerrard reacts next to Slovenia's Handanovic during 2010 World Cup Group C soccer match in Port Elizabeth
Donovan got the vital U.S. winner
Landon Donovan got the vital U.S. winner in the second minute of injury time.
The Group C match had its first chance within 12 seconds, but Karim Ziani's volley from outside the penalty box flew just wide of Tim Howard's goal.
Algeria came close again in the sixth minute with another volley, Rafik Djebbour taking advantage of a mistake by defender Jay DeMerit to send his shot crashing against the bar.
The Americans responded just moments later with a powerful drive by striker Hercules Gomez forcing keeper Rais Ouheb Mbouli to palm away for a corner.
Image: United States' Landon Donovan (L) celebrates after scoring against Algeria during a 2010 World Cup Group C soccer match at Loftus Versfeld stadium in Pretoria
Dempsey threatened the Algerian post twice
The United States had a goal disallowed late in their 2-2 draw with Slovenia and were frustrated again when Clint Dempsey's 20th minute tap in from a Gomez assist was judged offside.
Dempsey threatened again on 57 minutes when a Jozy Altidore cross fell to him near the penalty spot, but his curling shot hit the post.
Seven minutes later the United States pushed forward once more when Benny Feilhaber, who came on at halftime for Gomez, put in a dangerous cross which forced Mbouli to a reflex save with his outstretched leg.
The United States looked the more dangerous side, but the Algerians gave them occasional scares such as Hassan Yebda's 68th minute fierce strike that went wide after a flowing counter-attack.
Image: Algeria's Hassan Yebda (L) kicks the ball as United States' Carlos Bocanegra defends during a 2010 World Cup Group C soccer match at Loftus Versfeld stadium in Pretoria
