The Phillies had forged a 3-1 lead in the top of the 10th on Hunter Pence's two-run single off Javy Guerra, but the Dodgers came back in the bottom of the frame off Philadelphia closer Jonathan Papelbon.
Luis Cruz led off the half-inning with a double and came home on Bobby Abreu's pinch-hit single one out later. Abreu stole second, went to third on Tony Gwynn Jr.'s infield hit scored the tying run when Kemp beat out shortstop Jimmy Rollins' throw from deep in the hole for an RBI single.
Then in the 12th, Mark Ellis drew a one-out walk from Jake Diekman (1-1) and Kemp followed by blasting a pitch into the right field bleachers.
Juan Rivera added a solo shot as the Dodgers -- who came into the contest three games behind National League West-leading San Francisco -- broke a four-game skid.
Jamey Wright (4-2) earned the victory with a scoreless top of the 12th.
Dennis Rodman pledges to end trips to North Korea
Interpol investigating stolen passports on missing flight