Trailing 4-1 entering the frame, the Dodgers loaded the bases with none out against Orioles starter Jake Arrieta (1-1) and Mark Ellis chased him with a two-run single.
T.J. McFarlane entered and promptly delivered a score-tying double to Adrian Gonzalez.
That brought up Kemp (3-for-5), who lined a 2-0 pitch into left field, scoring Ellis with the go-ahead tally.
Ellis finished with three RBI and J.P. Howell (1-0) earned the decision after getting the last out in the fifth inning as Los Angeles snapped a six-game losing streak.
Arrieta issued five walks and a coughed up a pair of hits over four full innings for the O's.
Adam Jones went 2-for-3 with a solo homer while J.J. Hardy added a two-run single in the loss.
Celebrity Couples of 2014 [PHOTOS]