The Rangers were trailing 3-2 in the sixth when they erupted for six runs, ignited by A.J. Pierzynski's score-tying RBI double.
Elvis Andrus followed with a run-scoring single to put Texas ahead and Engel Betre's hit made it 5-3 before Kinsler unloaded a blast that one-hopped the left-field wall, sending home Andrus, Beltre and Leonys Martin.
He also drove in a run with third-inning single for Texas, which claimed the opener of a four-game series and stayed 1/2-game behind American League West-leading Oakland.
Derek Holland (7-4) earned the decision despite allowing five runs (three earned) on 10 hits over 6 2/3 innings. He struck out seven.
Orioles starter Scott Feldman (0-1) was lit up for seven runs on nine hits in a 5 1/3-inning stint.
Matt Wieters socked a two-run homer and Manny Machado went 3-for-5 with an RBI for Baltimore.