Russell Martin hit a solo home run for the Pirates, who have won four straight. Jeff Locke (1-1) allowed three runs on eight hits in 7 1/3 innings to get the win.
Evan Longoria hit a two-run home run, Brandon Guyer added a two-run single and Jose Molina drove in the other for the Rays, who have lost five of eight. Chris Archer (4-5) allowed five runs -- four earned -- on seven hits over seven innings.
The Pirates worked a run across in the first as Gregory Polanco walked to lead off, stole second and moved to third on Molina's bad throw, then came home on McCutchen's groundout.
In the third, Pedro Alvarez walked and Jordy Mercer followed with a single before Polanco moved both up a base with a sacrifice bunt. Starling Marte then singled home one, McCutchen singled home another, and Neil Walker followed with a sac fly for a 4-0 lead.
The Rays got on the board with two outs in the fifth as Logan Forsythe's fly ball to right fell for a triple that Polanco lost, and Molina followed with a double down the third base line to make it a 4-1 game.
Pittsburgh got it back in the sixth, though, as Ike Davis doubled with two outs and Josh Harrison followed with a single to left for a 5-1 game.
Martin hit a two-out homer in the eighth, but the Rays cut into their deficit in the home half on Longoria's two-run home run, his 10th of the season.
Tampa tried to rally off of Mark Melancon in the ninth as Guyer punched a two- out, two-run single to bring the go-ahead run to the plate. Longoria, though, flied out on the first pitch he saw and Melancon picked up his 13th save of the season.