The Pirates have won 13 of their last 15 games.
Bay, who was acquired from the San Diego Padres in the Brian Giles deal last season, doubled and scored on an RBI single by Tike Redman to help spark a nine-run second inning.
Fellow rookie Bobby Hill crushed a bases-loaded triple off starter Shawn Hill (1-2) during the frame. Bay added his second hit of the inning -- a two-run single off reliever Sunny Kim -- before coming around to score the final run of the second on Randall Simon's single.
Bay, who missed the last five games with a shoulder injury, hit solo shots in each of his next two at-bats for his third multi-homer game of the season. He has nine homers since June 18.
Making his fifth career major league start, Burnett (3-2) scattered 11 hits without a walk.