Van Riemsdyk added an assist to spoil Pittsburgh's home opener and James Reimer picked up the win with 28 saves.
Sidney Crosby scored for the Penguins to tie it at 2-2 with 12:43 remaining in the middle period, but van Riemsdyk tallied on a wrist shot with 5:26 left in the stanza to put the Maple Leafs back in front.
Marc-Andre Fleury surrendered all five Toronto goals on 24 shots. Pittsburgh had won its first two games of the season on the road.
Megyn Kelly: Santa Claus and Jesus are both white men
Susan Sarandon: I was stoned at almost every awards show