Van Riemsdyk added an assist to spoil Pittsburgh's home opener and James Reimer picked up the win with 28 saves.
Sidney Crosby scored for the Penguins to tie it at 2-2 with 12:43 remaining in the middle period, but van Riemsdyk tallied on a wrist shot with 5:26 left in the stanza to put the Maple Leafs back in front.
Marc-Andre Fleury surrendered all five Toronto goals on 24 shots. Pittsburgh had won its first two games of the season on the road.
Astronomers offer more expansive view of universe
Handler slams Piers Morgan: 'You're a terrible interviewer'