The Bruins will try to advance to the Eastern Conference semifinals when they host Game 5 Friday.
Boston overcame a two-goal deficit by producing three consecutive scores in the second period -- the last two coming from Krejci in a span of less than 4 minutes.
Toronto tied it on Clarke MacArthur's goal with 2:37 to go in the second period and it remained a 3-3 deadlock through the regulation 60 minutes.
Krejci's winning goal came after a defensive lapse that allowed him to break free down the left wing. He fired a low shot that beat James Reimer.
Toronto lost consecutive home games in a playoff series for the first time in nine years.
Krejci then took over and he finished the night with five post-season goals, most in the NHL this year.
Tuukka Rask made 45 saves to get the win. Reimer stopped 41 shots.
Exploding whale video goes viral on Internet
Couple mistakenly served bag of cash at McDonald's drive-thru