The Bruins will try to advance to the Eastern Conference semifinals when they host Game 5 Friday.
Boston overcame a two-goal deficit by producing three consecutive scores in the second period -- the last two coming from Krejci in a span of less than 4 minutes.
Toronto tied it on Clarke MacArthur's goal with 2:37 to go in the second period and it remained a 3-3 deadlock through the regulation 60 minutes.
Krejci's winning goal came after a defensive lapse that allowed him to break free down the left wing. He fired a low shot that beat James Reimer.
Toronto lost consecutive home games in a playoff series for the first time in nine years.
Krejci then took over and he finished the night with five post-season goals, most in the NHL this year.
Tuukka Rask made 45 saves to get the win. Reimer stopped 41 shots.