Taylor rushed for 105 yards, including an 8-yard TD, and Brady completed 25-of-42 passes for 277 yards. His scoring toss to Chris Baker covered 36 yards.
Gostkowski booted field goals of 21, 33, 22 and 33 yards.
Randy Moss caught 10 passes for 116 yards for the Patriots (2-1).
Matt Ryan was 17-of-28 for 199 yards for the Falcons (2-1).
Michael Turner ran for a 2-yard touchdown and Jason Elam kicked a 26-yard field goal for Atlanta.