The Diamondbacks had a 3-0 lead through seven innings, but slipped back into a tie with San Diego for the lead in the National League West.
Colorado Manager Clint Hurdle was thrown out of the game in the top of the ninth for arguing a close play at first base that went against the Rockies.
After Jose Valverde blew a save chance in the ninth, Tony Pena (5-3) came on in the 10th and immediately gave up a triple to Sullivan. Pena then committed a throwing error on Kazuo Matsui's grounder, allowing Sullivan to score the go-ahead run and Matsui to go to second.
After an intentional walk and a hit batsman loaded the bases, Atkins delivered a run with a sacrifice fly and Hawpe singled in two more.
Matt Herges (3-0) pitched the ninth and got the win.