Indianapolis trailed, 20-19, when it took over on its own 20-yard line with 2:22 left. The Colts faced fourth and 10 from their own 36 after a sack by Ravens linebacker Peter Boulware.
Peyton Manning was flushed out of the pocket and Boulware was again in his face when the Indianapolis quarterback lofted a pass down the left sideline in the direction of former Raven Qadry Ismail. But Baxter, a second-year player out of Baylor, clearly interfered although the ball may have not been catchable.
That advanced the ball 22 yards, after which Manning completed a 15-yard pass to Marvin Harrison and Edgerrin James ran 10 yards to put Indianapolis into field goal range. The Colts called a timeout with nine seconds left and Vanderjagt booted his third field goal of the quarter.
"I was just playing my man and (they) called it on me," Baxter said. "All I wanted to do was make sure I got him out of bounds."
"Sometimes you have to be a good actor and then look at the ref," Ismail said. "It was one of those situations when if you're not looking for the ball, I have the right to go up for the ball. If you push me away from the ball, that's pass interference."
Baltimore coach Brian Billick argued vehemently to no avail.
"The officials were beyond reproach," Billick said. "I can't afford the fine on the performance of the officials. After speaking with (Colts coach) Tony Dungy after the game, I'll let him make comments on the officials and that is from the winner talking.
"It's hard to learn a lesson when a game is taken out of your hands and that's what the officials did today."
Dungy had a different view.
"I thought it was a pretty easy call," he said. "He (Baxter) had his back turned and they made contact before the pass got there. But I told Brian that's a tough way to lose a game on a fourth-down pass interference call."
"They played coverage and I was just kind of looking for somebody to get open," Manning said. "I threw it to Qadry and I noticed that (Baxter) was not looking at me."
Manning, who directed his 10th fourth-quarter comeback, was 30 of 40 for 284 yards and one touchdown.
Harrison had 12 receptions for 150 yards, increasing his season totals to 44 catches for 583 yards.
Baltimore had taken a 20-19 lead on Jamal Lewis' second touchdown of the game, a five-yard run nearly six minutes into the fourth quarter. It capped a seven-play, 67-yard drive that featured a leaping catch by tight end Todd Heap for a 31-yard gain that set up first and goal at the Indianapolis four.
The Colts snapped a 13-13 tie on Vanderjagt's 43-yard field goal that opened the fourth quarter. Manning was solid on the drive as he had a 27-yard completion to Reggie Wayne and a 15-yard pass to Harrison.
Ravens quarterback Chris Redman fumbled on the next possesion and defensive end Raheem Brock recovered on the Baltimore 11.
But Indianapolis was unable to punch it in and settled for a 25-yard field goal by Vanderjagt for a 19-13 advantage.
After Baltimore took its 20-19 lead, Manning was intercepted by safety Anthony Mitchell, who returned it 32 yards to midfield. The Ravens got a first down at the Colts' 35-yard line before a sack by defensive end Chad Bratzke on third down forced another fumble by Redman, who recovered for a loss of six yards.
The Ravens played without All-Pro linebacker Ray Lewis, but still came up with five sacks on Manning, who was also intercepted once.
"They did some different things with Lewis out," Dungy said. "They showed us some different looks and different things and I thought we held our composure for the most part."
The Colts took the opening kickoff and Manning fashioned a five-play, 73-yard drive that he capped with an 11-yard pass to tight end Marcus Pollard. Baltimore got a pair of field goals from Matt Stover to climb within 7-6 after one quarter.
Vanderjagt booted a 39-yard field goal to make it 10-6 and Redman was intercepted by linebacker Mike Peterson with 33 seconds left in the half. That helped set up a 50-yard kick by Vanderjagt just before intermission.
The Ravens tied it on a five-yard run by Jamal Lewis to cap a 10-play, 82-yard drive in the third quarter. Bratzke had a 15-yard personal foul on a failed third and six attempt that helped keep the drive alive for Baltimore.
Redman completed 15 of 28 passes for 163 yards and Lewis had trouble getting going as he carried 21 times for 75 yards.
"I'm really proud of the ways the guys fought," Redman said. "We were able to keep our intensity up for the entire game."
Indianapolis was penalized nine times for 70 yards. Baltimore fumbled seven times, but lost just a pair.