The 24-year-old infielder smacked the two-run homer in the bottom of the ninth inning in the 2-1 win Wednesday at Citizens Bank Park.
"I don't know if I have words to describe that moment," Williams told reporters. "It was pretty incredible. I'm still trying to figure it all out."
The rookie was the only Phillies player to record multiple hits in the game.
Williams, who made his MLB debut Tuesday, stepped up to the plate with two outs and Andrew McCutchen on first base. The Phillies trailed 1-0.
Braves relief pitcher Will Smith painted the edge of the strike zone with an 81.9-mph slider for a called strike for his first throw to Williams.
The Phillies rookie then settled in and smacked an 0-1 slider to left field. The blast carried over the outfield fence for a 377-foot homer to win the game. Williams' walk-off shot had an exit velocity of 99.1-mph, according to Statcast.
Phillies players mobbed Williams when he reached home plate to seal the victory.
"We were like, 'He's going to do it right here. Captain America is going to end this right here,'" Phillies starter Zach Eflin told reporters.
Williams went 2 for 4 with two RBIs, a run scored and two strikeouts in the win. He went 1 for 1 with a run scored in his debut Tuesday in Philadelphia, a 9-5 loss to the Braves.
Braves second baseman Ozzie Albies went 3 for 4 in the loss. Braves starter Tucker Davidson allowed four hits and six scoreless innings, but was not on the record for a decision.
Eflin allowed eight hits and one run in six innings. Phillies relief pitcher Ranger Suarez earned the victory. He did not allow a hit or a run over the final three innings.
The Phillies host the Braves in the final game of the series at 1:05 p.m. EDT Thursday at Citizens Bank Park.