The announcement of his dismissal came a few hours after Lauren Holiday and Megan Rapinoe scored in a 2-0 American victory over China.
"We want to thank Tom for his service over the past year and half, but we felt that we needed to go in a different direction at this time," U.S. Soccer president Sunil Gulati said in a statement. "We will begin looking for a new coach immediately to guide our Women's National Team toward qualifying for the 2015 FIFA Women's World Cup."
Until they find a replacement, Jill Ellis, U.S. Soccer's director of development, will serve as interim coach and take over the post on April 10 just in time for the U.S.-China match in San Diego.
The U.S. women were undefeated last year but had a disappointing Algarve Cup in March with a loss to Sweden that ended the team's 43-game undefeated streak.
"I'm disappointed that things didn't work out, but I'd like to thank U.S. Soccer for the opportunity to have coached this team and also the staff and players for all their hard work,'' Sermanni said in the national governing body's statement.