Melzer, who opened the best-of-five series with a five-set win Friday over Igor Kunitsyn, topped Alex Bogomolov Jr. 6-2, 6-4, 6-1 on Sunday to give the Austrians three points.
Kunitsyn rebounded for a 6-4, 4-6, 7-6 (7-4) win over Andreas Haider-Maurer in Sunday's final match, making the final series score 3-2.
Austria earned a spot in the Davis Cup quarterfinals in April against Spain, which defeated Kazakhstan in its first-round series.
After falling behind 2-0 Friday, Russia answered with a five-set win Saturday in doubles by Nikolay Davydenko and Mikhail Youzhny, leaving the series to be decided in Sunday reverse singles.
But Melzer won half the points when receiving serve and had seven breaks in a straight-set win over Bogomolov that clinched the series win. Half of the 86 points he took came on winners.
Kunitsyn took a tight match from Haider-Maurer, who won a singles match over Bogomolov Friday, by winning five of the last six points in the tiebreaker. Haider-Maurer won 104 points to Kunitsyn's 103 in the match.