Speaking on "Fox News Sunday," Portman it's time to deal with the growing national debt, which stands at more than $17 trillion.
"If you look at his budget last year, it barely had deficit reduction relative to what would happen otherwise," Portman said. "In other words, there was a slight decrease in the deficit. You would hope there would be a significant one."
Portman noted the United States in currently in the slowest economic recovery since World War II, and increased taxes, which he said are recommended in the proposed budget, will not solve the issue of slow growth or the deficit.
Rep. Chris Van Hollen, D-Md., countered by pointing out the budget includes ending tax breaks for businesses moving jobs out of the country and using "those savings to invest in infrastructure here at home."
He added the budget would reduce long-term deficits to less than 2 percent of the gross domestic product, and the deficit is currently on a "downward trajectory at the end of (a) 10-year period."
Portman and Van Hollen also disagreed on a Pentagon plan to reduce the number of Army personnel by 21 percent, to the lowest level since 1940.
Van Hollen said the United States no longer needs an army "to support two overseas wars that we're winning at the same time. I think you can respond to the threats, as they arise, with a lower [smaller] army.
"Look at what is happened in the Ukraine," Portman countered. "Look at what is happening in the Middle East. We could not have done the surge in Iraq, in my view with an army that size."
Portman added it is important, though, to reduce "waste, fraud and abuse in every place, including the Pentagon."
Celebrity Breakups and divorces of 2014 [PHOTOS]