In an appearance on CBS' "Late Show with David Letterman," the president suggested Romney was "writing off a big chunk of the country" with comments that 47 percent of Americans "believe that they are victims, who believe the government has a responsibility to care for them."
Romney's comments emerged Monday with the release of a video of his address to campaign donors in Florida in May.
"When I won in 2008, 47 percent of the American people voted for John McCain," Obama said Tuesday in an interview with Letterman to be televised Tuesday night. "They didn't vote for me and what I said on election night was: 'Even though you didn't vote for me, I hear your voices, and I'm going to work as hard as I can to be your president.'"
Obama said people he meets as he travels the country believe in "the American dream."
"There are not a lot of people out there who think they're victims," he said. "There are not a lot of people who think they're entitled to something."
Romney has defended his remarks as "not elegantly stated."
In an interview Tuesday with Fox News Channel, he largely reiterated comments he made at a hastily called Monday night news conference in Costa Mesa, Calif., that his comments addressed "a question about direction for the country."
The question is, "Do you believe in a government-centered society that provides more and more benefits? Or do you believe instead in a free-enterprise society where people are able to pursue their dreams?" he told reporters.
"We have a very different approach, the president and I, between a government-dominated society and a society driven by free people pursuing their dreams," Romney said.
In the video, posted online by Mother Jones, a liberal magazine, Romney says, "There are 47 percent of the people who will vote for the president, no matter what.
"All right, there are 47 percent who are with him, who are dependent upon government, who believe that they are victims, who believe the government has a responsibility to care for them, who believe that they are entitled to healthcare, to food, to housing, to you-name-it -- that that's an entitlement -- and the government should give it to them. And they will vote for this president no matter what," Romney says in the video.
"These are people who pay no income tax," Romney says. "Forty-seven percent of Americans pay no income tax. So our message of low taxes doesn't connect. So he'll be out there talking about tax cuts for the rich. I mean, that's what they sell every four years.
"And so my job is not to worry about those people. I'll never convince them they should take personal responsibility and care for their lives. What I have to do is convince the 5 to 10 percent in the center that are independents, that are thoughtful, that look at voting one way or the other, depending upon, in some cases, emotion -- whether they like the guy or not."
White House press secretary Jay Carney told reporters Tuesday: "When you're president of the United States, you're president of all the people."
"What unites us as Americans is far greater than what divides us," he said.
Carney did not say whether President Barack Obama had seen the Romney video.
"Setting aside what Governor Romney thinks, I can tell you that the president certainly doesn't think that men and women on Social Security are irresponsible or victims," he said.
While conservatives including commentator Bay Buchanan and former New Hampshire Gov. John Sununu defended Romney, the leaked video has generated political difficulty for the campaign among other high-profile conservatives.
MSNBC host Joe Scarborough, a conservative former member of Congress, said, "You're going to start seeing suburban voters, swing voters, storm away from the campaign as quickly as possible unless he fixes it."
New York Times columnist David Brooks wrote Romney is "a kind, decent man who says stupid things because he is pretending to be something he is not -- some sort of cartoonish government-hater."
"But it scarcely matters. He's running a depressingly inept presidential campaign. Mr. Romney, your entitlement reform ideas are essential, but when will the incompetence stop?"
The author of the Mother Jones article told The Washington Post the fundraiser took place May 17 in Boca Raton, Fla., at a the home of Marc Leder, co-chief executive officer of Sun Capital Partners Inc., a private investment firm focused on leveraged buyouts, equity, debt and other investments.
Obama campaign manager Jim Messina said in a statement Monday evening it was "shocking" Romney would "go behind closed doors" to describe nearly half of the country the way he did.
"It's hard to serve as president for all Americans when you've disdainfully written off half the nation," Messina's statement said.
At the White House press briefing, Carney brushed off a question about a similar controversy involving Obama, when he said during an April 6, 2008, fundraiser people in rural Pennsylvania, bitter over economic circumstances, "cling to guns or religion or antipathy toward people who aren't like them" to explain their frustrations.
"That happened four years ago and was discussed in abundance at the time," Carney said, noting that the president had not said he was not concerned about such voters.
In the video, Romney predicts stock markets would likely rise if he wins in November.
"There will be a great deal of optimism about the future of this country," he says. "We'll see capital come back and we'll see -- without actually doing anything -- we'll actually get a boost in the economy."
In an audio clip of the same fundraiser, Romney jokes he would have had an easier time winning the election if his father had been born to Mexican parents.
"My dad, as you probably know, was the governor of Michigan and was the head of a car company," Romney says in the clip. "But he was born in Mexico and, uh, had he been born of, uh, Mexican parents, I'd have a better shot at winning this, but he was not," as the crowd laughs.
"But he was unfortunately born to Americans living in Mexico," Romney says. "He lived there for a number of years. And, uh, I say that jokingly, but it would be helpful to be, uh, Latino."
The recordings surfaced on the same day the Romney campaign pledged more specific policy details and rolled out a set of new TV ads.
2014: The Year in Music [PHOTOS]