Most of President Barack Obama's schedule Tuesday was closed to the public, so Brady's visit, along with his wife Sarah, drew major attention from the White House press corps, a media pool report said.
Brady was in a wheelchair, his familiar cane in his left hand. His disabilities, including slurred speech, are the result of a failed assassination attempt on President Ronald Reagan in 1981. Brady was shot in the head.
Shooter John Hinckley Jr. was found not guilty by reason of insanity, and has spent all his time since the attempt at a Washington mental hospital, freed only occasionally for supervised visits with his parents.
The report said Brady was at the White House to pay his respects to Robert Gibbs, Obama's press secretary, and talked with reporters in the briefing room named for him in 2000 for a few minutes before heading for the West Wing.
Brady told a story about how then-UPI White House correspondent Helen Thomas used to knock on the glass of his office early before he had his morning tea.
"I would say, 'Woman, have you no shame?' The answer was, 'No, do you think I would have gotten so far in a male-dominated world if I had any shame?'" Brady said.
Dennis Rodman pledges to end trips to North Korea
Jessica Simpson shares three-way kiss with friends in photo