Bush was to have afternoon tea with Queen Elizabeth II before talks with Brown, the White House said.
Flying to London, U.S. National Security Adviser Stephen Hadley told reporters aboard Air Force One there is no disagreement between Washington and London on Iraq troop levels.
He dismissed a report in The Observer newspaper that suggested Bush would warn Brown against a premature withdrawal from Iraq when they meet Sunday.
"There is no disagreement between us, between the president and Prime Minister Brown" on withdrawals, Hadley said.
Bush and Brown agree troop withdrawals from the war-torn country have to be made "on the advice of our military, and not according to any arbitrary schedule," Hadley said.
On Iran, he said the two men will discuss sanctions against Tehran over its nuclear program.