Children's Minister Beverly Hughes and Cabinet Officer Tom Watson announced Tuesday that they would be resigning, The Daily Mail reported.
Brown is facing a potential disaster in Thursday's European elections. A poll last week showed the Labor Party running third behind the Conservatives and the United Kingdom Independence Party.
While a number of Conservative members of Parliament have been caught taking advantage of rules allowing them to claim expenses for second homes either in their constituencies or in London, polls show the public tends to blame Labor, the party in power. All three major parties have lost ground to the UKIP, the Greens and the British National Party.
The Daily Mail said Smith told Brown two months ago she would be leaving. But the news leaked out Tuesday as Brown met with the Cabinet.
Celebrity Families of 2014 [PHOTOS]