"There come a time when one must attend diligently to personal and family matters," Jackson said. "Now is such a time for me."

Jackson said he has tried to "make America a better place to live, work and raise families" since 2004 when he was appointed HUD secretary.

Sens. Patty Murray, D-Wash., and Christopher Dodd, D-Conn., sent a letter to President George Bush, urging him to seek Jackson's resignation, saying accusations of wrongdoing made him ineffective, The Wall Street Journal reported. Jackson has denied any impropriety while leading HUD.

HUD runs the Federal Housing Administration, which insures mortgages for low-income homeowners and first-time home buyers. Congress and the White House have discussed expanding FHA's role in stabilizing the mortgage market.

The latest claim of wrongdoing by Jackson stems from a Philadelphia redevelopment deal. The city's housing authority has sued, charging Jackson tried to punish the agency for blocking a deal involving a friend. Jackson earlier was cleared of wrongdoing after he made remarks that could have been interpreted that some contracts were awarded based on political affiliation.