Towns, 77, was expected to face a competitive primary June 26, the date for House and Senate primaries, The Hill reported Sunday.
Several Democratic officials said Towns spent Sunday calling supporters to tell them he was dropping out of the race, leaving it open the two men who were running against him -- state Assemblyman Hakeem Jeffries and New York City Councilman Charles Barron, the New York Daily News reported.
Barron is a former Black Panther who unsuccessfully challenged Towns in 2006. Jeffries had picked up steam during the last few weeks, winning the backing of several key labor groups, The Hill said.
Jeffries did not comment to The Hill beyond praising Towns' three decades of service representing the borough of Brooklyn in Congress. Barron didn't immediately respond to a request for comment.
Towns was first elected to Congress in 1982.