Conrado Juarez, 52, of New York's Bronx borough was taken into custody Friday at a Greenwich Village restaurant where he worked, The New York Times reported.
NYPD Commissioner Raymond Kelly said Juarez confessed to sexually abusing and killing Anjelica Castillo, 4, in 1991, the newspaper said.
Kelly said Juarez told police one of his sisters helped him put the girl's body in a picnic cooler that was then left near the Henry Hudson Parkway.
The girl's remains were found by construction workers later that year.
The girl was referred to as Baby Hope through the ensuing years until investigators finally got the tip that led to her identification.
The pivotal information came from a woman who told police she recalled a conversation she once had with another woman who had said her younger sister had been murdered, the Times said. Investigators first tracked down that woman, who turned out to be Anjelica's adult sister, and eventually found Anjelica's mother.
From there, police learned Anjelica had been living in a Queens borough home with several relatives, including Juarez's sister.
Kelly said Juarez told investigators he visited the home and encountered Anjelica in a hallway, sexually assaulted her and smothered her to death.
2014: The Year in Music [PHOTOS]