Jonathan Aldridge said he was moving a cooler near the back of his 44-foot lobster vessel, Anna Mary, when a handle broke and he "fell off the back of the boat, just like that."
Aldridge said he stayed afloat by tucking his arms across his rubber boots as Coast Guard crews launched a rescue effort covering 780 square miles of ocean off New England.
"I was like, 'There is no way I'm dying this way. This is how I gotta go? No way,'" said Aldridge.
A helicopter crew spotted Aldridge 43 miles south of Montauk, N.Y. around 3 p.m. Wednesday and was able to fish him out of the water.
He spend the night in a hospital and was released after being treated for sunburn, sore throat and pain under his arms.
"When I knew that they saw me it was like, it was the best feeling in the world," he said. "I was craving a cheeseburger for some reason, I don't know why."
Swim Week Miami Beach 2014 [PHOTOS]