Gilson said he was using a mini-excavator to remove parts of the old foundation on Friday when he encountered a pair of strange objects.
"I thought they were weeds," Gilson told NJ.com. "I picked them up and just threw them aside and they went into the pile I was using for fill."
Gilson said it rained on Saturday, and when he returned to complete the work on Sunday the objects again caught his eye. He realized they were rolled-up paper bound with rubber bands.
"I got to look at the edge and it had a green tint to it and I said, 'This is money,'" Gilson said. "It looked like little mini-cigars all bound up together. As I broke it apart, I started to see what it was."
He said the bills were buried underneath where the porch used to be. He said the area was previously only accessible via crawlspace.
"It was pretty shallow, too," Gilson said. "Somebody had to crawl under there and dig a hole in that crawl space."
Suzanne Gilson said in a Facebook post that there was $1,000 total, and all of the $10 and $20 bills were marked as having been printed in 1934.
Rich Gilson said he suspects the cash -- the equivalent of more than $20,000 in the 1930s -- might have been tied to illegal activity.
"My sense is that something fishy happened," he said. "Somehow, somebody got new bills, rolled them up like that, put them in a jar. Somebody was hiding it, not just under their bed or in a wall for safe keeping."
Gilson said he plans to hold onto the money.
"I don't see myself spending this money," Gilson said. "The story's too good for what it's worth."