His affinity for what was once routinely thought of as "America's pastime" led him to produce two baseball books -- "Baseball Letters" and "Every Pitcher Tells a Story" -- in the 1990s. Now, Swirsky has turned out a third book on the subject he loves so much.
"Something to Write Home About" is a collection of letters about baseball from world-famous figures such as President George W. Bush, former president George H.W. Bush, musician Paul McCartney and actual baseball players such as Tom Seaver and Ernie Banks.
Some of the letters -- like the 1935 note from Babe Ruth to his agent's son -- are items that Swirsky has come across in his pursuit of baseball collectibles. Others -- like the ones from the 41st and 43rd presidents -- were written to Swirsky in response to his request for letters on the subject of their common interest.
Swirsky told United Press International he specifically asks for handwritten letters.
"When you write a hand-written letter, it forces you to say exactly what you feel," he said.
Besides that, Swirsky said that writing -- as opposed to e-mail or dictating to an assistant -- takes time. Just like baseball itself.
"I love baseball because it's such a relaxing sport," said Swirsky. "Because you're basically sitting in a park with somebody hopefully you feel close with -- father, wife, kids -- and baseball allows you to open up other conversations. It's one of the last things that we do just sitting in a park."
The letters offer personal glimpses of the writers not always available through mass-distributed, carefully managed media appearances.
For example, there's the elder Bush reminiscing about a great spring day in 1948 on the baseball diamond at Yale, where he was the captain of his college team. The legendary New York Yankees' slugger Babe Ruth came to visit and Bush got to meet him.
"I had a lot of wonderful moments as Vice President and as President -- great seats at great events," wrote the 79-year-old Bush, "but the Babe at Yale topped them all."
Tork, it turns out, is more than a fan of the sport.
"I found out he was a high school baseball coach after the Monkees broke up," said Swirsky.
Swirsky said he has received more than 20 gold and platinum records in his songwriting career, for such hits as "Love Is a Beautiful Thing" for Al Green, and "Tell It to My Heart" and "Prove Your Love" for Taylor Dayne. But he reached a point where he needed to do something else, so he turned to producing the baseball letter books.
"I decided the only way I was going to move on in my life was to remove the gold records from the wall," he said. "These books started to kind of happen."
As a collector, Swirsky is naturally proud to have letters from some of the most famous people of the era in his collection. But he said the books have also motivated lots of readers -- relatively anonymous people for the most part -- to write to him with their own favorite baseball stories.
"They're more valuable to me than the letters that were written by famous people," said Swirsky.
Still, it isn't every day you get a letter from Paul McCartney.
"It's pretty cool," said Swirsky. "The autograph magazines and collecting magazines ... think the autographs are the coolest things."