The 27-year-old country music singer performed the song remotely during Monday's episode of the CBS late-night show.
"A Country Song" appears on Ballerini's fourth studio album, Ballerini, released last week. Ballerini is Ballerini's second album of 2020, following Kelsea, released in March.
In the interview, Ballerini said it wasn't her original plan to release two albums this year.
"I wish I could tell you it was the plan all along, but it was definitely just the circumstances of releasing an album in late March, kind of when the world shut down. I think it was a weird time for anyone to release anything," she said, referencing the coronavirus (COVID-19) pandemic.
Ballerini ended up reworking Ballerini due to the health crisis.
"I think I needed a way to fall back in love with this album, because I kind of was bummed at all the plans changing," she said. "I just kind of remade it to listen in your living room, rather than to listen in a big arena with confetti cannons."
Ballerini will give a pre-taped performance during the Academy of Country Music Awards this week. She filmed at an empty Ryman Auditorium in Nashville, Tenn.
"I felt so rusty," the singer said. "It's odd, but it's nice to be back on any kind of stage, especially that one."