The BBC said Arroyo had been receiving treatment at the medical facility for about a month for heart, lung and kidney problems. He also had long battled an addiction to drugs.
A specific cause of death was not immediately disclosed for Arroyo, who was scheduled to be awarded a lifetime achievement award at this year's Latin Grammy Awards show.
Colombia Reports quoted the Archbishop of Barranquilla as saying the performer is to be buried Wednesday. His hometown of Cartagena will observe two days of official mourning and all flags are to be flown at half-staff.
Hundreds of fans had gathered at the hospital to pay tribute to the music icon, Colombia Reports said.
"The Archbishop of Barranquilla calls on the whole community touched by the artist to respectfully and serenely live through these moments of sadness in which the best tribute we can offer is to unite the prayers of Colombians so that the soul of our beloved and always remembered Joe Arroyo may rest in the peace of the Lord," the archbishop said in a statement.
2014: The Year in Music [PHOTOS]