Police said they found Khalaf Al Ali, 45, with a gunshot wound to the chest but still alive in the diplomatic district of Gulshan. He died in hospital later.
Ali was found near where he lived in Gulshan, which is also where the Saudi Arabian Embassy is located.
Police aren't speculating on who shot Ali or what the motive might have been, the BBC said.
Private news agency United News of Bangladesh quoted the ambassador saying he hopes "the government will catch the killers," the BBC said.
Relations between the two countries had been improving for more than a decade.
Saudi Arabia has sent more than 100 military officers to the Bangladesh's Defense Services Command and Staff College in Dhaka since 1981. Saudis make up the largest foreign student numbers at the college, the college's Web site states.
Bangladesh also has been trying to improve economic relations with oil- and cash-rich Saudi Arabia since the mid 2000s.
In August 2008, the Saudi newspaper Arab News reported that Bangladeshi Ambassador Fazlul Karim said there was an "urgent need to bolster bilateral economic relations."
Bangladeshi exports $74 million worth of goods to Saudi Arabia but imports $344 million, he said. Saudi Arabia could be an important market for Bangladeshi pharmaceutical products, ceramics and leather goods, much of which goes to Western countries.
"We hope to look for markets for these products in the Saudi Kingdom," said Karim. "Dhaka will try to bridge the balance of trade which is in the kingdom's favor."
He said Saudi investors could invest in Bangladeshi infrastructure development, road construction and communications projects.
"They will get attractive returns for their foreign investments in the country," he said.
But the relationship hit the buffers on Oct. 7 when eight Bangladeshi migrant workers were publicly beheaded in the Saudi capital Riyadh. They were convicted of killing an Egyptian security guard in an attempt to rob a warehouse in 2007.
Three other Bangladeshis were sentenced to prison terms and flogged in the case.
Amnesty International, along with other human rights organizations, condemned the beheadings.
"Court proceedings in Saudi Arabia fall far short of international standards for fair trial and news of these recent multiple executions is deeply disturbing," said Hassiba Hadj Sahraoui, Amnesty International's deputy director for Middle East and North Africa.
"The Saudi authorities appear to have increased the number of executions in recent months, a move that puts the country at odds with the worldwide trend against the death penalty. The government must establish an immediate moratorium on executions in the kingdom and commute all death sentences, with a view to abolishing the death penalty completely," she added.
The beheadings brought the number of executions in Saudi Arabia up to then to 58, more than double the 2010 total, Amnesty International said. Twenty of those executed, including the Bangladeshis, were foreign nationals.
Swim Week Miami Beach 2014 [PHOTOS]