The Moscow Times reported Kazakhstan's Prosecutor General's Office blamed the terrorist group Jund al-Khilafah, or Soldiers of the Caliphate.
The alleged bomber was the sole casualty.
Kazakh authorities said the Islamist insurgency had spread to the republic from North Caucasus.
The terror group said in a statement on Islamist Web sites after the Oct. 31 explosions they were retaliation for a legislative crackdown on religion.
Legislation adopted last month bans prayer rooms in state buildings and requires missionaries to obtain state registration and renew it annually.
President Nursultan Nazarbayev introduced the legislation, which he called an anti-extremism measure.
Before Wednesday, the Times said, Kazakh authorities had denied the existence of Soldiers of the Caliphate.
Authorities said three members of the group arrested last week said the failed attacks were not intended to be suicide bombings, but the victim died because he mishandled an explosive device.
Atyrau is home to offices of numerous oil companies, including ENI, Tengizchevroil and Exxon Mobil.
Man behind Doritos Locos Tacos passed away on Thanksgiving
Exploding whale video goes viral on Internet