The cause of the blast at Pemex remained unclear, The New York Times reported. Witnesses said they heard a sound like a bomb going off at about 4 p.m. Thursday. A paramedic told CNN the building was ripped "from the inside out."
Twenty women and 12 men were reported killed and another 121 people were injured.
Mexican President Enrique Pena Nieto, who took office Dec. 1, has promised to reorganize Pemex.
The explosion is expected to increase calls for a shakeup in the company.
"You pull all of this together and you say, well, if they can't even guarantee safety in their own building, their own headquarters, what does that tell us about the company?" Duncan Wood, director of the Mexico Institute at the Woodrow Wilson International Center for Scholars, told the Times. "It tells us there are things seriously wrong there. It tells you things need to be seriously shaken up."
Emergency responders said the explosion, which originate in the basement, collapsed two floors of a building adjacent to the company's office tower.
Rescuers found many people trapped.
At least one person was rescued more than 5 hours after the blast, Pena Nieto said on his Twitter page.
"People were screaming. ... You could see pieces of the wall falling to the ground," said Joaquin Borrell Valenzuela, an attorney with the Pemex comptroller's office, who was outside the building when the blast occurred.
The explosion was in an annex building just to the east of the tower, Foreign Minister Jose Antonio Meade said.
Officials evacuated the complex, which has been closed until further notice.
Thousands of people work at the Pemex headquarters, which includes a 54-story building that is one of Mexico City's tallest skyscrapers.
"We will get to the bottom of the causes in close coordination with the authorities," Pemex Director General Emilio Lozoya Austin said. "At this time, attending to the injured is the priority."
Mexico's attorney general's office will participate in the investigation of the explosion, officials said.
Pemex, short for Petroleos Mexicanos, was established after Mexico nationalized foreign oil operations in 1938.