Vettel, of Red Bull Racing, sped around the 3.4-mile course in Kuala Lumpur in 1 minute, 34.870 seconds, 1/10th of a second faster than McLaren's Hamilton.
Vettel said he was happy to get to the pole after what he said was not the most satisfying qualifying series.
"I was happy with the car, but we never really got into a rhythm," Vettel said. "It was very close in the end with the McLaren guys, so I'm happy to get the pole."
Vettel's teammate, Mark Webber, placed third in the qualifying followed by McLaren's Jenson Button and Fernando Alonso. Seven-time F1 champ Michael Schumaker from the Mercedes team will start 11th.
The race at Sepang International Circuit Sunday will include 24 cars.