A spokesperson said Mashaal , who led Hamas' headquarters in Damascus, has told the group's leadership council he won't seek re-election, The Christian Science Monitor reported.
Why he's stepping down or who's likely to be his successor remained unclear.
Mashaal 's decision comes as Islamist allies in the Middle East have made gains through election victories but also as Hamas has begun move staff out of Damascus, its headquarters, because of fighting in Syria.
The Monitor notes some believe Hamas wants to open a headquarters in Egypt and hopes to demonstrate it can be more moderate.
The Muslim Brotherhood won nearly half the seats in Parliament in elections in Egypt, and that should mean the country is more receptive to Hamas than former President Hosni Mubarak.
"We as Palestinians paid the price for the Arab dictatorship," said Fawzi Barhoum, a Hamas spokesperson in the Gaza Strip. "We believe that this result of the democratic process might mean full support for Palestinian rights and interest, now that [Arab governments'] hearts are with the people."
Mashaal had been pushing for reconciliation with Palestinian Authority President Mahmoud Abbas and his Fatah party, which support negotiations with Israel and reject military confrontation.
The change in Hamas leadership could affect its relations with Israel and the United States, both of which consider it a terrorist group.
"It is important to see whether this vacuum will be filled by the moderates or a hawk, because this will affect the future of Hamas and Palestinian politics," said Mohammed Dejani, a political science professor at al Quds University.