Abe asked the Tokyo Electric Power Co. to stop using nuclear Reactors 5 and 6 during a visit to the plant, CNN reported.
Reactors 5 and 6 were not in operation in 2011 when the tsunami and earthquake hit, and therefore didn't experience the damage four other reactors did.
TEPCO has already announced it would decommission the four damaged reactors, but hadn't discussed what would happen to Reactors 5 and 6.
Abe was visiting the plant to investigate efforts to contain leaks from tanks that hold toxic water.
His visit comes after the government announced it would spend $470 million to try to correct the contaminated water problem.
Abe also asked the plant operator to fix the contaminated water problem by 2014.