Mabus told an audience at Yokosuka Naval Base in Japan that some 3,000 mid-career sailors will lose their jobs later this year, Stars and Stripes reported.
He said the decision to create a selection board to trim the ranks was made at least partially because of problems with the Navy's Perform-to-Serve program created in 2003.
The program requires certain sailors to justify why they should be allowed to re-enlist.
"The problem we had with (Perform-to-Serve) was that it only came up when it was time to re-enlist," Mabus said. "What this meant was you kept some people that maybe we shouldn't have ... and we got rid of some people that perhaps we should have kept."
Mabus was in Japan to thank sailors and Marines for aiding the Japanese after the March 11 earthquake and tsunami.