He has been under pressure to deal with the scandal following a report by a body overseeing public finances.
The inspector of the government's accounts noted the rampant misuse of funds in at least seven ministries.
The ministers who have been dropped from cabinet all hold high-profile portfolios: Finance, energy, tourism, trade, transport and health.