The role of impurities in insultating transition metal oxides