Experimental test of McDougall's theory for the onset of convective instabilities in isothermal ternary systems