Multifrequency observations of blazars. II - The variability of the 1 micron to 2 mm continuum