Mid-infrared VISIR and Spitzer observations of the surroundings of the magnetar SGR 1806-20