The SWELLS survey. III. Disfavouring "heavy" initial mass functions for spiral lens galaxies