Structure of infrared singularities of gauge-theory amplitudes at three and four loops