The conventional spectrum subtraction algorithm cannot effectively suppress the noise under highly non-stationary environment and results in the remaining music noise is often heard in the enhanced speech. In order to improve the speech enhancement performance, a novel denoising algorithm is proposed, which is based on speech endpoint detection using spectrum variance and the dynamic spectrum subtraction in Bark bands. According to human auditory characteristics, the Bark bands spectrums of the noisy speech signal are firstly calculated, and the noise power spectrum of each Bark band is then tracked and estimated by the improved minima controlled recursive averaging method. This noise estimation is adjustable frame by frame and more accurate for non-stationary environment. The experiment results showed that the proposed method can suppress the noise more efficiently than the conventional spectrum subtraction and the remaining music noise is almost eliminated.