On a Conjecture on Automorphisms of Finite $p$-groups