We study the timescale of random telegraph noise (RTN) of nanomagnets in stochastic magnetic tunnel junctions (MTJs). From analytical and numerical calculations based on the Landau-Lifshitz-Gilbert and the Fokker-Planck equations, we reveal mechanisms governing the relaxation time of perpendicular easy-axis MTJs (p-MTJs) and in-plane easy-axis MTJs (i-MTJs), showing that i-MTJs can be made to have faster RTN. Superparamagnetic i-MTJs with small in-plane anisotropy and sizable perpendicular effective anisotropy show relaxation times down to 8 ns at negligible bias current, which is more than 5 orders of magnitude shorter than that of typical stochastic p-MTJs and about 100 times faster than the shortest time of i-MTJs reported so far. The findings give a new insight and foundation in developing stochastic MTJs for high-performance probabilistic computers.