$L^p(\mathbb{R}^d)$ boundedness for the Calder\'on commutator with rough kernel