Fourth moments reveal the negativity of the Wigner function