On the Fourier analytic structure of the Brownian graph