BPS Operators in N=4 SYM: Calogero Models and 2D Fermions