Mean flow in hexagonal convection: stability and nonlinear dynamics