An experimental study of nonlinear baroclinic instability and mode selection in a large basin