Enabling coordinated motion of multiple quadrotors is an active area of research in the field of small unmanned aerial vehicles (sUAVs). While there are many techniques found in the literature that address the problem, these studies are limited to simulation results and seldom account for wind disturbances. This paper presents the experimental validation of a decentralized planner based on multi-objective reinforcement learning (RL) that achieves waypoint-based flocking (separation, velocity alignment, and cohesion) for multiple quadrotors in the presence of wind gusts. The planner is learned using an object-focused, greatest mass, state-action-reward-state-action (OF-GM-SARSA) approach. The Dryden wind gust model is used to simulate wind gusts during hardware-in-the-loop (HWIL) tests. The hardware and software architecture developed for the multi-quadrotor flocking controller is described in detail. HWIL and outdoor flight tests results show that the trained RL planner can generalize the flocking behaviors learned in training to the real-world flight dynamics of the DJI M100 quadrotor in windy conditions.