Equipped with machine learning (ML) models, unmanned aerial vehicle (UAV) swarms can execute various applications like surveillance and target detection. However, the connections between UAVs and cloud servers cannot be guaranteed, especially when executing massive data. Thus, traditional cloud-centric approach will not be suitable, since it may cause high latency and significant bandwidth consumption. In this work, we propose a federated learning (FL) framework via non-orthogonal multiple access (NOMA) for a UAV swarm which is composed of a leader-UAV and a group of follower-UAVs. Specifically, each follower-UAV updates its local model by using its collected data, and then all follower-UAVs form a NOMA-group to send their respectively trained FL parameters (i.e., the local FL models) to the leader-UAV simultaneously. We formulate a joint optimization of the uplink NOMA-transmission durations, downlink broadcasting duration, as well as the computation-rates of the leader-UAV and all follower-UAVs, aiming at minimizing the latency in executing the FL iterations until reaching a specified accuracy. Numerical results are presented to verify the effectiveness of our proposed algorithm, and demonstrate that the proposed algorithm can outperform some baseline strategies.