We consider a mission-driven multiple unmanned aerial vehicles (multi-UAVs) network with millimeter-wave (mmWave) transmissions, where a leader UAV (LUAV) communicates with a large number of follower UAVs (FUAVs) simultaneously via a uniform planar array with only a limited number of radio frequency chains. Since only a few orthogonal beams are available and mission-driven UAV networks are autonomous and delay-sensitive, non-orthogonal multiple access (NOMA) over these beams is considered for agility and efficiency. In particular, aiming to address these challenges of highly dynamic mobility, we propose a machine learning framework to enable agile analog beam management for mmWave-NOMA transmissions. The proposed beam management aims to achieve efficient beam tracking and NOMA-grouping-aware analog beamforming optimization to facilitate mmWave-NOMA transmissions by judiciously utilizing angular domain information (ADI). More specifically, a Gaussian process machine learning-based ADI prediction scheme is proposed to track the angular dynamics of FUAVs, which facilitates fast beam-tracking for mmWave-NOMA transmissions. Moreover, by exploiting the predicted ADI, an unsupervised-learning-based FUAV grouping scheme is proposed to facilitate mmWave-NOMA transmissions with high radio-frequency chain efficiency, while a deep learning-based NOMA-grouping-aware fast transmit beamforming optimization scheme is proposed to improve the coverage of mmWave-NOMA transmissions in highly dynamic multi-UAVs networks. Simulation results validate the performance advantages of our proposed beam management scheme against state-of-the-art schemes.