Co12xMnx thin films epitaxially grown on GaAs~001! have been studied using reflection high-energy electron diffraction, x-ray diffraction, and the magneto-optical Kerr effect. The result shows that Co-rich films are body-centered cubic and ferromagnetic, and Mn-rich ones are face-centered cubic and antiferromagnetic. However, the film structure and magnetism at x;0.5 show strongly thickness-dependent behavior. The magnetic phases of ordered Co12xMnx alloys (x50,1/4,1/2,3/4,1) in both bcc and fcc structures have also been investigated by a first-principles linearized augmented plane-wave calculation in the local-spin-density approximation. With the help of the calculated results, the correlation established in experiment between the structure and magnetism of Co12xMnx alloys is better understood.