Revealing the formation histories of the first stars with the cosmic near-infrared background