Over the past few years, the decades-long slow but steady increase in participation in physics by undergraduate women has stalled. As the numbers of undergraduate majors in physics increase, the numbers of women are not keeping pace. Moreover, women of color represent a disproportionately small fraction of physicists. This means that women of color, and women in general, are an undertapped pool of talent. Significant variation in the participation of women from one institution to the next suggests that local factors, such as department culture, are important, rather than differences in aptitude, motivation, or preparation. Physicists in the U.S. must redouble their efforts to make physics departments and laboratories places where women and men of all backgrounds can thrive and produce exciting science.