Coronagraph design for an extreme adaptive optics system with spatially filtered wavefront sensing on segmented telescopes

High dynamic range coronagraphy targeted at discovering planets around nearby stars is often associated with monolithic, unobstructed aperture space telescopes. With the advent of extreme adaptive optics (ExAO) systems with thousands of sensing and correcting channels, the benefits of placing a near-infrared coronagraph on a large segmented mirror telescope become scientifically interesting. This is because increased aperture size produces a tremendous gain in achievable contrast at the same angular distance from a point source at Strehl ratios in excess of 90\% (and at lower Strehl ratios on future giant telescopes such as the Thirty Meter Telescope). We outline some of the design issues facing such a coronagraph, and model a band-limited coronagraph on an aperture with a Keck-like pupil. We examine the purely diffractive challenges facing the eXtreme AO Planetary Imager (XAOPI) given the Keck pupil geometry, notably its inter-segment gap spacing of 6~mm. Classical Lyot coronagraphs, with hard-edged occulting stops, are not efficient enough at suppressing diffracted light, given XAOPI's scientific goal of imaging a young Jupiter at a separation as close as 0.15 arcseconds (4λD at H on Keck) from its parent star. With a 4000 channel ExAO system using an anti-aliased spatially-filtered wavefront sensor planned for XAOPI, we wish to keep diffracted light due to coronagraphic design at least as low as the noise floor set by AO system limitations. We study the band-limited Lyot coronagraph (BLC) as a baseline design instead of the classical design because of its efficient light suppression, as well as its analytical simplicity. We also develop ways of investigating tolerancing coronagraphic mask fabrication by utilizing the BLC design's mathematical tractability.