How well does the chiral expansion converge in nuclear and neutron matter