Breaking a dark degeneracy: The gamma-ray signature of early matter domination