Systematic study of the response of single crystal diamond neutron detectors at high temperature