Phase spreading and temporal coherence of a pair-condensed Fermi gas at low temperature