Characterization of constant policies in optimal control