Nonlinearity and nonclassicality in a nanomechanical resonator