We approach the problem of stabilizing a dynamical system while optimizing a cost and satisfying safety and control constraints. For affine control systems and quadratic costs, it has been shown that Control Barrier Functions (CBF) guaranteeing safety and Control Lyapunov Functions (CLF) enforcing convergence can be used to reduce the optimal control problem to a sequence of Quadratic Programs. In this paper, we propose High Order CBFs (HOCBF), which can accommodate systems of arbitrary relative degree. We construct a set of controls that renders the intersection of a set of sets forward invariant for the system, which implies the satisfaction of the original safety constraint. Our notion of HOCBF is more general than the recently proposed exponential CBF. We formulate optimal control problems with constraints given by HOCBF and CLF, and propose two methods - the penalty and the parameterization methods - to address the feasibility problem. Finally, we show how our methodology can be extended for safe navigation in unknown environments with long-term feasibility. We illustrate the proposed framework on a cruise control system and on a robot control problem.