The particle swarm optimization (PSO) algorithm has been widely used in the optimization area when the problem to be optimized is non-convex and the gradient function is difficult to obtain. In this paper, a novel particle swarm optimization (NPSO) algorithm is proposed. Different from the standard PSO, in the NPSO algorithm the trajectories of the particles are limited to a set of orthogonal directions, one of which is pointed to the best position in the population. Each particle maintains its own orthogonal directions, which are updated in several iterations. The convergence property of the NPSO algorithm is analyzed with the help of the dynamic system theory. Numerical experiments on benchmark functions show that the NPSO algorithm can locate the global optimum fast and accurately and is superior to the standard PSO algorithm.