We investigate the optical trapping of a Rayleigh particle by a linearly or radially polarized Gaussian beam. The Mie theory is applied to obtain a full solution, with the incident beam being described by the mixed dipole model, which is beyond the paraxial approximation. We then obtain approximate analytical expressions for the optical force, equilibrium position, and trap stiffness for a Rayleigh particle. At equilibrium, the displacement for the particle from the focus scales like a(3) (where a is the radius) for a transparent particle, owing to scattering, whereas for absorptive particles it scales like C+Da(2), owing to absorption. The trap stiffness is found to be proportional to a(3), in agreement with the recent experiment. The radially polarized beam is found to be superior to the linearly polarized beam in the Rayleigh regime in terms of its ability to trap. It is found that the larger the ratio of epsilon(r)/epsilon(i), the closer the equilibrium to the focus, and thus higher stability.