Nighttime polarimetric measurements are often obtained very close to the limits of the instrumental capabilities. It is important to be aware of the possible sources of spurious polarization, and to adopt data reduction techniques that best compensate for the instrumental effects intrinsic to the design of the most common polarimeters adopted for nighttime observations. We define a self-consistent framework starting from the basic definitions of the Stokes parameters, and we present an analytical description of the data reduction techniques commonly used with a polarimeter (consisting of a retarder wave plate and a Wollaston prism) to explore their advantages and limitations. We first consider an ideal polarimeter in which all optical components are perfectly defined by their nominal characteristics. We then introduce deviations from the nominal behavior of the polarimetric optics, and develop an analytical model to describe the polarization of the outgoing radiation. We study and compare the results of two different data reduction methods, one based on the differences of the signals, and one based on their ratios, to evaluate the residual amount of spurious polarization. We show that data reduction techniques may fully compensate for small deviations of the polarimetric optics from their nominal values, although some important (first-order) corrections have to be adopted for linear polarization data. We include a detailed discussion of quality checking by means of null parameters. We present an application to data obtained with the FORS1 instrument of the ESO VLT, in which we have detected a significant amount of cross talk between circular and linear polarization. We show that this cross-talk effect is not due to the polarimetric optics themselves, but is most likely caused by spurious birefringence due to the instrument's collimator lens.