We report an experimental investigation of the dependence of the lowest exciton energy of CdS clusters (quantum dots) on the cluster size. The cluster diameter is varied from 60 to 10 \AA{}. X-ray diffraction provides definitive identification of the cluster phase and permits a determination of the average cluster size. We find that the observed size dependence cannot be explained by models based on the effective-mass approximation, but, for clusters larger than 20 \AA{} in diameter, it is in agreement with a recent tight-binding calculation [P. E. Lippens and M. Lannoo, Phys. Rev. B 39, 10 935 (1989)]. Below 20 \AA{}, where the cluster deviates from the bulk zinc-blende structure, some discrepancy between the tight-binding calculation and experiments becomes evident.