Constraints on dark matter particles from theory, galaxy observations and N-body simulations