Diminishing return of investment in genetic diversity