Double Cross-Validation

W hen the same test set is used both to tune a calibration algorithm and to give an assessment of its performance, the assessment will be over optimistic. one solution to this problem is to use three sets: calibration, tuning and validation sets. Cross-validation, often used as an alternative to removing a test set, suffers from the same problem of over optimism. Double cross-validation, described here, solves the problem, though at the expense of rather a lot of computation.