Over-the-shoulder (OTS) ratings by expert air traffic controllers are often used to evaluate the performance of air traffic control specialists (ATCS), however, these OTS ratings are subjective and time-consuming. This study examines whether alternative methods could be as effective as OTS ratings in measuring ATCS performance. OTS ratings were replaced by computer-derived measures and two types of behavior checklists (Behavioral and Event Checklist (BEC) and the Remaining Actions Form (RAF)). The effectiveness of computer-derived measures alone as well as the need for supplementation with behavior checklists was examined. The study also considered the possible redundancy of the two types of checklists. Subjects were 107 air traffic controllers, with 14 expert controllers serving as raters. A model including counts of mistakes and computer-derived performance measures predicted the OTS rating; a model containing only the computer-derived measures did not. Results showed that the RAF was not effective in predicting performance, but the BEC measure alone produced a model equivalent to the full model containing all the predictor variables.