The Use of Level Three Evaluation Data to Assess the Impact of Technology Training on Work Performance

The operational environment permitted the evaluation of new technology use comparing trained end-users. Data was collected to determine: • the impact of formal and informal training on tool use • the utilization of system features as a baseline • the questions asked of the help desk by trained and untrained users • the impact of training on job productivity via technology Data showed that a formal training program yielded greater use of the new technology and more questions of an advanced nature to the help desk. A job/task-focused technology training program, although short, did have significant positive impact on job skills of query formulation. When evaluating new technology and its support performance, technologists should: • use a team approach that includes diverse expertise • identify critical job performance metrics associated with effective tool use • get participation from the technology development and end-user communities • collect quantitative data which can be amplified by follow-up interviews and observations • get a commitment from management to use evaluation results for overall performance improvement • work toward a technology deployment process that routinely includes evaluation data.