Evaluating detection and treatment effectiveness of commercial anti-malware programs

Commercial anti-malware programs consist of two main components: detection and treatment. Detection accuracy is often used to rank effectiveness of commercial anti-malware programs with less emphasis on the equally important treatment component. Effectiveness measures of commercial anti-malware programs should consider equally detection and treatment. This can be achieved by standardized measurements of both components. This paper presents a novel approach to evaluate the effectiveness of a commercial anti-malware program's detection and treatment components against malicious objects by partitioning true positives to incorporate detection and treatment. This new measurement is used to evaluate the effectiveness of four commercial anti-malware programs in three tests. The results show that several anti-malware programs produced numerous incorrectly treated or untreated true positives and false negatives leaving many infected objects unresolved and thereby active threats in the system. These results further demonstrate that our approach evaluates the detection and treatment components of commercial anti-malware programs in a more effective and realistic manner than currently accepted measurements which primarily focus on detection accuracy.