MACHINE LEARNING REDUCTIONS

1. Why Reductions? Charlie and Susan1 each have a problem to solve: given information from car sensors, predict if a car repair is required before the car fails. Some standard characteristics of this problem include: (1) Dependent data. Sensor values are correlated in time. (2) Very rare events. Most of the time, cars don't fail, so data drawn from the natural measure will have only a small portion representing car failures. (3) Varying value events. Having the third rear brake light fail ( x at next opportunity ) is not nearly as catastrophic as a blowout ( stop the car RIGHT NOW ). Charlie solves the problem directly. He considers the problem carefully, constructing and tuning a model making predictions about car failures given the sensory inputs from a large number of examples. Charlie works for several years on this, and eventually gets a system that works reasonably well. Susan solves the problem indirectly. She rst notices that the problem is an instance of cost sensitive learning with dependent data. She applies a reduction from cost sensitive learning to classi cation, tries a few classi cation algorithms, and picks the best. Susan spends only a week, producing a system that works reasonably well. Susan's method (an advanced machine learning approach) is much faster, requires much less training, takes advantage of past work by many researchers, and is easily veri ed. Charlie's method (a statistics-like approach) is surer and might be superior in the long run (or not. What if Charlie's sensors are obsolete by the time he succeeds?). Reductions naturally aid Susan's method in several ways.