In this paper, we examine the use of machine learning to improve the robustness of systems for image analysis on the task of roof detection. We review the problem of analyzing aerial photographs, and describe an existing vision system that attempts to automate the identiication of buildings in aerial images. After this, we brieey review several well-known learning algorithms that represent a wide variety of inductive biases. We report three experiments designed to illuminate facets of applying machine learning methods to the image analysis task; one experiment focuses on within-image learning, another deals with the cost of diierent errors, and a third addresses between-image learning. Experimental results demonstrate that machine-learned classiiers meet or exceed the accuracy of handcrafted solutions and that useful generalization occurs when training and testing on data derived from diierent images.