In many practical classification problems it is important to distinguish false positive from false negative results when evaluating the performance of the classifier. This is of particular importance for medical diagnostic tests. In this context, receiver operating characteristic (ROC) curves have become a standard tool. Here we apply this concept to characterize the performance of a simple neural network. Investigating the binary classification of a perceptron we calculate analytically the shape of the corresponding ROC curves. The influence of the size of the training set and the prevalence of the quality considered are studied by means of a statistical-mechanics analysis.