Simultaneous learning of concepts and simultaneous estimation of probabilities

When can one simultaneously learn many concepts from the same sample? We call this “simultaneous learning.” This generalizes the notion of learning, which focuses on one concept at a time. We examine this issue along with an analogous notion for estimating the probabilities of sets, “simultaneous estimation.” In each case, an unknown distribution from a known class P of probabilities generates the samples. We show that simultaneous estimation of a class of concepts is possible if it can be simultaneously learned, and this, in turn, is possible if a somewhat richer class can be simultaneously estimated. We also characterize simultaneous learning and estimation in terms of a notion of sampling complexity. Specializing to the case where P is the set of all probabilities, we show that simultaneous learning or estimation is possible if and only if the VC dimension of the concept class is finite.