Unsupervised learning of distributions

We study unsupervised learning from non-uniformly distributed examples with a single symmetry-breaking orientation when both the distribution and the preferential direction are otherwise completely unknown. For asymptotically high dimensions N of the pattern space the distribution can be inferred exactly from p = O(N) examples up to a well-known remaining uncertainty in the preferential direction. We further discuss implications for supervised learning of a teacher perceptron with unknown transfer function, unsupervised learning with several preferential directions, and architecture optimization.