Existing data-driven fault diagnosis methods assume that the label sets of the training data and test data are consistent, which is usually not applicable for real applications since the fault modes that occur in the test phase are unpredictable. To address this problem, open set fault diagnosis (OSFD), where the test label set consists of a portion of the training label set and some unknown classes, is studied in this article. Considering the changeable operating conditions of machinery, OSFD tasks are further divided into shared-domain open set fault diagnosis (SOSFD) and cross-domain open set fault diagnosis (COSFD) in this article. For SOSFD, 1-D convolutional neural networks are trained for learning discriminative features and recognizing fault modes. For COSFD, due to the distribution discrepancy between the source and target domains, the deep model needs to learn domain-invariant features of shared classes and separate features of outlier classes. Thus, by utilizing the output of an additional domain classifier, a model named bilateral weighted adversarial networks is proposed to assign large weights to shared classes and small weights to outlier classes during the feature alignment. In the test phase, samples are classified according to the outputs of the deep model and unknown-class samples are rejected by the extreme value theory model. Experimental results on two bearing datasets demonstrate the effectiveness and superiority of the proposed method.