Fog computing is promising to solve the challenge caused by an extremely large amount of data on cloud computing. In this study, an efficient binary convolutional neural network with numerous skip connections (BNSC-Net) is proposed for fog computing to enable real-time smart industrial applications. This network features decomposition convolution kernels and concatenated feature maps. Moreover, the network performance is further improved through expanding the update interval of the straight-through estimator. To verify the performance, BNSC-Net is tested on two broadly used public data sets: 1) ImageNet and 2) CIFAR-10. An ablation study is first conducted to verify the effectiveness of the proposed improved operations, and results demonstrate that BNSC-Net can obviously increase the classification accuracy for both data sets. ImageNet-based classification results indicate that BNSC-Net can achieve 59.9% TOP-1 accuracy that is 2.6% higher than the state-of-the-art binary neural networks, such as projection convolutional neural networks (PCNNs). Finally, a subset with ten classes is selected from ImageNet to simulate the data collected in the smart industry with limited categories, based on which BNSC-Net also demonstrates an impressive classification performance with friendly memory and calculation requirements. Particularly, the receiver operating characteristic curves of BNSC-Net surpass that of the state-of-the-art algorithm DeepIns. Therefore, the proposed BNSC-Net is effective and efficient for building deep learning-enabled industrial applications on fog nodes.