We focus on a small object detection model for digital twin, aiming to realize the dynamic synchronization between a physical manufacturing system and its virtual representation. Three significant elements, including equipment, product, and operator, are considered as the basic environmental parameters to represent and estimate the dynamic characteristics and real-time changes in building a generic DT system of smart manufacturing workshop. A hybrid deep neural network model based on the integration of MobileNet-v2, YOLOv4, and Openpose, is constructed to identify the real-time status from physical manufacturing environment to virtual space. A learning algorithm is then developed to realize the multi-type small object detection, in order to facilitate the modeling, monitoring, and optimizing of the whole smart manufacturing process in DT system. Experiments and evaluations using a real-world dataset demonstrate the effectiveness and usefulness of our proposed method which can achieve a higher detection accuracy for digital twinning in smart manufacturing.