Parallel model processing method and device based on multiple graphics processing units

The invention relates to a parallel model processing method based on multiple graphics processing units (GPUs). The method includes the steps: creating multiple Workers used for respectively controlling multiple Worker Groups in a central processing unit (CPU), wherein each Worker Group comprises the GPUs; binding each Worker with one corresponding GPU; loading one Batch of training data from a nonvolatile memory into a GPU video memory corresponding to one Worker Group; transmitting data, needed by the GPUs for data processing, among the GPUs corresponding to one Worker Group in a Peer to Peer manner; controlling the GPUs to perform data processing in parallel through the Workers. By the method, efficiency of parallel data processing of the GPUs can be improved. Besides, the invention further provides a parallel data processing device.