Neuromorphic computing, which is evolved from the imitation of biological nervous systems, has recently attracted considerable attention for its amazing capability in recognition and classification at a fraction of power. Intensive research has been conducted in this field for developing artificial synapses and neurons, attempting to mimic the behaviors of these two fundamental units in biological neural networks. Typically, the implementation of neuromorphic computing systems relies on integrating many transistors, which however, sacrifices energy efficiency and integration density. Recently, magnetic skyrmion, a swirling topological spin configuration, has been studied as a promising information carrier candidate in future ultra-dense, low-power memory and logic devices for its outstanding merits of nanoscale size, low depinning current density, high motion velocity and particle-like stability etc. To date, the research of neuromorphic computing and magnetic skyrmions has achieved great advances, yet none can fully exploit the advantages of both. This paper introduces some striking designs of skyrmion-based neuromorphic computing devices, which enable some different design paradigms from previous studies. We will firstly give a brief review of the related works on this field and then introduce our recent research. We will illustrate how the single neuronal or synaptic device can be realized by probing into the behaviors of skyrmions and how the related circuits/systems are achieved.