Microwave absorbers with light weight and excellent microwave absorption performance are urgently needed in the microwave absorption field, which is still a challenge. Herein, N-doped carbon nanofibers decorated with nickel nanoparticles (Ni@CNFs) were synthesized by a facile electrospinning method combined with a two-step heat treatment, in which Ni nanoparticles are uniformly dispersed in carbon nanofibers. Benefitting from the special nanoarchitecture (including Ni nanoparticles encapsulated with graphitic carbon layers and carbon nanotube protrusions anchored on CNFs), three-dimensional conductive networks, the synergistic effect between suitable impedance matching and satisfactory electromagnetic (EM) attenuation ability, a superb comprehensive microwave absorption (MA) property is achieved for the optimal Ni@CNF sample with a rather low filler loading of 5 wt%. The optimal reflection loss (RL) reaches -66.3 dB at a small thickness of 3.1 mm and the maximum effective absorption bandwidth (EAB, RL < -10.0 dB) as wide as 4.56 GHz is obtained at 2.0 mm. This study demonstrates that the carefully designed Ni@CNF composites are superior to many previously reported magnetic carbon-based hybrid absorbers and can be applied as promising candidates for light weight and high-efficiency EM wave absorbers.