The existing multispectral imaging technologies usually utilize optical spectroscopy and multiple detectors to capture spectral images. These techniques suffer from complexity, a large amount of data and low efficiency. Addressing these deficiencies, in this paper, a spectral encoded computational ghost imaging technology based on orthogonal modulation model was proposed. The orthogonal spectral encoded matrices fused with Hadamard patterns were used to produce the illumination patterns that modulate the broadband light source. A single-pixel detector was utilized to collect the back-reflected signal from the imaging objects. The evolutionary compressive technology was applied to recover the mixed spectral image. The subsampled spectral channel images were obtained from the mixed spectral image by means of the orthogonality of the spectral encoded matrices. Then the group sparse compressed sensing algorithm was applied to reconstruct the full-sampling spectral channel images, which finally fused the multispectral image of the imaging object. The efficiency of the proposed method was verified by a numerical simulation and an experiment. The proposed technology simplifies the multispectral imaging configuration and greatly reduces the amount of data. The orthogonal spectral encoded strategy can extend to more spectral channels and also can be applied to polarization imaging, information encryption, and other many fields.