Human emotion recognition could greatly contribute to human-computer interaction with promising applications in artificial intelligence. One of the challenges in recognition tasks is learning effective representations with stable performances from electroencephalogram (EEG) signals. In this paper, we propose a novel deep-learning framework, named channel-fused dense convolutional network, for EEG-based emotion recognition. First, we use a 1D convolution layer to receive weighted combinations of contextual features along the temporal dimension from EEG signals. Next, inspired by state-of-the-art object classification techniques, we employ 1D dense structures to capture electrode correlations along the spatial dimension. The developed algorithm is capable of handling temporal dependencies and electrode correlations with effective feature extraction from noisy EEG signals. Finally, we perform extensive experiments based on two popular EEG emotion datasets. Results indicate that our framework achieves prominent average accuracies of 90.63% and 92.58% on the SEED and DEAP datasets respectively, which both receive better performances than most of the compared studies. The novel model provides an interpretable solution with excellent generalization capacity for broader EEG-based classification tasks.