Satellite image time series classification has become a research focus with the launch of new remote sensing sensors capable of capturing images with high spatial, spectral, and temporal resolutions. In particular, in the field of crop classification, time dimension information is particularly important. Although some advanced machine learning algorithms, such as random forests (RFs), can achieve good results, they often ignore the time series information. To make full use of temporal and spectral information in multitemporal remote sensing images, a channel attention-based temporal convolutional network (CA-TCN) is proposed in this letter. Specifically, the proposed method is composed of two main modules: temporal convolutional network and attention block. The temporal convolutional network can capture long-range dependence by using a hierarchy of temporal convolutional filters. To capture relevant information inside the sequence and enhance the important information, the attention block is used to enhance the important features in the channel dimension since not all bands contain equal information in crop type classification. The proposed CA-TCN can excavate deeper phenological characteristics. Compared to the temporal attention-based temporal convolutional network and other deep learning-based models, the proposed CA-TCN has achieved state-of-the-art performance in the Breizhcrops dataset with fewer parameters.