Vehicle detection in aerial images has a wide range of applications, and the majority of vehicle detection methods use the bounding-box approach for localization. But the bounding-box approach usually yields low precision and recall rates, especially under the dense vehicle density situations where the close spatial proximity of the vehicles confuses the bounding-box detectors. This letter proposes a Fully Convolutional Lightweight Pyramid Network (FCLPN) to detect vehicles in visible-spectrum aerial images. Unlike the bounding-box approach, FCLPN performs pixel-level localization and classification. FCLPN trained on the DLR-3K dataset is directly tested on DLR-3K, VEDAI, COWC, and VAID datasets to validate its generalization strength. Experimental results show that FCLPN performs better than state-of-the-art methods in aerial vehicle detection in terms of Precision, F1 score, and mAP.