Small aerial object detection plays an important role in numerous computer vision tasks, including remote sensing, early warning systems, and visual tracking. Despite existing moving object detection techniques that can achieve reasonable results in normal size objects, they fail to distinguish the small objects from the dynamic background. To cope with this issue, a novel method is proposed for accurate small aerial object detection under different situations. Initially, the block segmentation is introduced for reducing frame information redundancy. Meanwhile, a random projection feature (RPF) is proposed for characterizing blocks into feature vectors. Subsequently, a moving direction estimation based on feature vectors is presented to measure the motions of blocks and filter out the major directions. Finally, variable search region clustering (VSRC), together with the color feature difference, is designed for extracting pixelwise targets from the remaining moving direction blocks. The comprehensive experiments demonstrate that our approach outperforms the level of state-of-the-art methods upon the integrity of small aerial objects, especially on the dynamic background and scale variation targets.