A UAV aerial object detection algorithm based on modified YOLOv4 is proposed in order to address the high need of detection speed for UAV aerial object detection as well as the issue of missed detection and false detection when there are many small targets in aerial images. Firstly, the lightweight network MobileNetv3 is introduced to replace the main feature extraction network of YOLOv4, and the depth separable convolution is used to replace the 3×3 standard convolution of the network, which reduces the complexity of the model and improves the detection speed. Secondly, the 104×104 shallow detection layer for small targets is added, the three detection scales of the original feature extraction network are increased to four, and the number of feature fusion network layers is increased. These changes increase the algorithm′s accuracy in detecting small targets. Finally, the K-means++clustering technique is used to redesign the initial anchor frame, accelerating the network′s rate of convergence. The UAV aerial data set is used in a comparison experiment. The findings demonstrate that as compared to the original approach, the suggested technique not only ensures average detection accuracy but significantly enhances the detection accuracy of small targets. The detection time is 15.2% faster while the model parameters are lowered by 60%. It performs accurately and with good real-time performance.