For the issue of low accuracy, poor realtime performance and large network model parameters of the existing insulator defect detection technology, an insulator defect detection model based on improved YOLO v4 is proposed in this study. Firstly, a modified VGG convolutional neural network was applied in the backbone feature extraction. In addition, to reduce the complexity of the model, depthwise separable convolution was introduced in the enhanced feature extraction and prediction networks. Moreover, channel attention mechanism was utilized in the enhanced feature extraction network to enhance the important features. The object recognition ability of the model for insulator defect was further strengthened. Finally, employing Average Precision, Frames Per Second, Parameter Scale, etc. as the evaluation indicators, ablation and comparison experiments were conducted on our constructed dataset based on the public dataset CPLID. The results show that the detection accuracy of the improved YOLO v4 model is 98.35%, which is 6.4% higher than that of the traditional YOLO v4 model. Moreover, the detection speed and parameter scale of the improved model are 1.5 times and 37.5% of those of the traditional YOLO v4 model. Accurate and real-time detection of aerial insulator defect imagery can be realized. Furthermore, the improved model also has higher accuracy, higher speed, and smaller parameter scale compared with other mainstream models YOLO v5-M and Faster R-CNN.