Aiming at the problems of false detection and missing detection in the complex background of photovoltaic cell defect detection, an improved YOLOv8 based photovoltaic cell defect detection algorithm was proposed. Firstly, the bidirectional feature pyramid network is used as the feature fusion mechanism to achieve multi-scale feature fusion through top-down and top-down paths. Secondly, the context aggregation module is introduced into the neck network, and the context information of different receptive fields is obtained by using the cavity convolution of different cavity convolution rates, which helps the model to identify small targets more accurately, and thus improves the target detection performance of the model. Finally, the boundary frame loss function is optimized and its weight factor is adjusted continuously to improve the convergence speed and efficiency of the model. The experimental results show that compared with the detection network of YOLOv8 algorithm, the recall rate and average accuracy are respectively increased by 10.4% and 1.8%, and the detection frame rate reaches 270 frames /s, ensuring the lightweight requirements of real-time detection and subsequent deployment. The improved algorithm can carry out robust detection of photovoltaic cell defects under complex background.