The segmentation and extraction of PV panel region information from infrared images of PV panels can greatly improve the accuracy of PV panel fault detection. However, the traditional semantic segmentation algorithm is not effective in processing the boundary information of PV panels, and there are cases that the boundary of PV panels is wave-like, sticking to each other, and the background is mis-segmented. To solve this situation, this paper proposes a semantic segmentation algorithm model for PV panels based on improved DeepLabV3+, which changes the backbone network to MobileNetV2, introduces the Canny edge detection algorithm to output new shallow feature semantic information, and designs the SE-ASPP module to re-calibrate the feature channels to enhance the network expression capability, and increase the number of channels of shallow feature semantic information to strengthen the attention to shallow feature semantic information. Experimental results show that the precision, mIoU, recall and F1 score of the improved DeepLabV3+ algorithm model reach 99.50%, 99.21%, 99.61% and 99.55%, respectively, which are 2.24%, 1.58%, 1.57% and 1.72% higher than the original DeepLabV3+ model, respectively. Improved DeepLabV3+ model performs well in real segmentation tasks and has higher detection accuracy and reliability.