The traditional ORB-RANSAC algorithm has the problems of high error matching rate and poor stability of similar feature points, which can not meet the technological requirements of welding handle based on the direction of bottom label in aluminum pot production. An improved ORBRANSAC algorithm is proposed. Firstly, ORB algorithm was used to extract and match the feature points, and hamming distance threshold method was used to remove the matching points. Secondly, when RANSAC algorithm performs fine elimination of matching point pairs, kfold crossvalidation is added to achieve consistency prejudgment of the initial model. Finally, the classified point pairs in the previous iteration are removed during each iteration, and the sampling space is dynamically updated. The results show that the repeatability accuracy of the improved ORBRANSAC algorithm is improved by 6604% compared with the original algorithm in the interference environment, and the single frame calculation time is reduced by 613%. The Angle measurement error based on the improved ORBRANSAC algorithm is 0201° and the average detection time is 0255 s in the multitype pot bottom label measurement experiment, which meets the requirements of automatic production measurement accuracy and realtime.