The traditional CamShift only uses the color histogram of the target as the feature, so it may lead to inaccurate tracking or losing the target in the case of similar background, occlusion, high-speed motion and so on. In view of the above shortcomings, an improved CamShift tracking algorithm based on SIFT and perceptual hash is proposed. Firstly, transforming the image from RGB color space to HSV color space, then extracting the hue and saturation histograms and the edge gradient histogram of the image and combine the histograms to obtain the fusion histogram of the target. Secondly, using the fusion histogram of the target to obtain the optimal candidate target under the framework of CamShift algorithm. If the Bhattacharyya distance between the candidate target and the target template is larger than the threshold, using the improved perceptual hash algorithm to search the optimal candidate target. Then in the next frame search, using the SIFT algorithm to extract the feature points of the high information entropy part of both the target and video sequence, then matching the feature points to obtain the initial search window. If the SIFT algorithm fails to match, using the search box which is predicted by the Kalman filter as the initial search window to search the target. The algorithm is compared with other common tracking algorithms on OTB-100 dataset. The experimental results show that the algorithm can track the target accurately and the success rate reaches 90.1%. Then applying the algorithm to the task of face tracking and compared with other face tracking algorithms. The experimental results show that the algorithm has good performance and high accuracy, and the tracking success rate reaches 93.5%.