Aiming at the problem that the fixed fusion algorithm can not adapt to the difference between video frames in dynamic scene, which results in poor fusion effect and even failure of fusion, a fusion method of infrared and visible video mimicry based on differential feature driver is proposed. Firstly, three different features of infrared and visible video sequences are extracted respectively. Secondly, the fusion validity of different fusion algorithms is calculated by using the improved fusion validity formula, the fusion validity is weighted by entropy weight method, and then the decision score of the fusion algorithm is obtained, and the optimal fusion algorithm on different video sequences is determined. The experimental results show that the proposed method is better than the optimal fusion algorithms RP and MOD in the whole video sequence, compared with the fixed algorithm mentioned above, the comprehensive objective evaluation index is improved by 59.9249% and 2.7608% , which provides a new idea for infrared and visible light video fusion.