To overcome the difficulty of choosing the dynamic model and giving the strength of noise that applying the Kalman filter will face, this paper presents an algorithm of noise adaptive Kalman filtering for video point target tracking, based on the characteristic of video point targets. To better fit the dynamics of a video point target, the twostage dynamic model, which can changes its form with regard to the value of correlation time constant, has been chosen as the state transition model for Kalman filtering. Moreover, the process noise is estimated according to the dynamic model and the observation data. Meanwhile, the observation noise is estimated according to the grey value distribution in each image. Then the process noise and the observation noise are adaptive. According to the outfield experimental result, the methods we proposed could effectively ensure the tracking accuracy.