The median filter not only suppresses interference pulse and punctiform noise, but also keeps the image details better, therefore it plays an important role in many image processing. Moving object recognition technology needs higher and higher realtime quality in image preprocessing. In order to improve the preprocessing speed, we study the filtering algorithm and implementation scheme based on the median filter principle. And a fast image median filtering method based on the order statistics theory is proposed to realize realtime filtering. The results show that this method is the improvement of traditional median filtering sorting algorithm on hardware system, making 9 pixels of 2 dimension reduce to 5 pixels of 1 dimension to find the median in window 3×3. The comparing times, finding the median in single window, change from traditional sort of 36 times down to 16 times. Combined with the FPGA parallel processing ability and pipeline structure, filtering speed increases by almost four times, so as to satisfy the realtime image preprocessing.