In order to simultaneously display information from multiple sensors, we need to fuse the image information that is acquired from infrared sensors and CCD sensor. This article make the resolution of infrared image is consistent with the resolution of the CCD image after the infrared image interpolation amplification, and get the infrared color images by using colour coding arithmetic. In order to realize the 2 pair of color image fusion, we adopt the method of color space transformation which can split color and brightness information and use different fusion strategies for each channel to fuse the image information, and realize the image fusion algorithm in FPGA.Finally,we design a set of realtime image fusion system based on FPGA.