Research in moving object tracking has shown significant progress towards application in recent years. However, single sensors suffer from illumination variations for the vision sensor and low directional resolution for the radar. In this paper, we propose a moving object tracking method that fuses radar and vision data. First, false radar objects are filtered out. Second, an adaptive background subtraction method is used to detect candidate regions in images. Finally, moving objects are determined when the effective radar objects are in the candidate regions.