An object tracking algorithm in a varying complex background, swimming pool, has been studied in this thesis. And its accuracy has been verified by video clips. The proposed approach consists of two main stages: swimmer detection and swimmer tracking. The detection stage begins with employing the mean-shift algorithm to cluster input image, and then chooses the sets by graphical models to train the Gaussian mixture model. Finally, swimmers can be detected in a model-based way. The main purpose in this tracking stage is to compare the accuracy among Kalman filter, mean-shift tracking algorithm, and particle filter algorithms. The experimental results show that the proposed approach can track swimmers efficiently. In the future, we hope to apply it on drowning alarm systems in swimming pools.