This paper proposes a color classification and identification methods of moving objects on the real-time video surveillance. HSV color space was used in this study that lets the color more clearly to be defined. Use C-means clustering algorithm to simplify the colors for each frame, and then use statistical methods to define the representative color of each selected object. This paper combines with information feedback, thumbnail, etc. methods to enhance the execution speed to satisfy the requirement of real-time video surveillance.