Vision-based sport analysis becomes a popular research topic these years. The most challenging problem of a table tennis robot is to analyze and predict the trajectory of the table tennis ball. We utilize Kinect to capture the skeleton of the table tennis player. Combining with the detection and tracking of table tennis ball and racket, we recognize the spin serve type and predict the trajectory of the table tennis ball.