This paper describes the optical non-contact of handwriting recognition system. Use the CMOS camera and fingertip writing trajectory, transmission to the processing system to achieve the handwriting recognition’s result. The traditional handwriting recognition system needs to touch the screen to achieve the characters’ recognition is the most different between the optical non-contact of handwriting recognition system. We processed for the images after CMOS camera captured, such as median filter, erosion and expansion. And use the principle of background subtraction to do the image segmentation and find the fingertip position, the last by algorithm to recognize the hand-written characters. After experimental verification, the system has effective recognize for characters. It can be feasible in the future.