In recently years, the concept of health is popularized and promoted unceasingly. Smoking is harmful to our health seriously. The government had put the new “Tobacco Hazards Prevention and Control Act” into practice this year. It stipulates that smoking in the public places is prohibited except in the designated smoking areas. Thereby, we propose a vision-based system and operate this system in coordination with general camera to detect smoking event. Our system is based on face detection. We make use of color feature and various characteristics of smoking event to detect cigarette. And then, we analyze the correlation between cigarette and hands and the continuity of smoking event to recognize the behavior. Experimental results reveal that the performance of proposed system is well.