In medicine, surgical operations are important clinical procedure for patient treatment. Although robots and/or robot arms have been widely used in industry, they are not yet seen in surgical operations at present. This is partly because the technical challenge of implementing such a medical robot remains in resolving the vision problems. In this study, our objective is to develop a vision-based system for the automatic recognition and localization of surgical equipment. The technical approaches include: (1) image preprocessing; (2) region segmentation; and (3) object recognition. Using a set of surgical equipment, i.e., scalpels, acutenaculums, and haemostats, preliminary results showed that our system could achieve the recognition rate of over 83.8%. In conclusion, our system has the potentials to be incorporated in a robot and/or a robot arm to play the role of a surgical assistant, despite that this is beyond the scope of this study.