Obstacle avoidance is one of the fundamental and most important technologies for an autonomous mobile robot. However, most robots have the ability to detect and avoid obstacles are relied on help of laser or ultrasonic sensors, where the control structure is more complex and the cost is more expensive. The purpose of this paper is to develop an image processing scheme with a simple structure that only requires a stereo vision system for mobile robots to safely avoid static and dynamic obstacles in working environment. In other words, the stereo vision system consists of two cameras for detecting the static obstacles in working environment, and then it can precisely evaluate the size and relative position of obstacles by stereo vision algorithm. Finally, a safe path to the goal from the current position of the mobile robot is determined. Furthermore, as the mobile robot navigates the working environment, it can continuously rebuild the forward path of discovered moving obstacles. Using this path, the robot will avoid the dynamic obstacle to the goal.