Augmented reality (AR) is an increasingly discussed topic with major research efforts focused upon in the recent years. Recent developments in ideas, software and hardware opened up new opportunities, and the application of augmented reality for virtual assembly purposes has great potential. Compared to real world tasks, virtual assembly allows users to complete tasks with lower cost and without exposure to environmental danger. The simulation of realistic real world interactions within a virtual world therefore becomes an essential issue to solve. This research proposes an augmented reality based assembly system, which provides the user with an natural user interface that allows them to manipulate objects using their bare hands, without additional worn equipment. A flexible constraint-based assembly system is implemented to handle part assembly. To resolve issues with occlusion and increase the immersion and perceived realism of the system, an innovative approach proposed and implemented, minimizing computational load. User test results show that the system was robust and successful in improving user experience in virtual assembly.