Harmony is the important category in Chinese philosophy. It also plays a considerable role in Pre- Chin as well as in ancient Greek musical thought. Hence, I probe into the conception of harmony in Confucianism, Taoism and ancient Greek musical thought. In this paper, it is expected to reveal the generality and the difference in Chinese and Western musical thought to explore the deep inner trait in music aesthetics.