This article is devoted to explaining how ancient Chinese literati family perceived mathematics using Yan Zhitui and his Yan Shi Jia Xun (Clan Education of Yan Family) as example. While Yan Zhitui's major concern was how the competency of six arts of the literati family was taken advantage by the noble family, his viewpoint towards mathematics followed however the Confucian tradition, i.e., mathematics itself is only one of the six inferior arts. Viewed in this context of how Zu Chongzhi and Zu Geng (father and son) regarded their own notable mathematical accomplishment as by modern standard, one would better understand why the father and son were more concerned with whether the father's Da Ming Calendar was put into practice.