Due to the tax penalties are often-controversial in our country, even academia constantly criticized, but the field of legal practice are still conservative, so that the problem has not been resolved yet. This may be closely related to the overall tax system and regulations. This study, will review and analyze the requirements of various tax penalties to highlight the impropriety and contradiction of provisions of administrative penalty of tax evasion, then positioning the status of administrative penalty of tax evasion in the tax penalties system. In particular, this paper has an in depth research for the function and boundary of administrative penalty of tax evasion and criminal penalty of tax evasion. Research results would show that when the taxpayer deliberately violate tax evasion should be punished by criminal punishments, administrative penalty of tax evasion apply only to negligent commission. Finally, in the legislative level, put forward the proposal of amendment, and before of that, this paper also proposed a temporary solution for the legal practice.