The desire for learning the relationships from data has resulted in a lot of research on statistics and data mining. The two fields have developed separately for years. Many wonder about their similarities. This paper will discuss their relationships from the underlying mathematical point of views and also from the problems they intend to solve.