In recent years, mobile devices are nearly indispensable in our daily lives; for instance, message, Facebook, and credit card number. Therefore, privacy of personal information on mobile devices has become an important issue. Hence, this paper proposes a new approach continuously protecting the user information from an impostor’s attack. The approach detects whether the current users of mobile devices are actual owners or impostors. From the result of our experiment, we observed that our approach obtains the lower equal error rate than that of the related work, which make use of the popular classification algorithms (SVN, k-NN).