In order to help people to maintain a healthy life style in today’s busy world, hav- ing a device that can accurately and conveniently calculate calories consumption will be crucial. This research paper has found a reliable system that can accurately report a user’s calories consumption rate by using the accelerometer, gyroscope and GPS sen- sors in mobile phones. It has successfully overcome the orientation problem that trou- bled many previous scholars. Resolving this issue is critical because mobile phones can be placed anywhere by users in real life. Firstly, this system can accurately recognize the following six most common phys- ical activities of a user’s walking, running, cycling, going upstairs, going downstairs and idling (stationary). By computing the amplitude of 1) vertical components, 2) magnitude of horizontal components, 3) vibration of the vertical and horizontal com- ponents, and 4) angle of the vertical and horizontal components, this system can attain an average accuracy rate of 90.44%, better than 73.09% in Yang’s method. This high accuracy rate is significant because wrong recognition of activities will amplify the margin of error in calories consumption rate calculation. Secondly, this system will use the ACSM Metabolic Equations, published by Amer- ican College of Sports Medicine and approved by many scholars, to calculate calories consumption. By applying the results generated from the above method to these equa- tions, this system can determine a user’s calories consumption rate with a mere margin of mean absolute error of 11.45%. It is significantly better than the consumption rate generated with Yang’s method, which has a margin of mean absolute error of 24.97%. Together, this system can accurately recognize a user’s activity independent of orien- tation, which is significant in estimating accurately the calories consumption rate.