Applying correlation coefficient is a simple method to estimate the reliability coefficient, but many problems may sneak in it. The results of statistical simulation reveal that intraclass correlation coefficient is a good estimator of reliability, while Pearson correlation coefficient over estimates the reliability in many conditions. Therefore when we apply any method to estimate reliability, we must look at the assumption, the purpose and the required experimental design of it, otherwise some avoidable fallacy will arise from the inattentive situation.