Effect of Bias and Variance on Estimation and Classification Error for Prediction

預測誤差與變異效應之估計與分類

Mostafizur Rahman ； 朱建平(Jian-Ping Zhu) ； 鄭宇庭(Yu-Ting Cheng)

《Journal of Data Analysis》 1卷3期 (2006/06) Pp. 113-135

https://doi.org/10.6338/JDA.200606_1(3).0008

摘要

我們的目標是呈現出預測平方誤差與分類誤差的變異以及誤差的影響。我們發現估計平方誤差與分類誤差的變異以及偏誤是不同的。對於給予的誤差／變異，估計平方誤差是與各變異與誤差對稱的。分類誤差則是依誤差範圍的量而定。若誤差範圍是負的，那麼分類誤差則會不論預測誤差是否增加而減少。誤差範圍是正的情況下，分類誤差會增加預測距離的1/2。誤差範圍的影響在分類誤差中可以因小的變異而減少。相似的變異誤差則視誤差範圍的值而定。我們使用最方便的方法來使這些影響降至最小。

並列摘要

Our aim is to show the effect of bias and variance on squared estimation error and classification error. We found that the bias and variance affect squared estimation error and classification error differently. For a given bias/ variance, squared estimation error is proportional to variance/ bias respectively. Classification error depends on the relevant quantity of boundary bias. If the boundary bias is negative then classification error decreases with increasing irrespective of the estimation bias. For positive boundary bias the classification error increases with the distance of estimation from 1/2. The effect of boundary bias on classification error can be mitigated by low variance. Similarly the affect of the variance depends on the value of the boundary bias. And we use nearest neighbor methods for minimizing these effects.

參考文獻

Bellman, R. E.(1961).Adaptive Control Process.Princeton University Press.

(Bianca Zadrozny (2004). IBM T.J. Watson Research Center, Yorktown Heights, NY 10598.).

Breiman, L.(1996).Bias, variance and arcing classifiers.,未出版Dept. of Statistics, University of California.

Chow, W. S.,Chen, Y. C.(1992).A new fast algorithm for effective training of neural classifiers.Pattern Recognition.25,423-429.

Dietterich, T. G.,Kong, E. B.(1995).Machine learning bias, statistical bias and statistical variance of decision tree algorithms.,未出版Dept. of Computer Science, Oregon State University.