過度自信與相依性對專家機率判斷整合的影響

Effects of Overconfidence and Dependence on Aggregated Probability Judgments

黃思瑋(Ssu-Wei Huang)

指導教授 ： 林希偉

元智大學/管理學院/企業管理學系/碩士(2008年)

摘要

在決策與風險分析中，決策者透過機率理論來架構問題中的不確定性，並往往需要透過專家來取得這些主觀機率的估計值，然而，這些專家判斷的品質容易受到客觀機率論者的質疑。其中，過度自信可以算是主觀機率估計中最嚴重的一個議題，而除了過度自信之外，專家間往往存在正向相依。
有鑑於過去的研究並未深入且完整探討上述不同的因素對整合專家機率判斷所造成的影響，我們在本研究中，將過度自信、相依性、專家數目和種子問題數等因素同時納入考量，分析這些變數與整合方法的關係。
透過實證資料進行交叉驗證的評估，我們發現過度自信、相依性與專家數目均顯著影響整體模型的表現。此外在不同的相依性與過度自信水準之下，平均權重法整合法的表現皆優於 Cooke 的古典模型整合法或專家小組中的最適專家。
進一步的模擬分析中，我們驗證了一些因子之間的重要交互作用，專家數目與專家變異程度的交互作用指出 Cooke 古典模型必須在專家校準度的變異較高，且專家數目足夠時，始能展現出篩選的效果。而過度自信與相依性的交互作用指出當相依性與過度自信程度皆高時，基於加權平均概念的線性意見彙整將有其限制，運用較複雜的貝氏模型來整合甚或擴展區間估計可能會是較佳的方案。

關鍵字

過度自信 ； 相依性 ； 專家判斷 ； 機率估計 ； 交叉驗證

並列摘要

Expert’s subjective probability interval or distribution estimates can provide useful information for forecasting and decision making. However, the performance of mathematical aggregation methods for probability distribution judgments—especially whether or not these approaches can be robust under situations of high overconfidence and dependence—still gives rise to questions needing to be clearly answered, through the use of either better data sets or better analytical methods.

In this study, we thoroughly evaluate several major linear opinion pooling methods, and concentrate particularly on exploring the effect of expert overconfidence, dependence, the size of expert panel, and the number of seed question on the performance of aggregated results to identify the main factors that affect the calibration of the aggregated interval or distribution judgments. We also use a large data set on expert opinion and conduct cross-validation for out-of-sample evaluation of the performance, to verify the effectiveness of various aggregation approaches. Finally, simulation techniques are used to further explore the interaction effects between these factors.

Our analyses show that both Cooke’s classical model and equal weight approach outperform the best expert, indicating the need of inputs from multiple experts. The significant overconfidence effect suggests all aggregation approaches investigated cannot effectively counteract the phenomena of expert overconfidence. The interaction effect between overconfidence and dependence suggests that methods involving broadening of subjective confidence intervals or distributions may occasionally also be useful.

並列關鍵字

Overconfidence ； Dependence ； Expert Judgment ； Probability Estimate ； Cross Validation

參考文獻

35. 劉明祥，相依程度對整合專家意見之影響，元智大學企業管理研究所碩士論文，2007。

10. Delbecq, A. L., Van de Ven, A. H., and Gustafson, D. H. (1975), “Group Techniques for Program Planning,” Glenview, IL: Scott Foresman.

1. Bacharach, M. (1979), “Normal Bayesian dialogues,” Journal of the American Statistical Association, 74, 837 – 846.

2. Bier, V. M. (2004), “Implications of the research on expert overconfidence and dependence,” Reliability Engineering and System Safety, 85, pp. 321-329.

3. Brier, G. W. (1950), “Verification of forecasts expressed in terms of probability,” Monthly Weather Review, 78, 1–3