使用古典模型可以篩選與整合出較佳的 機率區間估計嗎？

Can Cooke’s Classical Model Sift Out And Aggregate A Better Probability Interval Estimate?

鄭治炘(Chih-Hsing Cheng)

指導教授 ： 林希偉

元智大學/管理學院/企業管理學系/碩士(2008年)

摘要

在決策與風險分析中，Cooke 的古典模型是整合專家機率分配估計時常用的方法，其主要的優點在於擁有嚴謹的數學架構和理論基礎，並根據個別專家在校準度與資訊度的表現給予不同的權重。然而，Cooke 的古典模型仍存在許多缺陷。Clemen （2008）認為古典模型有二個令人質疑的地方。首先，因為 Cooke 的校準度計分並非基於個別機率評量，因此可能存有讓專家策略性地提供（非反應真實主觀）機率分配或者機率區間預測之可能。其次，在評估整合模型時所使用的專家估計資料，一方面被用於計算專家的權重，另一方面又用於評估整合後之預測區間的優劣，這樣的評估難免造成對古典模型之偏頗。
本研究針對上述疑點，使用 Cooke 教授與其他的研究人員在執行專家機率判斷相關研究所收集的大型資料庫，然後利用交叉驗證中 leave-one-out 的方法來驗證上述的二個問題，並使用中位數分析法與 Jackknife 重抽樣法來檢視古典模型的精確度與穩定度。本研究發現雖然古典模型於樣本外（out-of-sample）的表現，並沒有如樣本內（in-sample）一樣優異，但 Cooke 古典模型的績效表現與精確度仍優於簡單平均法，本研究中的非線性迴歸模型同時指出種子問題數越多則權重變化的幅度越低，且其下降之趨勢呈（負）指數函數之關係。另一方面，簡單平均法的表現則較古典模型穩定，是以在實務使用上，仍有其重要性。

關鍵字

古典模型 ； 校準 ； 計分法則 ； 交叉驗證 ； 專家判斷

並列摘要

Cooke’s classical model is one of the most widely used methods for aggregating experts’ probability interval estimates in the field of decision and risk analysis. Cooke’s model is simple and mathematically sound, and it could also assign experts different weights based on their performance on calibration and information on seed questions. However, experts may have chance to obtain higher scores and hence receive more weights by dishonestly report their quantile estimates, because Cooke’s scoring rule is based on average probabilities. Furthermore, the data in Cooke’s studies are usually used for calculating experts’ weights and at the same time for assessing models. This may lead to results that favor Cooke’s classical model.

In this study, we use a larger data set to verify the superiority of Cooke’s model. In particular, we adopt the leave-one-out cross validation technique to perform out-of-sample comparison of Cooke’s classical model, equal weight linear pooling method, and the best expert approach, use median analysis to evaluate the accuracy of different models, and apply Jackknife re-sampling technique to examine the stability of the classical model. Our results indicate that the performance of classical model is much poorer after using out-of-sample analysis. However, classical model still performs better than equal weight approach. Furthermore, the non-linear regression model in this study also indicates that the stability of Cooke’s classical model will increase exponentially as the number of seed questions increases. These results verify the capability of Cooke’s model in sifting out and aggregating better probability interval estimates. On the other hand, the equal weight approach is still more stable than the classical model in the whole, and thus still plays an important role in practical application.

並列關鍵字

classical model ； calibration ； scoring rule ； cross validation ； expert judgment.

參考文獻

1. Bier, V., (2004), “Implications of the research on expert overconfidence and dependence,” Reliability Engineering and System Safety, 85, 321-329.

2. Chatfield, C., (2001), “Prediction Intervals for Time-series Forecasting,” in Principles of Forecasting: A Handbook for Researchers and Practitioners, edited by J. S. Amstrong, Kluwer Academic Publishers, 475 – 494.

3. Clemen, R. T., & Winkler, R. L., (1999), “Combining Probability Distributions From Experts in Risk Analysis,” Risk Analysis, 19, 2, 187-203.

4. Clemen, R. T., (2008), “Comment on Cooke’s classical method,” Reliability Engineering and System Safety, 93, 760-765.

5. Clemen, R. T., & Winkler, R. L., (1987), “Calibrating and combining precipitation probability forecasts,” In R. Viertl (Ed.), Probability and Bayesian Statistics, New York: Plenum, 97-110.