貝氏優化方法與機器學習分類方法於潛在類別迴歸模型的超參數選擇之應用

Bayesian Optimization and Machine Learning Classification for Hyper-parameter Selection in Latent Class Regression Models

許劭廷(Shao-Ting Hsu)

指導教授 ： 黃冠華

國立交通大學/理學院/統計學研究所/碩士(2020年)

摘要

潛在類別分析將各筆資料中多維度的測量指標分為若干類別。潛在類別迴歸延續潛在類別分析，利用個體之其他共變量來估計其潛在類別和多維度的測量指標。
在潛在類別迴歸中有三個超參數需要調整：潛在類別數、影響潛在類別分配之共變量、影響測量指標的條件機率之共變量。以往僅能使用逐步迴歸、窮舉法等方式，以模型預測值與資料之皮爾森殘差值作為模型與資料間適配度指標，以找到較佳的超參數組合。我們研發了一套利用貝氏優化與機器學習分類的超參數選擇方法，它可以提高選擇超參數之效率，並使用機器學習中常用之交叉驗證方法與交叉熵作為模型與資料間適配度指標。

關鍵字

貝氏優化 ； 分類 ； 超參數選擇 ； 潛在類別迴歸模型 ； 機器學習

並列摘要

Latent class analysis (LCA) can be used to classify people with multivariate observed indicators into several latent classes. Latent class regression is an extension of LCA, using people’s other covariates to estimate latent classes and multivariate observed indicators.
There are three types of hyper-parameters needing to be adjusted in LCR: the number of latent classes, covariates associated with latent prevalence and covariates associated with observed indicator probabilities conditional on class membership. In the past, one applied either stepwise regression or exhaustive search method to find out the optimal hyper-parameter combination in the Pearson’s residual sum of squares. This study adopts Bayesian optimization and machine learning classification approaches for hyper-parameter selection of the LCR mode, which makes the process of choosing hyper-parameter more efficient and uses cross validation method and cross entropy as the goodness-of-fit indicator.

並列關鍵字

Bayesian optimization ； Classification ； hyper-parameter selection ； latent class regression model ； machine learning

參考文獻

Ueno, Tsuyoshi & Rhone, Trevor David & Hou, Zhufeng & Mizoguchi, Teruyasu & Tsuda, Koji. (2016). COMBO: An Efficient Bayesian Optimization Library for Materials Science.

Bandeen-Roche, K., Miglioretti, D.L., Zeger, S.L., & Rathouz, P.J. (1997). Latent variable regression for multiple discrete outcomes. Journal of the American Statistical Association, 92, 1375–1386.

Batuwita, R., & Palade, V. (2013). Class imbalance learning methods for support vector machines. Imbalanced learning: Foundations, algorithms, and applications, 83.

Breiman, L. (2001). Random forests. Machine learning, 45(1), 5-32.

Chen, C. C. H., Chen, C. N., Lai, I. R., Huang, G. H., Saczynski, J. S., & Inouye, S. K. (2014). Effects of a modified Hospital Elder Life Program on frailty in individuals undergoing major elective abdominal surgery. Journal of the American Geriatrics Society, 62(2), 261-268.