A Solution to Separation and Multicollinearity in Multiple Logistic Regression

Jian-Zhao Shen ； Su-Juan Gao

《Journal of Data Science》 6卷4期 (2008/10) Pp. 515-531

https://doi.org/10.6339/JDS.2008.06(4).395

並列摘要

In dementia screening tests, item selection for shortening an existing screening test can be achieved using multiple logistic regression. However, maximum likelihood estimates for such logistic regression models often experience serious bias or even non-existence because of separation and multicollinearity problems resulting from a large number of highly correlated items. Firth (1993, Biometrika, 80(1),27-38) proposed a penalized likelihood estimator for generalized linear models and it was shown to reduce bias and the non-existence problems. The ridge regression has been used in logistic regression to stabilize the estimates in cases of multicollinearity. However, neither solves the problems for each other. In this paper, we propose a double penalized maximum likelihood estimator combining Firth's penalized likelihood equation with a ridge parameter. We present a simulation study evaluating the empirical performance of the double penalized likelihood estimator in small to moderate sample sizes. We demonstrate the proposed approach using a current screening data from a community-based dementia study.

並列關鍵字

Logistic regression ； maximum likelihood ； penalized maximum likelihood ； ridge regression ； item selection