Shrinkage estimators for mixture of linear and logistics regression models

Abstract

The mixture of regression models is one of the most common model-based techniques to incorporate the information of covariates into learning population heterogeneity. The multicollinearity problem is one of the most common problems in regression and a mixture of regression models where the covariates are highly correlated. This problem results in unreliable maximum likelihood estimates for the regression coefficients. In the first part of this thesis, we developed two shrinkage methods through an unsupervised learning approach to estimate the model coefficients in the presence of multicollinearity issues. These shrinkage methods include Ridge and Liu-type estimators. The estimation and prediction performance of the methods are evaluated via EM algorithms. In the second part of the thesis, we focus on extending the mixture analysis to the binary response in the presence of multicollinearity. The logistic regression model is one of the most powerful statistical methods for analysis of binary data. The logistic regression allows to use a set of covariates to explain the binary responses. The mixture of logistic regression models is used to fit heterogeneous populations through an unsupervised learning approach. This research developed Ridge and Liutype shrinkage methods to deal with the multicollinearity in a mixture of logistic regression models. Through extensive numerical studies, we show that the developed methods provide more reliable results in estimating the coefficients of the mixture models. We applied the shrinkage methods to analyze the bone disorder status of women aged 50 and older.