Doubly Robust Covariate Shift Regression with Semi-nonparametric Nuisance Models

In contemporary statistical learning, covariate shift correction plays an important role when distribution of the testing data is shifted from the training data. Importance weighting is used to adjust for this but is not robust to model misspecifcation or excessive estimation error. In this paper, we propose a doubly robust covariate shift regression approach that introduces an imputation model for the targeted response, and uses it to augment the importance weighting equation. With a novel semi-nonparametric construction for the two nuisance models, our method is less prone to the curse of dimensionality compared to the nonparametric approaches, and is less prone to model mis-specification than the parametric approach. To remove the overfitting bias of the nonparametric components under potential model mis-specification, we construct calibrated moment estimating equations for the semi-nonparametric models. We show that our estimator is root-n consistent when at least one nuisance model is correctly specified, estimation for the parametric part of the nuisance models achieves parametric rate, and the nonparametric components are rate doubly robust. Simulation studies demonstrate that our method is more robust and efficient than existing parametric and fully nonparametric (machine learning) estimators under various configurations. We also examine the utility of our method through a real example about transfer learning of phenotyping algorithm for bipolar disorder. Finally, we propose ways to improve the (intrinsic) efficiency of our estimator and to incorporate high dimensional or machine learning models with our proposed framework.