We study the problem of transfer learning, observing that previous efforts to understand its information-theoretic limits do not fully exploit the geometric structure of the source and target domains. In contrast, our study first illustrates the benefits of incorporating a natural geometric structure within a linear regression model, which corresponds to the generalized eigenvalue problem formed by the Gram matrices of both domains. We next establish a finite-sample minimax lower bound, propose a refined model interpolation estimator that enjoys a matching upper bound, and then extend our framework to multiple source domains and generalized linear models. Surprisingly, as long as information is available on the distance between the source and target parameters, negative-transfer does not occur. Simulation studies show that our proposed interpolation estimator outperforms state-of-the-art transfer learning methods in both moderate- and high-dimensional settings.
翻译:我们研究转移学习问题,指出先前为了解其信息理论界限所作的努力没有充分利用源和目标域的几何结构。相反,我们的研究首先说明了将自然几何结构纳入线性回归模型的好处,该模型与这两个域的格拉姆矩阵形成的普遍的精度值问题相对应。我们接下来将建立一个有限分布式小型下界,提出一个具有匹配上界的精细模型内插测仪,然后将我们的框架扩大到多个源域和通用线性模型。令人惊讶的是,只要有关于源和目标参数之间的距离的信息,就不会出现负转移。模拟研究表明,我们提议的内插估计值在中高层环境中都超越了最先进的转移学习方法。