Large pre-trained models are usually fine-tuned on downstream task data, and tested on unseen data. When the train and test data come from different domains, the model is likely to struggle, as it is not adapted to the test domain. We propose a new approach for domain adaptation (DA), using neuron-level interventions: We modify the representation of each test example in specific neurons, resulting in a counterfactual example from the source domain, which the model is more familiar with. The modified example is then fed back into the model. While most other DA methods are applied during training time, ours is applied during inference only, making it more efficient and applicable. Our experiments show that our method improves performance on unseen domains.
翻译:经过培训的大型模型通常对下游任务数据进行微调,并用不可见的数据进行测试。当火车和测试数据来自不同领域时,模型可能会挣扎,因为它不适应测试领域。我们提出新的领域适应方法(DA),使用神经层面的干预措施:我们修改每个测试示例在特定神经元中的表达方式,从源域得出一个反事实例子,而模型更熟悉这个例子。随后,修改后的示例被反馈到模型中。大多数其他指定官员的方法在培训期间应用,我们的方法只在推断期间应用,使其更有效和适用。我们的实验表明,我们的方法改善了在未知领域的绩效。