Deep transfer learning (DTL) has formed a long-term quest toward enabling deep neural networks (DNNs) to reuse historical experiences as efficiently as humans. This ability is named knowledge transferability. A commonly used paradigm for DTL is firstly learning general knowledge (pre-training) and then reusing (fine-tuning) them for a specific target task. There are two consensuses of transferability of pre-trained DNNs: (1) a larger domain gap between pre-training and downstream data brings lower transferability; (2) the transferability gradually decreases from lower layers (near input) to higher layers (near output). However, these consensuses were basically drawn from the experiments based on natural images, which limits their scope of application. This work aims to study and complement them from a broader perspective by proposing a method to measure the transferability of pre-trained DNN parameters. Our experiments on twelve diverse image classification datasets get similar conclusions to the previous consensuses. More importantly, two new findings are presented, i.e., (1) in addition to the domain gap, a larger data amount and huge dataset diversity of downstream target task also prohibit the transferability; (2) although the lower layers learn basic image features, they are usually not the most transferable layers due to their domain sensitivity.
翻译:深层传输学习(DTL)已形成一项长期的追求,使深神经网络(DNN)能够像人类一样有效地再利用历史经验。这种能力被称为知识可转让性。DTL通常使用的模式是首先学习一般知识(培训前),然后再利用(调整)这些知识进行具体的目标任务。对预先培训的DNN的可转让性有两种共识:(1) 培训前和下游数据之间更大的领域差距降低了可转让性;(2) 从低层(近输入)到更高层(近输出)的可转让性逐步减少。然而,这些共识基本上是从自然图像实验中得出的,这限制了其应用范围。这项工作的目的是从更广泛的角度研究和补充这些知识,提出一种方法来衡量事先培训后DNNN参数的可转让性。我们对12个不同图像分类数据集的实验得出与先前的共识相似的结论。更重要的是,提出了两个新的结论,即:(1) 除了领域差距外,更大的数据数量和下游目标任务的巨大数据多样性(通常不限制其基本可转让性)的可转让性。