小样本学习(Few-Shot Learning,以下简称 FSL )用于解决当可用的数据量比较少时,如何提升神经网络的性能。在 FSL 中,经常用到的一类方法被称为 Meta-learning。和普通的神经网络的训练方法一样,Meta-learning 也包含训练过程和测试过程,但是它的训练过程被称作 Meta-training 和 Meta-testing。

VIP内容

我们发现了在流行的小样本学习(FSL)方法中一直被忽视的一个缺陷: 预训练的知识确实是限制性能的一个混杂因素。这一发现源于我们的因果假设: 一个关于预训练的知识、样本特征和标签之间因果关系的结构性因果模型(SCM)。正因为如此,我们提出了一种新的FSL范式:干预少样本学习(IFSL)。具体来说,我们开发三个有效的IFSL算法,它本质上是一个因果干预SCM学习:目前在因果视图的上限。值得注意的是,IFSL的贡献与现有的基于微调和元学习的FSL方法是正交的,因此IFSL可以改进所有这些方法.

成为VIP会员查看完整内容
0
24

最新内容

Neural networks are known to suffer from catastrophic forgetting when trained on sequential datasets. While there have been numerous attempts to solve this problem for large-scale supervised classification, little has been done to overcome catastrophic forgetting for few-shot classification problems. We demonstrate that the popular gradient-based few-shot meta-learning algorithm Model-Agnostic Meta-Learning (MAML) indeed suffers from catastrophic forgetting and introduce a Bayesian online meta-learning framework that tackles this problem. Our framework incorporates MAML into a Bayesian online learning algorithm with Laplace approximation. This framework enables few-shot classification on a range of sequentially arriving datasets with a single meta-learned model. The experimental evaluations demonstrate that our framework can effectively prevent forgetting in various few-shot classification settings compared to applying MAML sequentially.

0
0
下载
预览

最新论文

Neural networks are known to suffer from catastrophic forgetting when trained on sequential datasets. While there have been numerous attempts to solve this problem for large-scale supervised classification, little has been done to overcome catastrophic forgetting for few-shot classification problems. We demonstrate that the popular gradient-based few-shot meta-learning algorithm Model-Agnostic Meta-Learning (MAML) indeed suffers from catastrophic forgetting and introduce a Bayesian online meta-learning framework that tackles this problem. Our framework incorporates MAML into a Bayesian online learning algorithm with Laplace approximation. This framework enables few-shot classification on a range of sequentially arriving datasets with a single meta-learned model. The experimental evaluations demonstrate that our framework can effectively prevent forgetting in various few-shot classification settings compared to applying MAML sequentially.

0
0
下载
预览
Top