In continual learning, the learner faces a stream of data whose distribution changes over time. Modern neural networks are known to suffer under this setting, as they quickly forget previously acquired knowledge. To address such catastrophic forgetting, many continual learning methods implement different types of experience replay, re-learning on past data stored in a small buffer known as episodic memory. In this work, we complement experience replay with a new objective that we call anchoring, where the learner uses bilevel optimization to update its knowledge on the current task, while keeping intact the predictions on some anchor points of past tasks. These anchor points are learned using gradient-based optimization to maximize forgetting, which is approximated by fine-tuning the currently trained model on the episodic memory of past tasks. Experiments on several supervised learning benchmarks for continual learning demonstrate that our approach improves the standard experience replay in terms of both accuracy and forgetting metrics and for various sizes of episodic memories.
翻译:在持续学习中,学习者面临一系列数据流,其分布会随着时间的变化而变化。现代神经网络在这种环境下会受到影响,因为它们很快会忘记以前获得的知识。为解决这种灾难性的遗忘问题,许多持续学习的方法都采用不同类型的经验回放,对储存在被称为“偶发记忆”的小型缓冲中以往数据进行再学习。在这项工作中,我们用我们称之为“锚点”的新目标来补充经验回放,在这个目标中,学习者使用双层优化来更新对当前任务的知识,同时保持对过去任务某些锚点的预测不变。这些锚点是使用基于梯度的优化来学习,以最大限度地忘却,这通过微调目前受过训练的关于过去任务偶发记忆的模型可以大致看出。关于若干受监管的连续学习基准的实验表明,我们的方法在准确性和遗忘指标以及不同大小的脑细胞记忆方面改进了标准重现经验。