We introduce the integrative task of few-shot classification and segmentation (FS-CS) that aims to both classify and segment target objects in a query image when the target classes are given with a few examples. This task combines two conventional few-shot learning problems, few-shot classification and segmentation. FS-CS generalizes them to more realistic episodes with arbitrary image pairs, where each target class may or may not be present in the query. To address the task, we propose the integrative few-shot learning (iFSL) framework for FS-CS, which trains a learner to construct class-wise foreground maps for multi-label classification and pixel-wise segmentation. We also develop an effective iFSL model, attentive squeeze network (ASNet), that leverages deep semantic correlation and global self-attention to produce reliable foreground maps. In experiments, the proposed method shows promising performance on the FS-CS task and also achieves the state of the art on standard few-shot segmentation benchmarks.
翻译:我们引入了小片分类和分解(FS-CS)的综合任务,目的是在给目标类别提供几个例子时,在查询图像中将目标对象分类和部分目标对象分类,这一任务结合了两个传统的小片学习问题、小片分类和分解。FS-CS将其概括为更现实的情况,有任意的图像配对,其中每个目标类别可能存在,也可能没有出现在查询中。为了完成这项任务,我们建议了FS-CS的综合小片学习(iFSL)框架,该框架培训了一位学习者,以构建多标签分类和像素分解的类前方图。我们还开发了有效的 iFSL模型、细微挤压网络(ASNet),利用深度的语义相关性和全球自用来生成可靠的前景图。在实验中,拟议方法显示了FS-CS任务的良好表现,并实现了标准小片分解基准的艺术状况。