Incremental few—shot
212645695
Incremental Few Shot Object Detection
Existing object detection methods typically rely on the availability of abundant labelled training samples per class and offline model training in a batch mode. These requirements substantially limit their scalability to open ended accommodation of novel classes with limited labelled training data, both in terms of model accuracy and training efficiency during deployment. We present the first study aiming to go beyond these limitations by considering the Incremental Few Shot Detection (iFSD) problem setting, where new classes must be registered incrementally (without revisiting base classes) and with few examples. To this end we propose OpeN ended Centre nEt (ONCE) a detector designed for incrementally learning to detect novel class objects with few examples. This is achieved by an elegant adaptation of the efficient CentreNet detector to the few shot learning scenario, and meta learning a class wise code generator model for registering novel classes. ONCE fully respects the incremental learning paradigm, with novel class registration requiring only a single forward pass of few shot training samples, and no access to base classes thus making it suitable for deployment on embedded devices, etc. Extensive experiments conducted on both the standard object detection (COCO, PASCAL VOC) and fashion landmark detection (DeepFashion2) tasks show the feasibility of iFSD for the first time, opening an interesting and very important line of research.
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Juan-Manuel Perez-Rua Xiatian Zhu Timothy M Hospedales Tao Xiang
52986657
Incremental Few Shot Learning with Attention Attractor Networks
Machine learning classifiers are often trained to recognize a set of pre defined classes. However, in many applications, it is often desirable to have the flexibility of learning additional concepts, with limited data and without re training on the full training set. This paper addresses this problem, incremental few shot learning, where a regular classification network has already been trained to recognize a set of base classes, and several extra novel classes are being considered, each with only a few labeled examples. After learning the novel classes, the model is then evaluated on the overall classification performance on both base and novel classes. To this end, we propose a meta learning model, the Attention Attractor Network, which regularizes the learning of novel classes. In each episode, we train a set of new weights to recognize novel classes until they converge, and we show that the technique of recurrent back propagation can back propagate through the optimization process and facilitate the learning of these parameters. We demonstrate that the learned attractor network can help recognize novel classes while remembering old classes without the need to review the original training set, outperforming various baselines.
NeurIPS
Mengye Ren Renjie Liao Ethan Fetaya Richard S Zemel
213005291
XtarNet: Learning to Extract Task Adaptive Representation for Incremental Few Shot Learning
Learning novel concepts while preserving prior knowledge is a long standing challenge in machine learning. The challenge gets greater when a novel task is given with only a few labeled examples, a problem known as incremental few shot learning. We propose XtarNet, which learns to extract task adaptive representation (TAR) for facilitating incremental few shot learning. The method utilizes a backbone network pretrained on a set of base categories while also employing additional modules that are meta trained across episodes. Given a new task, the novel feature extracted from the meta trained modules is mixed with the base feature obtained from the pretrained model. The process of combining two different features provides TAR and is also controlled by meta trained modules. The TAR contains effective information for classifying both novel and base categories. The base and novel classifiers quickly adapt to a given task by utilizing the TAR. Experiments on standard image datasets indicate that XtarNet achieves state of the art incremental few shot learning performance. The concept of TAR can also be used in conjunction with existing incremental few shot learning methods; extensive simulation results in fact show that applying TAR enhances the known methods significantly.
ICML
Sung Whan Yoon Do-Yeon Kim Jun Seo Jaekyun Moon
227248038
A Few Guidelines for Incremental Few Shot Segmentation
Reducing the amount of supervision required by neural networks is especially important in the context of semantic segmentation, where collecting dense pixel level annotations is particularly expensive. In this paper, we address this problem from a new perspective: Incremental Few Shot Segmentation. In particular, given a pretrained segmentation model and few images containing novel classes, our goal is to learn to segment novel classes while retaining the ability to segment previously seen ones. In this context, we discover, against all beliefs, that fine tuning the whole architecture with these few images is not only meaningful, but also very effective. We show how the main problems of end to end training in this scenario are i) the drift of the batch normalization statistics toward novel classes that we can fix with batch renormalization and ii) the forgetting of old classes, that we can fix with regularization strategies. We summarize our findings with five guidelines that together consistently lead to the state of the art on the COCO and Pascal VOC 2012 datasets, with different number of images per class and even with multiple learning episodes.
ArXiv
Fabio Cermelli Massimiliano Mancini Yongqin Xian Zeynep Akata Barbara Caputo
226292007
Incremental Few Shot Meta learning via Indirect Discriminant Alignment
We propose a method to train a model so it can learn new classification tasks while improving with each task solved. This amounts to combining meta learning with incremental learning. Different tasks can have disjoint classes, so one cannot directly align different classifiers as done in model distillation. On the other hand, simply aligning features shared by all classes does not allow the base model sufficient flexibility to evolve to solve new tasks. We therefore indirectly align features relative to a minimal set of "anchor classes." Such indirect discriminant alignment (IDA) adapts a new model to old classes without the need to re process old data, while leaving maximum flexibility for the model to adapt to new tasks. This process enables incrementally improving the model by processing multiple learning episodes, each representing a different learning task, even with few training examples. Experiments on fewshot learning benchmarks show that this incremental approach performs favorably compared to training the model with the entire dataset at once.
ECCV
Qing Liu Orchid Majumder Alessandro Achille Avinash Ravichandran Rahul Bhotika Stefano Soatto
173990610
Incremental Few Shot Learning for Pedestrian Attribute Recognition
Pedestrian attribute recognition has received increasing attention due to its important role in video surveillance applications. However, most existing methods are designed for a fixed set of attributes. They are unable to handle the incremental few shot learning scenario, i.e. adapting a well trained model to newly added attributes with scarce data, which commonly exists in the real world. In this work, we present a meta learning based method to address this issue. The core of our framework is a meta architecture capable of disentangling multiple attribute information and generalizing rapidly to new coming attributes. By conducting extensive experiments on the benchmark dataset PETA and RAP under the incremental few shot setting, we show that our method is able to perform the task with competitive performances and low resource requirements.
IJCAI
Liuyu Xiang Xiaoming Jin Guiguang Ding Jungong Han Leida Li
233394141
Incremental Few shot Text Classification with Multi round New Classes: Formulation, Dataset and System
Text classification is usually studied by labeling natural language texts with relevant categories from a predefined set. In the real world, new classes might keep challenging the existing system with limited labeled data. The system should be intelligent enough to recognize upcoming new classes with a few examples. In this work, we define a new task in the NLP domain, incremental few shot text classification, where the system incrementally handles multiple rounds of new classes. For each round, there is a batch of new classes with a few labeled examples per class. Two major challenges exist in this new task: (i) For the learning process, the system should incrementally learn new classes round by round without re training on the examples of preceding classes; (ii) For the performance, the system should perform well on new classes without much loss on preceding classes. In addition to formulating the new task, we also release two benchmark datasets in the incremental few shot setting: intent classification and relation classification. Moreover, we propose two entailment approaches, ENTAILMENT and HYBRID, which show promise for solving this novel problem.
NAACL
Congyin Xia Wenpeng Yin Yihao Feng P L H Yu
231940830
Incremental few shot learning via vector quantization in deep embedded space
The capability of incrementally learning new tasks without forgetting old ones is a challenging problem due to catastrophic forgetting. This challenge becomes greater when novel tasks contain very few labelled training samples. Currently, most methods are dedicated to class incremental learning and rely on sufficient training data to learn additional weights for newly added classes. Those methods cannot be easily extended to incremental regression tasks and could suffer from severe overfitting when learning few shot novel tasks. In this study, we propose a nonparametric method in deep embedded space to tackle incremental few shot learning problems. The knowledge about the learned tasks is compressed into a small number of quantized reference vectors. The proposed method learns new tasks sequentially by adding more reference vectors to the model using few shot samples in each novel task. For classification problems, we employ the nearest neighbor scheme to make classification on sparsely available data and incorporate intra class variation, less forgetting regularization and calibration of reference vectors to mitigate catastrophic forgetting. In addition, the proposed learning vector quantization (LVQ) in deep embedded space can be customized as a kernel smoother to handle incremental few shot regression tasks. Experimental results demonstrate that the proposed method outperforms other state of the art methods in incremental learning.
ICLR
Kuilin Chen Chi-Guhn Lee
227230475
A Two phase Prototypical Network Model for Incremental Few shot Relation Classification
Relation Classification (RC) plays an important role in natural language processing (NLP) Current conventional supervised and distantly supervised RC models always make a closed world assumption which ignores the emergence of novel relations in open environment. To incrementally recognize the novel relations, current two solutions (i.e, re training and lifelong learning) are designed but suffer from the lack of large scale labeled data for novel relations. Meanwhile, prototypical network enjoys better performance on both fields of deep supervised learning and few shot learning. However, it still suffers from the incompatible feature embedding problem when the novel relations come in. Motivated by them, we propose a two phase prototypical network with prototype attention alignment and triplet loss to dynamically recognize the novel relations with a few support instances meanwhile without catastrophic forgetting. Extensive experiments are conducted to evaluate the effectiveness of our proposed model.
COLING
Haopeng Ren Yi Cai Xiaofeng Chen Guohua Wang Qing Li
221680919
SUPPLEMENTAL MATERIAL XtarNet: Learning to Extract Task Adaptive Representation for Incremental Few Shot Learning
For the miniImageNet experiments of XtarNet, we utilized ResNet12, a residual network with 12 convolutional layers, as architecture of the backbone network. ResNet12 consists of four residual blocks having 64, 96, 128 and 256 respective channels. We use a LeakyReLu for activation function and the slope is 0.1 at x 0. Each residual block contains three 3 x 3 convolutional layers, where every convolutional layer is followed by a batch normalization and an activation function. For the last layer of the fourth residual block, we do not use an activation function. The residual connection of each block consists of a single 3 x 3 convolutional layer and a batch normalization layer. At the end of each residual block, 2 x 2 max pooling is applied. To reduce the dimension of the feature vector, a global average pooling is applied to the output of the last residual block. For every residual block, we use a dropout layer with a drop ratio of 0.2 after the max pooling layer. For all methods other than XtarNet, the difference is that we use a ReLu for activation function and do not remove the activation at the last layer of the fourth residual block.
Sung Whan Yoon Do-Yeon Kim Jun Seo Jaekyun Moon