AAAI 2020
Conference paper

Do not have enough data? Deep learning to the rescue!

Download paper


Based on recent advances in natural language modeling and those in text generation capabilities, we propose a novel data augmentation method for text classification tasks. We use a powerful pre-Trained neural network model to artificially synthesize new labeled data for supervised learning. We mainly focus on cases with scarce labeled data. Our method, referred to as language-model-based data augmentation (LAMBADA), involves fine-Tuning a state-of-The-Art language generator to a specific task through an initial training phase on the existing (usually small) labeled data. Using the fine-Tuned model and given a class label, new sentences for the class are generated. Our process then filters these new sentences by using a classifier trained on the original data. In a series of experiments, we show that LAMBADA improves classifiers performance on a variety of datasets. Moreover, LAMBADA significantly improves upon the state-of-The-Art techniques for data augmentation, specifically those applicable to text classification tasks with little data.