ZeroAE: Pre-trained Language Model based Autoencoder for Transductive Zero-shot Text Classification

Citations Over TimeTop 19% of 2023 papers

Abstract

Many text classification tasks require handling unseen domains with plenty of unlabeled data, thus giving rise to the self-adaption or the so-called transductive zero-shot learning (TZSL) problem. However, current methods based solely on encoders or decoders overlook the possibility that these two modules may promote each other. As a first effort to bridge this gap, we propose an autoencoder named ZeroAE. Specifically, the text is encoded with two separate BERT-based encoders into two disentangled spaces, i.e., label-relevant (for classification) and label-irrelevant respectively. The two latent spaces are then decoded by prompting GPT-2 to recover the text as well as to further generate text with labels in the unseen domains to train the encoder in turn. To better exploit the unlabeled data, a novel indirect uncertainty-aware sampling (IUAS) approach is proposed to train ZeroAE. Extensive experiments show that ZeroAE largely surpasses the SOTA methods by 15.93% and 8.70% on average respectively in the label-partially-unseen and label-fully-unseen scenario. Notably, the label-fully-unseen ZeroAE even possesses superior performance to the label-partially-unseen SOTA methods.

Related Papers

• → AEG: Automatic Exploit Generation(2018)209 cited
• → PExy: The Other Side of Exploit Kits(2014)24 cited
• → Performance Comparison of Three Types of Autoencoder Neural Networks(2008)28 cited
• → AEMB: An Automated Exploit Mitigation Bypassing Solution(2021)5 cited
• → A Comparative Evaluation of AutoEncoder-Based Unsupervised Anomaly Detection Methods Applied on Space Payload(2020)