Object-Driven Text-To-Image Synthesis via Adversarial Training

Citations Over TimeTop 1% of 2019 papers

Abstract

In this paper, we propose Object-driven Attentive Generative Adversarial Newtorks (Obj-GANs) that allow attention-driven, multi-stage refinement for synthesizing complex images from text descriptions. With a novel object-driven attentive generative network, the Obj-GAN can synthesize salient objects by paying attention to their most relevant words in the text descriptions and their pre-generated class label. In addition, a novel object-wise discriminator based on the Fast R-CNN model is proposed to provide rich object-wise discrimination signals on whether the synthesized object matches the text description and the pre-generated class label. The proposed Obj-GAN significantly outperforms the previous state of the art in various metrics on the large-scale MS-COCO benchmark, increasing the inception score by 27% and decreasing the FID score by 11%. A thorough comparison between the classic grid attention and the new object-driven attention is provided through analyzing their mechanisms and visualizing their attention layers, showing insights of how the proposed model generates complex scenes in high quality.

Related Papers

• → Differentially private generative decomposed adversarial network for vertically partitioned data sharing(2022)9 cited
• → Student Action Recognition Based on Deep Convolutional Generative Adversarial Network(2020)13 cited
• → GAN Using Capsule Network for Discriminator and Generator(2021)2 cited
• BRIDGING ADVERSARIAL SAMPLES AND ADVERSARIAL NETWORKS(2019)
• → Improving Super-Resolution GAN Performance through Discriminator using U-Net Structure and Auxiliary Classifier(2021)