Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation

Citations Over TimeTop 1% of 2016 papers

Abstract

Despite the recent advances in automatically describing image contents, their applications have been mostly limited to image caption datasets containing natural images (e.g., Flickr 30k, MSCOCO). In this paper, we present a deep learning model to efficiently detect a disease from an image and annotate its contexts (e.g., location, severity and the affected organs). We employ a publicly available radiology dataset of chest x-rays and their reports, and use its image annotations to mine disease names to train convolutional neural networks (CNNs). In doing so, we adopt various regularization techniques to circumvent the large normalvs-diseased cases bias. Recurrent neural networks (RNNs) are then trained to describe the contexts of a detected disease, based on the deep CNN features. Moreover, we introduce a novel approach to use the weights of the already trained pair of CNN/RNN on the domain-specific image/text dataset, to infer the joint image/text contexts for composite image labeling. Significantly improved image annotation results are demonstrated using the recurrent neural cascade model by taking the joint image/text contexts into account.

Related Papers

• → Overview of deep learning in medical imaging(2017)1,052 cited
• → Deep learning ensemble 2D CNN approach towards the detection of lung cancer(2023)173 cited
• → Research and Application of Ancient Chinese Pattern Restoration Based on Deep Convolutional Neural Network(2021)4 cited
• → Leaf Features Extraction for Plant Classification using CNN(2021)7 cited
• → Deep Convolution Neural Network for RBC Images(2022)2 cited