Automatic Generation of Medical Imaging Diagnostic Report with Hierarchical Recurrent Neural Network
Citations Over TimeTop 10% of 2019 papers
Abstract
Medical images are widely used in the medical domain for the diagnosis and treatment of diseases. Reading a medical image and summarizing its insights is a routine, yet nonetheless time-consuming task, which often represents a bottleneck in the clinical diagnosis process. Automatic report generation can relieve the issues. However, generating medical reports presents two major challenges: (i) it is hard to accurately detect all the abnormalities simultaneously, especially the rare diseases; (ii) a medical image report consists of many paragraphs and sentences, which are longer than natural image captions. We present a new framework to accurately detect the abnormalities and automatically generate medical reports. The report generation model is based on hierarchical recurrent neural network (HRNN). We introduce a topic matching mechanism to HRNN, so as to make generated reports more accurate and diverse. The soft attention mechanism is also introduced to HRNN model. Experimental results on two image-paragraph pair datasets show that our framework outperforms all the state-of-art methods.
Related Papers
- → Simulation-Based Procedure for Bottleneck Identification(2007)17 cited
- Computer-assisted language learning (CALL): Using paragraph punch software in developing EFL paragraph writing skills(2021)
- → An Intelligent Partner for Organizing a Paragraph(2009)
- Methods of Paragraph Writing in CET 4(2003)
- → To Teach or Not to Teach the Five-Paragraph Essay?(2021)