Finished · MSc

Neural Models for Generating Clinically Accurate Chest X-Ray Reports

Authored by André Leite

Supervised by Arlindo L. Oliveira, Bruno Martins

View the thesis →

Image captioning models have been increasing their performance comprehensively, having shown that artificial intelligence can achieve successful results in computer vision tasks. However, there are still some tasks within the range of image captioning that need more focus, including the automatic clinical report generation. The automatic generation of radiology reports based on radiology images has gathered an increasing amount of focus in the last few years. This is supported by the repetitive and exhaustive work that these clinical reports demand. Artificial neural networks that address this task have been changing over the years, starting as convolutional neural networks, changing over to transformer-based models. However, these existing methodologies focus more on one of two important aspects, that being the fluency and human-readability capacity of the generated text, over the clinical efficiency of the model. Consequently, in this dissertation we propose a model capable of achieving competitive results regarding the human readability of the reports, as well as improving clinical efficiency. We propose to adapt the MedCLIP model to have an image-text encoder capable of concatenating both image and text. We further propose that this model works with the assistance of an Information Retrieval mechanism, to retrieve reports resulting on similarity evaluation done on an input x-ray, obtaining the closest reports. On the MIMIC-CXR dataset, our model has improved on both natural language processing metrics and clinical efficiency, over well-established models. Finally, we further show that our model can lead to more human-readable reports, while keeping clinical actuality, over most state-of-the-art models.

← All dissertations