A Review of Automated Report Generation Technologies for Ophthalmic Medical Imaging

Yuanshuo Zheng; Pan Su

doi:10.54691/0xcw7739

Authors

Yuanshuo Zheng
Pan Su

DOI:

https://doi.org/10.54691/0xcw7739

Keywords:

Ophthalmic Medical Imaging; Deep Learning; Automated Report Generation.

Abstract

The integration of medical and technology has significantly improved diagnostic accuracy, but it has also led to certain challenges. Hospitals and clinics generate, store, and examine large volumes of medical imaging data daily, placing increasing pressure on radiologists. As a result, there is a growing need for technology-assisted report generation. Automated medical report generation technology can reduce physician workload, minimize diagnostic errors, and expedite the clinical diagnostic process. Different types of medical images are generated based on various imaging principles, each with its own advantages and limitations in aiding diagnosis, thus requiring distinct approaches for processing. This paper explores the application prospects of deep learning in the generation of ophthalmic medical reports.

Downloads

Download data is not yet available.

References

[1] Baoyu Jing, Pengtao Xie, and Eric Xing. On the Automatic Generation of Medical Imaging Reports.In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume1: Long Papers), 2018, pages 2577–2586.

[2] Yuan Li, Xiaodan Liang, Zhiting Hu, and Eric P Xing. Hybrid Retrieval-Generation Reinforced Agent for Medical Image Report Generation. In Advances in neural information processing systems, 2018, pages 1530–1540.

[3] Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chihying Deng, Roger G Mark, and Steven Horng. MIMIC-CXR: A large publicly available database of labeled chest radiographs. 2019, arXiv preprint arXiv:1901.

[4] Guanxiong Liu, Tzu-Ming Harry Hsu, Matthew McDermott, Willie Boag, Wei-Hung Weng, Peter Szolovits, and Marzyeh Ghassemi. Clinically Accurate Chest X-Ray Report Generation. In Machine Learning for Healthcare Conference, 2019, pages 249–269.

[5] Baoyu Jing, Zeya Wang, and Eric Xing. Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-ray Reports. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pages 6570–6580.

[6] Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and Tell: A Neural Image Caption Generator. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pages 3156–3164.

[7] Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pages 6077–6086.

[8] Marcella Cornia, Matteo Stefanini, Lorenzo Baraldi, and Rita Cucchiara. M2: Meshed-Memory Transformer for Image Captioning. 2019, arXiv preprint arXiv:1912.08226.

[9] CHEN Zhihong, et al. Cross-modal memory networks for radiology report generation. 2022, arXiv preprint arXiv:2204.13258.

[10] WANG Xiaosong, et al. Tienet: Text-image embedding network for common thorax disease classification and reporting in chest x-rays. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2018, p. 9049-9058.

[11] LU Jiasen, et al. Knowing when to look: Adaptive attention via a visual sentinel for image captioning. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017, p. 375-383.

[12] ZENG Xianhua, et al. Generating diagnostic report for medical image by high-middle-level visual information incorporation on double deep learning models. Computer methods and programs in biomedicine, 2020, 197: 105700.

[13] CHEN Zhihong, et al. Generating radiology reports via memory-driven transformer. 2020, arXiv preprint arXiv:2010.16056.

[14] LI Mingjie, et al. Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023, p. 3334-3343.

[15] LI Yuan, et al. Hybrid retrieval-generation reinforced agent for medical image report generation. Advances in neural information processing systems, 2018, 31.

[16] WANG Zhanyu, et al. A self-boosting framework for automated radiographic report generation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021, p. 2433-2442.

[17] VINYALS, Oriol, et al. Show and tell: A neural image caption generator. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2015, p. 3156-3164.

[18] LIN Tsung-Yu, ROYCHOWDHURY Aruni, MAJI Subhransu. Bilinear CNN models for fine-grained visual recognition. In: Proceedings of the IEEE international conference on computer vision. 2015, p. 1449-1457.

[19] WANG Zhanyu, et al. A medical semantic-assisted transformer for radiographic report generation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022, p. 655-664.