AI Foundation Models in radiology
Main research areas
Recent advances in the field of artificial intelligence - particularly through the development of deep neural networks, so-called foundation models and large language models (LLMs) - are opening up new application possibilities in healthcare. The introduction of transformer-based architectures and attention mechanisms has significantly improved language understanding and generative capabilities and forms the basis for versatile AI applications in a wide range of areas.
Especially in radiology - a specialty inherently based on the translation of visual data into text-based findings - these technologies offer great potential. The potential applicability of LLMs in radiology workflows has already been demonstrated for various sub-areas such as findings optimization, clinical decision support and medical education.
The further development towards multimodal foundation models (MFMs) further expands the range of applications, as these models can process multiple input modalities - text, image and audio - within a single model. This opens up new possibilities for AI-supported precision diagnostics and improved interdisciplinary communication.
The aim of our research group is to investigate how AI tools - especially multimodal LLMs - can contribute to the optimization of radiological workflows. A central focus is on evaluating their integration into the clinical infrastructure and assessing their potential for use in daily radiology practice.
Dr. Roman Johannes Gertz
Funded by the Cologne Clinician Scientist Program (CCSP) / Faculty of Medicine / University of Cologne
Funded by the German Research Foundation (DFG), project number 413543196
Supported by the Else-Kröner-Fresenius Foundation
Gertz RJ, Bunck AC, Lennartz S, Dratsch T, Iuga AI, Maintz D, Kottlors J. GPT-4 for Automated Determination of Radiological Study and Protocol based on Radiology Request Forms: A Feasibility Study. Radiology. 2023 Jun;307(5):e230877. doi:10.1148/radiol.230877. PMID: 37310247.
Gertz RJ, Dratsch T, Bunck AC, Lennartz S, Iuga AI, Hellmich MG, Persigehl T, Pennig L, Gietzen CH, Fervers P, Maintz D, Hahnfeldt R, Kottlors J. Potential of GPT-4 for Detecting Errors in Radiology Reports: Implications for Reporting Accuracy. Radiology. 2024 Apr;311(1):e232714. doi: 10.1148/radiol.232714. PMID: 38625012.
Laukamp KR, Terzis RA, Werner JM, Galldiks N, Lennartz S, Maintz D, Reimer R, Fervers P, Gertz RJ, Persigehl T, Rubbert C, Lehnen NC, Deuschl C, Schlamann M, Schönfeld MH, Kottlors J. Monitoring Patients with Glioblastoma by Using a Large Language Model: Accurate Summarization of Radiology Reports with GPT-4. Radiology. 2024 Jul;312(1):e232640. doi: 10.1148/radiol.232640. PMID: 39041936.
Kaya K, Gietzen C, Hahnfeldt R, Zoubi M, Emrich T, Halfmann MC, Sieren MM, Elser Y, Krumm P, Brendel JM, Nikolaou K, Haag N, Borggrefe J, Krüchten RV, Müller-Peltzer K, Ehrengut C, Denecke T, Hagendorff A, Goertz L, Gertz RJ, Bunck AC, Maintz D, Persigehl T, Lennartz S, Luetkens JA, Jaiswal A, Iuga AI, Pennig L, Kottlors J. Generative Pre-trained Transformer 4 analysis of cardiovascular magnetic resonance reports in suspected myocarditis: A multicenter study. J Cardiovasc Magn Reson. 2024 Winter;26(2):101068. doi: 10.1016/j.jocmr.2024.101068. Epub 2024 Jul 28. PMID: 39079602; PMCID: PMC11414660.
Fervers P, Hahnfeldt R, Kottlors J, Wagner A, Maintz D, Pinto Dos Santos D, Lennartz S, Persigehl T. ChatGPT yields low accuracy in determining LI-RADS scores based on free-text and structured radiology reports in German language. Front Radiol. 2024 Jul 5;4:1390774. doi: 10.3389/fradi.2024.1390774. PMID: 39036542; PMCID: PMC11257913.
Kottlors J, Bratke G, Rauen P, Kabbasch C, Persigehl T, Schlamann M, Lennartz S. Feasibility of Differential Diagnosis Based on Imaging Patterns Using a Large Language Model. Radiology. 2023 Jul;308(1):e231167. doi: 10.1148/radiol.231167. PMID: 37404149.
Kottlors J, Hahnfeldt R, Görtz L, Iuga AI, Fervers P, Bremm J, Zopfs D, Laukamp KR, Onur OA, Lennartz S, Schönfeld M, Maintz D, Kabbasch C, Persigehl T, Schlamann M. Large Language Models-Supported Thrombectomy Decision-Making in Acute Ischemic Stroke Based on Radiology Reports: Feasibility Qualitative Study. J Med Internet Res. 2025 Feb 13;27:e48328. doi: 10.2196/48328. PMID: 39946168; PMCID: PMC11888093.