Foundation Models for Medical Image Segmentation: Technical Progress, Clinical Translation, and Governance Challenges
DOI:
https://doi.org/10.54691/85dt0r63Keywords:
Medical Image Segmentation; Foundation Models; Segment Anything Model; Deep Learning; Multimodal Imaging; Clinical Translation; Artificial Intelligence.Abstract
Medical image segmentation is moving from task-specific convolutional models toward foundation models that can be adapted across organs, modalities, and clinical tasks with fewer manual labels. This transition has been accelerated by self-supervised pretraining, vision-language learning, and promptable segmentation frameworks such as the Segment Anything Model and its medical derivatives. However, the clinical value of these systems cannot be inferred from technical novelty alone. Medical images differ from natural images in dimensionality, intensity statistics, acquisition protocols, disease prevalence, and safety requirements, and recent evaluations show that naive zero-shot transfer remains inconsistent across modalities and lesion types. This narrative review synthesizes literature published up to May 22, 2026, on foundation models for medical image segmentation, with emphasis on technical evolution, application scenarios, validation strategies, and governance needs. Current evidence suggests that foundation models are most promising when they are deployed as interactive, auditable components within human-in-the-loop workflows, where they can reduce annotation burden, support rapid draft segmentation, and improve consistency across large imaging studies. Their translation into routine practice requires external validation, uncertainty-aware quality control, prospective workflow evaluation, bias assessment, and transparent reporting under medical AI guidelines. Future work should prioritize patient-level multimodal modeling, 3D and longitudinal segmentation, federated evaluation, and clinically meaningful endpoints rather than isolated benchmark gains.
Downloads
References
[1] Litjens, G., Kooi, T., Ehteshami Bejnordi, B., Setio, A. A. A., Ciompi, F., Ghafoorian, M., et al. (2017). A survey on deep learning in medical image analysis. Medical Image Analysis, 42, 60–88. https:// doi. org/ 10.1016/j.media.2017.07.005. DOI: https://doi.org/10.1016/j.media.2017.07.005
[2] Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science, 234–241. https://doi.org/10.1007/978-3-319-24574-4_28. DOI: https://doi.org/10.1007/978-3-319-24574-4_28
[3] Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021). nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods, 18(2), 203–211. https://doi.org/10.1038/s41592-020-01008-z. DOI: https://doi.org/10.1038/s41592-020-01008-z
[4] Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B. A., et al. (2022). The Medical Segmentation Decathlon. Nature Communications, 13(1), 4128. https://doi.org/ 10. 1038/ s41467-022-30695-9. DOI: https://doi.org/10.1038/s41467-022-30695-9
[5] Zhou, Z., Sodha, V., Pang, J., Gotway, M. B., & Liang, J. (2021). Models Genesis. Medical Image Analysis, 67, 101840. https://doi.org/10.1016/j.media.2020.101840. DOI: https://doi.org/10.1016/j.media.2020.101840
[6] Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., et al. (2023). Foundation models for generalist medical artificial intelligence. Nature, 616(7956), 259–265. https: // doi.org/10.1038/s41586-023-05881-4. DOI: https://doi.org/10.1038/s41586-023-05881-4
[7] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., et al. (2023). Segment Anything [Preprint]. arXiv. https://arxiv.org/abs/2304.02643 DOI: https://doi.org/10.1109/ICCV51070.2023.00371
[8] Ma, J., He, Y., Li, F., Han, L., You, C., Wang, B., et al. (2024). Segment anything in medical images. Nature Communications, 15(1), 654. https://doi.org/10.1038/s41467-024-44824-z. DOI: https://doi.org/10.1038/s41467-024-44824-z
[9] Huang, Y., Yang, X., Liu, L., Zhou, H., Chang, A., Zhou, X., et al. (2024). Segment anything model for medical images? Medical Image Analysis, 92, 103061. https://doi.org/ 10. 1016/ j. media. 2023. 103061. DOI: https://doi.org/10.1016/j.media.2023.103061
[10] Mazurowski, M. A., Dong, H., Gu, H., Yang, J., Konz, N., Zhang, Y., et al. (2023). Segment anything model for medical image analysis: An experimental study. Medical Image Analysis, 89, 102918. https: // doi. org/10.1016/j.media.2023.102918. DOI: https://doi.org/10.1016/j.media.2023.102918
[11] Zhang, Y., Shen, Z., & Jiao, R. (2024). Segment anything model for medical image segmentation: Current applications and future directions. Computers in Biology and Medicine, 171, 108238. https://doi.org/10.1016/j.compbiomed.2024.108238. DOI: https://doi.org/10.1016/j.compbiomed.2024.108238
[12] Wasserthal, J., Breit, H. C., Meyer, M. T., Pradella, M., Hinck, D., Sauter, A. W., et al. (2023). TotalSegmentator: Robust Segmentation of 104 Anatomic Structures in CT Images. Radiology: Artificial Intelligence, 5(5), e230024. https://doi.org/10.1148/ryai.230024. DOI: https://doi.org/10.1148/ryai.230024
[13] Gu, Y., Wu, Q., Tang, H., Mai, X., Shu, H., Li, B., et al. (2024). LeSAM: Adapt Segment Anything Model for Medical Lesion Segmentation. IEEE Journal of Biomedical and Health Informatics, 28(10), 6031–6041. https://doi.org/10.1109/JBHI.2024.3406871. DOI: https://doi.org/10.1109/JBHI.2024.3406871
[14] Shen, Y., Li, J., Shao, X., Romillo, B. I., Jindal, A., Dreizin, D., et al. (2024). FastSAM3D: An Efficient Segment Anything Model for 3D Volumetric Medical Images. Lecture Notes in Computer Science, 542–552. https://doi.org/10.1007/978-3-031-72390-2_51. DOI: https://doi.org/10.1007/978-3-031-72390-2_51
[15] Li, H., Liu, H., Hu, D., Wang, J., & Oguz, I. (2024). PROMISE: Prompt-Driven 3D Medical Image Segmentation Using Pretrained Image Foundation Models. In 2024 IEEE International Symposium on Biomedical Imaging (ISBI) (pp. 1–5). https://doi.org/10.1109/ISBI56570.2024.10635207. DOI: https://doi.org/10.1109/ISBI56570.2024.10635207
[16] Archit, A., Freckmann, L., Nair, S., Khalid, N., Hilt, P., Rajashekar, V., et al. (2025). Segment Anything for Microscopy. Nature Methods, 22(3), 579–591. https://doi.org/10.1038/s41592-024-02580-4. DOI: https://doi.org/10.1038/s41592-024-02580-4
[17] Wu, X. T., Chen, X. D., Wu, W., Ma, W., & Song, H. (2026). Accelerating vision foundation model for efficient medical image segmentation. Medical Physics, 53(1). https://doi.org/10.1002/mp.70193. DOI: https://doi.org/10.1002/mp.70193
[18] Zhang, R., Huang, M., & Li, R. (2026). MedZeroSeg: Zero-shot medical image segmentation via vision foundation models. PLOS ONE, 21(3), e0344978. https://doi.org/10.1371/journal.pone.0344978. DOI: https://doi.org/10.1371/journal.pone.0344978
[19] Tiu, E., Talius, E., Patel, P., Langlotz, C. P., Ng, A. Y., Rajpurkar, P., et al. (2022). Expert-level detection of pathologies from unannotated chest X-ray images via self-supervised learning. Nature Biomedical Engineering, 6(12), 1399–1406. https://doi.org/10.1038/s41551-022-00936-9. DOI: https://doi.org/10.1038/s41551-022-00936-9
[20] Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., et al. (2024). A visual-language foundation model for computational pathology. Nature Medicine, 30(3), 863–874. https:// doi. org/ 10. 1038/s41591-024-02856-4. DOI: https://doi.org/10.1038/s41591-024-02856-4
[21] Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., et al. (2024). Towards a general-purpose foundation model for computational pathology. Nature Medicine, 30(3), 850–862. https://doi.org/10.1038/s41591-024-02857-3. DOI: https://doi.org/10.1038/s41591-024-02857-3
[22] Zhang, K., Zhou, R., Adhikarla, E., Yan, Z., Liu, Y., Yu, J., et al. (2024). A generalist vision-language foundation model for diverse biomedical tasks. Nature Medicine, 30(11), 3129–3141. https:// doi.org/10.1038/s41591-024-03185-2. DOI: https://doi.org/10.1038/s41591-024-03185-2
[23] Christensen, M., Vukadinovic, M., Yuan, N., & Ouyang, D. (2024). Vision-language foundation model for echocardiogram interpretation. Nature Medicine, 30(5), 1481–1488. https://doi.org/ 10. 1038/ s41591-024-02959-y. DOI: https://doi.org/10.1038/s41591-024-02959-y
[24] Xiang, J., Wang, X., Zhang, X., Xi, Y., Eweje, F., Chen, Y., et al. (2025). A vision-language foundation model for precision oncology. Nature, 638(8051), 769–778. https://doi.org/10.1038/s41586-024-08378-w. DOI: https://doi.org/10.1038/s41586-024-08378-w
[25] Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., Oermann, E. K., et al. (2018). Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. PLOS Medicine, 15(11), e1002683. https://doi.org/ 10.1371/ journal. pmed. 1002683. DOI: https://doi.org/10.1371/journal.pmed.1002683
[26] Larrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E. (2020). Gender imbalance in medical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences, 117(23), 12592–12594. https://doi.org/ 10.1073/ pnas. 1919 012117. DOI: https://doi.org/10.1073/pnas.1919012117
[27] Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021). Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature Medicine, 27(12), 2176–2182. https://doi.org/ 10. 1038/ s4 1591-021-01595-0. DOI: https://doi.org/10.1038/s41591-021-01595-0
[28] Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. BMC Medicine, 17(1), 195. https://doi.org/ 10. 1186/ s12916-019-1426-2. DOI: https://doi.org/10.1186/s12916-019-1426-2
[29] Rajpurkar, P., Chen, E., Banerjee, O., & Topol, E. J. (2022). AI in health and medicine. Nature Medicine, 28(1), 31–38. https://doi.org/10.1038/s41591-021-01614-0. DOI: https://doi.org/10.1038/s41591-021-01614-0
[30] Mongan, J., Moy, L., & Kahn, C. E. Jr. (2020). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A Guide for Authors and Reviewers. Radiology: Artificial Intelligence, 2(2), e200029. https://doi.org/10.1148/ryai.2020200029. DOI: https://doi.org/10.1148/ryai.2020200029
[31] Tejani, A. S., Klontzas, M. E., Gatti, A. A., Mongan, J. T., Moy, L., Park, S. H., et al. (2024). Checklist for Artificial Intelligence in Medical Imaging (CLAIM): 2024 Update. Radiology: Artificial Intelligence, 6(4), e240300. https://doi.org/10.1148/ryai.240300. DOI: https://doi.org/10.1148/ryai.240300
[32] Norgeot, B., Quer, G., Beaulieu-Jones, B. K., Torkamani, A., Dias, R., Gianfrancesco, M., et al. (2020). Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nature Medicine, 26(9), 1320–1324. https://doi.org/10.1038/s41591-020-1041-y. DOI: https://doi.org/10.1038/s41591-020-1041-y
[33] Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., & Denniston, A. K. (2020). Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension. BMJ, 370, m3164. https://doi.org/10.1136/bmj.m3164. DOI: https://doi.org/10.1136/bmj.m3164
[34] Cruz Rivera, S., Liu, X., Chan, A. W., Denniston, A. K., & Calvert, M. J. (2020). Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI Extension. BMJ, 370, m3210. https://doi.org/10.1136/bmj.m3210. DOI: https://doi.org/10.1136/bmj.m3210
[35] Vasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., et al. (2022). Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine, 28(5), 924–933. https://doi.org/10.1038/s41591-022-01772-9. DOI: https://doi.org/10.1038/s41591-022-01772-9
[36] Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., et al. (2024). TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 385, e078378. https://doi.org/10.1136/bmj-2023-078378. DOI: https://doi.org/10.1136/bmj-2023-078378
[37] Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., et al. (2025). FUTURE-AI: international consensus guideline for trustworthy and deployable artificial intelligence in healthcare. BMJ, 388, e081554. https://doi.org/10.1136/bmj-2024-081554 DOI: https://doi.org/10.1136/bmj-2024-081554
[38] Xu, Y., Peng, Y., Zhang, C., Jiang, K., She, X., Feng, L., et al. (2026). Application of transformer models in medical image segmentation: a narrative review. Quantitative Imaging in Medicine and Surgery, 16(5), 421. https://doi.org/10.21037/qims-2025-aw-2381. DOI: https://doi.org/10.21037/qims-2025-aw-2381
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Scientific Journal of Technology
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
