講演情報
[E4-3]CRCFound: A Colorectal Cancer CT Image Foundation Model Based on Self-Supervised Learning
Du Cai1, Jing Yang2, Junwei Liu3, Zhenfeng Zhuang4, Yibing Zhao5, Feng-Ao Wang6, Chenghang Li7, Chuling Hu1, Baowen Gai1, Yiping Chen8, Yixue Li9, Liansheng Wang4, Feng Gao1, Xiaojian Wu1 (1.Department of General Surgery (Colorectal Surgery), The Sixth Affiliated Hospital, Sun Yat-sen University, 2.National Institute for Data Science in Health and Medicine, Xiamen University, 3.Guangzhou National Laboratory, 4.Department of Computer Science at the School of Informatics, Xiamen University, 5.Department of Colorectal Surgery, Ningbo Medical Center Lihuili Hospital (Affiliated Lihuili Hospital of Ningbo University), 6.Key Laboratory of Systems Health Science of Zhejiang Province, School of Life Science, Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, 7.Artificial Intelligence Thrust, The Hong Kong University of Science and Technology, 8.School of Geospatial Engineering and Science, Sun Yat-Sen University, 9.Shanghai Institute of Nutrition and Health, Chinese Academy of Sciences)
Background: Existing deep learning models perform poorly in the preoperative diagnosis for colorectal cancer (CRC) and lack generalizability due to insufficient annotated data. To address these issues, we propose CRCFound, a self-supervised learning-based CT image foundational model for CRC.
Methods: A total of 6,332 CRC patients with preoperative CT images were collected from 2008 to 2019 at the Sixth Affiliated Hospital of Sun Yat-sen University. Among them, 5,137 unlabeled CT images were used for pretraining, and 1,195 images were reserved for fine-tuning and validation. Comprehensive benchmark tests were conducted on six diagnostic and two prognosis tasks in comparison with other models. Model interpretation was also explored to gain a more profound understanding of the model’s behavior and decision-making process.
Results: For preoperative diagnosis of TNM stage, the average AUC of CRCFound for T, N, M, and overall TNM stage reached 0.889, 0.847, 0.830, and 0.774. In the MSI and CMS diagnosis tasks, it also showed good predictive performance (AUC 0.952 and 0.810, respectively). For prognosis prediction, CRCFound_CT was an independent risk factor for disease-free survival and overall survival . Feature visualization using the t-SNE algorithm demonstrated that CRCFound could effectively differentiate samples in multiple tasks. Visualization of the attention map indicates that CRCFound can focus more effectively on the critical areas of the tumor than traditional supervised learning models, thus providing more accurate predictions.
Conclusion: CRCFound addresses the challenge of insufficient annotated data and performs well in a wide range of downstream tasks, offering a promising solution for accurate diagnosis and personalized treatment of CRC patients.
Methods: A total of 6,332 CRC patients with preoperative CT images were collected from 2008 to 2019 at the Sixth Affiliated Hospital of Sun Yat-sen University. Among them, 5,137 unlabeled CT images were used for pretraining, and 1,195 images were reserved for fine-tuning and validation. Comprehensive benchmark tests were conducted on six diagnostic and two prognosis tasks in comparison with other models. Model interpretation was also explored to gain a more profound understanding of the model’s behavior and decision-making process.
Results: For preoperative diagnosis of TNM stage, the average AUC of CRCFound for T, N, M, and overall TNM stage reached 0.889, 0.847, 0.830, and 0.774. In the MSI and CMS diagnosis tasks, it also showed good predictive performance (AUC 0.952 and 0.810, respectively). For prognosis prediction, CRCFound_CT was an independent risk factor for disease-free survival and overall survival . Feature visualization using the t-SNE algorithm demonstrated that CRCFound could effectively differentiate samples in multiple tasks. Visualization of the attention map indicates that CRCFound can focus more effectively on the critical areas of the tumor than traditional supervised learning models, thus providing more accurate predictions.
Conclusion: CRCFound addresses the challenge of insufficient annotated data and performs well in a wide range of downstream tasks, offering a promising solution for accurate diagnosis and personalized treatment of CRC patients.