Presentation Information
[INTS5-3]Can Local AI Match the Surgeon Pilot Study of Five AI Models in Colorectal Cancer Operative Decision-Making
*Takeshi Yanagita1,2, Shinichiro Tsuru1, Ryoji Umeki1, Sawamura Naoki1, Shuji Kurata1, Futoshi Teranishi2 (1. Department of Surgery, Shonan Fujisawa Tokushukai Hospital, Japan, 2. Department of Surgery, Toyokawa City Hospital)
Background: AI is transforming medicine, including colorectal surgery. Large Language Models—the technology behind ChatGPT—now run locally without internet connection, offering privacy-preserving support. This pilot study evaluated five Local AI models for colorectal cancer operative decision-making in remote islands and rural communities.
Methods: Preoperative data from 89 prospectively collected patients (13 non-operated; 76 operated) were retrospectively analyzed. Variables (BMI, ASA-PS, ECOG PS, nutritional indices, comorbidities, ADL/IADL, tumor location, TNM stage) were submitted to Local AI from OpenAI, Google, Alibaba, Meta, and DeepSeek. Each selected one of four strategies: curative resection with/without anastomosis, reduced resection, palliative stoma/bypass, or non-operative management. Selections were compared to attending surgeons’ decisions as the reference standard.
Results: Four models (OpenAI, Google, Alibaba, Meta) agreed with surgeons’ actual decisions in 71.9% of cases: 75–83% among operated patients, dropping to 8–54% among non-operated patients. DeepSeek showed uniform 46% agreement regardless of operative status. Inter-model agreement was 66–80% within the cluster but only 22–43% between DeepSeek and others.
Discussion: Four of five models showed meaningful procedure-selection agreement but favored curative over reduced or palliative options—underweighting the balance between oncological goals and patient tolerance. Paradoxically, DeepSeek’s more complex cumulative-risk algorithm showed the lowest, most uniform agreement (46%), suggesting algorithmic sophistication alone does not ensure clinical alignment. Discordant cases warrant prospective reassessment.
Conclusions: Four Local AI models can meaningfully support operative procedure selection with high inter-model consistency. The decision of whether to operate—and how aggressively—demands irreplaceable bedside judgment. This pilot supports a privacy-preserving AI system for specialist-level guidance in remote and underserved communities.
Methods: Preoperative data from 89 prospectively collected patients (13 non-operated; 76 operated) were retrospectively analyzed. Variables (BMI, ASA-PS, ECOG PS, nutritional indices, comorbidities, ADL/IADL, tumor location, TNM stage) were submitted to Local AI from OpenAI, Google, Alibaba, Meta, and DeepSeek. Each selected one of four strategies: curative resection with/without anastomosis, reduced resection, palliative stoma/bypass, or non-operative management. Selections were compared to attending surgeons’ decisions as the reference standard.
Results: Four models (OpenAI, Google, Alibaba, Meta) agreed with surgeons’ actual decisions in 71.9% of cases: 75–83% among operated patients, dropping to 8–54% among non-operated patients. DeepSeek showed uniform 46% agreement regardless of operative status. Inter-model agreement was 66–80% within the cluster but only 22–43% between DeepSeek and others.
Discussion: Four of five models showed meaningful procedure-selection agreement but favored curative over reduced or palliative options—underweighting the balance between oncological goals and patient tolerance. Paradoxically, DeepSeek’s more complex cumulative-risk algorithm showed the lowest, most uniform agreement (46%), suggesting algorithmic sophistication alone does not ensure clinical alignment. Discordant cases warrant prospective reassessment.
Conclusions: Four Local AI models can meaningfully support operative procedure selection with high inter-model consistency. The decision of whether to operate—and how aggressively—demands irreplaceable bedside judgment. This pilot supports a privacy-preserving AI system for specialist-level guidance in remote and underserved communities.