Comment on: Concordance of ChatGPT artificial intelligence decision-making in colorectal cancer multidisciplinary meetings: retrospective study

Corresponding author: Hinpetch Daungsupawong (email: hinpetchdaung@gmail.com)

Private Academic Consultant

Phonhong

Laos

BJS Open, https://doi.org/10.1093/bjsopen/zraf040, published 07 May 2025

In the article by Chatziisaak et al.1, the methodological and statistical critiques of this work highlight various limitations that must be recognized. The adoption of a retrospective, single-center design may result in selection bias and limit generalizability to a larger population. Furthermore, there is inadequate detail regarding the data given into ChatGPT, which is an essential component that may influence the accuracy of recommendations. For example, if sensitive clinical data like imaging results or the patient's socioeconomic status are not included, the artificial intelligence (AI) model's recommendations may be inaccurate or clinically irrelevant.

While providing results in terms of percentages of concordance (full, partial, and discordant concordance) provides a quick overview, it does not include an in-depth investigation of statistical accuracy. Using statistics like Cohen's kappa or weighted kappa may result in a higher level of agreement than random matching. Furthermore, no model confidence levels are presented, nor is it determined whether the inconsistency is caused by cancer type, stage, or individual patient characteristics.

A bigger concern is: what role could AI, such as ChatGPT, play in clinical decision-making in circumstances when medical resources are limited? Or in countries with weaker multidisciplinary team (MDT) systems? This question sparks debates about crucial concerns such as faith in automation, healthcare equity, and AI's potential to lessen the strain on staff, particularly in developing health systems.

Future methods should concentrate on building multicenter prospective studies and creating AI models that can better interpret individual data, such as comorbidities, social backgrounds, and patient demands in various circumstances. Platforms should also be developed to enable MDTs and AI to collaborate in ways that supplement, rather than replace, the requirement for informed, humanized decision-making.