Evaluating the Impact of Large Language Model AI on Acute Pancreatitis Management: A Chatgpt-Based Investigation
Keywords:
Social integration, responsibilities, relationships and mentorship, Metastasis., resistance, Genomics, sequencing, prosigna, transduction., In Vitro Fertilization, Meta-Analysis, uterine cancer, infertility, ovulation induction, alcohol, dementia, elderly., management, ChatGPT, Acute Pancreatitis, Large Language ModelAbstract
Background: Evidence-based management of acute pancreatitis (AP) is important for patient outcomes. The present study evaluated suggestions by artificial intelligence (AI) chatbot system, ChatGPT, for the management of acute pancreatitis, its alignment with clinical guidelines, and assistance in clinical decision-making.
Methods: Six questions on pancreatitis management were curated by experienced RACS-qualified general surgeons and were put forth to ChatGPT. The chatbot was also asked to provide five high-level evidence references to support each of its responses. Each response was analyzed for its accuracy and comprehensiveness with respect to current internationally recognized guidelines and by two Board-Certified General Surgeons for acute pancreatitis management, as well as for its spelling, grammar, and reference quality. A five-point Likert Scale was utilized to analyze ChatGPT’s responses, with scores ranging from 1 (strongly disagree) to 5 (strongly agree). Ten questions were designed to assess accuracy, consistency, informativeness, reliability, and coherence. These were independently rated by three junior doctors and two General Surgeons, with any scoring discrepancies resolved through consensus.
Results: ChatGPT successfully adhered to clinical guidelines when generating recommendations for the management of acute pancreatitis. The depth of information remained general and non-specific but was presented in an academic manner with appropriate grammar, spelling and sentence structure. ChatGPT missed pertinent references, with some being totally fabricated or erroneous.
Conclusion: ChatGPT holds promise for delivering prompt and accessible medical information to non-experts, which may benefit in situations where medical professionals and resources may be scarce or patients are reluctant to seek such services. The inclusion of aberrant or fabricated references is a challenge for researchers and clinicians and breaches academic integrity. Ethically, it is imperative for researchers to exercise prudence when utilizing ChatGPT for research purposes.
References
M S Cappell (2008) Acute pancreatitis: etiology, clinical presentation, diagnosis, and therapy. 92, 889-923. https://doi.org/10.1016/j.mcna.2008.04.013
M Nesvaderani, G D Eslick, M R Cox (2015) Acute pancreatitis: update on management. 202, 420-423. https://doi.org/10.5694/mja14.01333
M A Mederos, H A Reber, M D Girgis (2021) Acute pancreatitis: a review. 325, 382-390. https://doi.org/10.1001/jama.2020.20317
S M Van DIjk, N D Hallensleben, H C van Santvoort, et al. (2017) Acute pancreatitis: recent advances through randomised trials. 66, 2024-2032. https://doi.org/10.1136/gutjnl-2016-313595
I Seth, A Rodwell, R Tso, J Valles, G Bulloch, N Seth (2023) A Conversation with an Open Artificial Intelligence Platform on Osteoarthritis of the Hip and Treatment. 5, 112-120.
Y Wu, A Q Jiang, W Li, et al. (2022) Autoformalization with large language models. 35, 32353-32368.
B D Lund, T Wang (2023) Chatting about ChatGPT: how may AI and GPT impact academia and libraries?.
M Liebrenz, R Schleifer, A Buadze, D Bhugra, A Smith (2023) Generating scholarly content with ChatGPT: ethical challenges for medical publishing. 5, e105-e106. https://doi.org/10.1016/S2589-7500(23)00019-5
G-J Wang, C-F Gao, D Wei, C Wang, S-Q Ding (2009) Acute pancreatitis: etiology and common pathogenesis. 15, 1427. https://doi.org/10.3748/wjg.15.1427
J Gapp, S Chandra (2023) Continuing Education Activity.
A Oláh, L Romics Jr (2014) Enteral nutrition in acute pancreatitis: a review of the current evidence. 20, 16123. https://doi.org/10.3748/wjg.v20.i43.16123
S Tenner, J Baillie, J DeWitt, S S Vege (2013) American College of Gastroenterology guideline: management of acute pancreatitis. 108, 1400-1415. https://doi.org/10.1038/ajg.2013.218
P A Banks, M L Freeman, Gastroenterology PPCotACo (2006) Practice guidelines in acute pancreatitis. 101, 2379-2400. https://doi.org/10.1111/j.1572-0241.2006.00856.x
C Boumitri, E Brown, M Kahaleh (2017) Necrotizing Pancreatitis: Current Management and Therapies. 50, 357-365. https://doi.org/10.5946/ce.2016.152
A Haj-Mirzaian, B N Patel, E K Fishman, A Zaheer (2020) Value of multidisciplinary collaboration in acute and chronic pancreatitis. 45, 1458-1467. https://doi.org/10.1007/s00261-019-02320-9
D W Da Costa, S A Bouwense, N J Schepers, et al. (2015) Same-admission versus interval cholecystectomy for mild gallstone pancreatitis (PONCHO): a multicentre randomised controlled trial. 386, 1261-1268. https://doi.org/10.1016/S0140-6736(15)00274-3
W H Nealon, J Bawduniak, E M Walser (2004) Appropriate timing of cholecystectomy in patients who present with moderate to severe gallstone-associated acute pancreatitis with peripancreatic fluid collections. 239, 741. https://doi.org/10.1097/01.sla.0000128688.97556.94
P Silva-Vaz, A M Abrantes, M Castelo-Branco, A Gouveia, M F Botelho, J G Tralhão (2020) Multifactorial scores and biomarkers of prognosis of acute pancreatitis: applications to research and practice. 21, 338. https://doi.org/10.3390/ijms21010338
Y Liu, Z Wan, D Liao (2023) Efficacy of enteral nutrition for patients with acute pancreatitis: A systematic review and meta-analysis of 17 studies. 1-12. https://doi.org/10.3892/etm.2023.11883
M Sallam (2023) ChatGPT Utility in Health Care Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. 11, 887. https://doi.org/10.3390/healthcare11060887
T-J Chen (2023) ChatGPT and other artificial intelligence applications speed up scientific writing. https://doi.org/10.1097/JCMA.0000000000000900
P G Lankisch, M Apte, P A Banks (2015) Acute pancreatitis. 386, 85-96. https://doi.org/10.1016/s0140-6736(14)60649-8
J Scherer, V P Singh, C S Pitchumoni, D Yadav (2014) Issues in hypertriglyceridemic pancreatitis: an update. 48, 195-203. https://doi.org/10.1097/01.mcg.0000436438.60145.5a
C J Nitsche, N Jamieson, M M Lerch, J V Mayerle (2010) Drug induced pancreatitis. 24, 143-155. https://doi.org/10.1016/j.bpg.2010.02.002
D M Parenti, W Steinberg, P Kang (1996) Infectious causes of acute pancreatitis. 13, 356-371. https://doi.org/10.1097/00006676-199611000-00005
P A Banks, T L Bollen, C Dervenis, H G Gooszen, C D Johnson, M G Sarr, et al. (2013) Classification of acute pancreatitis-2012: revision of the Atlanta classification and definitions by international consensus. 62(1), 102-11. https://doi.org/10.1136/gutjnl-2012-303579
IAP/APA (2013) IAP/APA evidence-based guidelines for the management of acute pancreatitis. e1-15. https://doi.org/10.1016/j.pan.2013.07.063
C E Forsmark, S S Vege, C M Wilcox (2016) Acute Pancreatitis. 375, 1972-1981. https://doi.org/10.1056/NEJMra1505202
E de-Madaria, G Soler-Sala, J Sánchez-Payá, et al. (2011) Influence of fluid therapy on the prognosis of acute pancreatitis: a prospective cohort study. 106, 1843-1850. https://doi.org/10.1038/ajg.2011.236
R Mounzer, C J Langmead, B U Wu, et al. (2012) Comparison of existing clinical scoring systems to predict persistent organ failure in patients with acute pancreatitis. 142, 1476-1482; quiz e1415-1476. https://doi.org/10.1053/j.gastro.2012.03.005
O J Bakker, S van Brunschot, H C van Santvoort, et al. (2014) Early versus on-demand nasoenteric tube feeding in acute pancreatitis. 371, 1983-1993. https://doi.org/10.1056/NEJMoa1404393
R Isenmann, M Rünzi, M Kron, S Kahl, D Kraus, W Jung, et al. (2004) Prophylactic antibiotic treatment in patients with predicted severe acute pancreatitis: a placebo-controlled, double-blind trial. 126(4), 997-1004. https://doi.org/10.1053/j.gastro.2004.03.067
H C van Santvoort, M G Besselink, O J Bakker, et al. (2010) A step-up approach or open necrosectomy for necrotizing pancreatitis. 362, 1491-1502. https://doi.org/10.1056/NEJMoa0908821
S S Vege, M J DiMagno, C E Forsmark, M Martel, A N Barkun (2018) Initial Medical Treatment of Acute Pancreatitis: American Gastroenterological Association Institute Technical Review. 154, 1103-1139. https://doi.org/10.1053/j.gastro.2018.01.031
A Aboulian, T Chan, A Yaghoubian, et al. (2010) Early cholecystectomy safely decreases hospital stay in patients with mild gallstone pancreatitis: a randomized prospective study. 251, 615-619. https://doi.org/10.1097/SLA.0b013e3181c38f1f
M C Van Baal, M G Besselink, O J Bakker, et al. (2012) Timing of cholecystectomy after mild biliary pancreatitis: a systematic review. 255, 860-866. https://doi.org/10.1097/SLA.0b013e3182507646
K Ito, H Ito, E E Whang (2008) Timing of cholecystectomy for biliary pancreatitis: do the data support current guidelines?. 12, 2164-2170. https://doi.org/10.1007/s11605-008-0603-y
D W Da Costa, S A Bouwense, N J Schepers, et al. (2015) Same-admission versus interval cholecystectomy for mild gallstone pancreatitis (PONCHO): a multicentre randomised controlled trial. 386, 1261-1268.
J Mayerle, M Sendler, E Hegyi, G Beyer, M M Lerch, M Sahin-Tóth (2019) Genetics, Cell Biology, and Pathophysiology of Pancreatitis. 156, 1951-1968.e1951. https://doi.org/10.1053/j.gastro.2018.11.081
N J Schepers, O J Bakker, M G Besselink, et al. (2019) Impact of characteristics of organ failure and infected necrosis on mortality in necrotising pancreatitis. 68, 1044-1051. https://doi.org/10.1136/gutjnl-2017-314657
A S Gukovskaya, I Gukovsky, H Algül, A Habtezion (2017) Autophagy, inflammation, and immune dysfunction in the pathogenesis of pancreatitis. 153, 1212-1226.
D C Whitcomb (2004) Mechanisms of disease: Advances in understanding the mechanisms leading to chronic pancreatitis. 1, 46-52.
M S Petrov, B P Loveday, R D Pylypchuk, K McIlroy, A R Phillips, J A Windsor (2009) Systematic review and meta-analysis of enteral nutrition formulations in acute pancreatitis. 96, 1243-1252. https://doi.org/10.1002/bjs.6862
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Authors and Global Journals Private Limited
This work is licensed under a Creative Commons Attribution 4.0 International License.
