Several members of UNC Health Enterprises participated in the study titled “Utilizing Artificial Intelligence and Chat Generative Pretrained Transformer to Answer Questions About Clinical Scenarios in
Neuroanesthesiology.”
The study was published in the Journal of Neurosurgical Anesthesiology, Oct 2024 (Vol 36. Issue 4). It investigates the application of advanced AI technologies to improve decision-making and efficiency in neuroanesthesiology clinical scenarios. The lead investigator is Dr. Samuel N. Blacker, Associate Professor of Anesthesiology and former Physician Innovator at UNC Health Enterprises. Among the co-authors, were three UNC Health Enterprises teammates: Carol Lewis, Michael Zimmer, and Brad Wilson.
The study tested the ability of ChatGPT, an artificial intelligence chatbot, to answer questions relevant to scenarios covered in three clinical guidelines published by the Society for Neuroscience in Anesthesiology and Critical Care (SNACC).
Four neuroanesthesiologists independently assessed whether ChatGPT could apply 52 high-quality recommendations (HQRs) included in the three SNACC guidelines. HQRs were deemed present in the ChatGPT responses if noted by at least three of the four reviewers. Reviewers also identified incorrect references, potentially harmful recommendations, and whether ChatGPT cited the SNACC guidelines.
The study concluded that ChatGPT should not currently be considered a reliable source of information for clinical decision-making. Future iterations of ChatGPT may refine algorithms to improve its reliability as a source of clinical information.
The full article can be found at online here.