Former Fellow Chris Evans (Class of ’22), along with his classmate and former PD, Carlton Moore published this study along with an additional one in the Annals of Emergency Medicine
A Natural Language Processing and Machine Learning Approach to Identification of Incidental Radiology Findings in Trauma Patients Discharged from the Emergency Department
Christopher S. Evans, MD, MPH, Hugh D. Dorris, MD, Michael T. Kane, MD, Jane H. Brice, MD, MPH, Benjamin Gray, BS, Carlton Moore, MD, MS
Published: October 31, 2022, DOI: https://doi.org/10.1016/j.annemergmed.2022.08.450
Study objective
Patients undergoing diagnostic imaging studies in the emergency department (ED) commonly have incidental findings, which may represent unrecognized serious medical conditions, including cancer. Recognition of incidental findings frequently relies on manual review of textual radiology reports and can be overlooked in a busy clinical environment. Our study aimed to develop and validate a supervised machine learning model using natural language processing to automate the recognition of incidental findings in radiology reports of patients discharged from the ED.
Methods
We performed a retrospective analysis of computed tomography (CT) reports from trauma patients discharged home across an integrated health system in 2019. Two independent annotators manually labeled CT reports for the presence of an incidental finding as a reference standard. We used regular expressions to derive and validate a random forest model using open-source and machine learning software. Final model performance was assessed across different ED types.
Results
The study CT reports were divided into derivation (690 reports) and validation (282 reports) sets, with a prevalence of incidental findings of 22.3%, and 22.7%, respectively. The random forest model had an area under the curve of 0.88 (95% confidence interval [CI], 0.84 to 0.92) on the derivation set and 0.92 (95% CI, 0.88 to 0.96) on the validation set. The final model was found to have a sensitivity of 92.2%, a specificity of 79.4%, and a negative predictive value of 97.2%. Similarly, strong model performance was found when stratified to a dedicated trauma center, high-volume, and low-volume community EDs.
Conclusion
Machine learning and natural language processing can classify incidental findings in CT reports of ED patients with high sensitivity and high negative predictive value across a broad range of ED settings. These findings suggest the utility of natural language processing in automating the review of free-text reports to identify incidental findings and may facilitate interventions to improve timely follow-up.