Development of a refractory septic shock prediction score by interpretable machine learning

Sejin Heo, Daun Jeong, Minyoung Choi, Inkyu Kim, Ye Rim Lee, Byuk Sung Ko, Seung Mok Ryoo, Eunah Han, Hyunglan Chang, Chang June Yune, Hui Jai Lee, Gil Joon Suh, Sung-Hyuk Choi, Sung Phil Chung, Tae Ho Lim, Won Young Kim, Kyuseok Kim, Sung Yeon Hwang, Jong Eun Park, Gun Tak Lee, Tae Gun Shin

Resumen


Purpose: The aim of this study was to develop an interpretable machine learning-based scoring system to predict refractory septic shock utilizing clinical parameters available during initial resuscitation.

Methods: This was a retrospective, multicenter study including suspected infection patients from the Marketplace for Medical Information in Intensive Care (MIMIC)-IV database for derivation and internal validation, an emergency department (ED) septic shock validation cohort, and an intensive care unit (ICU) sepsis validation cohort. We developed the REfractory Septic Shock (RESS) score using the Autoscore framework. Performance was assessed through area under the receiver operating characteristic curve (AUROC) and calibration plots. Primary outcome was the development of refractory septic shock within 24 h from ICU admission or ED arrival, defined as requirement for a norepinephrine equivalent dose greater than 0.5 µg/kg/min.

Results: We analyzed data of 29,618 patients from the MIMIC-IV database, 3,113 from the ED cohort, and 1,015 from the ICU cohort. The RESS score was developed using six variables (lactate, systolic blood pressure, heart rate, body temperature, arterial pH, and white blood cell count). The AUROC was 0.873 (95% CI: 0.846–0.900) in the internal validation, 0.705 (95% CI: 0.678–0.733) upon ED arrival, 0.781 (95% CI: 0.757-0.805) at the time of hypoperfusion or hypotension in the ED, and 0.822 (95% CI: 0.787-0.857) in ICU sepsis patients. Calibrations were fair across all cohorts. 

Conclusions: We developed a predictive scoring system using interpretable machine learning to identify patients at risk of refractory septic shock and demonstrated that this system had fair diagnostic accuracy across multiple validation cohorts including those drawn from the ICU and ED.


Palabras Clave


Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). Jama. 2016;315:801-810.

Namgung M, Ahn C, Park Y, Kwak IY, et al. Mortality among adult patients with sepsis and septic shock in Korea: a systematic review and meta-analysis. Clin Exp Emerg Med. 2023;10:157-171.

Kadri SS, Rhee C, Strich JR, Morales MK, et al. Estimating Ten-Year Trends in Septic Shock Incidence and Mortality in United States Academic Medical Centers Using Clinical Data. Chest. 2017;151:278-285.

Evans L, Rhodes A, Alhazzani W, Antonelli M, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive Care Med. 2021;47:1181-1247.

Suh GJ, Shin TG, Kwon WY, Kim K, et al. Hemodynamic management of septic shock: beyond the Surviving Sepsis Campaign guidelines. Clin Exp Emerg Med. 2023;10:255-264.

Teja B, Bosch NA, Walkey AJ. How We Escalate Vasopressor and Corticosteroid Therapy in Patients With Septic Shock. Chest. 2023;163:567-574.

Guerci P, Belveyre T, Mongardon N, Novy E. When to start vasopressin in septic shock: the strategy we propose. Critical care. 2022;26:125.

Antonucci E, Polo T, Giovini M, Girardis M, et al. Refractory septic shock and alternative wordings: A systematic review of literature. J Crit Care. 2023;75:154258.

Nandhabalan P, Ioannou N, Meadows C, Wyncoll D. Refractory septic shock: our pragmatic approach. Critical care. 2018;22:215.

Jentzer JC, Vallabhajosyula S, Khanna AK, Chawla LS, et al. Management of Refractory Vasodilatory Shock. Chest. 2018;154:416-426.

Evans L, Rhodes A, Alhazzani W, Antonelli M, et al. Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021. Intensive care medicine. 2021;

Johnson AEW, Bulgarelli L, Shen L, Gayles A, et al. MIMIC-IV, a freely accessible electronic health record dataset. Sci Data. 2023;10:1.

Lee HJ, Ko BS, Ryoo SM, Han E, et al. Modified cardiovascular SOFA score in sepsis: development and internal and external validation. BMC medicine. 2022;20:263.

Park H, Shin TG, Kim WY, Jo YH, et al. A quick Sequential Organ Failure Assessment-negative result at triage is associated with low compliance with sepsis bundles: a retrospective analysis of a multicenter prospective registry. Clin Exp Emerg Med. 2022;9:84-92.

Ko BS, Ryoo SM, Han E, Chang H, et al. Modified Cardiovascular Sequential Organ Failure Assessment Score in Sepsis: External Validation in Intensive Care Unit Patients. Journal of Korean medical science. 2023;38:e418.

Vincent JL, Moreno R, Takala J, Willatts S, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707-710.

Johnson AE, Stone DJ, Celi LA, Pollard TJ. The MIMIC Code Repository: enabling reproducibility in critical care research. J Am Med Inform Assoc. 2018;25:32-39.

Jenkins CR, Gomersall CD, Leung P, Joynt GM. Outcome of patients receiving high dose vasopressor therapy: a retrospective cohort study. Anaesth Intensive Care. 2009;37:286-289.

Bassi E, Park M, Azevedo LC. Therapeutic strategies for high-dose vasopressor-dependent shock. Crit Care Res Pract. 2013;2013:654708.

Lambden S, Laterre PF, Levy MM, Francois B. The SOFA score-development, utility and challenges of accurate assessment in clinical trials. Crit Care. 2019;23:374.

Xie F, Chakraborty B, Ong MEH, Goldstein BA, Liu N. AutoScore: A Machine Learning-Based Automatic Clinical Score Generator and Its Application to Mortality Prediction Using Electronic Health Records. JMIR Med Inform. 2020;8:e21798.

Xie F, Ning Y, Liu M, Li S, et al. A universal AutoScore framework to develop interpretable scoring systems for predicting common types of clinical outcomes. STAR Protoc. 2023;4:102302.

Kim DS, Park JE, Hwang SY, Jeong D, et al. Prediction of vasopressor requirement among hypotensive patients with suspected infection: usefulness of diastolic shock index and lactate. Clin Exp Emerg Med. 2022;9:176-186.

Capp R, Horton CL, Takhar SS, Ginde AA, et al. Predictors of patients who present to the emergency department with sepsis and progress to septic shock between 4 and 48 hours of emergency department arrival. Crit Care Med. 2015;43:983-988.

Noritomi DT, Soriano FG, Kellum JA, Cappi SB, et al. Metabolic acidosis in patients with severe sepsis and septic shock: a longitudinal quantitative study. Crit Care Med. 2009;37:2733-2739.

Leisman DE, Doerfler ME, Schneider SM, Masick KD, et al. Predictors, Prevalence, and Outcomes of Early Crystalloid Responsiveness Among Initially Hypotensive Patients With Sepsis and Septic Shock&ast. Critical care medicine. 2018;46:189-198.

Conrad M, Perez P, Thivilier C, Levy B. Early prediction of norepinephrine dependency and refractory septic shock with a multimodal approach of vascular failure. J Crit Care. 2015;30:739-743.

Belok SH, Bosch NA, Klings ES, Walkey AJ. Evaluation of leukopenia during sepsis as a marker of sepsis-defining organ dysfunction. PLoS One. 2021;16:e0252206.

Brown SM, Lanspa MJ, Jones JP, Kuttler KG, et al. Survival after shock requiring high-dose vasopressor therapy. Chest. 2013;143:664-671.

Brand DA, Patrick PA, Berger JT, Ibrahim M, et al. Intensity of Vasopressor Therapy for Septic Shock and the Risk of In-Hospital Death. J Pain Symptom Manage. 2017;53:938-943.

Brask AL, Shemanski SM, Barnes TE, Holmes AK. Timing of Vasopressin Addition to Norepinephrine and Efficacy Outcomes in Patients With Septic Shock. Ann Pharmacother. 2023;57:521-526.

Sacha GL, Lam SW, Wang L, Duggal A, et al. Association of Catecholamine Dose, Lactate, and Shock Duration at Vasopressin Initiation With Mortality in Patients With Septic Shock. Crit Care Med. 2022;50:614-623.

Russell JA, Walley KR, Singer J, Gordon AC, et al. Vasopressin versus norepinephrine infusion in patients with septic shock. N Engl J Med. 2008;358:877-887.

Zhang L, Gu WJ, Huang T, Lyu J, Yin H. The Timing of Initiating Hydrocortisone and Long-term Mortality in Septic Shock. Anesth Analg. 2023;137:850-858.

Park HY, Suh GY, Song JU, Yoo H, et al. Early initiation of low-dose corticosteroid therapy in the management of septic shock: a retrospective observational study. Crit Care. 2012;16:R3.

Sacha GL, Chen AY, Palm NM, Duggal A. Evaluation of the Initiation Timing of Hydrocortisone in Adult Patients With Septic Shock. Shock. 2021;55:488-494.