The impact of laboratory data missingness on sepsis diagnosis timeliness

Abstract

Objective

To investigate the impact of missing laboratory measurements on sepsis diagnostic delays.

Materials and Methods

In adult patients admitted to 2 University of California San Diego (UCSD) hospitals from January 1, 2021 to June 30, 2024, we evaluated the relative time of organ failure (T_OF) and time of clinical suspicion of sepsis (T_suspicion) in patients with sepsis according to the Centers for Medicare & Medicaid Services (CMS) definition.

Results

Of the patients studied, 48.7% (n = 2017) in the emergency department (ED), 30.8% (n = 209) in the wards, and 14.4% (n = 167) in the intensive care unit (ICU) had T_OF after T_suspicion. Patients with T_OF after T_suspicion had significantly higher data missingness of 1 or more of the 5 laboratory components used to determine organ failure. The mean number of missing labs was 4.23 vs 2.83 in the ED, 4.04 vs 3.38 in the wards, and 3.98 vs 3.19 in the ICU.

Discussion

Our study identified many sepsis patients with missing laboratory results vital for the identification of organ failure and the diagnosis of sepsis at or before the time of clinical suspicion of sepsis. Addressing data missingness via more timely laboratory assessment could precipitate an earlier recognition of organ failure and potentially earlier diagnosis of and treatment initiation for sepsis.

Conclusions

More prompt laboratory assessment might improve the timeliness of sepsis recognition and treatment.

Introduction

Sepsis is a life-threatening condition resulting from dysregulated host response to infection, affecting nearly 1.7 million adults in the United States per year.¹ While not every infection progresses toward sepsis, the cost in U.S. hospitals for those infections that do progress is substantial, especially if treatment is delayed, making early identification and prompt treatment of sepsis an important priority.²

A cornerstone of sepsis treatment is the timely administration of appropriate antibiotics; however, most sepsis care protocols also include additional interventions such as volume status assessment, fluid resuscitation, serial serum lactate measurement, and vasopressor support when appropriate.³ However, in clinical practice, the ability to differentiate a stable, appropriately managed infection from sepsis may be delayed based on the availability of laboratory results.⁴ For example, there may be delays in collecting, processing, and interpreting data from lab tests that are needed to meet the Centers for Medicare & Medicaid Services (CMS) defined organ failure criteria. This delay due to data missingness may worsen patient outcomes as evidence has suggested that every 1-hour delay in antibiotic administration leads to a 4%-7% increase in odds of mortality risk for sepsis.^5–7 Thus, understanding data missingness in patients with sepsis might be important for improving the timing of sepsis diagnosis and treatment.

The objective of this research was to measure the extent of incomplete data among hospitalized patients with sepsis, focusing on the timing of organ failure as determined by CMS via the evaluation of vital signs and lab tests recorded in electronic health records (EHRs). We hypothesize that septic patients with a time of organ failure (T_OF) after time of clinical suspicion of sepsis (T_suspicion) will have increased data missingness compared to patients where T_OF occurred before T_suspicion. According to the CMS SEP-1 quality measure, timelines for the 3- and 6-hour bundle should begin when a clinician makes a diagnosis of severe sepsis or septic shock or as soon as systemic inflammatory response syndrome (SIRS), organ failure, and infection criteria are all met within a 6-hour window.⁸ If organ failure criteria are not met because of missing values, then SEP-1 bundle initiation and completion may be delayed until the time of organ failure as opposed to the earlier documented time of infection in retrospective analysis. This concept also applies to the clinical implementation of the sepsis-3 criteria, which requires access to laboratory results to establish organ failure through the calculation of the Sequential Organ Failure Assessment (SOFA) score before diagnosing sepsis.⁹

Methods

Patient population

We conducted an observational retrospective multi-hospital cohort study of adult patients admitted to 2 hospitals between January 1, 2021 and June 30, 2024 within the University of California San Diego (UCSD) Health system. Patients who met the CMS criteria for sepsis were tagged and included for further analysis. The UCSD Institutional Review Board approval was obtained with a waiver of informed consent (#805726, AIVIS: Next Generation Vigilant Information Seeking Artificial Intelligence-based Clinical Decision Support for Sepsis, approved November 15, 2022). All study procedures were followed in accordance with the ethical standards of the responsible committee on human experimentation and with the Helsinki Declaration of 1975.

Feature extraction

Data were extracted from the EHR through Epic Clarity (Epic Systems). Extracted variables included patient demographics, care unit locations, timestamps of antibiotic and blood culture orders, vital sign measurements (heart rate, respiration rate, temperature, mean arterial pressure, and systolic blood pressure), and laboratory measurements [white blood cell count, bands, total bilirubin, creatinine, lactate, platelets, and partial thromboplastin time (PTT)].

We identified T_OF using the CMS criteria for sepsis.¹⁰ CMS organ failure is defined as the presence of 2 or more SIRS criteria¹¹ combined with at least 1 element of CMS-defined organ dysfunction as assessed through vitals, laboratory measurements, urine output, and the presence of acute respiratory failure requiring ventilatory support (Figure 1). Time of clinical suspicion of sepsis (T_suspicion) was defined as the minimum of blood culture tests and antibiotic initiation (for at least 4 days, excluding prophylactic use) within 24 or 72 hours, depending on whether culturing or antibiotic administration occurred first, respectively.^12,13

Patient encounters were binned into emergency department (ED), intensive care unit (ICU), or wards based on care unit location and further divided into 2 cohorts within each location category depending on whether T_OF occurred before T_suspicion (the “before” cohort) or T_OF occurred after T_suspicion (the “after” cohort). Missingness of the 5 CMS laboratory components was determined from whether the lab was present within the 6 hours prior to T_suspicion because CMS SEP-1 guidelines require an element of organ failure, SIRS criteria, and clinical suspicion of sepsis (ie, an infection) within a 6-hour window. A laboratory component was labeled as “contributory” if the laboratory measurement (1) met or exceeded the CMS threshold and (2) ultimately contributed to organ dysfunction at the considered T_OF. This categorization aimed to increase specificity in identifying missing data potentially linked to delayed recognition of organ failure.

Statistical analysis

P-values were calculated between the “before” and “after” groups using the chi-square test for individual laboratory components and t-test for overall missingness between means of the total number of missing CMS laboratory components. A P-value < .05 was considered statistically significant.

Results

We identified a total of 5985 patients who developed sepsis either in the ED, ICU, or wards (Table S1). Among patients with sepsis, 48.7% in the ED had T_OF after T_suspicion, compared to 14.4% in the ICU and 30.8% in the wards (Table 1). For patients with T_OF after T_suspicion, the median (IQR) time difference was 1.7 (0.8-6.9) hours in the ED, 6.0 (1.5-12.5) hours in the ICU, and 6.8 (1.8-15.1) hours in the wards. In all units, data missingness across the 5 CMS laboratory measurements was significantly greater in patients with T_OF after T_suspicion. We did not observe missing values in the vitals and ventilation components of the CMS sepsis definition and excluded them from further analysis.

Missingness of each CMS laboratory value was significantly greater in the “after” cohort for all laboratory values in the ED, wards, and the ICU 6 hours prior to T_suspicion (Table S3). When evaluating only the CMS laboratory components that contributed to organ dysfunction in the 6 hours prior to T_OF, missingness in each laboratory feature—except PTT—was significantly greater in the “after” cohort within the ED. In the ICU, only serum lactate and creatinine showed significantly greater missingness, while in the wards, only lactate was significantly greater (Table S4).

Discussion

Our study identified that there were many patients in whom the recognition of organ failure occurred after the time of clinical suspicion of sepsis (ie, T_OF after T_suspicion). Nearly half of the patients in the ED had T_OF after T_suspicion, likely reflecting the characteristics of patients who present to the ED, the delays in lab reporting, and a practice location where clinicians maintain a high suspicion for sepsis. In contrast, there were proportionally fewer patients in the ICU who had T_OF after T_suspicion, which might reflect more frequent laboratory testing for this population. Thus, in theory, more prompt laboratory assessment that would reveal organ dysfunction might improve the timeliness of the recognition and treatment of sepsis.

Sepsis early warning systems

The implications of these findings can be extended to the field of automated in-hospital warnings and alerts where notifications coming at or after clinical suspicion are not useful.^14,15 A number of machine learning-based early warning systems attempt to reduce the time to sepsis recognition by sending alerts for predicted at-risk patients before clinical suspicion.^16–20 However, models often wait for physicians to order labs after clinical suspicion, leading to scenarios where such alerts are not sent preemptively for at-risk patients.^14,15 Given our findings, it is likely that some of these potential false negatives may be mitigated by including more recent lab results or more rapid tests (eg, point of care testing) and that approaches for advancing the targeted collection of critical labs in high-risk patients (ie, not every patient with an infection may benefit) may play an important role in improving early notification, especially for sepsis. Our future work will attempt to address this need through the development of an automated alerting system that leverages artificial inteligence (AI) and will be capable of submitting laboratory order sets on high-risk patients when it detects a suspected sepsis diagnosis before human clinical suspicion. While the sampling frequency of EHR measurements can vary according to unit locations, patient acuity, and workflow practices,²¹ generalizable prediction tools need to possess an ability to function reliably in settings with missing data as well as complete data.

Limitations

It is possible that during this retrospective analysis, patients may have had a time of sepsis or time of organ failure different from the ground truth or suffer from a condition that mimics but is not sepsis. Secondly, we binned all care unit stays that were not in the ED or ICU into the wards cohort, meaning there is considerable heterogeneity in the units considered as part of this categorization. It is also unknown whether our results are consistent beyond the 3.5-year timeframe. Finally, there was no way to assess if patients had organ failure prior to the eventually measured lab value, but organ failure is often caused by an underlying process that generally occurs more than a few hours prior to the time of laboratory assessment.

Conclusions

In this study, we found significantly higher levels of data missingness in laboratory values used for the CMS sepsis definition across all levels of care. For septic patients where time of organ failure occurred after time of clinical suspicion of sepsis, a delayed creatinine measurement contributed to organ failure in the ED and ICU. Additionally, across all levels of care, a delayed serum lactate contributed to organ failure. Diagnostic timing and patient-centered outcomes for septic patients could be improved by ordering laboratory results earlier.

Acknowledgments

The authors would like to acknowledge the support provided by the Joan & Irwin Jacobs Center for Health Innovation at UC San Diego Health.

Author contributions

Jonathan Y. Lam, Aaron Boussina, Shamim Nemati, and Christopher S. Josef supported the study design. Jonathan Y. Lam analyzed the data and synthesized the results. Aaron Boussina supported data processing and analysis. Christopher S. Josef and Robert L. Owens provided clinical interpretation of the results. All authors contributed to manuscript preparation, critical revisions, and have read and approved the manuscript.

Supplementary material

Supplementary material is available at JAMIA Open online.

Funding

This work was supported by grants from the National Institute of Allergy and Infectious Diseases (#1R42AI177108-1 to J.Y.L., A.B., S.N., R.L.O., and C.S.J.), the National Library of Medicine (#2T15LM011271-11 to A.B. and #R01LM013998 to S.N.), the National Heart, Lung, and Blood Institute (#R01HL157985 to S.N.), and the National Institute of General Medical Sciences (#R35GM143121 to S.N.). The opinions or assertions contained herein are the private ones of the author and are not to be construed as official or reflecting the views of the NIH or any other agency of the US Government.

Conflicts of interest

A.B., S.P.S., S.N. are co-founders and J.Y.L. and C.S.J. are employees of a UCSD startup, Healcisio, Inc, which is focused on the commercialization of advanced analytical decision support tools and formed in compliance with UCSD conflict of interest policies. The remaining authors declare no competing interests.

Data availability

Access to the de-identified UCSD cohort can be made available by contacting the corresponding author and via approval from the UCSD Institutional Review Boards (IRB) and Health Data Oversight Committee (HDOC).

References

Author notes