Reliability of IL-6 Alone and in Combination for Diagnosis of Late Onset Sepsis: a Systematic Review

1. Introduction

The definition of late onset sepsis (LOS) includes presentation after the first 72h of life and association with the postnatal nosocomial or community environment [1]. Neonates at the NICU are prone to LOS due to their immaturity and their missing maternal protection by maternal antibody transfer in case of very preterm infants [2]. Coagulase-negative staphylococci (Gram-positive cocci) represent the most common organisms causing nosocomial infections followed by Gram-negative bacilli and fungi [1,3]. Risk factors for the development of LOS besides immaturity are mechanical ventilation, intravascular catheterization, failure of early enteral feeding with breast milk, prolonged duration of parenteral nutrition, surgery, underlying respiratory and cardiovascular diseases, and hospitalization [4]. In high--income countries, the mortality rate due to neonatal sepsis (including both early and late onset sepsis) ranges from 5% to 20%, higher mortality rates of over 70% can be observed in some low-- and middle--income countries (LMICs) [4]. Early and efficient treatment reduces both mortality and morbidity in neonates with suspected sepsis [5]. Hence, there is great need for biological markers that react rapidly after the onset of inflammation [6].

Released within 2h after the onset of bacteremia, the pro-inflammatory cytokine Interleukin-6 (IL-6) rises earlier than both PCT and CRP in neonatal septic patients [7,8]. IL-6 levels have been shown to be significantly elevated up to 48 h prior to clinical signs of sepsis [9]. Measured at the time of sepsis suspicion IL-6 levels were found to be associated with sepsis severity and mortality risk in preterm infants [7]. Combinations of IL-6 with later and more specific biomarkers (e.g., CRP) have been reported [10].

The aim of this systematic review was to determine accuracy of IL-6, both alone and combined with other markers, for the diagnosis of LOS by reviewing studies published between 1990 and 2020 and to explore affecting factors. In this meta-analysis we decided to focus solely on LOS due to the fact that type of sepsis had previously been recognized as a source of heterogeneity [11].

2. Material and Methods

We used the Pubmed database to search for diagnostic accuracy studies of IL-6 in neonates published between 1990 and 2020. The search terms we used in combination were the following: (Interleukin-6 OR IL-6) AND (neonatal sepsis OR neonatal infection OR sepsis) AND (late onset sepsis OR LOS OR LONS). We did not need any PubMed filters or language restrictions.

Potentially suitable studies were identified by screening titles and abstracts. The following criteria had to be fulfilled by reviewing the abstract: only neonates presenting with culture proven and/or clinically suspected sepsis and IL-6 (alone or combined with other inflammatory markers) being evaluated regarding its potential for the diagnosis of LOS. We excluded all studies dealing with early-onset sepsis or other bacterial infections, all studies written in other languages than English or German, animal and in vitro studies. In line with the PRISMA criteria (see Figure 1), full text articles were screened for other potentially relevant studies. The following data were extracted from all full-text studies included in the analysis: First author, country, year of publication, definition of LOS, number of neonates, recruitment characteristics, reference standards, analysis of blood samples, and time of sample collection. Finally, the IL-6 test method and its use alone or combined with other markers was documented. All analyses based on previously published studies, thus, no informed consent, ethic committee approval, or institutional review board were required.

We used an adapted STARD checklist for septic neonates as published by Chiesa et al. to assess the study quality [12]. This checklist includes 25 items from the key domains; descriptions of participant recruitment, reference standard and index test, which are answered with either yes or no [12].

We explored causes for heterogeneity by means of subgroup analysis. The influence of gestational age was evaluated by comparing subgroups of preterm and mixed populations. Timing of sample collection and its influence on IL-6 accuracy was analysed by dividing the studies into those reporting sample collection at the time when sepsis was suspicious, and those reporting collection times earlier than 12h, earlier than 24h and earlier than 48h after suspicion of sepsis. Biomarker combinations were assessed if at least three studies were found.

3. Results

Figure 1 depictures the search strategy that finally identified 107 records, and two further articles were found in the reference list from the selected studies. After exclusion of studies based on their title, 47 abstracts were screened and 23 full text articles assessed. Finally 16 studies [2,3,9,10,13,14,15,16,17,18,19,20,21,22,23,24] were eligible for meta-analysis. One study [22] only analyzed the biomarker combination IL-6 and CRP, leaving 15 studies including a total of 1306 infants for subgroup analysis of IL-6 as a single marker.

Almost all studies (n=13) defined LOS as sepsis occurring after the first 72h of life [2,10,13,14,15,16,17,18,19,20,21,22,23]. Two studies did not provide a definition but all included infants were older than 3 days [3,24]. Only one study defined LOS as sepsis >48 hours [9]. Eight studies studied preterm infants [9,10,13,19,20,21,24,25] ,while the other eight had a mixed study population [2,3,14,15,16,17,22,23]. All studies measured IL-6 levels in peripheral blood. The majority of studies included proven and clinical sepsis cases [2,3,9,10,13,15,16,17,22,24]. In two studies, the sepsis group consisted only of culture-proven cases [20,21]. One study performed separate analyses for cases of culture positive and cases of clinical sepsis [14].

The IL-6 sensitivities and specificities ranged from 68% to 100% and 28% to 100%, respectively; and the median values were 85.7% and 82%, see Figure 2. Pooled sensitivity was 88% (95% CI: 85%-90%), pooled specificity 78% (75%-81%), see Figure 3. We summarized all the data extracted from the selected studies in Table 1 for IL-6 as single marker and in Table 2 for IL-6 in combined with other biomarkers.

Subgup analyses are shown in Table 3. Sensitivity was higher in the preterm population (87% vs 82%), while specificity was the same for both study populations (86%). Eleven studies [3,13,14,15,16,17,18,19,20,21,23] collected blood samples at the time of sepsis suspicion (0 h). Three studies [14,18,24] collected their samples earlier than 12 hours after initial sepsis suspicion, 6 studies [10,14,18,19,21,24] earlier than 24h and 3 studies [10,14,21] earlier than 48h. One study [9] collected samples within a certain time interval rather than at a specific time point, and thus could not be assigned to one of the subgroups. Collecting the sample at the time of sepsis suspicion showed the highest sensitivity (84%), but the lowest specificity (86%) when compared to the later collection times: sensitivities and specificities of 57% and 94% before 12h, 54% and 88% before 24h and 67% and 92% before 48h.

Seven studies reported results of biomarker combinations including IL-6 [3,10,13,15,18,19,22]. Four studies combined the early sepsis marker IL-6 with either CRP [3,10,13,22]. Combinations with early markers sTREM-1 (soluble Triggering Receptor Expressed on Myeloid Cells-1 [15] and CD64 (Cluster of Differentiation 64, n=1) [18] were studied by one study each. Combinations of up to three biomarkers including, in addition to IL-6, the markers IP-10 (Interferon gamma-induced protein 10), IL-10 (Interleukin-10), CRP and TNF-α (Tumor necrosis factor-α) have been investigated by Ng et al. [10,19]. The positivity criterion of the test was defined by Ng et al. [10,18,19] as any one marker above the cut-off level, by Dillenseger et al. [22], as one of the two above cut-off level and not specified in the remaining studies. In the four studies analysing a combination of IL-6 and CRP at sepsis suspicion [3,10,13,22], cut-off values ranged from 21.7 to 60 pg/mL and 4.05 to 14 mg/L, respectively, sensitivities ranged between 78.12 and 100%, specificities between 41 and 96%. The biomarker combination of IL-6 and CRP, measured at the time of sepsis suspicion, had the highest overall sensitivity (92%), but the lowest overall specificity (79%) in the subgroup analysis.

Table 4 summarizes the quality assessment of the studies according to the adapted STARD criteria. All 16 articles [2,3,9,10,13,14,15,16,17,18,19,20,21,22,23,24] were studies on the diagnostic accuracy of IL-6, and the majority came from single perinatal centers. In 12 studies (75%) enrolment of patients was solely based on clinical signs suspicious for sepsis. In two studies cases were already diagnosed or had been excluded. Twelve studies (75%) were found to have different reference standards for diagnosis of LOS and for verification of index test results, thus, we documented verification bias [2,3,10,13,15,16,17,19,22,23,24,25]. Only four studies (25%) used a composite reference standard for exclusion of LOS [13,18,22,24]. In two studies we found CRP being a comparator of the index test and being part of the reference standard [20,22]. Clinical and demographic data was reported in 15 (94%) studies [3,9,10,13,14,15,16,17,18,19,20,21,22,23,24]. Four studies (25%) reported on the number of neonates fulfilling inclusion criteria that failed to undergo the index tests and/or the reference standard [9,13,21,23]. All studies defined their cut-off values post hoc. Three studies (19%) reported details of the persons who executed the data analysis (number, training and expertise); additionally four studies provided blinding information [2,21]. Measures of statistical uncertainty was reported in six (38%) studies [3,13,14,15,22,24]. Five studies (31%) provided information on calculation methods for test reproducibility [3,9,10,13,18]. Two studies included a cross-tabulation of the results [13,23], and only one study reported on the process how analyses were performed in case of indeterminate results, missing responses or outliers of index tests [2]. None of the studies reported on illness severity scores and their distribution in neonates with and without LOS.

4. Discussion

Our systematic review revealed a satisfying pooled sensitivity of IL-6 as a single marker of 88% (95% CI: 85%-90%), and a lower pooled specificity of 78% (75%-81%). Another review including 31 studies with 1448 infants reported on a global sensitivity and specificity of 82% (77%-86%) and 88% (83%-92%), respectively [26]. We had only nine of the 31 studies (29%) [3,9,10,13,14,15,16,19,21] from this review (26) included in our review. This fact was mainly due to the missing differentiation between early and late onset sepsis in their meta-analysis. Other differences to our meta-analysis were the selection process, missing differentiation by gestational age and time of sampling, as well as combinations of IL-6 with other markers.

Fifteen studies analysed the diagnostic accuracy of IL-6 as a single marker. Most studies measured IL-6 levels at the time of first signs and symptoms of sepsis. Küster et al. [9] in turn investigated the time course of IL-6 expression and its prognostic power in sepsis diagnostics. IL-6 was found to be superior to CRP in the prediction of sepsis 1 or more days before clinical diagnosis. The proven-sepsis group showed a significant increase in IL-6 levels from median baseline values of 7.5 pg/mL to 89.7 pg/mL on day -2, i.e., 2 days before clinical diagnosis [9]. Multiple studies found that IL-6 was able to differentiate between sepsis and no sepsis at the onset solely and had limited potential for diagnosis later during the course of sepsis [15,24]. This is logical due to early eruption of IL-6 and its short half-life time. Lusyati et al. [14] made serial determinations of IL-6 levels (0, 4, 12, 24, and 48 hours). Despite decreasing IL-6 values at all five time points, significantly higher values were found in the proven sepsis group than in the control group for all five measurement points [14]. In the study by Panero et al. [23] all 51 patient controls had IL-6 concentrations <15 pg/mL, while the 17 patients with LOS had IL-6 levels strikingly greater than 15 pg/mL at presentation, corresponding to sensitivity and specificity of 100% for IL-6. Gonzales et al. [21] found that IL-6 had a sensitivity of 75%, specificity of 68%, a NPV of 87% and PPV of 50% on day 0 of the sepsis episode. On day 1 specificity and NPV improved to 90% [21]. However, their cut-off value of 18 pg/mL was defined solely on inspection of the data [21].

Seven studies included in the meta-analysis reported results of biomarker combinations including IL-6 [3,10,13,15,18,19,22]. Raynor et al. [2], analysing seven cytokines, found IL-6 to be the best-performing individual cytokine. IL-6 at a cut-off of 130 pg/mL discriminated with 100% sensitivity and 52% PPV between patients with sepsis and those with sepsis (clinical or culture proven) [2]. Testing all 127 possible cytokine combinations for ruling out sepsis, revealed that adding any other cytokine to IL-6 did not result in a higher PPV [2]. Ng et al. [10] identified IL-6, TNF-α and CRP as the best three markers for LOS diagnosis. A comparison of the diagnostic value of the individual markers versus a combination or panel of markers revealed higher sensitivity and better negative predictive values for the latter [10]. Serial measurements of inflammatory markers further is able to improve diagnostic accuracy. The highest sensitivities (98%) and specificities (91%) were reached when CRP and IL-6 were measured at day 0 combined with either TNF-α (day 1) or CRP (day 2) [10]. In a later study Ng et al. [18] combined IL-6 and CRP at day 0 with CD64 at 24 (day1), which resulted in sensitivity and specificity of 100% and 86%, respectively. Sarafidis et al. [15] found the diagnostic accuracy of IL-6 combined with sTREM-1 (sensitivity and specificity 90% and 62%, respectively) not superior to that of IL-6 alone (sensitivity 80% and specificity 81%). The combination of IL-6 and CRP at time point 0 was superior to other markers and possible combinations in a study by Dillenseger et al. [22], however, sensitivity of 78% and specificity of 76% were not sufficient. Comparing two cut-off points, IL-6 at 60 pg/mL was shown to have good specificity (96%), but low sensitivity (67%), while a lower cut-off of 30 pg/mL, had excellent sensitivity (100%) but only average specificity (74%) [13]. Combining the sensitive IL-6 (cut-off of 30 pg/mL) with the more specific CRP, sensitivity and specificity for sepsis prediction improved to 100% and 96% [13]. Comparing the diagnostic potential of the three markers CD64, IL-6, and CRP in combinations versus individual markers revealed only marginally improvement of sensitivity and negative predictive value [18].

Subgroup analysis was used to analyse the influence of the gestational age and the time of sample collection. One study [2] modified their cut-off criteria in order to achieve a sensitivity of 100%. To prevent introducing a bias, this study [2] was excluded from the subgroup analysis. Some groups provided multiple results, e.g., for varying cut-off levels. To avoid introducing the same study population multiple times when comparing preterm versus mixed study populations, each study was included only once. We chose analyses including the whole study population and in case of different scenarios those that yielded the best results. [27].

Chiesa et al. [12] analysed IL-6 diagnostic accuracy studies and found the majority being suboptimal due to missing information on key elements like study design, conduct, analysis and interpretation of test accuracy, as suboptimal [27]. We used the adapted STARD checklist [12] to assess the quality of the included studies. Twelve of the 16 included studies used different reference standards for diagnosing LOS and verifying index test results [2,3,10,13,15,16,17,19,22,23,24,25]. The majority of studies included proven and clinical sepsis cases [2,3,9,10,13,14,15,16,17,22,24]. In two studies, the sepsis group consisted only of culture-proven cases [20,21]. None of the studies included illness severity scores in their study design. As an inflammatory marker CRP, it serves as an important comparator of the index test, however in two studies it also formed part of the reference standard for sepsis diagnosis [20,22]. All studies included defined cut-offs post hoc, most of them using ROC analysis. In one study the cut-off was chosen solely on inspection of the data [21] and one study did not provide information on the origin of their cut-off value [23]. For further information on the importance of each item we refer to a recent publication of our study group [27].

Regarding the clinical applicability of IL-6 for sepsis diagnosis, Dillenseger et al. [22] stated that cytokine assays require a minimum time of 85min to obtain the results, which would be compatible with clinical decision making but nonetheless should be shortened. Compared to CRP, determination of cytokines is more elaborate and their assays more expensive, therefore many hospital laboratories are not able to perform these assays [3]. Most laboratories are not able to perform these expensive tests in test batteries that further hamper their clinical usefulness as early markers [18]. Others like Değirmencioğlu et al. [20] already implemented IL-6 into clinical routine. Raynor et al. [2] argue that it is unlikely to achieve a 100% diagnostic accuracy via cytokines, since a robust systemic inflammatory response might be absent in some cases of clinical or Gram-positive sepsis. Verboon et al. [3] measured IL-6 levels after 48h of antibiotic treatment to find out whether IL-6 might support the decision about duration of antibiotic treatment (7 to 14 days) in cases of confirmed bacterial sepsis and clinical recovery. They found that a rapid decrease of IL-6 at 48h would justified the early discontinuation of antibiotics [3]. Findings of Ng et al. [10] led to the same conclusion for a serial measurement of IL-6 and CRP measured at the day of sepsis suspicion and CRP measured again two days later. While withholding of antibiotic treatment at the onset of sepsis is not recommended, high sensitivity (98%) and negative predictive values (98%) of this combination indicate that antibiotics could be confidently discontinued at 48 hours without waiting for microbiological results, provided that the infants were in good clinical condition [10].

Strengths of the study: We eliminated the factor of uncertainty in many studies between early or late-onset sepsis by including only cases of LOS. Subgroup analysis identified the type of sepsis as a significant source of heterogeneity [11,27]. Limitations of the study: We investigated a heterogeneous number of studies in order to gain information (subgroup analyses) on IL-6 performance and possible influencing factors. This might have influenced the precision of the study negatively. It might be subject to future research to analyse individual factors causing heterogeneity within otherwise homogenous subgroups. Unfortunately, only a few studies looked at biomarker combinations.

In conclusion, we identified 15 studies analysing diagnostic accuracy of IL-6 for diagnosis of LOS and one study combining Il-6 with CRP between 1990 and 2020 – in total including 1306 infants. IL-6 sensitivities and specificities were between 68% to 100% and 28% to 100% with median values of 85.7% and 82%, respectively. Sensitivity (87% vs 82%), but not specificity (both 86%), was better in preterm infants than in the mixed study population. Sample collection at the time point 0 (were sepsis was first suspected) had the highest sensitivity (84%), but the lowest specificity (86%) when compared to later time points. The biomarker combination of IL-6 and CRP, measured at the time of sepsis suspicion, had the highest overall sensitivity (92%), but the lowest overall specificity (79%).