Abstract

Background

Cellulitis is a common disease in the elderly, and detecting etiologic organisms with blood cultures is difficult because of the low positive rate and occasional skin contamination. Therefore, routine blood cultures are not recommended for uncomplicated cellulitis. However, it is unclear whether blood culture collection for the diagnosis of cellulitis in elderly patients is useful.

Methods

This single hospital–based observational study was performed between April 2012 and March 2015 in Okinawa, Japan. All enrolled patients were aged 15 years or older and admitted to the Division of Infectious Diseases with suspected cellulitis, erysipelas, and cutaneous abscess. Two routine sets of blood cultures were obtained.

Results

Two hundred and twenty-one patients were enrolled. The median age was 77 years. The proportion of bacteremia was 21.7% for all patients (48/221), 8.5% (4/47) for those <65 years, and 25.3% (44/174) for those ≥65 years old (P = .013). The skin contamination rate was 0.9% (2/221). The most common pathogen was Streptococcus dysgalactiae (62.5%). Gram-negative bacteremia not susceptible to cefazolin was detected in 8.3%. Cefazolin and ampicillin were the first- and second-most commonly used therapies. Anti–methicillin-resistant Staphylococcus aureus therapy was required in 3.6% of patients. In addition to age and severe infection, shaking chills and white blood count ≥13 000 cells/µL were independent risk factors of bacteremia.

Conclusions

Two routine sets of blood cultures are recommended for the precise diagnosis and appropriate treatment of cellulitis in elderly patients, especially in patients with shaking chills or leukocytosis.

Cellulitis is a common bacterial infection of the skin and soft tissue, and the number of hospitalizations has increased over the last decade, especially in the elderly [1, 2]. The diagnosis and treatment of cellulitis is challenging [1] and the use of routine blood cultures for identifying the causative agents of cellulitis is controversial [3, 4]. The Infectious Diseases Society of America recommends against performing routine blood cultures [5], based on a clinical study in which specific organisms could be identified from blood cultures in 2.0% of cases, and skin contaminants could be isolated in 3.6% of cases [6]. However, the average number of blood cultures obtained per patient was 1.3 sets, and recent antibiotic use and blood puncture sites were not investigated [6]. This could cause a low prevalence of bacteremia if only 1 set of blood culture was obtained.

Japan is now a “super-aging society” in which the proportion of the elderly population (aged ≥65 years) reached 25% in 2013 [7], and increased to 28.6% in 2020 [8]. Our previous studies revealed that blood culture positivity was higher among elderly patients (aged ≥80 years) [9] and that the blood culture–positive rate of cellulitis was 17.2% (10/58) in this population (mean age, 69.2 years) [10]. Thus, our hypothesis was that the positivity rate of 2 routine blood cultures would be higher in elderly patients, especially those without recent antibiotic use. The skin contamination rate could be lower if physicians used an appropriate sampling procedure, avoiding blood collection from femoral vessels [11].

Two routine sets of blood cultures were obtained from vessels other than femoral vessels, and the patient’s antibiotic use was investigated. Our findings indicated that obtaining 2 routine sets of blood cultures in especially elderly patients with cellulitis ensures a more frequent diagnosis of the causative pathogen and a better selection of appropriate antibiotic treatment. The objective of this study was to clarify the true-positive rate of bacteremia in the cellulitis of elderly patients.

METHODS

Study Design and Setting

This was a single-center, retrospective, observational study. The study setting was Okinawa Chubu Hospital (OCH), which is located in the central area of Okinawa, a subtropical region of Japan. Appropriately 39 000 patients visit the emergency room (ER) annually and nearly 14 000 patients are hospitalized each year [10]. The University of Hawaii has supported the clinical education of the staff at this hospital with a Postgraduate Medical Education Program since 1966 [12]. Most patients with suspected cellulitis are initially examined in the ER, and those who need to be hospitalized are admitted to the Division of Infectious Diseases. Patients with skin and soft tissue infections requiring surgical interventions, such as necrotizing fasciitis, are admitted to the Department of Surgery.

Case Definitions and Data Collection

Cellulitis was considered if a patient had fever or chills and local inflammation of any skin region. Erysipelas is a term used for superficial cellulitis [13], but the distinction between cellulitis and erysipelas is frequently confusing [5, 14]. Therefore, erysipelas was included with cellulitis in this study.

Patients with cellulitis were divided into the following 3 groups: uncomplicated cellulitis, cutaneous abscess, and complicated cellulitis [2]. Uncomplicated cellulitis was defined as an acute skin or soft tissue bacterial infection that required only systemic antibiotic treatment. Cutaneous abscess was defined as a skin abscess that required skin puncture, or in which pus naturally drained from cellulitis, and in which a pus culture was identified as the causative pathogen. Various definitions of complicated cellulitis have been published [2, 14, 15]. In this study, complicated cellulitis was defined as cellulitis with complications, including severe infection (septic shock, suspected meningoencephalitis, requirement for debridement, or intensive care), underlying diseases related to cellulitis (chronic skin ulcer, deep venous thrombosis, fracture, gout, hematoma, or trauma), atypical cellulitis (periorbital cellulitis, odontogenic disease with facial cellulitis, atypical bacterial infection such as Helicobacter cinaedi), and other comorbid infections.

Comorbidities of cancer, diabetes mellitus, liver cirrhosis, ipsilateral leg paralysis, congestive heart failure, chronic kidney disease, hemodialysis, steroid use (oral prednisolone), and immunosuppressant or chemotherapy use (methotrexate, tacrolimus, everolimus, tamoxifen, exemestane) were considered potential risk factors of cellulitis.

Patient information was collected from the medical charts of patients admitted to the Division of Infectious Diseases between April 2012 and March 2015. These data were derived from the hospital discharge summary database. The inclusion criteria were patients aged 15 years or older with suspected with cellulitis. Elderly patients were considered to be 65 years or older. Nosocomial infection cases were not included.

The exclusion criteria were as follows: (1) not diagnosed with cellulitis after admission; (2) unclear diagnosis; (3) discharge from ER; and (4) insufficient data.

Blood Cultures

In OCH, blood cultures are obtained by resident physicians (mainly in their first or second postgraduate year) from the veins of the upper or lower limbs but not from the femoral vessels to minimize skin contamination. If blood collection from veins is difficult, collection from the radial artery is an option. Only isopropyl alcohol (not povidone iodine or chlorhexidine) was used for skin disinfection before needle puncture, based on the findings of our previous research [16]. At least 2 sets of blood cultures and at least 10 mL in each set were encouraged to be drawn and captured in aerobic and anaerobic bottles with Bactec Plus resin medium (Becton, Dickinson and Company, Franklin Lakes, New Jersey). All bottles were incubated for at least 5 days using the Bactec 9240 system (Becton, Dickinson and Company) [16]. If any bacteria grew in blood cultures, the microbiology technicians called the physicians who obtained the blood culture and told them the result. All cultured bacteria were identified by VITEK 2 (bioMérieux Japan Ltd) using biochemical examination.

All blood culture positives were reviewed monthly by the infection control team composed of attending physicians and microbiology technicians. Coagulase-negative staphylococci, Bacillus, Propionibacterium, Micrococcus, Clostridium, and α-hemolytic streptococci were considered to be potential skin contaminants [16]. The skin contamination rate was calculated and reported on a monthly basis to medical staff.

Antimicrobial Selection and Treatment Duration

Empiric treatment was defined as antimicrobial choice in the ER. Targeted treatment was defined as antimicrobial choice after culture results or serum anti-streptolysin O (ASLO) titer was revealed. Ampicillin, ampicillin-sulbactam, cefazolin, and cefotiam (a second-generation cephalosporin, alternative to cefuroxime in Japan) were defined as narrow-spectrum, carbapenems and anti–methicillin-resistant Staphylococcus aureus (MRSA) drugs such as vancomycin and daptomycin as broad-spectrum, and all other antibiotics as intermediate-spectrum antibiotics [10].

The first empiric choice for treating cellulitis was cefazolin 1 g every 6 hours intravenously (IV), which is active against Streptococcus spp and methicillin-sensitive S aureus (MSSA). Clindamycin 600 mg every 8 hours IV or vancomycin 1 g every 12 hours IV was added to cefazolin or substituted if MRSA was suspected based on previous culture results. The doses were adjusted for age, body weight, and renal function. When cutaneous pus was obtained or other infection sites were coinfected, antibiotics were selected according to point-of-care Gram stain results [10, 17–19].

If skin inflammation had a clear borderline and superficial fresh redness, erysipelas was suspected. In such cases, serum ASLO was measured when blood culture did not detect Streptococcus pyogenes or Streptococcus dysgalactiae. If ASLO titer was >240 IU/mL, S pyogenes or S dysgalactiae infection was indicated, and ampicillin was selected as a targeted therapy.

Antibiotic treatment was recommended to continue until 3 days after acute inflammation disappeared. After a patient’s condition stabilized, IV antibiotics were switched to oral antibiotics and the patient was discharged from the hospital.

Outcome Measures

The primary outcome was the proportion of bacteremia that was blood culture positive in the initial 2 sets of bottles, excluding skin contaminants. The secondary outcome was the skin contamination rate.

Sample Size Calculation

To estimate an adequate sample size, we referred to recent research [20] in which the study population was 351 and the blood culture–positive rate was 9%. We expected that the primary outcome would be 17% (10/58) based on our previous study [10]. Therefore, assuming that the blood culture–positive rate would be 17%, with 80% power, and a 2-sided α level of .05, 219 patients would be required.

Statistical Analysis

The χ2 or Fisher exact test was used for categorical variables, and Mann-Whitney U test was used for continuous variables. Additionally, P < .05 was considered to be significant. A multiple logistic regression model was used to investigate the association between the risk of blood culture positivity and other variables. Statistical analysis was performed using Stata software version 16.1 (StataCorp, College Station, Texas).

Patient Consent Statement

The study proposal was approved by the Ethics Committee of OCH (number 49, 2014). Because this was a retrospective observational study, and because Japanese national observational study guidelines do not require individual consent from subjects, the Ethics Committee of OCH waived the consent requirement for this study.

RESULTS

Two hundred and seventy-six patients were screened, and 55 patients were excluded due to the following exclusion criteria: 29 other infections (9 decubitus, 8 herpes zoster, 3 bursitis, 3 osteomyelitis, 2 septic arthritis, 1 each of myositis, necrotizing fasciitis, panniculitis, and trauma); 10 unclear diagnoses; 7 discharges from ER; 6 noninfections (2 contact dermatitis, 2 pseudogout, 1 gout, and 1 statis dermatitis); and 3 data insufficient.

Finally, 221 patients with cellulitis were enrolled. Median age was 77 (interquartile range [IQR], 67–87). Table 1 shows a comparison between blood culture positives and negatives. The median age of patients with positive and negative blood cultures was 80 and 76, respectively (P = .0365). The positive rate of blood cultures was 21.7% (48/221 patients) for all ages, 8.5% (4/47) for those <65 years, and 25.3% (44/174) for the elderly (P = .013). After excluding recent antibiotic use within 48 hours, the rate of bacteremia was 26.0% (47/181). The skin contamination rate was 0.9% (2/221).

Table 1.

Comparison Between Blood Culture–Positive and –Negative Patients

CharacteristicBlood Culture PositiveBlood Culture NegativeP Value
(n = 48)(n = 173)
Age, y, median (IQR)80 (74–89)76 (65–86).0365a
Male sex16 (33.3)68 (39.3).451
Medical history
 Symptoms, d, median (IQR)1 (0–3)2 (1–4).0108a
 Recent antibiotic use within 48 h1 (2.1)39 (22.5)<.001a
 Shaking chills16 (33.3)35 (20.4).059
Comorbidities
 Cancer10 (20.8)24 (13.9).237
 Diabetes mellitus8 (17.4)38 (22.0).547
 Congestive heart failure7 (14.6)25 (14.5)1.000
 Ipsilateral leg paralysis6 (12.5)13 (7.5).259
 Chronic kidney disease3 (6.3)4 (2.3).176
 Liver cirrhosis2 (4.2)8 (4.6)1.000
 Steroid user1 (2.1)7 (4.1)1.000
 Hemodialysis03 (1.7)1.000
 Immunosuppressant user02 (1.2)1.000
Location of infection
 Face018
 Chest13
 Back16
 Groin or buttock310
 Arm119
 Leg42118
Physical examinations
 Lymphedema of arm or leg12/43 (27.9)36/137 (26.3).833
 Femoral lymph node tender10/42 (20.8)28/118 (23.7).992
 Tinea pedis32/42 (76.2)82/118 (69.5).410
Laboratory tests
 WBCs/μL, median (IQR)15 000 (10 000–19 000)10 500 (7400–15 000).001a
 CRP, mg/dL, median (IQR)2.2 (1–9)5.3 (1.5–12).0753
Initial diagnosis change after admission10 (20.8)14 (8.1).012a
Final diagnosis classification
 Uncomplicated cellulitis28124
 Cutaneous abscess414
 Complicated cellulitis1635
Bacterial coinfection other than cellulitis11 (22.9)14 (8.1).004a
Severe infectionb3 (6.3)3 (1.7).118
Antibiotic treatment days in hospital, median (IQR)14 (13–15)8 (6–10)<.0001
Death1 (2.1)1 (0.6).388
CharacteristicBlood Culture PositiveBlood Culture NegativeP Value
(n = 48)(n = 173)
Age, y, median (IQR)80 (74–89)76 (65–86).0365a
Male sex16 (33.3)68 (39.3).451
Medical history
 Symptoms, d, median (IQR)1 (0–3)2 (1–4).0108a
 Recent antibiotic use within 48 h1 (2.1)39 (22.5)<.001a
 Shaking chills16 (33.3)35 (20.4).059
Comorbidities
 Cancer10 (20.8)24 (13.9).237
 Diabetes mellitus8 (17.4)38 (22.0).547
 Congestive heart failure7 (14.6)25 (14.5)1.000
 Ipsilateral leg paralysis6 (12.5)13 (7.5).259
 Chronic kidney disease3 (6.3)4 (2.3).176
 Liver cirrhosis2 (4.2)8 (4.6)1.000
 Steroid user1 (2.1)7 (4.1)1.000
 Hemodialysis03 (1.7)1.000
 Immunosuppressant user02 (1.2)1.000
Location of infection
 Face018
 Chest13
 Back16
 Groin or buttock310
 Arm119
 Leg42118
Physical examinations
 Lymphedema of arm or leg12/43 (27.9)36/137 (26.3).833
 Femoral lymph node tender10/42 (20.8)28/118 (23.7).992
 Tinea pedis32/42 (76.2)82/118 (69.5).410
Laboratory tests
 WBCs/μL, median (IQR)15 000 (10 000–19 000)10 500 (7400–15 000).001a
 CRP, mg/dL, median (IQR)2.2 (1–9)5.3 (1.5–12).0753
Initial diagnosis change after admission10 (20.8)14 (8.1).012a
Final diagnosis classification
 Uncomplicated cellulitis28124
 Cutaneous abscess414
 Complicated cellulitis1635
Bacterial coinfection other than cellulitis11 (22.9)14 (8.1).004a
Severe infectionb3 (6.3)3 (1.7).118
Antibiotic treatment days in hospital, median (IQR)14 (13–15)8 (6–10)<.0001
Death1 (2.1)1 (0.6).388

Data are presented as No. (%) unless otherwise indicated.

Abbreviations: CRP, C-reactive protein; IQR, interquartile range; WBC, white blood cell.

a

P < .05.

b

Septic shock, suspected meningoencephalitis, requirement for debridement, or intensive care.

Table 1.

Comparison Between Blood Culture–Positive and –Negative Patients

CharacteristicBlood Culture PositiveBlood Culture NegativeP Value
(n = 48)(n = 173)
Age, y, median (IQR)80 (74–89)76 (65–86).0365a
Male sex16 (33.3)68 (39.3).451
Medical history
 Symptoms, d, median (IQR)1 (0–3)2 (1–4).0108a
 Recent antibiotic use within 48 h1 (2.1)39 (22.5)<.001a
 Shaking chills16 (33.3)35 (20.4).059
Comorbidities
 Cancer10 (20.8)24 (13.9).237
 Diabetes mellitus8 (17.4)38 (22.0).547
 Congestive heart failure7 (14.6)25 (14.5)1.000
 Ipsilateral leg paralysis6 (12.5)13 (7.5).259
 Chronic kidney disease3 (6.3)4 (2.3).176
 Liver cirrhosis2 (4.2)8 (4.6)1.000
 Steroid user1 (2.1)7 (4.1)1.000
 Hemodialysis03 (1.7)1.000
 Immunosuppressant user02 (1.2)1.000
Location of infection
 Face018
 Chest13
 Back16
 Groin or buttock310
 Arm119
 Leg42118
Physical examinations
 Lymphedema of arm or leg12/43 (27.9)36/137 (26.3).833
 Femoral lymph node tender10/42 (20.8)28/118 (23.7).992
 Tinea pedis32/42 (76.2)82/118 (69.5).410
Laboratory tests
 WBCs/μL, median (IQR)15 000 (10 000–19 000)10 500 (7400–15 000).001a
 CRP, mg/dL, median (IQR)2.2 (1–9)5.3 (1.5–12).0753
Initial diagnosis change after admission10 (20.8)14 (8.1).012a
Final diagnosis classification
 Uncomplicated cellulitis28124
 Cutaneous abscess414
 Complicated cellulitis1635
Bacterial coinfection other than cellulitis11 (22.9)14 (8.1).004a
Severe infectionb3 (6.3)3 (1.7).118
Antibiotic treatment days in hospital, median (IQR)14 (13–15)8 (6–10)<.0001
Death1 (2.1)1 (0.6).388
CharacteristicBlood Culture PositiveBlood Culture NegativeP Value
(n = 48)(n = 173)
Age, y, median (IQR)80 (74–89)76 (65–86).0365a
Male sex16 (33.3)68 (39.3).451
Medical history
 Symptoms, d, median (IQR)1 (0–3)2 (1–4).0108a
 Recent antibiotic use within 48 h1 (2.1)39 (22.5)<.001a
 Shaking chills16 (33.3)35 (20.4).059
Comorbidities
 Cancer10 (20.8)24 (13.9).237
 Diabetes mellitus8 (17.4)38 (22.0).547
 Congestive heart failure7 (14.6)25 (14.5)1.000
 Ipsilateral leg paralysis6 (12.5)13 (7.5).259
 Chronic kidney disease3 (6.3)4 (2.3).176
 Liver cirrhosis2 (4.2)8 (4.6)1.000
 Steroid user1 (2.1)7 (4.1)1.000
 Hemodialysis03 (1.7)1.000
 Immunosuppressant user02 (1.2)1.000
Location of infection
 Face018
 Chest13
 Back16
 Groin or buttock310
 Arm119
 Leg42118
Physical examinations
 Lymphedema of arm or leg12/43 (27.9)36/137 (26.3).833
 Femoral lymph node tender10/42 (20.8)28/118 (23.7).992
 Tinea pedis32/42 (76.2)82/118 (69.5).410
Laboratory tests
 WBCs/μL, median (IQR)15 000 (10 000–19 000)10 500 (7400–15 000).001a
 CRP, mg/dL, median (IQR)2.2 (1–9)5.3 (1.5–12).0753
Initial diagnosis change after admission10 (20.8)14 (8.1).012a
Final diagnosis classification
 Uncomplicated cellulitis28124
 Cutaneous abscess414
 Complicated cellulitis1635
Bacterial coinfection other than cellulitis11 (22.9)14 (8.1).004a
Severe infectionb3 (6.3)3 (1.7).118
Antibiotic treatment days in hospital, median (IQR)14 (13–15)8 (6–10)<.0001
Death1 (2.1)1 (0.6).388

Data are presented as No. (%) unless otherwise indicated.

Abbreviations: CRP, C-reactive protein; IQR, interquartile range; WBC, white blood cell.

a

P < .05.

b

Septic shock, suspected meningoencephalitis, requirement for debridement, or intensive care.

In terms of cellulitis classification, the positive rate was 18.4% (28/152) for uncomplicated cases, 22.2% (4/18) for cutaneous abscess, and 31.4% (16/51) for complicated cellulitis. There was no significant difference in comorbidities between blood culture positives and negatives. Even if all comorbidities were negative, the blood culture yield was 25.0% (21/84) in uncomplicated cellulitis.

With regard to the location of infection, the leg was the most common, and the blood culture yield was 26.3% (42/160). Some cellulitis in the chest or groin was spread from the arm or leg, respectively. Cellulitis in the back or buttock was mainly caused by bedridden status or pressure sores.

The initial diagnoses of 24 patients changed after admission, and the proportion of change was higher among blood culture positives than negatives (20.8% vs 8.1%, P = .012). In 16 of them, local inflammation of skin was not recognized in the ER, and their initial diagnoses were as follows: 8 unknown origin, 5 urinary tract infection, 3 pneumonia, 2 upper respiratory infection. The remaining 8 patients were initially considered to have uncomplicated cellulitis or cutaneous abscess in the ER, but their diagnoses were later changed to complicated cellulitis after admission: 2 with infective endocarditis (1 of these with cerebral septic embolism), 1 chronic dacryoadenitis with periorbital cellulitis, 1 odontogenic disease with facial cellulitis, 1 lumbar diskitis with leg cellulitis, 1 adjacent ischial tuberosity osteomyelitis from a buttock cutaneous abscess, 1 radius fracture under arm cellulitis, and 1 H cinaedi infection.

Complicated cellulitis consisted of 36 other infections (15 urinary tract infections, 8 herpes zoster, 5 respiratory infections, 4 osteomyelitis, 2 infective endocarditis, 1 cerebral septic embolism, 1 ventriculoperitoneal shunt infection); 10 underlying diseases (3 chronic skin diseases, 2 fractures, 2 gout, 2 traumas, and 1 deep venous thrombosis); 6 severe infections; and 5 atypical cellulitis (3 odontogenic diseases, 1 periorbital cellulitis, and 1 atypical bacteria). Among the 36 other infections, 5 patients had distant infection cites other than leg cellulitis caused by the same bacteria: 2 pyelonephritis caused by Citrobacter koseri or Morganella morganii, 1 infective endocarditis and cerebral septic embolism caused by S dysgalactiae, 1 infective endocarditis caused by Enterococcus faecalis, and 1 lumbar diskitis caused by Streptococcus agalactiae.

Empiric treatment was as follows: cefazolin in 172 (65.9%), clindamycin in 21 (8.0%), vancomycin in 16 (6.1%), aztreonam in 13 (5.0%), ampicillin-sulbactam in 9 (3.4%), ampicillin in 8 (3.1%), ceftriaxone in 7 (2.7%), cefotaxime in 4 (1.5%), cefotiam in 3 (1.1%), ceftazidime in 2 (0.8%), doxycycline in 2 (0.8%), imipenem-cilastatin in 2 (0.8%), cefmetazole in 1 (0.4%), and ciprofloxacin in 1 (0.4%). Blood culture positives had longer antibiotic treatment days than negatives. In uncomplicated cellulitis, the empiric treatment of 4 patients were not effective: 1 was Shewanella cultured by blood, 1 was MRSA cultured by pus, and 2 cases were unknown pathogen.

Table 2 shows the identified pathogens in blood and pus cultures and targeted treatment based on cultures and ASLO results. The most common pathogen was S dysgalactiae; 27 of 30 (90.0%) S dysgalactiae were identified as S dysgalactiae subsp equisimilis, and the remaining 3 (10.0%) as S dysgalactiae subsp dysgalactiae. Two cases of Staphylococcus epidermidis cultured in only 1 set of samples were considered to be skin contamination (0.9% [2/221]). With regard to the location of infection caused by Streptococcus spp, S dysgalactiae infected 27 legs, 1 arm, 1 back, 1 groin, and 1 buttock; S agalactiae infected 6 legs, 1 chest and back, and 1 buttock; and S pyogenes infected 1 leg. Among 7 gram-negative pathogens, 4 of them were resistant to cefazolin.

Table 2.

Identified Pathogens in Blood and Pus Cultures and Targeted Treatment During Hospitalization

CultureNo. (%)Targeted Treatment (n = 225)
AmpicillinCefazolinCefotaxime/
Ceftriaxone
ClindamycinVancomycin/
Daptomycin
Othersa
(n = 56)(n = 122)(n = 10)(n = 10)(n = 8)(n = 19)
Blood cultures (n = 48)
Gram-positive41 (85.4)
  Streptococcus dysgalactiae30 (62.5)2523
  Streptococcus agalactiae8 (16.7)71
  Enterococcus faecalisb1 (2.1)11
  MRSA1 (2.1)1
  Streptococcus pyogenes1 (2.1)1
Gram-negative7 (14.6)
  Aeromonas hydrophila1 (2.1)1
  Citrobacter koseri1 (2.1)1
  Helicobacter cinaedi1 (2.1)1
  Morganella morganii1 (2.1)1
  Serratia marcescens1 (2.1)1
  Shewanella algae1 (2.1)1
  Vibrio alginolyticus1 (2.1)1
Pus cultures (n = 19)
MSSA7 (38.9)43
MRSA3 (16.7)3
 Streptococcus dysgalactiae3 (16.7)3
 Serratia marcescens2 (11.1)2
 Streptococcus agalactiae2 (11.1)11
 Shewanella algae1 (5.7)1
 Streptococcus pyogenes1 (5.7)1
CultureNo. (%)Targeted Treatment (n = 225)
AmpicillinCefazolinCefotaxime/
Ceftriaxone
ClindamycinVancomycin/
Daptomycin
Othersa
(n = 56)(n = 122)(n = 10)(n = 10)(n = 8)(n = 19)
Blood cultures (n = 48)
Gram-positive41 (85.4)
  Streptococcus dysgalactiae30 (62.5)2523
  Streptococcus agalactiae8 (16.7)71
  Enterococcus faecalisb1 (2.1)11
  MRSA1 (2.1)1
  Streptococcus pyogenes1 (2.1)1
Gram-negative7 (14.6)
  Aeromonas hydrophila1 (2.1)1
  Citrobacter koseri1 (2.1)1
  Helicobacter cinaedi1 (2.1)1
  Morganella morganii1 (2.1)1
  Serratia marcescens1 (2.1)1
  Shewanella algae1 (2.1)1
  Vibrio alginolyticus1 (2.1)1
Pus cultures (n = 19)
MSSA7 (38.9)43
MRSA3 (16.7)3
 Streptococcus dysgalactiae3 (16.7)3
 Serratia marcescens2 (11.1)2
 Streptococcus agalactiae2 (11.1)11
 Shewanella algae1 (5.7)1
 Streptococcus pyogenes1 (5.7)1

Abbreviations: MRSA, methicillin-resistant Staphylococcus aureus; MSSA, methicillin-sensitive Staphylococcus aureus.

a

Ampicillin-sulbactam (n = 5), ceftazidime (n = 5), cefmetazole (n = 2), doxycycline (n = 2), piperacillin (n = 2), tobramycin (n = 2), streptomycin (n = 1).

b

Treated with both ampicillin and streptomycin.

Table 2.

Identified Pathogens in Blood and Pus Cultures and Targeted Treatment During Hospitalization

CultureNo. (%)Targeted Treatment (n = 225)
AmpicillinCefazolinCefotaxime/
Ceftriaxone
ClindamycinVancomycin/
Daptomycin
Othersa
(n = 56)(n = 122)(n = 10)(n = 10)(n = 8)(n = 19)
Blood cultures (n = 48)
Gram-positive41 (85.4)
  Streptococcus dysgalactiae30 (62.5)2523
  Streptococcus agalactiae8 (16.7)71
  Enterococcus faecalisb1 (2.1)11
  MRSA1 (2.1)1
  Streptococcus pyogenes1 (2.1)1
Gram-negative7 (14.6)
  Aeromonas hydrophila1 (2.1)1
  Citrobacter koseri1 (2.1)1
  Helicobacter cinaedi1 (2.1)1
  Morganella morganii1 (2.1)1
  Serratia marcescens1 (2.1)1
  Shewanella algae1 (2.1)1
  Vibrio alginolyticus1 (2.1)1
Pus cultures (n = 19)
MSSA7 (38.9)43
MRSA3 (16.7)3
 Streptococcus dysgalactiae3 (16.7)3
 Serratia marcescens2 (11.1)2
 Streptococcus agalactiae2 (11.1)11
 Shewanella algae1 (5.7)1
 Streptococcus pyogenes1 (5.7)1
CultureNo. (%)Targeted Treatment (n = 225)
AmpicillinCefazolinCefotaxime/
Ceftriaxone
ClindamycinVancomycin/
Daptomycin
Othersa
(n = 56)(n = 122)(n = 10)(n = 10)(n = 8)(n = 19)
Blood cultures (n = 48)
Gram-positive41 (85.4)
  Streptococcus dysgalactiae30 (62.5)2523
  Streptococcus agalactiae8 (16.7)71
  Enterococcus faecalisb1 (2.1)11
  MRSA1 (2.1)1
  Streptococcus pyogenes1 (2.1)1
Gram-negative7 (14.6)
  Aeromonas hydrophila1 (2.1)1
  Citrobacter koseri1 (2.1)1
  Helicobacter cinaedi1 (2.1)1
  Morganella morganii1 (2.1)1
  Serratia marcescens1 (2.1)1
  Shewanella algae1 (2.1)1
  Vibrio alginolyticus1 (2.1)1
Pus cultures (n = 19)
MSSA7 (38.9)43
MRSA3 (16.7)3
 Streptococcus dysgalactiae3 (16.7)3
 Serratia marcescens2 (11.1)2
 Streptococcus agalactiae2 (11.1)11
 Shewanella algae1 (5.7)1
 Streptococcus pyogenes1 (5.7)1

Abbreviations: MRSA, methicillin-resistant Staphylococcus aureus; MSSA, methicillin-sensitive Staphylococcus aureus.

a

Ampicillin-sulbactam (n = 5), ceftazidime (n = 5), cefmetazole (n = 2), doxycycline (n = 2), piperacillin (n = 2), tobramycin (n = 2), streptomycin (n = 1).

b

Treated with both ampicillin and streptomycin.

If Streptococcus infection was suspected because of a clear borderline and superficial fresh redness, ASLO was measured in 84 cases. Median ASLO was 412 IU/mL (IQR, 92–771) in ampicillin selected (n = 36), and 65 IU/mL (IQR, 23–141) in not selected (n = 48).

Of the 48 blood culture positives, the number of positive bottles was 15 for 1 bottle (31.3%), 9 for 2 bottles (18.8%) (2 patients were within the same set), 6 for 3 bottles (12.5%), and 18 for 4 bottles, which meant 2 aerobic and 2 anaerobic (37.5%). Therefore, nearly one-third of the patients had only 1 set of blood culture positives (35.4% [17/48]). Among 1 or 2 bottle positives, 10 were anaerobic bottles.

Of the 19 pus cultures, both MSSA and S agalactiae were identified in 1 patient. Serratia marcescens was detected in 2 cases, 1 of which was also detected in blood culture. The 2 Serratia patients had been admitted to another hospital 1 month earlier, and they were suspected of having nosocomial infections. Including these 2 cases, there was no venous line at the site of cellulitis on arrival to the ER.

In targeted therapy, narrow-spectrum antibiotics such as cefazolin (54.2%) and ampicillin (24.9%) were frequently used, and anti-MRSA therapy was needed in 3.6% of patients. Two combination therapies were used for 4 patients, and the total targeted antibiotics were 225, not 221.

Oral antibiotic treatments after discharge were as follows: 58 cephalexin, 21 amoxicillin, 5 doxycycline, 2 amoxicillin-clavulanic acid, 2 clindamycin, 2 ciprofloxacin, 1 levofloxacin, and 1 trimethoprim-sulfamethoxazole.

Table 3 shows the odds ratios of blood culture positivity with their associated 95% confidence intervals. Age, recent antibiotic use within 48 hours before arrival to the ER, and shaking chills were sequentially assigned to the model, because these factors were shown to be correlated with blood culture positivity [9]. Next, leukocytosis was added to the model to include the possibility of another correlating variable, because it differed significantly between the blood culture positives and negatives. Finally, severe infection, comorbidities, and bacterial coinfection other than cellulitis were added to adjust confounding. As a result, age, recent antibiotic use within 48 hours, shaking chills, high white blood cell counts, and severe infection were independent risk factors for blood culture prevalence. However, comorbidities and bacterial coinfection other than cellulitis were not. Eighteen cases of bacteremia caused shaking chills (9 S dysgalactiae, 5 S agalactiae, 1 C koseri, 1 H cinaedi, 1 M morganii, and 1 Shewanella algae).

Table 3.

Odds Ratios of Blood Culture Positivity

CharacteristicUnadjusted OR
(95% CI)
Adjusted OR
(95% CI)
P Value
Age1.03 (1.01–1.06)1.04 (1.01–1.07).004a
Recent antibiotic use within 48 h.07 (.01–.55).08 (.01–.60).014a
Shaking chills1.96 (.97–3.96)2.63 (1.12–6.15).025a
WBC count ≥13 000/μL2.84 (1.45–5.57)2.81 (1.34–5.92).006a
Severe infectionb3.78 (.74–19.4)8.54 (1.26–58.0).028a
Comorbiditiesc1.04 (.54–1.99)1.07 (.53–2.14).86
Bacterial coinfection other than cellulitis3.38 (1.42–8.03)1.54 (.59–4.02).38
CharacteristicUnadjusted OR
(95% CI)
Adjusted OR
(95% CI)
P Value
Age1.03 (1.01–1.06)1.04 (1.01–1.07).004a
Recent antibiotic use within 48 h.07 (.01–.55).08 (.01–.60).014a
Shaking chills1.96 (.97–3.96)2.63 (1.12–6.15).025a
WBC count ≥13 000/μL2.84 (1.45–5.57)2.81 (1.34–5.92).006a
Severe infectionb3.78 (.74–19.4)8.54 (1.26–58.0).028a
Comorbiditiesc1.04 (.54–1.99)1.07 (.53–2.14).86
Bacterial coinfection other than cellulitis3.38 (1.42–8.03)1.54 (.59–4.02).38

Abbreviations: CI, confidence interval; OR, odds ratio; WBC, white blood cell.

a

P < .05.

b

Septic shock, suspected meningoencephalitis, requirement for debridement, or intensive care.

c

Cancer, diabetes mellitus, liver cirrhosis, ipsilateral leg paralysis, congestive heart failure, chronic kidney disease, hemodialysis, steroid use, immunosuppressant use.

Table 3.

Odds Ratios of Blood Culture Positivity

CharacteristicUnadjusted OR
(95% CI)
Adjusted OR
(95% CI)
P Value
Age1.03 (1.01–1.06)1.04 (1.01–1.07).004a
Recent antibiotic use within 48 h.07 (.01–.55).08 (.01–.60).014a
Shaking chills1.96 (.97–3.96)2.63 (1.12–6.15).025a
WBC count ≥13 000/μL2.84 (1.45–5.57)2.81 (1.34–5.92).006a
Severe infectionb3.78 (.74–19.4)8.54 (1.26–58.0).028a
Comorbiditiesc1.04 (.54–1.99)1.07 (.53–2.14).86
Bacterial coinfection other than cellulitis3.38 (1.42–8.03)1.54 (.59–4.02).38
CharacteristicUnadjusted OR
(95% CI)
Adjusted OR
(95% CI)
P Value
Age1.03 (1.01–1.06)1.04 (1.01–1.07).004a
Recent antibiotic use within 48 h.07 (.01–.55).08 (.01–.60).014a
Shaking chills1.96 (.97–3.96)2.63 (1.12–6.15).025a
WBC count ≥13 000/μL2.84 (1.45–5.57)2.81 (1.34–5.92).006a
Severe infectionb3.78 (.74–19.4)8.54 (1.26–58.0).028a
Comorbiditiesc1.04 (.54–1.99)1.07 (.53–2.14).86
Bacterial coinfection other than cellulitis3.38 (1.42–8.03)1.54 (.59–4.02).38

Abbreviations: CI, confidence interval; OR, odds ratio; WBC, white blood cell.

a

P < .05.

b

Septic shock, suspected meningoencephalitis, requirement for debridement, or intensive care.

c

Cancer, diabetes mellitus, liver cirrhosis, ipsilateral leg paralysis, congestive heart failure, chronic kidney disease, hemodialysis, steroid use, immunosuppressant use.

DISCUSSION

The proportion of bacteremia was 21.7% for overall, and 25.3% for those ≥65 years old, which was higher than previous studies on cellulitis (2.0%–10.8%) [6, 14, 15, 20, 21]. The skin contamination rate was 0.9%, which was lower than the findings of other studies (1.6%–4.8%) [6, 14, 15, 20, 22]. In the targeted treatment, narrow-spectrum antibiotics were commonly selected. There are 3 possible reasons for the high rate of bacteremia.

The first reason is the older age of our cohort: The median age of patients in this study was 77 years, which was the highest among the studies we reviewed [2, 14, 15, 20, 22–24]. Elderly patients generally have more complications, and nearly 70% of patients (114/160) had tinea pedis in our study, although most of these diagnoses were based on inspection and not confirmed microbiologically. Skin barrier damage caused by tinea pedis is a significant risk factor of nonpurulent cellulitis of the legs [25]. Okinawa belongs to a subtropical region, and local people prefer to wear sandals on bare feet. Given this, they may tend to have small injuries of their toes. Okinawa is also endemic for human T-lymphotropic virus 1 (HTLV-1) [26], which can cause skin disorder [27]. In addition, the presentation of bacteremia is often atypical or nonspecific [28], and many patients who have cognitive impairment are unable to complain of their symptoms. As a result, caregivers are slow to notice a change and this can lead to the progression of cellulitis [9].

Second, 2 routine sets of blood culture were drawn using appropriate sampling methods. A previous study found that the number of positive blood culture bottles was relatively low in cellulitis [22], and our result confirmed that half of the patients with bacteremia had only 1 or 2 positive bottles among 2 sets of blood cultures. If only 1 set of blood cultures is obtained in cellulitis, false-negatives are likely to occur, and the prevalence of bacteremia can easily be low [6]. There have been some reports in which blood culture positivity was >18% [22–24]. However, their blood cultures were not obtained in mild cases, which caused selection bias.

Third, recent antibiotic use was considered in order to make a precise evaluation of bacteremia. The proportion of bacteremia was low in cellulitis, and the most common pathogen was S dysgalactiae, for which antimicrobial resistance is uncommon. Patients who were exposed to antibiotics before arrival at the ER had a low odds ratio of bacteremia, which was consistent with another study [22]. These results suggest that any recent antibiotic use could reduce bacteremia. Thus, blood cultures should always be obtained before starting antibiotic treatment.

We recognize that collecting blood samples from sites other than femoral vessels led to the low rate of skin contamination. In OCH, clinical research proved that isopropyl alcohol as a skin disinfectant was sufficient [16]. In addition, the surveillance team has a mandate to provide feedback to the physicians who obtained the blood samples, which potentially contributed to decreased contamination rates.

This research confirmed that a cellulitis diagnosis was not always straightforward. Among the 221 patients in this study with confirmed cellulitis, 16 cases were not discovered in the ER, and 8 of these cases were diagnosed with cellulitis after blood culture detected Streptococcus spp. The initial physical examinations of these cases may have been insufficient. However, in some cellulitis cases, fever may appear hours before skin abnormalities appear [5]. Therefore, cellulitis may have presented as other conditions in the ER until skin inflammation appeared clearly.

Cellulitis rarely occurs as a result of bacteremia distant from the initial site [13], and this phenomenon was suspected with 5 cases of secondary cellulitis from the primary site such as infective endocarditis, lumbar diskitis, and pyelonephritis cases in our study. Blood cultures were essential for the diagnoses of these cases.

The initial diagnoses of 24 patients changed after admission, and the proportion was higher in blood culture positives than negatives. This indicates that cellulitis is frequently misdiagnosed and that a blood culture pathogen can point toward the correct diagnosis. In addition, IV antibiotic treatment days were longer for blood culture positives than negatives.

The most common pathogen identified in blood culture was S dysgalactiae, which is in agreement with other recent studies [20, 21, 29]. This resulted in the common use of ampicillin as a targeted therapy. Staphylococcus aureus was cultured in cutaneous abscesses, which is consistent with previous reports [2, 24]. The organism was detected in only 1 blood culture (2.1%), and this result was similar to that of another recent study (4.3%) [21]. Negative blood culture results prompted physicians to discontinue anti-MRSA therapy. As a result, anti-MRSA antibiotics were used in 3.6% of targeted therapy in our study.

Without blood cultures, many cases of bacteremia may be overlooked, and diagnosis of severe underlying diseases, such as infective endocarditis or osteomyelitis, will be delayed. If the etiologic pathogen is unknown, empiric broad-spectrum antibiotics must be continued. Cellulitis with gram-negative bacteremia is diagnosed only by blood culture that avoids inappropriate treatment with cefazolin or vancomycin.

There were some limitations to this study. First, this was a retrospective chart review study and some data were missing. There was uncertainty regarding the volume and collection method of the blood cultures. Although medical staff were encouraged to obtain at least 10 mL in each set and not from femoral vessels, it was impossible to determine the exact volume and collection sites. Second, our strategy is not directly applicable to mild cellulitis because these data were derived from adult patients who required hospitalization. However, if a patient had shaking chills or leukocytosis, it is strongly recommended that blood cultures be obtained. Third, our targeted therapy based on culture results cannot be directly applied to other geographic regions because the prevalence of antimicrobial resistance may differ among regions. Fourth, body mass index was not recorded, and obesity was not investigated for analysis.

CONCLUSIONS

Using 2 routine sets of blood cultures obtained from sites other than femoral vessels, we found a high proportion of bacteremia and a low rate of skin contamination in cellulitis in especially elderly Japanese patients. Blood cultures are encouraged for the precise diagnosis of cellulitis and can promote the use of narrow-spectrum antibiotics as a targeted therapy.

Notes

Acknowledgments. We thank all resident doctors at OCH who obtained blood cultures in the emergency room. We are grateful to Dr Tomokazu Kishaba, the former chief of the Division of Infectious Diseases of OCH, who retired in 2006. He reviewed this manuscript and gave us extensive advice.

Potential conflicts of interest. All authors: No reported conflicts of interest.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References

1

Raff
AB
,
Kroshinsky
D
.
Cellulitis: a review
.
JAMA
2016
;
316
:
325
37
.

2

Jenkins
TC
,
Sabel
AL
,
Sarcone
EE
,
Price
CS
,
Mehler
PS
,
Burman
WJ
.
Skin and soft-tissue infections requiring hospitalization at an academic medical center: opportunities for antimicrobial stewardship
.
Clin Infect Dis
2010
;
51
:
895
903
.

3

Coburn
B
,
Morris
AM
,
Tomlinson
G
,
Detsky
AS
.
Does this adult patient with suspected bacteremia require blood cultures?
JAMA
2012
;
308
:
502
11
.

4

Fabre
V
,
Sharara
SL
,
Salinas
AB
,
Carroll
KC
,
Desai
S
,
Cosgrove
SE
.
Does this patient need blood cultures? A scoping review of indications for blood cultures in adult nonneutropenic inpatients
.
Clin Infect Dis
2020
;
71
:
1339
47
.

5

Stevens
DL
,
Bisno
AL
,
Chambers
HF
, et al.
Practice guidelines for the diagnosis and management of skin and soft tissue infections: 2014 update by the Infectious Diseases Society of America
.
Clin Infect Dis
2014
;
59
:
e10–52
.

6

Perl
B
,
Gottehrer
NP
,
Raveh
D
,
Schlesinger
Y
,
Rudensky
B
,
Yinnon
AM
.
Cost-effectiveness of blood cultures for adult patients with cellulitis
.
Clin Infect Dis
1999
;
29
:
1483
8
.

7

Arai
H
,
Ouchi
Y
,
Toba
K
, et al.
Japan as the front-runner of super-aged societies: perspectives from medicine and medical care in Japan
.
Geriatr Gerontol Int
2015
;
15
:
673
87
.

8

Statics Bureau of Japan
.
Basic complete tabulation on population and households of the 2020 population census of Japan
. https://www.stat.go.jp/english/info/news/20211228.html. Accessed 4 June 2022.

9

Taniguchi
T
,
Tsuha
S
,
Takayama
Y
,
Shiiki
S
.
Shaking chills and high body temperature predict bacteremia especially among elderly patients
.
SpringerPlus
2013
;
2
:
624
.

10

Taniguchi
T
,
Tsuha
S
,
Shiiki
S
,
Narita
M
.
Gram-stain-based antimicrobial selection reduces cost and overuse compared with Japanese guidelines
.
BMC Infect Dis
2015
;
15
:
458
.

11

Ota
K
,
Oba
K
,
Fukui
K
, et al.
Sites of blood collection and topical antiseptics associated with contaminated cultures: prospective observational study
.
Sci Rep
2021
;
11
:
6211
.

12

Maeshiro
M
,
Izutsu
S
,
Connolly
KK
.
Medical school hotline: a history of the University of Hawai'i Postgraduate Medical Education Program at Okinawa Chubu Hospital, 1966–2012
.
Hawaii J Med Public Health
2014
;
73
:
191
4
.

13

Swartz
MN
.
Cellulitis
.
N Engl J Med
2004
;
350
:
904
12
.

14

Bauer
S
,
Aubert
CE
,
Richli
M
,
Chuard
C
.
Blood cultures in the evaluation of uncomplicated cellulitis
.
Eur J Intern Med
2016
;
36
:
50
6
.

15

Paolo
WF
,
Poreda
AR
,
Grant
W
,
Scordino
D
,
Wojcik
S
.
Blood culture results do not affect treatment in complicated cellulitis
.
J Emerg Med
2013
;
45
:
163
7
.

16

Kiyoyama
T
,
Tokuda
Y
,
Shiiki
S
,
Hachiman
T
,
Shimasaki
T
,
Endo
K
.
Isopropyl alcohol compared with isopropyl alcohol plus povidone-iodine as skin preparation for prevention of blood culture contamination
.
J Clin Microbiol
2009
;
47
:
54
8
.

17

Fukuyama
H
,
Yamashiro
S
,
Kinjo
K
,
Tamaki
H
,
Kishaba
T
.
Validation of sputum Gram stain for treatment of community-acquired pneumonia and healthcare-associated pneumonia: a prospective observational study
.
BMC Infect Dis
2014
;
14
:
534
.

18

Taniguchi
T
,
Tsuha
S
,
Shiiki
S
,
Narita
M
.
Point-of-care urine Gram stain led to narrower-spectrum antimicrobial selection for febrile urinary tract infection in adolescents and adults
.
BMC Infect Dis
2022
;
22
:
198
.

19

Taniguchi
T
,
Tsuha
S
,
Shiiki
S
,
Narita
M
.
Point-of-care cerebrospinal fluid Gram stain for the management of acute meningitis in adults: a retrospective observational study
.
Ann Clin Microbiol Antimicrob
2020
;
19
:
59
.

20

Lee
CY
,
Kunin
CM
,
Chang
C
,
Lee
SS
,
Chen
YS
,
Tsai
HC
.
Development of a prediction model for bacteremia in hospitalized adults with cellulitis to aid in the efficient use of blood cultures: a retrospective cohort study
.
BMC Infect Dis
2016
;
16
:
581
.

21

Tay
EY
,
Thirumoorthy
T
,
Pang
SM
,
Lee
HY
.
Clinical outcomes of bacteraemia in cellulitis of the leg
.
Clin Exp Dermatol
2014
;
39
:
683
8
.

22

Peralta
G
,
Padrón
E
,
Roiz
MP
, et al.
Risk factors for bacteremia in patients with limb cellulitis
.
Eur J Clin Microbiol Infect Dis
2006
;
25
:
619
26
.

23

Collazos
J
,
de la Fuente
B
,
de la Fuente
J
, et al.
Factors associated with sepsis development in 606 Spanish adult patients with cellulitis
.
BMC Infect Dis
2020
;
20
:
211
.

24

Carratala
J
,
Roson
B
,
Fernandez-Sabe
N
, et al.
Factors associated with complications and mortality in adult patients hospitalized for infectious cellulitis
.
Eur J Clin Microbiol Infect Dis
2003
;
22
:
151
7
.

25

Quirke
M
,
Ayoub
F
,
McCabe
A
, et al.
Risk factors for nonpurulent leg cellulitis: a systematic review and meta-analysis
.
Br J Dermatol
2017
;
177
:
382
94
.

26

Sasaki
Y
,
Taniguchi
T
,
Kinjo
M
, et al.
Meningitis associated with strongyloidiasis in an area endemic for strongyloidiasis and human T-lymphotropic virus-1: a single-center experience in Japan between 1990 and 2010
.
Infection
2013
;
41
:
1189
93
.

27

Okajima
R
,
Oliveira
ACP
,
Smid
J
,
Casseb
J
,
Sanches
JA
, Jr
.
High prevalence of skin disorders among HTLV-1 infected individuals independent of clinical status
.
PLoS Negl Trop Dis
2013
;
7
:
e2546
.

28

Pfitzenmeyer
P
,
Decrey
H
,
Auckenthaler
R
,
Michel
JP
.
Predicting bacteremia in older patients
.
J Am Geriatr Soc
1995
;
43
:
230
5
.

29

Rantala
S
.
Streptococcus dysgalactiae subsp. equisimilis bacteremia: an emerging infection
.
Eur J Clin Microbiol Infect Dis
2014
;
33
:
1303
10
.

This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs licence (https://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial reproduction and distribution of the work, in any medium, provided the original work is not altered or transformed in any way, and that the work is properly cited. For commercial re-use, please contact [email protected]

Comments

0 Comments
Submit a comment
You have entered an invalid code
Thank you for submitting a comment on this article. Your comment will be reviewed and published at the journal's discretion. Please check for further notifications by email.