Evaluation of Digital Auscultation to Diagnose

Original Article

Evaluation of Digital Auscultation to Diagnose Pneumonia in Children 2 to 35 Months of Age in a Clinical Setting in Kathmandu, Nepal: A Prospective Case–Control Study C. G. Scrafford1 S. C. Basnet2 I. Ansari3 S. K. Khatry5 W. Checkley4,6 S. Basnet2 V. Todi3 J. M. Tielsch7

L. Shrestha2 S. Shrestha3 R. Ghimire2 J. Katz4 M. Shrestha2 S. B. Thapa2 P. Kansakar3 S. Puree3

1 Health Sciences, Exponent, Inc., Washington, District of Columbia,

United States 2 Department of Pediatrics, Institute of Medicine, Tribhuvan University, Kathmandu, Nepal 3 Department of Pediatrics, Patan Hospital, Lalitpur, Kathmandu, Nepal 4 Department of International Health, Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University, Baltimore, Maryland, United States 5 Nepal Nutrition Intervention Project, Sarlahi, Kathmandu, Nepal 6 Division of Pulmonary and Critical Care, Johns Hopkins University, Baltimore, Maryland, United States 7 Department of Global Health, George Washington University, Washington, District of Columbia, United States

Address for correspondence Carolyn G. Scrafford, PhD, MPH, Health Sciences, Exponent, Inc., 1150 Connecticut Avenue NW, Washington, DC 20036, United States (e-mail: [email protected]).

J Pediatr Infect Dis 2016;11:28–36.

Abstract

Keywords

► auscultation ► children ► pneumonia

Objective The objective of this study was to determine the diagnostic validity of digital chest auscultation to improve the differentiation of chest sounds associated with pneumonia in children. Methods This is a prospective case–control study at two hospitals in Nepal. Cases had World Health Organization-defined pneumonia and were classified as radiologically confirmed or nonconfirmed based on radiographic findings. Controls had no respiratory complaints. The presence of crepitations in recorded lung sounds defined pneumonia. Radiologically confirmed pneumonia was the reference standard. Results Sensitivity and specificity of digital auscultation were 56% (95% confidence interval [CI], 40–70%) and 73% (95% CI, 70–76%), respectively. Conclusion Digital auscultation in conjunction with standardized grading of digital lung sounds has the potential to improve the specificity of pneumonia diagnosis, but further development of objective interpretation of lung sounds is needed.

Background Pneumonia was the cause of 922,000 child deaths or 15% of all deaths in children younger than 5 years of age in 2015,1 yet the ability to accurately diagnose this condition is limited. Pneumonia can be diagnosed through a physical examination including

received July 6, 2016 accepted after revision September 6, 2016 published online October 21, 2016

chest auscultation by a trained clinician and, if necessary, chest X-ray or other imaging techniques. In low-resource settings, these advanced tests are not routinely available and the diagnosis of pneumonia is reliant on clinical signs such as tachypnea and lower chest wall indrawing by community health workers using the World Health Organization (WHO) algorithm with

Copyright © 2016 by Georg Thieme Verlag KG, Stuttgart · New York

DOI http://dx.doi.org/ 10.1055/s-0036-1593749. ISSN 1305-7707.

Downloaded by: Karolinska Institutet. Copyrighted material.

28

Evaluation of Digital Auscultation to Diagnose Pneumonia in Children 2 to 35 Months of Age

Methods Study Design and Population This study was designed as a prospective, hospital-based case– control validation study in an urban clinical setting in

29

Kathmandu, Nepal from March 2011 through June 2013. Kathmandu is in the central hill region of Nepal and stands at an elevation of approximately 1,400 m.17 Recruitment of children 2 to 35 months of age occurred at two outpatient pediatric clinics at Tribhuvan University Teaching Hospital (March 29, 2011,–June 30, 2013) and Patan Hospital (February 12, 2012,–June 30, 2013). Nepal is a country where community-based management of respiratory infections is prevalent and access to imaging technology such as radiography and ultrasound as well as trained clinicians is limited. Historically, Nepal has been the site for groundbreaking research in maternal and child health and previous studies have been used to generalize to broader populations within the region and developing world. Therefore, results from this population would be generalizable to other low-resource settings where IMCI is the standard of care for many communities.

Case and Control Definitions Cases of pneumonia, as defined by the IMCI guidelines, had reported difficulty in breathing and/or cough in the 2 weeks that led to seeking care at the clinic and an elevated respiratory rate according to the current WHO age-specific definition for nonsevere pneumonia.2 Cases were further categorized as “radiologically confirmed” pneumonia cases based on a positive radiograph finding. Cases with negative radiographic findings were categorized as “radiologically negative” pneumonia cases with WHO-defined pneumonia ranging from nonsevere to very severe.3 For every two cases, one control was selected among children who presented with nonrespiratory symptoms. The majority of the controls enrolled in this study were attending the clinic for vaccinations (63%) as part of the routine vaccination days held in the two outpatient pediatric clinics, while the remaining controls were there for diarrhea (7.6%), skin infections (7.6%), or other nonrespiratory illnesses including routine checkups (22%). Controls were matched to cases on age in two strata based on the WHO respiratory rate cutoffs (i.e., 2–11 and 12–35 months). Controls provided recordings to understand the natural variability in normal chest sounds and allow for a baseline comparison to children with respiratory illnesses as well as the evaluation of the ability of the device and the listener to accurately classify chest sounds in children with no respiratory symptoms. The sample size for this study was 250 children per each case and control groups for a total of 750 children based on our a priori hypothesis that the specificity of this device will reach a minimum of 80%. Any value less than that would not be meaningful or useful to improve upon the current approach to diagnosing pneumonia in low-resource settings.

Data Collection Information on symptoms, other recent illnesses, and treatment obtained prior to the visit was collected from the child’s caregiver using a medical history questionnaire. The child’s demographic and household information was collected along with questions related to the socioeconomic status of the caregivers such as education level, income, and literacy. Journal of Pediatric Infectious Diseases

Vol. 11

No. 2/2016


limited tools to differentiate from other illnesses such as malaria, asthma, and other bacterial infections.2 The WHO algorithm emphasizes sensitivity (i.e., ability of the device to accurately classify those individuals with the disease that truly have the disease or “true positives”) over specificity (i.e., ability of the device to accurately classify those individuals without the disease that truly do not have the disease or “true negatives”) and current recommendations treat all diagnosed children with antibiotics under the assumption that pneumonia has a bacterial etiology.3 Studies evaluating the Integrated Management of Childhood Illnesses (IMCIs) program guidelines show the algorithm to be highly sensitive with low specificity for the classification of respiratory illnesses.4–8 Among these studies, the sensitivity of pneumonia classification was as high as 97% with specificity as low as 49%. Interest in analysis of chest auscultation recordings from a digital stethoscope to classify illness has increased due to the potential for objectivity, decreased reliance on the memory and skill of the examiner, and limiting exposure to radiation from chest X-rays. Acoustic stethoscopes are designed to attenuate high frequencies of sounds, while digital stethoscopes can be designed to amplify these ranges; an important factor to consider in pulmonary auscultation where lung sounds are characterized by a higher frequency spectrum.9 Interpretation of acoustic sounds has been shown to be unreliable between individuals with studies showing poor agreement among independent observers when classifying the presence or absence of abnormal chest sounds as well as the type of adventitious sound.10,11 Computerized acoustic signal analysis is a field that has been advancing with improvements in technology and understanding and is a potential approach to objectively classify auscultatory findings.12 Auscultation is not included in the WHO algorithm and IMCI guidelines. However, if implemented with a standardized approach, we hypothesize digital auscultation has the potential to improve the specificity of diagnosis. One study found that when auscultation was added to rapid breathing and lower chest wall indrawing to diagnose pneumonia, the specificity increased from 67 to 84%.13 Community health workers typically treat cases empirically with antibiotics,4 lacking the resources to distinguish between viral and bacterial pneumonia. A more specific diagnostic tool, that is, one that can identify those children who truly do not have pneumonia and do not require treatment, could result in decreased use of antibiotics to treat nonbacterial respiratory illness in the community and may reduce the high prevalence of antibiotic resistance seen throughout the developing world.14–16 The primary objective of this study was to determine the ability of digital auscultation combined with standardized interpretation to accurately classify pneumonia using radiographic pneumonia as a gold standard in young children in a low-resource setting.

Scrafford et al.

Evaluation of Digital Auscultation to Diagnose Pneumonia in Children 2 to 35 Months of Age Information on known or potential risk factors for pneumonia including number of people living in the home, smoking status of adults in the home, and immunization status of the child was collected during the interview. Immunization status was defined as “complete” if the caregiver reported the child has received all age-appropriate recommended immunizations. The pneumococcal conjugate vaccine was not included in the immunization schedule in Nepal at the time of this study but the Haemophilus influenzae type b (Hib) vaccine was available with high coverage. The study assistants measured each child’s respiratory rate, blood oxygen saturation level using the Masimo Rad-5v hand-held, batterypowered oximeter with a pediatric probe, height, weight, and axillary temperature. Each child’s respiratory rate was counted twice and if the two counts were > 5 breaths difference, then the study assistant counted a third time. The lowest respiratory rate of the two or three counts was used to determine an elevated respiratory rate and eligibility as a case in the study. Height, pulse, and oxygen saturation were measured in triplicate and the median measurement was used in the data analysis. Body weight was measured once to the nearest 10 g using a digital infant scale (Tanita model BD-585, Arlington Heights, Illinois, United States). The attending pediatrician examined the child to provide routine care in addition to recording their own single count of the child’s respiratory rate and the presence of lower chest wall indrawing and WHO-defined danger signs of very severe pneumonia.3 The attending pediatrician’s primary diagnosis of the child based on a routine clinical examination with an acoustic stethoscope was recorded and categorized as either nonrespiratory infections, upper respiratory infections, recurrent wheeze, bronchiolitis, or pneumonia (etiology not specified).

Test Device: Digital Auscultation Techniques The digital recording device used to record chest sounds was a commercially available stethoscope (ThinkLabs, Inc., Electronic model ds32a, Centennial, Colorado, United States).

Scrafford et al.

Study pediatricians collected chest sound recordings at six sites on the chest and back of every child (►Fig. 1). The child was auscultated in either a sitting or lying position. The stethoscope was connected to a mp3 player that recorded the chest sounds. The recordings lasted for 15 seconds at each site for a total of 90 seconds and the resulting sound files were downloaded and passed through a band pass filter with low and high cutoff frequency of 100 and 2,500 Hz, respectively, to remove extraneous noises outside the typical range of chest sounds and amplify the sounds within that range.

Gold Standard: Radiographic Methods Chest radiography was performed on every case and radiographs were independently read by one pediatrician and one radiologist masked to the clinical diagnosis of the child following the methods used by the WHO pneumonia vaccine trial investigator’s group.18 The controls did not receive chest radiographs and were assumed to have negative findings. According to the WHO protocol, radiological pneumonia was defined by the presence of consolidation or pleural effusion at any location on the radiograph. If there was disagreement on radiographic findings between the two readers, the readers discussed the radiograph and came to consensus.

Primary Outcome The findings of the digital auscultation recording review were the primary outcome. Abnormal lung sounds limited to crepitations in any location in the chest (i.e., at any time during the recording) were used to define pneumonia and that child was identified as being “auscultation positive.” Therefore, any child with no finding of crepitations at any location in the chest was identified as “auscultation negative.” Wheeze alone was not considered a pneumonia finding due to the strong association with asthma, bronchiolitis, and viral-induced wheezing. Chest sounds were graded by a pediatrician and a trained nonclinician. These two individuals reviewed the sounds independently and were masked to the clinical diagnosis and case status. Both listeners documented their findings for each 15-second interval within a child indicating one or more of the following: normal, crepitations, wheezes, conductive sounds, sounds not distinguishable. Disagreement in the overall finding of pneumonia within a child between the two listeners was adjudicated.

Statistical Methods

Fig. 1 Order of auscultation recording by digital stethoscope. A 15-second recording was taken at each site. Journal of Pediatric Infectious Diseases

Vol. 11

No. 2/2016

Summary statistics describing the children and their household within each case and control group were derived and associations explored using logistic regression. Odds ratios (ORs) with 95% confidence intervals (CIs) were estimated to compare child characteristics, clinical measures, and household characteristics between the radiologically confirmed and the radiologically negative pneumonia cases to the control. Interlistener agreement was estimated using the kappa (k) coefficient.19 The sensitivity and specificity of the process were estimated along with 95% CIs. Sensitivity was calculated as the number of children with auscultation-positive recordings and radiologically confirmed pneumonia (“true positives”) divided by the total number of


30


31

Results A total of 1,785 children were screened for enrollment from March 29, 2011, through June 30, 2013. More than half were ineligible due to report of cough or difficulty breathing with a low respiratory rate (n ¼ 943), or missing chest radiograph (n ¼ 6), auscultation recording (n ¼ 20), or both (n ¼ 3) (►Fig. 2). Among eligible children, 549 (67.5%) were cases and 264 (32.5%) were controls. Forty-five (8.2%) of cases had radiologically confirmed pneumonia. Fifty-one percent of both cases and controls were 12 to 35 months of age. When compared with controls, radiologically negative pneumonia cases were less likely to be female (OR ¼ 0.69; 95% CI, 0.51–0.95), while the radiologically confirmed cases were more likely to be in the older age group (12–35 months; OR ¼ 2.39; 95% CI, 1.15–5.18) (►Table 1). There were no significant differences in the physical characteristics of the radiologically confirmed pneumonia cases compared with the controls. Clinically, both groups of cases had a significantly higher mean respiratory rate and axillary temperature, a lower blood oxygen saturation level, and more likely to have history of wheeze in the past year compared with the controls based on the child’s caregiver report.

Interlistener Agreement Of the 813 enrolled children, both listeners agreed there were 170 (20.9%) with crepitations (►Table 2). There was moderate agreement between the two listeners (k ¼0.58 [95% CI, 0.52– 0.64]) for interpretation of all recorded sounds (►Table 3). The age of the child did not significantly modify the agreement of the listeners on the interpretation of crepitations heard in the digital recordings (2–11 months: k ¼ 0.56 [95%

Fig. 2 Standards for reporting of diagnostic accuracy flow chart for enrollment and determination of case status. RR, respiratory rate; CXR, chest X-ray. Journal of Pediatric Infectious Diseases

Vol. 11

No. 2/2016


radiologically confirmed pneumonia cases. Specificity was calculated as the number of cases and controls with negative chest X-ray readings and auscultation-negative recordings (“true negatives”) divided by the total number of radiologically negative cases and controls. Controls were included in the validity calculations, as these are the true negative children who will allow for the complete evaluation of the ability of the device and the listener to accurately classify chest sounds in children with no respiratory symptoms. The potential for the transfer of sounds from upper respiratory infections or other environmental background noise to be interpreted as abnormal chest sounds is important to include in this evaluation of the validity of the device in a controlled setting. These analyses were repeated and stratified by age to investigate possible effect modification. The validity of the process when the study population was restricted to cases only (radiologically confirmed and radiologically negative) with the exclusion of the controls was also estimated. Considering the limitations of our gold standard, we also evaluated performance of the device using the attending pediatrician’s primary diagnosis of pneumonia as the gold standard. All statistical analyses used STATA 13.0 (Stat Corp, College Station, Texas, United States). The study protocol for this research project was approved by the Institutional Review Boards at the Johns Hopkins Bloomberg School of Public Health, Johns Hopkins University; Institute of Medicine, Tribhuvan University; and Patan Academy of Health Sciences. Oral informed consent was obtained from the parent or caregiver of the study participants prior to the collection of any data or chest sounds used for the analyses in this study. This study adhered to the Standards for Reporting of Diagnostic Accuracy criteria.20

Scrafford et al.


Scrafford et al.

Table 1 Selected characteristics of radiologically confirmed pneumonia cases, radiologically negative pneumonia cases, and controls Radiologically confirmed pneumonia cases

Radiologically negative pneumonia cases

Controls

Radiologically confirmed vs. controls

Radiologically negative vs. controls

N ¼ 45

N ¼ 504

N ¼ 264

Univariate OR (95% CI)

Univariate OR (95% CI)

19 (42.2)

186 (36.9)

121 (45.8)

0.86 (0.43, 1.71)

0.69 (0.51, 0.95)

Child characteristics Female Age, 12–35 mo

32 (71.1)

248 (49.2)

134 (50.8)

2.39 (1.15, 5.18)

0.94 (0.69, 1.28)

a

6 (13.3)

66 (13.1)

29 (11.0)

1.25 (0.40, 3.33)

1.22 (0.75, 2.02)

b

5 (11.1)

43 (8.5)

20 (7.6)

1.53 (0.42, 4.51)

1.14 (0.64, 2.09)

Body weight (kg) (mean) (SE)

9.1 (0.3)

8.8 (0.1)

9 (0.1)

1.03 (0.89, 1.20)

0.96 (0.89, 1.04)

Exclusively breastfed (up to 6 mo)

19 (44.2)

221 (43.9)

124 (47.3)

0.88 (0.43, 1.77)

0.87 (0.64, 1.19)

Respiratory rate, mean (SE)

58 (1.4)

55 (0.4)

35 (0.6)

1.26 (1.18, 1.35)

1.26 (1.22, 1.31)

Blood O2 saturation (%), median (IQR)

96 (93, 97)

96 (94, 98)

98 (98, 99)

0.47 (0.39, 0.58)

0.53 (0.48, 0.59)

Fever ( 100.4°F)

24 (53.3)

136 (27.1)

4 (1.5)

74.3 (22.0, 311)

24.2 (9.02, 90.8)

Chest indrawing

14 (31.1)

127 (25.2)

0 (0)

–

–

Head nodding

4 (8.9)

19 (3.8)

0 (0)

–

–

Refusal/difficulty feeding

3 (6.7)

23 (4.6)

4 (1.5)

4.64 (0.65, 28.3)

3.11 (1.05, 12.5)

Central cyanosis

0 (0)

1 (0.19)

0 (0)

–

–

Lethargy/unconscious

0(0)

5 (0.99)

0 (0)

–

–

Convulsions

0 (0)

0 (0)

0 (0)

–

–

Recent antibiotic use (past 2 wk)

15 (33.3)

88 (17.5)

3 (1.1)

43.5 (11.1, 242)

18.4 (5.98, 91.7)

Incomplete immunizationc

1 (2.2)

15 (3.0)

3 (1.1)

1.98 (0.04, 25.2)

2.67 (0.75, 14.5)

History of hospitalization for pneumonia in prior year

7 (15.6)

83 (16.5)

22 (8.3)

2.03 (0.68, 5.33)

2.17 (1.30, 3.74)

History of wheeze in prior year

18 (40.0)

240 (47.6)

54 (20.5)

2.59 (1.24, 5.29)

3.54 (2.47, 5.09)

Stunting Wasting

Clinical examination

Medical history

Abbreviations: CI, confidence interval; IQR, interquartile range; OR, odds ratio; SE, standard error. a Stunting ¼ Moderate and severe (< 2 standard deviations [SD] below median weight for height); z-scores calculated based on the WHO Child Growth Standards (2006). b Wasting ¼ Moderate and severe (< 2 SD below median height for age); z-scores calculated based on the WHO Child Growth Standards (2006). c Based on age-appropriate immunization schedule.

CI, 0.47–0.65]; 12–35 months: k ¼ 0.59 [95% CI, 0.51–0.68]) (►Tables 2 and 3).

Sensitivity and Specificity Among the radiologically confirmed pneumonia cases, 55.6% (25/45) were found to have crepitations (i.e., auscultation positive) compared with 27.1% (208/768) of the radiologically negative cases and controls (►Fig. 2 and ►Table 4). Within the radiologically confirmed pneumonia cases, 50% (16/32) of the older children had crepitations compared with 69% (9/13) of the younger children (►Tables 4). Among the radiologically negative pneumonia cases and controls, 30% (116/382) of the Journal of Pediatric Infectious Diseases

Vol. 11

No. 2/2016

older children had crepitations compared with 24% (92/386) of the younger children. The sensitivity and specificity of the digital auscultation combined with standardized interpretation were 56% (95% CI, 40–70%) and 73% (95% CI, 70–76%), respectively (►Table 5). When controls were excluded from the analysis, the sensitivity of this process was the same (56%; 95% CI, 40–70%), but the specificity was lower at 59% (95% CI, 55–63%) (►Table 5). No statistical difference in sensitivity and specificity was observed between the two age groups. In the analysis of the attending pediatrician’s primary diagnosis as the gold standard, the specificity increased to 84% (95% CI, 81–87%) (►Table 5).


32


Listener #2

Listener #1 No crepitations

Crepitations

Total

Children 2–35 mo No crepitations

501

76

577 (0.71)

Crepitations

66

170

236 (0.29)

Total

567 (0.7)

246 (0.3)

813


262

33

295 (0.74)

Crepitations

34

69

103 (0.26)

Total

296 (0.74)

102 (0.26)

398


239

43

282 (0.68)

Crepitations

32

101

133 (0.32)

Total

271 (0.65)

144 (0.35)

415

Discussion Summary of Findings This study evaluates a device with the potential to improve upon the current approach to diagnose pneumonia in lowresource community settings and provides a collection of recorded chest sounds from young children for further objective computerized analysis. Based on the results of this study, digital auscultation with standardized interpretation is more specific than many previously measured performance estimates of the WHO algorithm for classifying pneumonia.4–8 However, subjectivity in human classification of chest sounds remains. Interlistener agreement statistics shows only moderate agreement in the human interpretation of recorded chest sounds, even with standardized training and removal of

Table 3 Interlistener agreement statistics for presence of crepitations in auscultation recordings % Agreement

Overall

Kappa (95% CI)

Positive

Negative

88%

88%

83%

0.58 (0.52–0.64)

2–11 mo

67%

89%

83%

0.56 (0.47–0.65)

12–35 mo

73%

86%

82%

0.59 (0.51–0.68)

Total Age group

Abbreviation: IQR, interquartile range.

33

potential biases such as the clinical examination of the child. Research on computerized analysis of sounds collected in this study may help improve test performance statistics. The results suggest that standardized human interpretation of auscultation is promising but alone may not be sufficient to diagnose pneumonia in a community setting.

Key Considerations and Limitations The prevalence of radiologically confirmed pneumonia among the cases in our study population was low. Differences in our study population may help explain this low prevalence compared with ranges in the literature21,22; recruitment occurred in outpatient pediatric departments where children with upper respiratory illnesses and asthma were likely to seek care along with less severe cases of pneumonia; a significant proportion of children had been treated with antibiotics before presenting for care at the participating clinics; and Nepal introduced Hib vaccine in 2009 resulting in a widely perceived drop in consolidated pneumonia cases. Although the efficacy of the Hib vaccine for Hib pneumonia is unknown, a recent meta-analysis23 reported its efficacy against radiologically confirmed consolidated pneumonia to range from 15 to 28%. Further, the definition of pneumonia in the current study requires all cases to have fast breathing as defined by the WHO guidelines. Therefore, children without fast breathing but with cough and who may have had chest indrawing (i.e., severe pneumonia) or another respiratory danger sign (i.e., very severe pneumonia) were excluded as cases in the study population. Review of the cases in the current study population for the presence of danger signs of severe pneumonia indicates a relatively small proportion of children in both the radiologically negative and positive case groups. Among the radiologically negative pneumonia cases, severe pneumonia indicated by chest indrawing was observed in 25.2% of the children compared with 31.1% of the radiologically confirmed pneumonia cases (p ¼ 0.38). Signs of very severe pneumonia were observed in 48 children (9.5%) among the radiologically negative pneumonia cases compared with seven children (15.6%) among the radiologically confirmed cases. Hypoxia defined as < 90% blood oxygen saturation was measured in eight children (1.6%) in the radiologically negative pneumonia group. The potential exclusion of these children without elevated respiratory rate, who could be the more severe cases and likely to have positive chest radiographs, would result in a lower proportion of radiologically confirmed pneumonia. Consolidated pneumonia is strongly associated with bacterial pneumonia,24 and therefore, was used as the gold standard in our study. We did not collect specimens such as blood, nasopharyngeal swabs, sputum, or lung aspirates from the children in this study to allow confirmation of bacterial pneumonia. The diagnostic accuracy of chest radiography when used as the gold standard for clinical signs and symptoms of pneumonia in children is low to moderate ranging from 10 to 69% of chest radiographs showing pneumonia using signs such as tachypnea, fever, pulse oximetry, and other clusters of pulmonary findings.13,25–27 In a pneumococcal vaccine trial among children < 30 months of age in the Journal of Pediatric Infectious Diseases

Vol. 11

No. 2/2016


Table 2 Agreement between standardized listeners for pneumonia based on auscultation recordings

Scrafford et al.


Scrafford et al.

Table 4 Agreement between digital auscultation device and chest radiography for end point pneumonia Digital auscultation

Chest radiography

Total

Positive

Negative

Positive

25 (0.56)

208 (0.27)

233 (0.29)

Negative

20 (0.44)

560 (0.73)

580 (0.71)

Total

45

768

813

Positive

9 (0.69)

92 (0.24)

101 (0.25)

Negative

4 (0.31)

294 (0.76)

298 (0.75)

Total

13

386

399

Positive

16 (0.50)

116 (0.30)

132 (0.32)

Negative

16 (0.50)

266 (0.70)

282 (0.68)

Total

32

382

414

Children 2–35 mo

Children 2–11 mo

Children 12–35 mo

Gambia, crepitations were seen more frequently among children with consolidation or other infiltrates/abnormalities than children with no radiographic findings; however, there were still 29.4% of children with crepitations who had negative radiographic findings.24 In the present study, crepitations were detected in 41% of children enrolled with WHO-defined pneumonia but negative radiographic findings. The specificity of the digital auscultation combined with standardized interpretation focusing on this group of children with controls excluded was lower at 59% (►Table 5). Radiograph timing and accuracy may affect the performance of the process. A retrospective study in adults showed that five out of nine patients with an initial negative chest radiograph developed infiltrates within 48 hours of admission for pneumonia.28 The radiographs in the current study were taken the same day as auscultation and since this study was conducted in outpatient clinics, most children were not admitted. Thus, we were not able to follow-up with subsequent radiographs to investigate this issue. In addition, recent research using ultrasound imaging techniques to diagnose

pneumonia has shown a significant proportion of consolidated pneumonia that can be missed if diagnosis is based on radiography due to under detection of small consolidations (< 1 cm).29 The lack of sensitivity of the gold standard in this study would attenuate the sensitivity and specificity estimates of the auscultation device by systematic misclassification of the true pneumonia cases in our study population.

Areas for Future Research This study relied on human interpretation of lung sounds from small infants and children who were often crying during recording. The recordings further contained background noises from a crowded clinic as is typical in these settings. Many sounds associated with the device itself were difficult to discern from true biological sounds. These included static and movement noise when the stethoscope diaphragm was moved on the skin which may have often been incorrectly interpreted as a crepitation and thus affect the true measured performance of the process. The adjudication process aimed to facilitate the standardization and definition of these

Table 5 Digital auscultation device performance characteristics with radiologically confirmed pneumonia and pediatrician’s diagnosis as a reference standard Age group

a

Radiologically confirmed pneumonia

Pediatrician’s diagnosisa

Cases and controls

Cases only

Cases and controls

Cases only

% (95% CI)

% (95% CI)

% (95% CI)

% (95% CI)

Sensitivity

Specificity

Sensitivity

Specificity

Sensitivity

Specificity

Sensitivity

Specificity

2–35 mo (n ¼ 813)

56 (40, 70)

73 (70, 76)

56 (40, 70)

59 (55, 63)

54 (48, 60)

84 (81, 87)

54 (48, 60)

69 (64, 75)

2–11 mo (n ¼ 399)

69 (39, 91)

76 (72, 80)

69 (39, 91)

64 (58, 70)

49 (40, 58)

85 (80, 89)

49 (40, 58)

71 (63, 78)

12–35 mo (n ¼ 414)

50 (32, 68)

70 (65, 74)

50 (32, 68)

54 (47, 60)

57 (49, 65)

84 (79, 88)

57 (49, 65)

67 (58, 75)

Attending pediatrician’s diagnosis of pneumonia (type not specified).

Journal of Pediatric Infectious Diseases

Vol. 11

No. 2/2016


34


Conclusion This study suggests that subjectivity in interpretation of chest sounds for the diagnosis of pneumonia remains even when based on standardized and trained listeners of digital lung sounds. The performance of the device when using standardized human interpretation fell below the goal of 80% specificity to diagnose pneumonia among young children, the level determined a priori to be the minimum level at which this device would be useful and could improve upon the current

35

approach to community-based management of pneumonia. Research is currently underway to increase the objectivity of the device through the use of computerized digital signal analysis. With improved objectivity and automation of a device similar to the device presented in this study, there is a potential to improve upon the current algorithm for treatment of children with respiratory infections in low-resource settings where access to trained clinicians and imaging technology is limited or nonexistent and management by community health workers is the standard of care.

Competing Interest The authors declare that they have no competing interest. Authors’ Contribution C.G.S. designed the study and data collection instruments, coordinated and supervised data collection at the two sites, reviewed the sound files, performed the initial analyses, and drafted the initial article. J.K. and J.M.T. conceptualized and designed the study, reviewed and finalized the data collection instruments, and reviewed and revised the article. J.M.T. obtained the funding for the study. S.B. coordinated data collection, served as a study pediatrician at the Institute of Medicine, Tribhuvan University, reviewed the sound files and radiographs, and reviewed and revised the article. L.S. coordinated data collection, served as a study pediatrician at the Institute of Medicine, Tribhuvan University, and reviewed and revised the article. S.S. and I.A. served as study pediatricians at the Patan Hospital, and reviewed and revised the article. R.G. reviewed the radiographs, and reviewed and revised the article. S.K.K. coordinated and supervised staff at the two sites and reviewed the article. W.C. reviewed and finalized the data collection instruments, participated in study design, and reviewed and revised the article. S.B., M.S., and S.B.T. served as study pediatricians at the Institute of Medicine, Tribhuvan University and reviewed and revised the article. P.K., S.P., and V.T. served as study pediatricians at the Patan Hospital, and reviewed and revised the article. All authors read and approved the final article. Acknowledgments This study was supported by the Bill and Melinda Gates Foundation (OPP1017682) and the National Institutes of Health Training Grant (T32HD046405). The funding sources played no role in the study design, data analysis, writing of the report, or decision to submit the article for publication.

References 1 World Health Organization. Pneumonia. Fact Sheet No. 331.

Available at: http://www.who.int/mediacentre/factsheets/fs331/ en. Accessed July 6, 2016 2 World Health Organization (WHO). Technical Bases for the WHO Recommendations on the Management of Pneumonia in Children at First-Level Health Facilities. Geneva: World Health Organization; 1991 Journal of Pediatric Infectious Diseases

Vol. 11

No. 2/2016


extraneous sounds. Focusing on sounds that were part of the inhale/exhale breathing pattern was intended to help limit this misclassification. However, chance-corrected interlistener agreement, while moderate (k ¼ 0.58), points to residual subjectivity in the human interpretation of chest sounds even after training and standardization. It is important to note that there is current research underway that seeks to address extraneous sounds from busy clinic settings. A recent analysis of digital pediatric lung sounds collected in a noisy clinic in West Africa reported on an algorithm to subtract out extraneous noise while preserving the lung sounds.30 This work will help minimize human subjectivity in interpretation of lung sounds. Eighty-nine percent (208/233) of children with crepitations had normal radiographic findings (i.e., lack of consolidation or pleural effusion). Among these 208 children, 62% (128/208) had pediatrician-diagnosed pneumonia indicating substantial disagreement between the pediatrician’s diagnosis and the radiographic findings. The use of antibiotics in 2 weeks prior to the child’s visit does not explain this lack of radiographic findings; the proportion of children with radiologically confirmed pneumonia that reported use was significantly higher compared with the radiologically negative cases (33 vs. 17%; 2 p-value ¼ 0.016) and the mean duration taken was similar between the two case groups (3.7 days). A prior history of wheeze in the past year as reported by the child’s caregiver was significantly higher among both case groups compared with the controls indicating that many of those children with WHO-defined signs of pneumonia may actually have asthma, bronchiolitis, viral pneumonia, or recurrent wheeze. Among the cases, a slightly higher proportion with negative radiographic findings reported history of wheeze, though not statistically significant. The characteristics of the children with findings of crepitations in the auscultation recordings but normal radiographs need to be further investigated to understand the risk factors and predictors for abnormal chest sounds in this group of children and what it means in terms of treatment success and health outcomes when managed in a community setting. In lowresource settings with a large proportion of children with nonsevere pneumonia, chest radiographs would not only be unavailable but also unnecessary. Our evaluation of the device when the attending pediatrician’s primary diagnosis was the gold standard resulted in an increased specificity of 84% (95% CI, 81–87%). This suggests the potential for this auscultation device to be beneficial in a community environment where access to a physician and a complete clinical exam is limited.

Scrafford et al.

Evaluation of Digital Auscultation to Diagnose Pneumonia in Children 2 to 35 Months of Age 3 World Health Organization (WHO). Pocketbook of Hospital Care

4

5

6

7

8

9

10

11 12 13

14

15

16

17

for Children: Guidelines for the Management of Common Childhood Illnesses, 2nd ed. Geneva: World Health Organization; 2013 Gove S; The WHO Working Group on Guidelines for Integrated Management of the Sick Child. Integrated Management of Childhood Illness by outpatient health workers: technical basis and overview. Bull World Health Organ 1997;75(Suppl 1):7–24 Weber MW, Mulholland EK, Jaffar S, Troedsson H, Gove S, Greenwood BM. Evaluation of an algorithm for the Integrated Management of Childhood Illness in an area with seasonal malaria in the Gambia. Bull World Health Organ 1997;75(Suppl 1):25–32 Perkins BA, Zucker JR, Otieno J, et al. Evaluation of an algorithm for Integrated Management of Childhood Illness in an area of Kenya with high malaria transmission. Bull World Health Organ 1997;75 (Suppl 1):33–42 Simoes EA, Desta T, Tessema T, Gerbresellassie T, Dagnew M, Gove S. Performance of health workers after training in Integrated Management of Childhood Illness in Gondar, Ethiopia. Bull World Health Organ 1997;75(Suppl 1):43–53 Kolstad PR, Burnham G, Kalter HD, Kenya-Mugisha N, Black RE. The Integrated Management of Childhood Illness in western Uganda. Bull World Health Organ 1997;75(Suppl 1):77–85 Grenier MC, Gagnon K, Genest J Jr, Durand J, Durand LG. Clinical comparison of acoustic and electronic stethoscopes and design of a new electronic stethoscope. Am J Cardiol 1998;81(5):653–656 Brooks D, Thomas J. Interrater reliability of auscultation of breath sounds among physical therapists. Phys Ther 1995;75(12): 1082–1088 Pasterkamp H, Wiebicke W, Fenton R. Subjective assessment vs computer analysis of wheezing in asthma. Chest 1987;91(3):376–381 Earis J, Cheetham B. Current methods used for computerized sound analysis. Eur Respir Rev 2000;10(77):586–590 Palafox M, Guiscafré H, Reyes H, Munoz O, Martínez H. Diagnostic value of tachypnoea in pneumonia defined radiologically. Arch Dis Child 2000;82(1):41–45 Mastro TD, Nomani NK, Ishaq Z, et al. Use of nasopharyngeal isolates of Streptococcus pneumoniae and Haemophilus influenzae from children in Pakistan for surveillance for antimicrobial resistance. Pediatr Infect Dis J 1993;12(10):824–830 Jain A, Kumar P, Awasthi S. High nasopharyngeal carriage of drug resistant Streptococcus pneumoniae and Haemophilus influenzae in North Indian schoolchildren. Trop Med Int Health 2005;10(3): 234–239 Coles CL, Sherchand JB, Khatry SK, et al. Nasopharyngeal carriage of S. pneumoniae among young children in rural Nepal. Trop Med Int Health 2009;14(9):1025–1033 Ministry of Health and Population (MOHP), Nepal, New ERA, and ICF International Inc. 2012. Nepal Demographic and Health Survey

Journal of Pediatric Infectious Diseases

Vol. 11

No. 2/2016

18

19 20

21

22

23

24

25

26

27 28

29

30

Scrafford et al.

2011. Kathmandu, Nepal: Ministry of Health and Population. New ERA and ICF International, Calverton, Maryland. Cherian T, Mulholland EK, Carlin JB, et al. Standardized interpretation of paediatric chest radiographs for the diagnosis of pneumonia in epidemiological studies. Bull World Health Organ 2005; 83(5):353–359 Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159–174 Bossuyt PM, Reitsma JB, Bruns DE, et al; Standards for Reporting of Diagnostic Accuracy. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. BMJ 2003; 326(7379):41–44 Lynch T, Bialy L, Kellner JD, et al. A systematic review on the diagnosis of pediatric bacterial pneumonia: when gold is bronze. PLoS One 2010;5(8):e11989 Hazir T, Nisar YB, Qazi SA, et al. Chest radiography in children aged 2-59 months diagnosed with non-severe pneumonia as defined by World Health Organization: descriptive multicentre study in Pakistan. BMJ 2006;333(7569):629 Watt JP, Wolfson LJ, O’Brien KL, et al; Hib and Pneumococcal Global Burden of Disease Study Team. Burden of disease caused by Haemophilus influenzae type b in children younger than 5 years: global estimates. Lancet 2009;374(9693):903–911 Enwere G, Biney E, Cheung YB, et al. Epidemiologic and clinical characteristics of community-acquired invasive bacterial infections in children aged 2-29 months in the Gambia. Pediatr Infect Dis J 2006;25(8):700–705 Leventhal JM. Clinical predictors of pneumonia as a guide to ordering chest roentgenograms. Clin Pediatr (Phila) 1982; 21(12):730–734 Zukin DD, Hoffman JR, Cleveland RH, Kushner DC, Herman TE. Correlation of pulmonary signs and symptoms with chest radiographs in the pediatric age group. Ann Emerg Med 1986;15(7): 792–796 Wilkins TR, Wilkins RL. Clinical and radiographic evidence of pneumonia. Radiol Technol 2005;77(2):106–110 Hagaman JT, Rouan GW, Shipley RT, Panos RJ. Admission chest radiograph lacks sensitivity in the diagnosis of community-acquired pneumonia. Am J Med Sci 2009;337(4): 236–240 Shah VP, Tunik MG, Tsung JW. Prospective evaluation of point-of-care ultrasonography for the diagnosis of pneumonia in children and young adults. JAMA Pediatr 2013;167(2): 119–125 Emmanouilidou D, McCollum ED, Park DE, Elhilali M. Adaptive noise suppression of pediatric lung auscultations with real applications to noisy clinical settings in developing countries. IEEE Trans Biomed Eng 2015;62(9):2279–2288


36