An alternative standard for radiological pneumonia in children

Abstract

Background: Although the WHO standard for paediatric radiological pneumonia has been useful in vaccine trials, it lacks sensitivity for some significant pneumonic lesions and interobserver agreement may be limited. We assessed inter-observer variation in interpretation of WHO standardised radiographs and evaluated an alternative standard. Methods: Four clinicians were trained using the first 122/222 WHO classified radiographs. The remaining radiographs were used to assess agreement between clinicians and WHO. The alternative standard used WHO definitions for quality, other infiltrate and pleural effusion, while the alveolar infiltrate category (end-point consolidation, EPC) was separated into two categories: EPC-A (alveolar infiltrate spanning “2 intercostal spaces), and EPC-B (alveolar infiltrate spanning “1 & <2 intercostal spaces). The alternative standard was applied to the first 100 WHO films. Clinicians then undertook a consensus review of the alternative standard using the first 100 films. The alternative standard was then applied to the final 100 WHO films. Prevalence and kappa values were calculated. Results: Kappa for agreement between readers and WHO was 0.55-0.88 (EPC) and 0.16-0.52 (other infiltrate). Inter-observer kappa for EPC was 0.56-0.86 and 0.03-0.53 for other infiltrates. Prevalence of EPC in the first 100 WHO classified films was 23% (right) and 6% (left). Prevalence of EPC-A was 21-32% (right) and 3-8% (left) and prevalence of EPC-B was 2-18% (right) and 1-5% (left). Prevalence of combined EPC-A and B was 24-50% (right) and 4-12% (left). Alternative standard inter-observer kappa was 0.33-0.73 (EPC-A), -0.02-0.38 (EPC-B), and 0.27-0.77 for combined EPC-A and B. Prevalence of EPC among the final 100 WHO classified films was 37% (right) and 10% (left). Prevalence of EPC-A was 27-43% (right) and 6-7% (left) and prevalence of EPC-B was 5-9% (right) and 0-4% (left). Prevalence of combined EPC-A and B was 36-50% (right) and 6-21% (left). Alternative standard inter-observer kappa was 0.47-0.85 (EPC-A), -0.02-0.66 (EPC-B), and 0.43-0.92 for combined EPC-A and B. Conclusion: Compared to WHO, the alternative standard showed greater sensitivity for alveolar infiltrates and similar levels of agreement. Improved agreement with training suggests the alternative standard may be used in a training package. This alternative standard should be evaluated for further use. Abstracts for SupplementInternational Journal of Infectious DiseasesVol. 14Preview Full-Text PDF Open Archive

Related Papers

• Understanding interobserver agreement: the kappa statistic.(2005)
• → Statistical methods in epidemiology. v. Towards an understanding of the kappa coefficient(2000)161 cited
• → The Kappa Coefficient and the Prevalence of a Diagnosis(1988)70 cited
• → Count on kappa(2014)52 cited
• → The Problems with the Kappa Statistic as a Metric of Interobserver Agreement on Lesion Detection Using a Third-reader Approach When Locations Are Not Prespecified(2018)11 cited