Journal Information
Share
Share
Download PDF
More article options
Visits
341
Original Article
Full text access
Available online 10 July 2024
The Accuracy of PUMA Questionnaire in Combination With Peak Expiratory Flow Rate to Identify At-risk, Undiagnosed COPD Patients
Visits
341
Kang-Cheng Sua,b,c, Yi-Han Hsiaoa,c, Hsin-Kuo Koa,c, Kun-Ta Choua,b,c, Tien-Hsin Jengd, Diahn-Warng Pernga,c,
Corresponding author
dwperng@vghtpe.gov.tw

Corresponding author.
a Department of Chest Medicine, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
b Center of Sleep Medicine, Taipei Veterans General Hospital, Taipei, Taiwan, ROC
c School of Medicine, College of Medicine, National Yang Ming Chiao Tung University, Taipei, Taiwan, ROC
d Medical Department, Ditmanson Medical Foundation, Chia-Yi Christian Hospital, Chia-Yi, Taiwan, ROC
Highlights

  • The Chinese PUMA questionnaire (C-PUMA) is linguistically and clinically validated.

  • The best C-PUMA cutoff score is ≥6 (sensitivity/specificity/NNS=77%/64%/3).

  • C-PUMA ≥5 is adequate to detect more COPD patients in high-incidence primary care.

  • The PUMA-PEFR model is more accurate and cost-effective than the C-PUMA alone.

  • The best PCOPD cutoff value is ≥0.39 (sensitivity/specificity/NNS=79%/88%/2).

This item has received
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (5)
Show moreShow less
Tables (4)
Table 1. Reliability and Validity of Translated Chinese PUMA in the Pilot Test and Validation Cohort.
Table 2. Baseline Characteristics in the Validation Cohort.
Table 3. Predictive Performance of Different Diagnostic Modalities in the Validation Cohort.
Table 4. Case Demonstration in the PUMA-PEFR Prediction Model in the Validation Cohort.
Show moreShow less
Additional material (3)
Abstract
Introduction

The English PUMA questionnaire emerges as an effective COPD case-finding tool. We aimed to use the PUMA questionnaire in combination with peak expiratory flow rate (PEFR) to improve case-finding efficacy in Chinese population.

Methods

This cross-sectional, observational study included two stages: translating English to Chinese PUMA (C-PUMA) questionnaire with linguistic validation and psychometric evaluation, followed by clinical validation. Eligible participants (with age ≥40 years, respiratory symptoms, smoking history ≥10 pack-years) were enrolled and subjected to three questionnaires (C-PUMA, COPD assessment test [CAT], and generic health survey [SF-12V2]), PEFR measurement, and confirmatory spirometry. The C-PUMA score and PEFR were incorporated into a PUMA-PEFR prediction model applying binary logistic regression coefficients to estimate the probability of COPD (PCOPD).

Results

C-PUMA was finalized through standard forward–backward translation processes and confirmation of good readability, comprehensibility, and reliability. In clinical validation, 240 participants completed the study. 78/240 (32.5%) were diagnosed with COPD. C-PUMA exhibited significant validity (correlated with CAT or physical component scores of SF-12V2, both P<0.05, respectively). PUMA-PEFR model had higher diagnostic accuracy than C-PUMA alone (area under ROC curve, 0.893 vs. 0.749, P<0.05). The best cutoff values of C-PUMA and PUMA-PEFR model (PCOPD) were ≥6 and ≥0.39, accounting for a sensitivity/specificity/numbers needed to screen of 77%/64%/3 and 79%/88%/2, respectively. C-PUMA ≥5 detected more underdiagnosed patients, up to 11.5% (vs. C-PUMA ≥6).

Conclusion

C-PUMA is well-validated. The PUMA-PEFR model provides more accurate and cost-effective case-finding efficacy than C-PUMA alone in at-risk, undiagnosed COPD patients. These tools can be useful to detect COPD early.

Keywords:
Diagnostic accuracy
Linguistic validation
Peak expiratory flow rate
Prediction model
Predictive performance
PUMA questionnaire
Abbreviations:
C-PUMA
CAT
COPD
DA
FEV1
FVC
MCS
NNS
NPV
PC
PCOPD
PCS
PEFR
PPV
SF-12V2
%PEFR
Full Text
Introduction

Chronic obstructive pulmonary disease (COPD) is the seventh leading cause of poor health and the third leading mortality disease worldwide.1 Despite increasing awareness of COPD burden, COPD remains highly underdiagnosed worldwide, ranging from 65% to 80% in the community,2 and a substantially high proportion of underdiagnosis occurs in primary care (PC).3–5 Underuse of spirometry is the strongest predictor for underdiagnosis.6 Low symptom perception and insufficient awareness of risk factors by the patients and their clinicians are also important contributing factors.2,7 Additionally, a large-scale study (n=30,874) by Lamprecht et al. discovered that COPD underdiagnosis was also associated with male gender, younger age, never and current smoking, lower education, no previous spirometry, and less severe airflow limitation.6 Hence, an effective COPD case-finding strategy to identify at-risk persons for confirmatory spirometry is urgently needed.

Currently, screening for COPD in asymptomatic adults is not recommended by the US Preventive Services Task Force.8,9 However, case-finding for early COPD identification in those with respiratory symptoms or exposure risks is advocated.10 The latter often present to healthcare facilities, particularly in PC, where are the optimal places to catch COPD early. Case-finding tools in PC include questionnaires, handheld devices, or a combination of both.11,12 Symptom-based questionnaires, such as the COPD diagnostic questionnaire,13,14 COPD Population Screener questionnaire (COPD-PS),15,16 and lung function questionnaire,17 are commonly used tools. These questionnaires exerted a poor to fair diagnostic accuracy (DA, indicated by the area under the receiver operating characteristic curve [AUROC]=0.64–0.71), a wide range of sensitivity (63–91%), and specificity (45–70%). Compared with questionnaires, case-finding tools using handheld devices, such as simplified spirometers,14,16 or peak expiratory flow meters18,19 exhibited better DA (AUROC=0.80–0.88), similar sensitivity (76–88%) and higher specificity (72–95%). Additionally, combined questionnaires and handheld devices improved DA (AUROC=0.87–0.91).14,18,19 Taken together, questionnaires are simple and sensitive but less specific case-finding tools. A questionnaire needs linguistic validation before being widely used in people who speak different languages. Applying handheld devices can elevate specificity and reduce the numbers needed to screen (NNS). Different countries should establish their own case-finding strategies.

Recently, the English PUMA questionnaire (7 items, score 0–9, higher scores indicating a higher diagnostic rate) has emerged as a new effective COPD case-finding tool with acceptable DA and predictive performance (AUROC=0.76, sensitivity/specificity/NNS=74%/65%/4 at the cutoff score ≥5) in Latin American countries.20 Additionally, Au-Doung et al. reported that a translated Chinese PUMA version (for Cantonese) exhibited similar performance (AUROC=0.753; sensitivity/specificity=77%/63% at the cutoff score ≥6) in Hong Kong.21 In Taiwan, a recent nationwide telephone interview survey of the general population revealed that COPD was largely underdiagnosed.22 Taiwan lacks an effective, validated Chinese questionnaire for early identification of COPD so far. Thus, Chinese PUMA (for Taiwanese) can be a feasible solution. Additionally, monitoring peak expiratory flow rate (PEFR) is widely used in PC for asthma control in Taiwan. The present study's primary objective was to form a linguistically validated Chinese PUMA (C-PUMA, Mandarin version) and examine its DA with or without the combination of PEFR in Taiwan. The other objectives included the reliability and validity of C-PUMA and the predictive performance of different diagnostic modalities.

MethodsStudy Design

This cross-sectional, observational study was conducted in a medical center in Taiwan, from December 2019 and April 2022. The study comprised two stages (Fig. 1), including linguistic validation and psychometric evaluation based on the guideline23,24 to form the finalized C-PUMA, which was applied for clinical validation in the targeted population (validation cohort). The linguistic validation involved the translation of the English PUMA to the traditional Chinese version and the adaption of Chinese culture. The psychometric evaluation examined the reliability and validity of the translated questionnaire. The targeted population was invited to pulmonary outpatient clinics, where their demographic information, questionnaires of C-PUMA, COPD assessment test (CAT, Chinese version, licensed from the Mapi Research Trust organization), short-form 12-item health survey (SF-12V2, traditional Chinese version 2, licensed from the QualityMetric Inc.), PEFR measurements, and confirmatory post-bronchodilation spirometry were obtained (Fig. 2A). All participants completed the study flow on the same day. This study was approved by the Institutional Review Board (approval number: 2019-05-009CC and 2019-08-007AC). All participants signed informed consents.

Fig. 1.

Framework for linguistic and clinical validation. C-PUMA: the finalized version of the Chinese PUMA questionnaire; EnV1: English version 1; ChV0: Chinese version 0, and so forth; PEFR: peak expiratory flow rate.

(0.44MB).
Fig. 2.

Study flowchart of participants (A) and the distribution of participants categorized by the cutoff values of the PUMA questionnaire and peak expiratory flow rate (B). An abnormal CXR includes overt bronchiectasis (n=6), interstitial lung disease (n=5), pleural effusion (n=2), lobar infiltration (n=4), pneumoconiosis (n=1), and lung mass (n=4). Questionnaires include a COPD assessment test, a Chinese PUMA questionnaire (C-PUMA), and a short-form 12-item health survey, Chinese version 2. PUMA+ and PEFR+ indicate C-PUMA scores ≥6 and PEFR <79% predicted value, respectively.

(0.73MB).
Study Participants

The targeted population was at-risk persons without a previous diagnosis of COPD. They presented to our medical center-affiliated pulmonary clinics because of chronic respiratory symptoms (e.g. cough, dyspnea, phlegm). These persons might come from the community without any referrals or being referred by non-pulmonologists. They were enrolled if they met all the following criteria: aged ≥40 years; current or ex-smokers with a smoking history of ≥10 pack-years, or ≥50pipes/year, or ≥50cigars/year, and/or biomass smoke (wood or coal, for cooking or heating) exposure ≥100h/year; reporting any respiratory symptoms. The exclusion criteria are listed in Appendix A. The PEFR was measured using a Mini-Wright peak flow meter (Micropeak, Micro Medical Limited, Rochester, UK) before bronchodilation according to the ERS recommendations.25 The best PEFR was adopted from three correct blows when participants exerted maximal expiratory efforts in a standing position. Spirometry measurement was in accordance with the standards from the American Thoracic Society/European Respiratory Society (Appendix A). The diagnosis of COPD, stage of airflow limitation, and COPD grouping were based on the 2017 Global Initiative for Chronic Obstructive Lung Disease (GOLD) report.10

Translation and Formation of the Chinese PUMA Questionnaire

The original English PUMA questionnaire (Appendix B) was translated to the Chinese versions using forward–backward translation methods with the assistance of the professional TransPerfect Translations, Inc. (New York, USA). The translation process is illustrated in Fig. 1 and detailed in Appendix A. The final format of C-PUMA was determined after the pilot test.

Reliability and Validity of the Translated PUMA Questionnaire

The reliability was examined by test–retest reproducibility and internal consistency, and the validity was presented as concurrent and predictive validity.24 The details are shown in Table 1. In the pilot test, the confusing wording was revised through the face-to-face feedback from the first 10 respondents, and the test–retest reproducibility was examined in 35 eligible participants who repeated to answer the draft Chinese PUMA (ChV2, Fig. 1) one or two weeks apart. Afterward, the C-PUMA format was finalized. In the validation cohort, the concurrent validity tested the associations of the C-PUMA with accepted standards, such as the CAT, because CAT is a COPD-specific questionnaire representing symptom burdens,10 and CAT was reported to act as a COPD case-finding tool.18,26 The predictive validity indicates the ability of C-PUMA to predict health status, represented by the SF-12V2. The SF-12V2 measures generic health-related quality of life and is presented with physical and mental component summaries (PCS and MCS, respectively; higher scores indicate better health, Appendix A).

Table 1.

Reliability and Validity of Translated Chinese PUMA in the Pilot Test and Validation Cohort.

  Numbers  Statistical Analysis  Coefficient  P-Value  Grade  Remark 
Reliability
Test–retest reproducibilitya35 (pilot test)  Pearson correlation  0.83  <0.001  Very strong  Test 1 correlated with test 2 
35 (pilot test)  Intraclass correlation  0.90  <0.001  Excellent  Test 1 correlated with test 2 
Internal consistencyb35 (pilot test – test 1)  Cronbach's α  0.64  –  Moderate  The average inter-correlation among the C-PUMA items 
35 (pilot test – test 2)  Cronbach's α  0.72  –  High   
240 (validation cohort)  Cronbach's α  0.63  –  Moderate   
Validity
Concurrent validityc  240 (validation cohort)  Spearman rho  0.25  <0.001  Weak  C-PUMA correlated with CAT 
Predictive validityd240 (validation cohort)  Spearman rho  −0.30  <0.001  Weak  C-PUMA correlated with SF-12V2-PCS 
240 (validation cohort)  Spearman rho  0.24  0.711    C-PUMA correlated with SF-12V2-MCS 

CAT: COPD assessment test; MCS: mental component score; PCS: physical component score; SF-12V2: short-form 12-item health survey version 2.

a

The test–retest reproducibility, indicating the temporal stability of the translated PUMA (ChV2), reflected the extent to which the participants’ responses remained consistent on different occasions.

b

The internal consistency examined the homogeneity of the ChV2, which reflected whether the questionnaire items were consistent in measuring the same construct (the probability of being diagnosed with COPD).

c

The concurrent validity tested the associations of the C-PUMA with accepted standards, such as CAT, for COPD-specific symptoms.

d

The predictive validity indicates the ability of C-PUMA to predict health status, such as SF-12V2.

Combined Variables to Generate a COPD Prediction Model

A binary logistic regression model using the enter method was applied to examine the independent variables related to COPD diagnosis and to generate an equation for estimating the probability of COPD (PCOPD). Subsequently, a logit model was generated using the independent variables to calculate PCOPD. The log odds ratio of participants with or without COPD is expressed as follows:

where β0 is the coefficient of the constant, and βi is the coefficient(s) of the independent variable(s) Xi. This equation can be transformed as follows27:
where PCOPD can be directly calculated by computer software, and this model can serve as a case-finding tool.

Statistical Analysis

Based on the recent review for cross-cultural adaption and psychometric validation research,24,28,29 and the epidemiological data from our similar study,18 the sample size was 35 for the pilot test and 240 for the validation cohort (Appendix A). The test–retest reproducibility was examined by the Pearson correlation coefficient and the intraclass correlation coefficient (ICC, deemed excellent between 0.75 and 0.9).29,30 The internal consistency was determined by Cronbach's α coefficient (moderate if >0.5; high if >0.7).29 The concurrent and predictive validity was determined by Spearman's rank correlation coefficient29,31 between the C-PUMA and CAT or SF-12V2, respectively.

Data are presented as means±SD or median (interquartile) or number (%), as appropriate. Continuous variables are compared using a t-test or Mann–Whitney U test. Categorical data were evaluated by a Chi-square test. The performance of different diagnostic modalities was determined and compared using AUROC analysis. An AUROC of 0.6–0.7, 0.7–0.8, 0.8–0.9, and 0.9–1.0 indicates poor, fair, good, and excellent DA, respectively.32 The best cutoff value of different modalities was calculated using the Youden index to determine the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and NNS. Statistical analysis was performed using SPSS Statistics for Windows, Version 20.0 (IBM Corp., Armonk, NY, USA). The comparisons of AUROC values were performed using MedCalc version 17.5.5 (MedCalc Software bvba, Ostend, Belgium). A two-sided P-value <0.05 was considered significant.

ResultsLinguistic Validation and Psychometric Evaluation of C-PUMA

During the translation process, the expert panels selected the plainest Chinese expressions to maximize their readability and comprehensibility. In the pilot test, the first 10 participants indicated the ChV2 was clear, easily understood, and had no comments on its format during face-to-face feedback, and 35 participants confirmed its reliability. Thereby, the C-PUMA format was finalized (Appendix C). The reliability and validity are summarized in Table 1.

Patient Characteristics in the Validation Cohort

Two hundred eighty-six consecutive participants were invited, and 240 completed the study (Fig. 2A) without missing data. Seventy eight of 240 (32%) were newly diagnosed COPD patients, who were categorized by stage I (37, 47%), II (34, 44%), and III (7, 9%) or by group A (60, 77%), B (15, 19%), C (2, 3%) and D (1, 1%), respectively. Compared with non-COPD cases, COPD patients were older and presented higher C-PUMA scores, more symptom burdens (higher CAT and less SF-12-PCS scores), and lower lung function parameters (Table 2). Based on the cutoff values of C-PUMA and PEFR, the participants were categorized into four subgroups: PUMA−/PEFR−, PUMA+/PEFR−, PUMA−/PEFR+, and PUMA+/PEFR+ (Fig. 2B), and their baseline characteristics were shown in Table A1 in Appendix A. The PUMA+/PEFR+ subgroup identified the most COPD patients (43 of 78, 55%) and contained the highest COPD proportion (43 of 49, 87.8%) across these four subgroups. Additionally, the majority of participants (132, 55%) directly came from the community without any referrals, followed by referrals from PC (86, 35.8%) and from non-pulmonary clinics (22, 9.2%) in the same hospital, respectively.

Table 2.

Baseline Characteristics in the Validation Cohort.

  Total  Non-COPD  COPD  P-Valuea 
Numbers (%)  240  162 (68)  78 (32)   
Age, years  64±12  61±11  69±11  <0.001 
Male, n (%)  224 (93)  150 (93)  74 (95)  0.592 
BMI  25.7 (23.7, 28.3)  26 (24.2, 28.7)  25.1 (23.1, 27.3)  0.186 
Current smoker, n (%)  76 (32)  52 (32)  24 (31)  0.118 
Smoking pack-years  34 (19, 50)  30 (16, 42)  41 (26, 56)  0.001 
Biomass exposure, n (%)  16 (6.7)  8 (4.9)  8 (10.3)  0.165 
Questionnaire score
C-PUMA  5 (4, 7)  5 (4, 6)  6 (6, 7)  <0.001 
CAT  4 (2, 6)  3 (1, 6)  4 (3, 8)  <0.001 
SF-12V2 PCS  63 (52, 68)  65 (58, 69)  53 (43, 65)  <0.001 
SF-12V2 MCS  88 (74, 97)  88 (71, 95)  88 (78, 99)  0.184 
Peak flow meter
Best PEFR, L/min  460 (363, 530)  500 (440, 560)  330 (245, 400)  <0.001 
PEFR, % pred.  91 (77, 104)  98 (89, 107)  70 (54, 81)  <0.001 
Pre-BD spirometry
FEV1, L  2.45±0.72  2.77±0.54  1.79±0.58  <0.001 
FEV1, % pred.  92±21  101±15  74±21  <0.001 
FVC, L  3.31±0.77  3.49±0.69  2.93±0.81  <0.001 
FVC, % pred.  92±16  95±13  86±19  <0.001 
FEV1/FVC ratio  74±11  79±61±10  <0.001 
Post-BD spirometry
FEV1, L  2.54±0.71  2.83±0.55  1.93±0.6  <0.001 
FEV1, % pred.  95±20  103±14  79±20  <0.001 
FVC, L  3.42±0.75  3.55±0.69  3.16±0.82  <0.001 
FVC, % pred.  95±16  96±15  93±17  <0.001 
FEV1/FVC ratio  74±11  80±61±10  <0.001 
Bronchoreversibility (+)b  28 (12)  4 (3)  24 (31)  <0.001 

Data are presented as numbers (%) for categorical variables, or median (interquartile range) for continuous non-parametric variables, or mean±SD for continuous parametric variables. BD: bronchodilation; BMI: body mass index; C-PUMA: Chinese PUMA questionnaire; CAT: COPD assessment test; FEV1: forced expiratory volume in first second; FVC: forced vital capacity; MCS: mental component score; PCS: physical component score; PEFR: peak expiratory flow rate; SF-12V2: short-form 12-item health survey questionnaire; % pred.: percent predicted value.

The bold words highlight the items with significant P values.

a

P-value is estimated with Chi-square test for categorical variables, Mann–Whitney U test for continuous non-parametric variables, or t-test for continuous parametric variables, respectively.

b

Positive bronchoreversibility indicates an increase in FEV1 and/or FVC ≥200mL and ≥12% of baseline level after bronchodilation.

Diagnostic Accuracy and Predictive Performance of C-PUMA and PEFR

C-PUMA demonstrated an acceptable DA (AUROC=0.749) (Fig. 3) and the best cutoff score ≥6 (Table 3). Compared with score ≥5, which was the best cutoff score reported by the original English PUMA,20 score ≥6 demonstrated lower sensitivity, higher specificity, less NNS (Table 3), and higher accuracy (true positive plus true negative rate, 68.3% vs. 54.6%), but misdiagnosed more COPD patients up to 11.5% (false negatives, 18 vs. 9, Fig. 4A and B). Looking deeply into the participants’ score distribution (Fig. 4C), 88.5% of COPD patients had scores ≥5, and 84.6% of non-COPD cases presented scores ≤6. Interestingly, a similar portion of COPD and non-COPD cases had their scores overlapped (score 5–6: 43.6% vs. 46.3%; score 4–7: 75.6% vs. 79%). Compared with C-PUMA, PEFR exhibited higher DA (Fig. 3) and predictive performance (Table 3) with the cutoff value of percent predicted PEFR (% PEFR) <79%, which value was recently validated in another similar case-finding study in Taiwan.18

Fig. 3.

Diagnostic accuracy according to the ROC curve analysis. The ROC curve and AUROC value of the three diagnostic modalities in the validation cohorts. *P<0.05, vs. PUMA; #P<0.05 vs. %PEFR. Statistical evaluations were performed using MedCalc. ROC: receiver operating characteristic curve; AUROC: area under the ROC; CI: conference interval; %PEFR: percent predicted peak expiratory flow rate; PCOPD: probability of COPD.

(0.16MB).
Table 3.

Predictive Performance of Different Diagnostic Modalities in the Validation Cohort.

  Sensitivity  Specificity  PPV  NPV  Youden Index  NNS 
PUMA score (C-PUMA)
PUMA ≥4  97  17  36  93  0.15 
PUMA ≥5  88  38  41  87  0.27 
PUMA ≥6a  77  64  51  85  0.41  3 
PUMA ≥7  45  85  58  76  0.29 
PEFR predicted value
%PEFR <79%b  84  86  69  94  0.70 
PUMA-PEFR prediction model
PCOPD ≥0.38  79  87  75  90  0.67 
PCOPD ≥0.39a  79  88  77  90  0.68  2 
PCOPD ≥0.40  77  88  76  89  0.65 

C-PUMA: Chinese PUMA questionnaire; NNS: numbers needed to screen; NPV: negative predictive value; PCOPD: probability of COPD; PEFR: peak expiratory flow rate; PPV: positive predictive value.

The bold words indicate the optimal values determined in this study.

a

Indicates the best cutoff value determined by the Youden index.

b

The cutoff value of %PEFR was based on our recent study, which had an identical study design and similar participants.15

Fig. 4.

Distributions of study participants based on the PUMA cutoff score of 5 (A), 6 (B), and individual PUMA scores from 1 to 9 (C). The proportion of summation numbers in participants with PUMA scores ranging from 5 to 6 and from 4 to 7 in COPD or non-COPD groups is shown in the left upper corner of panel C.

(0.48MB).
Integrate PUMA and PEFR to form a Prediction Model

The independent factors to predict the diagnosis of COPD are listed in Table A2 (Appendix A). The univariate logistic regression indicated that age, smoking pack-years, best PEFR, %PEFR, CAT, C-PUMA, and SF-12V2-PCS scores were significant variables. Among these, collinearity existed in the following pairs: best PEFR-%PEFR, C-PUMA-age, C-PUMA-smoking pack-years, C-PUMA-CAT, C-PUMA-SF-12-PCS. Thus, we adopted the C-PUMA score and %PEFR for multivariate logistic regression, and both variables were statistically independent and remained in the model (Table A1). The logit model, which incorporated PUMA score and %PEFR into a prediction model, namely the PUMA-PEFR model, was expressed as follows:

The aforementioned equation was transformed as follows:

The estimated PCOPD can be readily calculated by entering the two variables into preset computer software, as demonstrated in Table 4. The AUROC comparisons indicated that the PUMA-PEFR model (PCOPD) exhibited the best DA (Fig. 3). The best cutoff value for the PUMA-PEFR model is PCOPD ≥0.39, accounting for high accuracy (85.4%, Appendix A, Fig. A2) and good predictive performance (Table 3).

Table 4.

Case Demonstration in the PUMA-PEFR Prediction Model in the Validation Cohort.

  Independent VariablesPCOPDa  COPD Diagnosis  Post-Bronchodilation Value
  C-PUMA  % PEFR      FEV1/FVC  % FEV1 
Median value of different group
Non-COPD  98  0.102  No  0.80  103 
COPD  70  0.703  Yes  0.61  79 
Demonstration in selected cases from the validation cohort
Case 1  87  0.103  No  0.72  107 
Case 2  49  0.785  Yes  0.64  73 
Case 3  84  0.288  No  0.83  97 
Case 4  79  0.694  Yes  0.61  89 
Case 5  102  0.257  No  0.79  102 
Case 6  66  0.901  Yes  0.69  96 

% PEFR: percent predicted peak expiratory flow rate; PCOPD: probability of COPD.

a

Entering the values of PUMA and %PEFR into a preset computer program immediately calculates the probability of COPD.

Discussion

The present study applied the standard forward–backward translation processes to form the C-PUMA, followed by confirmation of reliability, validity, and clinical performance. C-PUMA alone showed fair DA, which was enhanced by the PUMA-PEFR prediction model. These tools might act as effective COPD case-finding tools in PC.

Linguistic validation and psychometric evaluation are necessary before a translated questionnaire is widely used in the targeted population. Although another Chinese PUMA questionnaire (HK-PUMA, for Cantonese) was reported by Au-Doung PLW et al. in Hong Kong,21 the details of the validation process were not reported. In the present study, excellent test–retest reproducibility (Table 1) indicates that C-PUMA measures a non-biased concept over time, and moderate-to-high internal consistency reflects that each item of C-PUMA constantly links to COPD diagnosis. This process ensured that C-PUMA was qualified before use in clinical validation. Additionally, C-PUMA exhibited weak but significant association with CAT (concurrent validity) and SF-12V2-PCS (predictive validity), indicating C-PUMA potentially links to COPD-specific symptom burdens and generic physical health status, respectively. Therefore, the translated C-PUMA is well-linguistically validated and suitable for use in the Chinese population.

Currently, PUMA has been reported in Latin American countries (original and externally validated English version),20,33 and in Hong Kong (Chinese validation for Cantonese).21 Compared with these PUMA studies (see the details in Table A3, Appendix A), our study had a similar study design except for the two major differences – respiratory symptoms and study settings. It might be concerned that in addition to age and smoke (tobacco or biomass) exposure, this study added respiratory symptoms to the inclusion criteria. The previous PUMA studies usually recruited subjects during routine spontaneous or scheduled appointments unrelated to the study in PC, irrespective of respiratory symptoms. However, the US Preventive Services Task Force recommended against screening for COPD in asymptomatic adults in 2016,8 which was reaffirmed in 2022.9 In contrast, the GOLD guideline recommended case-finding in symptomatic patients.10 The present study was conducted in a medical center, and patients seldom presented to our pulmonologist clinics without respiratory symptoms. Moreover, these symptomatic patients might benefit from timely diagnosis and treatment. This concept was proved in a very recent multicenter, randomized, controlled trial reported by Aaron et al.34 They used a case-finding method to identify adults in the community with respiratory symptoms without a previous diagnosis of COPD or asthma. Patients with confirmed diagnosis was assigned to receive either pulmonologist-directed treatment or usual care for 1 year. They found that the interventional group had less healthcare utilization for respiratory illness in comparison with the usual-care group. Therefore, respiratory symptoms might play an important role while initiating a case-finding strategy for early COPD detection, and these case-finding tool-detected patients had beneficial health outcomes from pulmonologist care.

The other difference is clinical settings. The impact of clinical settings on the case-finding efficacy of any specific questionnaire has not been reported to date. Interestingly, the DA of our hospital-based PUMA study (AUROC 0.749) was similar to that in previously PC-based (AUROC 0.704–0.753) or population-based (AUROC 0.734) studies (Table A3). As to predictive performance, in terms of the best cutoff score/sensitivity/specificity, our data (≥6/77%/64%) are almost identical to that in the HK-PUMA cohort (≥6/76.5%/63.3%).21 However, there are variations in sensitivity (51.5–77%), specificity (62.1–81.6%), PPV (39.1–59.9%), and NPV (63.3–88%) across these PUMA studies (Table A3). Different clinical settings and COPD incidence (18.7–45.1%) are the possible contributing factors. Collectively, PUMA exhibited similar DA but variable performance, indicating PUMA can be reliably used in English and Chinese populations, but different countries/regions must establish their own cutoff value before being widely used.

In the present study, the best C-PUMA cutoff score is ≥6, which had a more cost-effective NNS of 3 (vs. ≥5, NNS=4), but at a cost of missing more COPD patients up to 11.5%. Recently, CH Lin et al. reviewed different case-finding approaches and found that COPD incidence in PC ranged from 10% to 26% in those aged ≥40 years with a smoking history.11 Considering the high incidence of COPD in at-risk patients (e.g., 32% in the present study and 48.8% in our previous report18) and large underdiagnosis in the general population in Taiwan,22 we suggest the C-PUMA cutoff score ≥5 is appropriate for generalized use in PC to reduce COPD underdiagnosis if applying C-PUMA alone. Similarly, the HK-PUMA study also proposed that a cutoff score ≥5, which elevated sensitivity (91.2%) and NPV (92.7%), was more suitable than the Youden index-determined cutoff score ≥6 for their early detection strategy.21 Interestingly, the best English PUMA cutoff score also varied between ≥5 and ≥6 (Table A3).20,33 This might be speculated based on our data that a substantial and equal portion of COPD and non-COPD cases had their scores overlapped between this range. That is, the cutoff score of 5 or 6 has limited discrimination between COPD and non-COPD cases. Therefore, a secondary tool, such as PEFR, can enhance case discrimination in a case-finding approach.

This study found that PEFR was more accurate than C-PUMA, which was consistent with the recent studies that reported PEFR was also better than different questionnaires.18,19,35 However, the cutoff values of PEFR might largely vary across different studies, such as %PEFR <79%18 or <80%,36 PEFR <350, or <250L/min (for males and females, respectively).19 By contrast, COPD-PS was found to be more accurate than PEFR (cutoff value ≤2.2l/sm2).37 These discrepancies might be ascribed to different peak flow meters, clinical settings (PC vs. hospital), and various COPD incidences (17.4–49%) (summarized in Table A4, Appendix A). Additionally, Perez-Padilla et al. reported that %PEFR ≥70% could effectively exclude those with severe airflow limitation (FEV1 <50% predicted).38 Jithoo et al. suggested using PEFR with appropriate cutoff values could effectively reduce confirmatory spirometry numbers.39 Take together, PEFR is a useful case-finding tool but needs validation accordingly.

Combined various questionnaires and PEFR has been reported to serve as effective case-finding approaches in different study designs. In Spain, Soriano et al. reported that a combined modality selecting those with COPD-PS score ≥4 and pre-bronchodilation PEFR ≤2.2l/sm2 exhibited higher DA (AUROC 0.761) than that with COPD-PS (AUROC 0.715) or PEFR (AUROC 0.664) alone. This combined modality can reduce 90% of spirometry tests.37 In the US, Martinez et al. developed the novel CAPTURE questionnaire and classified the risk levels by scores (low: 0–1; middle: 2–4; high: 5–6) or by PEFR (high: males <350L/min, females <250L/min) for identification of clinically significant COPD (those with predicted FEV1 <60% and/or exacerbation risk).19 They reported that the combined modality resulted in the best DA (AUROC 0.9057), followed by PEFR (AUROC 0.8783) and CAPTURE (AUROC 0.7954). They concluded that those with middle scores should measure PEFR and those with high scores or middle scores with low PEFR should undergo spirometry. Identical case-finding protocol using the translated Spanish CAPTURE plus PEFR in those who spoke Spanish replicated identical results.40 Following these case-finding approaches, clinicians should independently evaluate two separate parameters (questionnaire scores and PEFR) to determine who should be subjected to confirmatory spirometry. This approach is likely to miss COPD patients who lack symptom perception (less symptom scores, such as case 2 in Table 4) or have preserved or mildly impaired lung function (less PEFR reduction, such as case 4 in Table 4). A prediction model possibly minimizes this risk. A prediction model can incorporate different factors into a single number, which number represents an integrated evaluation. Various COPD prediction models have been used to predict the risks of underdiagnosis41,42 and the probability of diagnosis.18 Recently, we incorporated age, CAT scores, smoking pack-years, and %PEFR to predict PCOPD.18 This integrated model exhibited the best DA (AUROC 0.866), compared to CAT (AUROC 0.666) or % PEFR (AUROC 0.832) alone. In this study, the integrated PUMA-PEFR model exhibited the best DA again. However, regarding combined questionnaires and PEFR, the evidence of whether a prediction model exerts better case-finding efficacy than evaluating two independent variables is lacking. Clinicians can develop their own combined modality based on available tools.

The strength of this study is that C-PUMA was well-translated and validated before use in clinical validation. There are some limitations. First, the data were driven in a medical center, not in PC. Actually, the healthcare system in Taiwan lacks mandatory referral regulations, and any outpatient has the freedom to visit any specialist in any hospital without a referral.43 Therefore, in the present study, 90.8% of participants came from the community, that is, our data will be very similar to those generated in PC. Nevertheless, validation in genuine PC is required to expand its generalizability. Second, a substantial portion of newly diagnosed COPD patients had mild disease (group A 77% or stage I 47%). Although new evidence has shown that case-finding tool-detected patients might benefit from specialist care,34 the optimal medication choice has not been well-established.

Conclusions

Our data are in line with the conclusions of a recent review by Aaron SD et al. that the use of combined tools has shown better diagnostic accuracy than either tool alone and that the combined use of a questionnaire and a handheld device is a more effective strategy for identifying individuals at increased risk for COPD, although questionnaires used alone are still valuable tools to predict COPD.12 In Taiwan, C-PUMA is well-linguistically and clinically validated. The best cutoff score is ≥6. However, the cutoff score ≥5 is considered appropriate to detect more undiagnosed COPD patients if applying C-PUMA alone. Furthermore, the PUMA-PEFR prediction model exhibits the best DA. Applying these new tools in PC can provide an opportunity to reduce COPD underdiagnosis.

Ethical Approval

This study was approved by the Institutional Review Board of Taipei Veterans General Hospital (approval number: 2019-05-009CC and 2019-08-007AC). All participants signed informed consent.

Funding

This work was supported by AstraZeneca Inc. The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Authors’ Contributions

Conception and design: D.W. Perng, K.C. Su, Y.H. Hsiao; grant application: D.W. Perng and K.C. Su; acquisition, analysis, or interpretation of data: K.C. Su, H.K. Ko, K.T. Chou, Y.H. Hsiao, and T.H. Jeng; drafting of the manuscript: K.C. Su, H.K. Ko, Chou, Y.H. Hsiao, and T.H. Jeng; critical revision of the manuscript for important intellectual content: D.W. Perng. All the authors discussed the results and approved the final version of the manuscript.

Conflicts of Interest

None declared.

Data Availability Statement

The datasets generated or analyzed during this study are available from the corresponding author upon reasonable request.

Prior Abstract Presentation

Poster presentation, 2023 ATS conference, Washington, DC, May 21, 2023

Acknowledgments

The authors thank all study participants. We are grateful to the statistical modeling consultant, Dr. Jack Chen, from the ESTAT Statistical Consulting Co., Ltd., for his assistance with statistical analysis. We also thank TransPerfect Translations, Inc. for their professional language translation and the Mapi Research Trust organization for the kind permission of CAT to use in this study.

Appendix B
Supplementary Data

The following are the supplementary data to this article:

References
[1]
World Health Organization.
Chronic obstructive pulmonary disease (COPD).
(2023),
[2]
N. Diab, A.S. Gershon, D.D. Sin, W.C. Tan, J. Bourbeau, L.P. Boulet, et al.
Underdiagnosis and overdiagnosis of chronic obstructive pulmonary disease.
Am J Respir Crit Care Med, 198 (2018), pp. 1130-1139
[3]
T. Takahashi, M. Ichinose, H. Inoue, K. Shirato, T. Hattori, T. Takishima.
Underdiagnosis and undertreatment of COPD in primary care settings.
Respirology, 8 (2003), pp. 504-508
[4]
M. Bednarek, J. Maciejewski, M. Wozniak, P. Kuca, J. Zielinski.
Prevalence, severity and underdiagnosis of COPD in the primary care setting.
Thorax, 63 (2008), pp. 402-407
[5]
A. Casas Herrera, M. Montes de Oca, M.V. Lopez Varela, C. Aguirre, E. Schiavi, J.R. Jardim, et al.
COPD underdiagnosis and misdiagnosis in a high-risk primary care population in four Latin American countries. A key to enhance disease diagnosis: the PUMA study.
PLoS One, 11 (2016), pp. e0152266
[6]
B. Lamprecht, J.B. Soriano, M. Studnicka, B. Kaiser, L.E. Vanfleteren, L. Gnatiuc, et al.
Determinants of underdiagnosis of COPD in national and international surveys.
Chest, 148 (2015), pp. 971-985
[7]
T. Ho, R.P. Cusack, N. Chaudhary, I. Satia, O.P. Kurmi.
Under- and over-diagnosis of COPD: a global perspective.
Breathe (Sheff), 15 (2019), pp. 24-35
[8]
US Preventive Services Task Force, A.L. Siu, K. Bibbins-Domingo, D.C. Grossman, K.W. Davidson, J.W. Epling Jr., et al.
Screening for chronic obstructive pulmonary disease: US preventive services task force recommendation statement.
JAMA, 315 (2016), pp. 1372-1377
[9]
US Preventive Services Task Force, C.M. Mangione, M.J. Barry, W.K. Nicholson, M. Cabana, A.B. Caughey, et al.
Screening for chronic obstructive pulmonary disease: US preventive services task force reaffirmation recommendation statement.
JAMA, 327 (2022), pp. 1806-1811
[10]
C.F. Vogelmeier, G.J. Criner, F.J. Martinez, A. Anzueto, P.J. Barnes, J. Bourbeau, et al.
Global strategy for the diagnosis, management, and prevention of chronic obstructive lung disease 2017 report. GOLD executive summary.
Am J Respir Crit Care Med, 195 (2017), pp. 557-582
[11]
C.H. Lin, S.L. Cheng, C.Z. Chen, C.H. Chen, S.H. Lin, H.C. Wang.
Current progress of COPD early detection: key points and novel strategies.
Int J Chron Obstruct Pulmon Dis, 18 (2023), pp. 1511-1524
[12]
S.D. Aaron, M. Montes de Oca, B. Celli, S.P. Bhatt, J. Bourbeau, G.J. Criner, et al.
Early diagnosis and treatment of chronic obstructive pulmonary disease: the costs and benefits of case finding.
Am J Respir Crit Care Med, 209 (2024), pp. 928-937
[13]
A.J. Stanley, I. Hasan, A.J. Crockett, O.C. van Schayck, N.A. Zwar.
Validation of the COPD Diagnostic Questionnaire in an Australian general practice cohort: a cross-sectional study.
Prim Care Respir J, 23 (2014), pp. 92-97
[14]
L. Sichletidis, D. Spyratos, M. Papaioannou, D. Chloros, A. Tsiotsios, V. Tsagaraki, et al.
A combination of the IPAG questionnaire and PiKo-6(R) flow meter is a valuable screening tool for COPD in the primary care setting.
Prim Care Respir J, 20 (2011), pp. 184-189
[15]
C.Z. Chen, C.C. Sheu, S.L. Cheng, H.C. Wang, M.C. Lin, W.H. Hsu, et al.
Performance and clinical utility of various chronic obstructive pulmonary disease case-finding tools.
Int J Chronic Obstr, 16 (2021), pp. 3405-3415
[16]
M. Llordés, E. Zurdo, A. Jaén, I. Vázquez, L. Pastrana, M. Miravitlles.
Which is the best screening strategy for COPD among smokers in primary care?.
[17]
N.A. Hanania, D.M. Mannino, B.P. Yawn, D.W. Mapel, F.J. Martinez, J.F. Donohue, et al.
Predicting risk of airflow obstruction in primary care: validation of the lung function questionnaire (LFQ).
Respir Med, 104 (2010), pp. 1160-1170
[18]
K.C. Su, H.K. Ko, K.T. Chou, Y.H. Hsiao, V.Y. Su, D.W. Perng, et al.
An accurate prediction model to identify undiagnosed at-risk patients with COPD: a cross-sectional case-finding study.
NPJ Prim Care Respir Med, 29 (2019), pp. 22
[19]
F.J. Martinez, D. Mannino, N.K. Leidy, K.G. Malley, E.D. Bacci, R.G. Barr, et al.
A new approach for identifying patients with undiagnosed chronic obstructive pulmonary disease.
Am J Respir Crit Care Med, 195 (2017), pp. 748-756
[20]
M.V. Lopez Varela, M. Montes de Oca, A. Rey, A. Casas, R. Stirbulov, V. Di Boscio, et al.
Development of a simple screening tool for opportunistic COPD case finding in primary care in Latin America: the PUMA study.
Respirology, 21 (2016), pp. 1227-1234
[21]
P.L.W. Au-Doung, C.K.M. Wong, D.C.C. Chan, J.W.H. Chung, S.Y.S. Wong, M.K.W. Leung.
PUMA screening tool to detect COPD in high-risk patients in Chinese primary care – a validation study.
PLoS One, 17 (2022), pp. e0274106
[22]
S.L. Cheng, M.C. Chan, C.C. Wang, C.H. Lin, H.C. Wang, J.Y. Hsu, et al.
COPD in Taiwan: a National Epidemiology Survey.
Int J Chron Obstruct Pulmon Dis, 10 (2015), pp. 2459-2467
[23]
D.E. Beaton, C. Bombardier, F. Guillemin, M.B. Ferraz.
Guidelines for the process of cross-cultural adaptation of self-report measures.
Spine (Phila Pa 1976), 25 (2000), pp. 3186-3191
[24]
S. Tsang, C.F. Royse, A.S. Terkawi.
Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine.
Saudi J Anaesth, 11 (2017), pp. S80-S89
[25]
P.H. Quanjer, M.D. Lebowitz, I. Gregg, M.R. Miller, O.F. Pedersen.
Peak expiratory flow: conclusions and recommendations of a Working Party of the European Respiratory Society.
Eur Respir J Suppl, 24 (1997), pp. 2S-8S
[26]
N. Raghavan, Y.M. Lam, K.A. Webb, J.A. Guenette, N. Amornputtisathaporn, R. Raghavan, et al.
Components of the COPD Assessment Test (CAT) associated with a diagnosis of COPD in a random population sample.
[27]
V. Bewick, L. Cheek, J. Ball.
Statistics review 14: logistic regression.
Crit Care, 9 (2005), pp. 112-118
[28]
J.A. Campos, M.C. Spexoto, S.V. Serrano, J. Maroco.
Psychometric characteristics of the Functional Assessment of Cancer Therapy-General when applied to Brazilian cancer patients: a cross-cultural adaptation and validation.
Health Qual Life Outcomes, 14 (2016), pp. 8
[29]
A. Aithal, S. Aithal.
Development and validation of survey questionnaire and experimental data – a systematical review-based statistical approach.
IJMTS, 5 (2020), pp. 233-251
[30]
T.K. Koo, M.Y. Li.
A guideline of selecting and reporting intraclass correlation coefficients for reliability research.
J Chiropr Med, 15 (2016), pp. 155-163
[31]
N.H. Meng, F.N. Chen, S.F. Lo, W.E. Cheng.
Reliability and validity of the Taiwan (Mandarin Chinese) version of the chronic respiratory questionnaire.
Qual Life Res, 20 (2011), pp. 1745-1751
[32]
F. Li, H. He.
Assessing the accuracy of diagnostic tests.
Shanghai Arch Psychiatry, 30 (2018), pp. 207-212
[33]
M.V. Lopez Varela, M. Montes de Oca, F.C. Wehrmeister, C. Rodriguez, L. Ramirez, A. Menezes.
External validation of the PUMA COPD diagnostic questionnaire in a general practice sample and the PLATINO study population.
Int J Chron Obstruct Pulmon Dis, 14 (2019), pp. 1901-1911
[34]
S.D. Aaron, K.L. Vandemheen, G.A. Whitmore, C. Bergeron, L.P. Boulet, A. Cote, et al.
Early diagnosis and treatment of COPD and asthma – a randomized, controlled trial.
N Engl J Med, 390 (2024), pp. 2061-2073
[35]
S.J. Ronaldson, L. Dyson, L. Clark, C.E. Hewitt, D.J. Torgerson, B.G. Cooper, et al.
Determining the optimal approach to identifying individuals with chronic obstructive pulmonary disease: the DOC study.
J Eval Clin Pract, 24 (2018), pp. 487-495
[36]
Y.T. Thorat, S.S. Salvi, R.R. Kodgule.
Peak flow meter with a questionnaire and mini-spirometer to help detect asthma and COPD in real-life clinical practice: a cross-sectional study.
NPJ Prim Care Respir Med, 27 (2017), pp. 32
[37]
J.B. Soriano, J. Molina, M. Miravitlles.
Combining case-finding methods for COPD in primary care: a large, two-stage design study.
Int J Tuberc Lung Dis, 22 (2018), pp. 106-111
[38]
R. Perez-Padilla, W.M. Vollmer, J.C. Vazquez-Garcia, P.L. Enright, A.M.B. Menezes, A.S. Buist, et al.
Can a normal peak expiratory flow exclude severe chronic obstructive pulmonary disease?.
Int J Tuberc Lung Dis, 13 (2009), pp. 387-393
[39]
A. Jithoo, P.L. Enright, P. Burney, A.S. Buist, E.D. Bateman, W.C. Tan, et al.
Case-finding options for COPD: results from the Burden of Obstructive Lung Disease study.
Eur Respir J, 41 (2013), pp. 548-555
[40]
W.A. Quezada, B.A. Whippo, P.A. Jellen, N.K. Leidy, D.M. Mannino, K.J. Kim, et al.
How well does CAPTURE translate? An exploratory analysis of a COPD case-finding method for Spanish-speaking patients.
[41]
S. Haroon, P. Adab, R.D. Riley, D. Fitzmaurice, R.E. Jordan.
Predicting risk of undiagnosed COPD: development and validation of the TargetCOPD score.
Eur Respir J, 49 (2017), pp. 1602191
[42]
D.W. Mapel, H. Petersen, M.H. Roberts, J.S. Hurley, F.J. Frost, J.P. Marton.
Can outpatient pharmacy data identify persons with undiagnosed COPD?.
Am J Manag Care, 16 (2010), pp. 505-512
[43]
T.-Y. Wu, A. Majeed, K.N. Kuo.
An overview of the healthcare system in Taiwan.
London J Prim Care (Abingdon), 3 (2010), pp. 115-119
Copyright © 2024. The Author(s)
Archivos de Bronconeumología
Article options
Tools

Are you a health professional able to prescribe or dispense drugs?