Polish Adaptation and Psychometric Validation of the PREM-C9 Questionnaire Dedicated for Patients with Chronic Obstructive Pulmonary Disease

Introduction: Chronic Obstructive Pulmonary Disease (COPD) is a common, preventable and treatable disease. The first PREM (Patient Reported Outcome Measure) type questionnaire that has been dedicated to asses experience of care in COPD is the PREM-C9. Aim: To create a Polish version of the PREM-C9 and determine its the psychometric characteristics. Methods: The validation procedure involved forward and back translation and included 42 COPD patients. The psychometric properties were assessed using Cronbach's alpha, Bartlett's test, Kaiser-Meyer-Olkin and Spearman correlation coefficient. The validity of the questionnaire was assessed using Principal Component Analysis for the extracted principal components. The validity of the factor analysis was demonstrated by the Bartlett's sphericity test and the Kaiser-Meyer-Olkin (KMO). Factor analysis was performed using the Oblimin and Varimax rotation. The reliability of the questionnaire was assessed using the Cronbach's alpha. Results: The Polish version of the PREM-C9 questionnaire met all validation criteria: face, translation, psychometric, functional and reconstruction equivalence. The Spearman correlation results between Polish PREM-C9&CAT: rho= 0,44, p=0,003539; HADS-Anxiety: rho= 0.370864, p=0,015612, HADS-Depression: rho= 0,387405, p=0,011253. Conclusion: The developed Polish PREM-C9 questionnaire is a reliable and valid tool that assesses Polish COPD patients' experiences of their disease and the care they receive.

Keywords:

Subject: Public Health and Healthcare - Health Policy and Services

1. Introduction

COPD is a common, preventable, and treatable disease. About 384 million people worldwide are affected by COPD. According to the latest GOLD report, COPD is “a heterogeneous lung condition characterized by chronic respiratory symptoms (dyspnea, cough, expectoration, exacerbations) due to abnormalities of the airways (bronchitis, bronchiolitis) and/or alveoli (emphysema) that cause persistent, often progressive, airflow obstruction” [1]. Causes and risk factors of COPD have been recently described with the GETomic acronym, which stands for dynamic, cumulative and repeated gene (G) - environment (E) interactions over the lifetime (T) that damage lungs or change their normal ageing processes [2]. The most important environmental risk factor is cigarette smoking, but this group also includes occupational exposure [3] and air pollution [4].

Commonly, the assessment of clinical effectiveness of interventions uses well-defined endpoints. According to the guidelines of the Polish Agency for Health Technology Assessment, these are endpoints related to mortality, morbidity (endpoints related to the course/exacerbation of the disease), health-related quality of life (HRQoL), unwanted events and adverse effects (divided into severe and other) [5]. However, the same guideline underlines that endpoints relevant from the patient's perspective should also be considered. These endpoints include Patient-Reported Outcome Measures (PROMs) and Patient-Reported Experience Measures (PREMs). The patient's perspective is paramount. Principle 5 of the COPD Patients Charter refers to the patient's right to appropriate specialist care whenever the patient needs it [6]. Improving patient care requires using appropriate tools to assess care for patients with COPD.

According to the Food and Drug Administration (FDA), PROMs can be defined as "any reports of the status of a patient's health condition that comes directly from the patient (without interpretation of the patient's response by a clinician or anyone else)"[7]. Such information is collected through standardized questionnaires. PROMs were originally developed and implemented for use in clinical trials as a way to include the patient's voice in the evaluation of clinical interventions [8]. However, their use has gradually become more common, and today, they are considered an integral part of assessing clinical effectiveness and patient-centred care [9].

PREMs allow to determine what is the patient's perception of their experience with received care. Typically, they are structured questionnaires, but unlike PROMs, they do not assess care outcomes but determine how patients perceive their experience of using care. They can be used as indicators of quality of care (determined from the patients’ perspective) at any level of healthcare system [10] and used by entitled agencies to assess medical technologies in the reimbursement process [11].

Like PROMs, PREMs also deliver information on the quality of care during the intervention, which allows for immediate response and adjustments. Several publications have shown that taking into account the experience of patients is a key factor in strengthening health systems. This data may be used as a basis to identify quality, performance or security issues [12,13].

The first PREM-type questionnaire that has been dedicated to asses experience of care in COPD is the PREM-C9 [14]. It is a simple questionnaire validated for use in the population of patients with mild to severe COPD. However, this tool has not been validated into Polish conditions yet. Therefore, the study aims are:

-: To create a Polish adaptation of the PREM-C9.
-: To determine the psychometric characteristics of the PREM-C9 questionnaire used in Polish conditions.
-: To provide knowledge regarding the characteristics and the role of the PREM-C9 in non-English speaking countries, allowing for cross-cultural comparisons and determining further possibilities of using PREM-C9 and generalizing the results obtained to other populations.

We believe that the Polish validation of the PREM-C9 test will stimulate Polish research on the role of the patient's perspective in the treatment of COPD.

Determining the usefulness of this questionnaire requires an examination of its validity and reliability. Thus, these parameters were assessed in a study on the Polish version of the PREM-C9. Questionnaire validity is understood as the level of agreement with which the assessed questionnaire measures what it was designed for. Reliability is defined as a questionnaire's ability to reflect the true value of the characteristic that has been evaluated [15].

2. Materials and Methods

2.1. The adaptation procedure

The PREM-C9 questionnaire was developed by Hodson et al. [14]. The procedure involved the use of both forward and back translation methods in accordance with ISPOR guidelines [16] and Breslin [17].

Forward translation of the PREM-C9 into Polish by a professional bilingual translator with Polish as a native language.
Reviewing the translated version of the PREM-C9 by the team, including pneumonologists and the research team members. This stage did not provide any significant changes in the Polish version of the questionnaire.
Back translation made a professional bilingual translator with English as a native language.
Comparison of the back-translated version with the original text. This stage did not provide any significant changes; only minor revisions were introduced.
Evaluation of the comprehensibility of the translated version in a group of 20 healthy volunteers who assessed whether each of the questions was understandable to them.
The first use of the PREM-C9 questionnaire in the group of 42 COPD patients was to validate the tool and test its psychometric features. Material was collected from patients by a trained interviewer, or patients answered by themselves by filling out a paper version of the questionnaires. The questionnaire was made available both in paper form and digital version. The use of the electronic version of the questionnaire was caused by the COVID-19 pandemic and resulted from the need to limit interpersonal contacts.

The result of conducting the adaptation procedure was the preparation of the Polish version of the PREM-C9 questionnaire, which was used for further testing of the adapted version. The adaptation procedure is presented in Figure 1, and the translated questionnaire is in Figure 2.

2.1. Participants

The study was approved by the Bioethics Committee of the Medical University of Gdańsk, and consent was obtained from the first author of the validated questionnaire for its use. In addition, permission to use the other questionnaires in the study was obtained as well.

A total of 42 Polish-speaking participants were tested. The inclusion criteria were:

-: a patient diagnosed with COPD according to GOLD criteria from 2022;
-: a patient expressing informed consent to participate in the study;
-: a patient not hospitalized at the time of the study due to COPD exacerbation or other reasons;
-: a patient who is able to answer questions.

Most of the patients participating in the study (n=31; 73.81%) confirmed that dyspnea makes them walk slower than their peers or makes them stop to catch a breath when they walk at their own pace. According to mMRC (Modified British Medical Research Council) [18] this is equivalent to at least grade 2 dyspnoea. The basic characteristics of the study group with their socio-demographic characteristics are presented below - (Table 1.)

2.2. Measures

The questionnaire

PREM-C9 (hereinafter also abbreviated as PC-9) is a simple, 9-item tool of relational PREMs type that has been divided into three sections:

-: my everyday life with COPD (4 questions),
-: usual care in COPD (3 questions),
-: COPD exacerbation/Flare up (2 questions).

The patient evaluates the statements using a graphical Likert scale with the two extreme labels marked - 0 (the lowest score), which describes the best situation and 5 (the highest score) dedicated to the worst situation. In the Polish adaptation and validation of the PREM-C9 questionnaire, we used the same questionnaires as in the original validation process (CATTM and HADS) with one additional general PROM – EuroQol EQ-5D 5-level version (EQ-5D-5L). They are briefly described below.

COPD Assessment Test (CAT^TM) [19] (Polish version) is a disease-specific PROM type questionnaire dedicated to people with COPD that assesses how the disease impacts a person's life and whether it changes over time. It consists of 8 items, and the answering scale is from 0 to 5, where 5 is the worst rating. CAT considers presence and severity of cough, phlegm, sense of tightness in the chest, breathlessness, limitations in performing everyday activities, self-confidence when leaving home, quality of sleep, and life energy level. Five points mean the upper limit of normal in healthy non-smokers, <10 points means that COPD has a minor influence on the patient's life, 10-20 points average impact, >20 severe impact, >30 very severe impact of COPD on the patient's life - the disease makes it impossible to perform any everyday activities, and the patient never has good days. It is worth noting that the construction and graphic layout of PREM-C9 and CATTM are very similar.

Hospital Anxiety and Depression Scale modified version (HADS-m) [20] (Polish version) is a PROM type questionnaire measuring anxiety and depression in patients who are somatically ill. It consists of three subscales - depression (HADS-D), anxiety (HADS-A) and irritability (two questions from the last subscale were not used in the original version and subsequently neither in this study). The whole questionnaire consists of 16 questions, which are evaluated on a 0-3 scale. Obtained results can be interpreted as follows: ≤7 = Normal, from 8 to 10 Borderline abnormal (borderline case), from 11 Abnormal (case). Since in the study, only two subscales (A and D) were used, hence hereafter, the questionnaire will be referred to as HADS (as the modification was the addition of the third unused subscale).

EuroQol EQ-5D 5-level version (EQ-5D-5L) [21] (Polish version) is a general PROM-type questionnaire consisting of 5 questions (considering: mobility, self-care, usual activities, pain and discomfort, anxiety and depression) that can be evaluated in 5 levels: no problems (1 point), slight problems, moderate problems, severe problems and extreme problems (5 points) and visual analogue scale (EQ-VAS) to describe the general health status on the day of attending the study. EQ-5D-5L is available in more than 150 languages and is one of the most frequently used generic questionnaire assessing health-related quality of life (HRQoL) [22].

2.3. Statistical methods and calculations

All calculations were performed using the Microsoft Excel spreadsheet questionnaire, the StatSoft Inc. statistical package Statistica and the SPSS program. In the statistical description of quantitative data, classical measures of position such as arithmetic mean, median, and standard deviation were used as measures of variability. To evaluate the psychometric properties of the PREM-C9 questionnaire, Cronbach's alpha, Bartlett's test of sphericity, Kaiser-Meyer-Olkin coefficient and Spearman correlation coefficient were used. In all statistical tests, p<0.05 was taken as the level of statistical significance of differences.

The validity of the questionnaire was assessed using PCA (Principal Component Analysis) for the extracted principal components. The validity of the factor analysis was demonstrated by the Bartlett's sphericity test and the Kaiser-Meyer-Olkin coefficient (KMO), with p<0.05 and KMO> 0.6 as the cut-off level, respectively. Factor analysis was performed using the Oblimin and Varimax rotation method, assuming 0.4 as the threshold value. The reliability of the questionnaire was assessed using the Cronbach's alpha index, assuming 0.70 as the cut-off value.

3. Results

The Polish language version of the questionnaire was named identically to the original version - PREM-C9. Like the original version, the Polish questionnaire consists of 9 questions divided into three sections. Developing a Polish version was proceeded in accordance with the principle of facade equivalence (test graphics, instructions).

This questionnaire and the Polish versions of the CAT, HADS and EQ-5D-5L questionnaires were given to the patients participating in the study for completion. The results obtained in the study group using the individual tests are presented in Table 2.

In the overall assessment of current health status using the EQ-VAS, a mean score of 60.56% of the ideal value was obtained (Me=60; SD=22.32). The mean scores obtained for the whole group on the CAT scale should be interpreted as the average impact of COPD on patients' lives, which was higher than in the well-controlled patients (CAT>10 points). The results of the HADS scale indicated that, overall, patients did not seem to suffer from depression or anxiety (mean score for the whole group <8 points). However, four respondents had a score greater than or equal to 8 for the HADS-D scale and six patients on the HADS-D scale, which should be interpreted as borderline abnormal or even abnormal values.

Assessment of psychometric characteristics
Validity of the PREM-C9 questionnaire

In order to examine the theoretical validity of the test, each part of the PREM-C9 was subjected to factor analysis. The validity of the factor analysis was proven by Bartlett's sphericity test, with a significant result (p=0.000), which proved the presence of a correlation between the components. Also, the Kaiser-Meyer-Olkin (KMO) test results of >0.5 proved that there were relationships between the components and that they were selected appropriately, so that factor analysis was justified (Table 3).

The PCA (Principal Component Analysis) factor extraction method using the Kaiser normalisation criterion, with both Varimax and Oblimin rotation, identified three components (the same as in the English-language version). In all questions, a correlation value of at least 0.6 was obtained between the individual questions and the questionnaire as a whole, which means that each question in the Polish-language version of the PREM-C9 is valid (contributes relevant content) and should not be removed. The lowest correlation value was obtained for question 4 (I am happy to talk about the future), but this was not below the accepted criterion and was noticeably higher when Oblimin was used as the rotation method.

The next stage of the validity analysis was the convergent relevance analysis, for which Spearman's correlation was used between the PREM-C9 questionnaire assessed and the other questionnaires, i.e. HADS, CAT, EQ-5D-5L and EQ-VAS (Table 4). In each case, a statistically significant (p<0.05) low or moderate correlation (a correlation of at least pronounced strength) was obtained.

The reliability of PREM-C9 questionnaire

The assessment of the reliability of the Polish version of the PREM-C9 questionnaire showed that the Cronbach's Alpha index was 0.743, which confirms that the assessed questionnaire is reliable (value >0.70).

In order to analyse the correlation of individual items with the remaining considered as "a total", the reliability assessment was repeated, but after removing individual questions from the questionnaire. It was shown that the elimination of individual statements did not significantly affect the Cronbach's alpha coefficient value for the rest of the questionnaire. The values obtained ranged from 0.690 to 0.767. The greatest reduction in the index was found when question 6 (I have enough information about my condition) was removed, which should be understood as the weakest correlation of this question with the others, but the value was still close to 0.70. The constructed correlation matrix of each question (Table 5) also showed that question 6, after removal of which the lowest value of Cronbach's Alpha was obtained, did not correlate significantly with the other questions. Question 2 was also the question that did not correlate with the others, but after removing it, Cronbach's Alpha only decreased by 0.032. However, as the aim of the study was not to modify the original tool, questions 6 and 2 were left in their Polish version, as in the original structure of the PREM-C9.

Also, a reliability analysis using Klein's criterion that individual variables should correlate with the total score for the whole scale at a level of at least 0.4 confirmed the reliability of the whole questionnaire, with all nine questions retained (Table 6). A comparison of the Spearman rank order correlation results between PREM-C9, CAT and HADS with the results of the original version (Hodson 2019) and the first use in the original language version (Jones 2020) is presented in Table 7.

4. Discussion

The Polish version of the PREM-C9 questionnaire met all validation criteria, i.e. face equivalence (test graphics, instructions), translation equivalence (question content), psychometric equivalence (similar correlation with questionnaires also used in psychometric analysis of the English language version of PREM-C9)), functional equivalence (suitability for the same purposes), reconstruction equivalence (checking reliability and validity). Validation of the PREM-C9 into Polish will allow the use of the questionnaire in the assessment of care for patients with COPD in Poland. The use of PREM indicators is still not a common practice, which previous studies have proven [23]. and which limits the possibility of comparing the results obtained in terms of accuracy and reliability of the Polish version of the PREM-C9 questionnaire with other works apart from the original assessment in the English language version. There are known studies using the PREM-C9 questionnaire in its original version. For example, a study by Jones et al. [24] measured patients' experiences of living with COPD and the medical care they received, and the results were compared with other scales - some of which were also used in the validation study of the Polish-language version (CAT and HADS). The correlation rho of the PREM-C9 (for all domains combined) with the CAT questionnaire score was 0.27 (thus less than in the present study), but it was a statistically significant relationship (p=0.03). However, only a result measuring towards significance was obtained for the correlation with the HADS-D subscale (p=0.09). The correlation level obtained in the present study between the PREM-C9 test and the CAT was 0.44, which is similar to the values obtained in the original validation of the tool (0.42). Similar results were also obtained for the HADS-A, slightly lower for the HADS-D [14]. In addition to the English language version, the PREM-C9 was translated into Catalan and Spanish [25]. Ten patients participated in this study, however, this study did not determine the questionnaire's psychometric values but the authors only translated it. Chaplin et al. define a meaningful change in PREM-9 following pulmonary rehabilitation. The change in PREM-9 for responders (defined as HADS anxiety MID ≥ -1.5) was -5.26 (SD 8.33). Sensitivity and specificity analysis using ROC using HADS anxiety anchor yielded a change of -7.5 units. The minimum important difference for the PREM-9 is calculated between -3.67 and -7.5 units [26].

The PCA analysis using the Kaiser normalisation criterion with Varimax and Oblimin rotation obtained the lowest correlation value for question 4 (I am happy to talk about the future). It should be acknowledged that the nature of the question itself may have influenced the result, as it is of a general type, going beyond the context of the disease itself, hence it can be assumed that the results obtained for this question differed slightly from the others.

An assessment of the reliability of the Polish version of the PREM-C9 questionnaire confirmed that it is a reliable tool and, therefore, correctly (accurately) reflects the actual condition. The Cronbach's alpha index was 0.743. According to various sources, even 0.6 can be considered a satisfactory level, although more often, it is 0.7 [27,28]. Slightly higher values were obtained in the validation of PREM-type tools dedicated to other conditions, e.g. rheumatic conditions [29], oncological conditions [30] or hypertension [31]. However, this was not a study conducted in a Polish setting; such studies, apart from this one, have not yet been produced. Also, a literature review performed in Germany in 2022 showed that none of the PREM-type questionnaires analysed were fully evaluated in German conditions [32].

5. Implications and limitations of the study

The benefits of using patient perspective in shaping effective health care are well accepted. For example, the addition of these data in the care of patients with metastatic cancer to those collected in a standard way resulted in increased survival compared to traditionally managed care [33], and in a group of patients with arthritis, it improved self-perceived health [34]. Using PROM and PREM type data also reduces the utilisation of healthcare system resources by improving symptom control, increasing patient satisfaction and ultimately - improving HRQOL [35].

Much attention has been paid over the years to the reliability of questionnaires of the PROM and PREM type while at the same time pointing to the need to develop new validated research tools [36]. This study has just resulted in the development, in accordance with current principles, of a PREM-type questionnaire, the psychometric properties of which have been meticulously examined and confirmed. This study may inspire and encourage other researchers to develop and use this type of questionnaire in patient care. What may be of note in the study is the small study group (42 patients). However, it should be clear that there are no specific guidelines on the minimum number of respondents for this type of study [37]. In the publication by Tsang et al. presenting Guidelines for developing, translating, and validating questionnaires, guidance can be found that it should be between 30 and 50 people, similar guidance is found in a broader study by Aithal [38].

6. Conclusions

The developed Polish version of the PREM-C9 questionnaire is a reliable and valid tool that assesses Polish patients' experiences of their disease (COPD) and the care they receive.

The questionnaire can be used to conduct follow-up among Polish patients and for comparative studies in non-English speaking countries, allowing for cross-cultural comparisons and determining further possibilities of using PREM-C9 and generalising the results to other populations.

Author Contributions

Conceptualization, IDK; EB and EJ. methodology, WC; TB formal analysis IDK, WC, TB investigation, WC, MOK, IDK; data curation EB, TB writing—original draft preparation IDK, EB, EJ, DB, MOK.; writing—review and editing DB, MOK, EJ, EB; visualization, DB, WC.; supervision, EB. All authors have read and agreed to the published version of the manuscript.”.

Funding

“This research received no external funding.

Institutional Review Board Statement

The study was approved by the Bioethics Committee of the Medical University of Gdańsk, and consent was obtained from the first author of the validated questionnaire for its use.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Acknowledgments

The authors wish to thank Matthew Hodson PhD, the author of the original version of the PREM-C9 questionnaire and Grzegorz Romanowicz PhD with his team - for support in conducting the study.

Conflicts of Interest

The authors declare no conflict of interest.

References

Agusti, A.; Celli, B.R.; Criner, G.J.; Halpin, D.; Anzueto, A.; et al. Global initiative for Chronic Obstructive Lung Disease 2023 Report: GOLD Executive Summary. Am J Respir Crit Care Med 2023, 207, 819–837. [Google Scholar] [CrossRef] [PubMed]
Agusti, A.; Melén, E.; DeMeo, D.L.; Breyer-Kohansal, R.; Faner, R. Pathogenesis of chronic obstructive pulmonary disease: understanding the contributions of gene-environment interactions across the lifespan. Lancet Respir Med 2022, 10, 512–524. [Google Scholar] [CrossRef] [PubMed]
De Matteis, S.; Jarvis, D.; Darnton, A.; Hutchings, S.; Sadhra, S.; Fishwick, D.; et al. The occupations at increased risk of COPD: analysis of lifetime job-histories in the population-based UK Biobank Cohort. Eur Respir J 2019, 54, 1900186. [Google Scholar] [CrossRef]
Bourbeau, J.; Doiron, D,; Biswas, S.; Smith, B.M.; Benedetti, A.; Brook, J.R.; et al. Ambient air pollution and dysanapsis: Associations with lung function and chronic obstructive pulmonary disease in the Canadian Cohort Obstructive Lung Disease Study. Am J Respir Crit Care Med 2022, 206, 44–55. [CrossRef]
Wytyczne oceny technologii medycznych. Agencja Oceny Technologii Medycznych i Taryfikacji, Warsaw 2016. Available online: https://www.aotm.gov.pl/media/2020/07/20160913_Wytyczne_AOTMiT-1.pdf (accessed on 15 August 2023).
Patient Charter. Global Allergy & Airways Patient Platform. Available online: https://gaapp.org/diseases/copd/patient-charter/?_gl=1*att4co*_ga*MTc2NzQ3MzU1OS4xNjkyODEwNTc0*_ga_8GH0BEJX5C*MTY5MjgxMDU3NC4xLjEuMTY5MjgxMDU3NS41OS4wLjA.&_ga=2.182628398.901915781.1692810578-1767473559.1692810574 (accessed on 22 August 2023).
Food and Drug Administration. Value and Use of Patient-Reported Outcomes (PROs) in Assessing Effects of Medical Devices CDRH Strategic Priorities. CDRH Strategic Priorities 2016–2017. Available online: https://www.fda.gov/downloads/AboutFDA/CentersOffices/OfficeofMedicalProductsandTobacco/CDRH/CDRHVisionandMission/UCM588576.pdf (accessed on 18 August 2023).
Valderas, J.M.; Alonso, J.; Guyatt; G.H. Measuring patient-reported outcomes: moving from clinical trials into clinical practice. Med J Aust 2008, 189, 93–94. [CrossRef]
Greenhalgh, J.; Gooding, K.; Gibbons, E.; Dalkin, S.; Wright, J.; Valderas, J.; et al. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J Patient Rep Outcomes 2018, 2, 42. [Google Scholar] [CrossRef] [PubMed]
Jamieson Gilmore, K.; Corazza, I.; Coletta, L.; Allin, S. The uses of Patient Reported Experience Measures in health systems: A systematic narrative review. Health Policy 2023, 128, 1–10. [Google Scholar] [CrossRef]
Sarri, G.; Freitag, A.; Szegvari, B.; Mountian, I.; Brixner, D.; Bertelsen, N.; et al. The role of patient experience in the value assessment of complex technologies – Do HTA Bodies need to reconsider how value is assessed? Health Policy 2021, 125, 593–601. [Google Scholar] [CrossRef] [PubMed]
Flott, K.M.; Graham, C.; Darzi, A.; Mayer, E. The challenges of using patient-reported feedback and how they might be addressed. BJM Qual Saf 2017, 26, 502–507. [Google Scholar] [CrossRef]
De Rosis, S.; Cerasuolo, D.; Nuti, S. Using patient-reported measures to drive change in healthcare: the experience of the digital, continuous and systematic PREMs observatory in Italy. BMC Serv Res 2020, 20, 315. [Google Scholar] [CrossRef]
Hodson, M.; Roberts, C.M.; Andrew, S.; et al. Development and first validation of a patient-reported experience measure in chronic obstructive pulmonary disease (PREM-C9). Thorax 2019, 74, 600–603. [Google Scholar] [CrossRef]
Młyńczak, K.; Golicki, D. Przegląd właściwości psychometrycznych kwestionariuszy oceny jakości życia związanych ze zdrowiem (HRQOL). Pol Prz Nauk Zdr 2016, 4, 415–421. [Google Scholar]
Wild, D.; Grove, A.; Martin, M.; et al. Principles of good practice for the translation and cultural adaptation process for Patient-Reported outcomes (PRO) Measures: report of the ISPOR Task Force for translation and cultural adaptation. Value Health 2005, 8, 94–104. [Google Scholar] [CrossRef] [PubMed]
Brislin, R.W. Back-translation for cross-cultural research. J Cross-Cult Psychol 1970, 1, 185–216. [CrossRef]
Fletcher, C.M. The clinical diagnosis of pulmonary emphysema—an experimental study. Proc R Soc Med 1952, 45, 577–584. [Google Scholar] [PubMed]
Jones, P.W.; et al. Development and first validation of the COPD assessment test. Eur Resp J 2009, 34, 648–654. [Google Scholar] [CrossRef] [PubMed]
Majkowicz, M.; de Walden-Gałuszko, K.; Chojnacka-Szawłowska, G. ; Model oceny jakości opieki paliatywnej realizowanej w warunkach stacjonarnych, Akademia Medyczna w Gdańsku: Gdańsk, Poland, 2021, pp. 94.
EQ-5D-5L. Available online: https://euroqol.org/eq-5d-instruments/eq-5d-5l-about/ (accessed on 20 August 2023).
Gerlinger, C.; Bamber, L.; Leverkus, F.; et al. Comparing the EQ-5D-5L utility index based on value sets of different countries: impact on the interpretation of clinical study results. BMC Res Notes 2019, 12, 18. [Google Scholar] [CrossRef] [PubMed]
Bandurska, E. The voice of patients really matters: using Patient-Reported Outcomes and Experiences Measures to assess effectiveness of home-based integrated care—a scoping review of practice. Healthcare 2023, 11, 98. [Google Scholar] [CrossRef] [PubMed]
Jones, M.A.; Evans, R.A.; Greening, N.J.; Roberts, M.C.; Toms, N.; Hodson, M.; Steiner, M.C. Measuring the experience of living with COPD and receiving medical care in people with COPD. Eur Respir J 2020, 56, 956. [Google Scholar] [CrossRef]
Moharra, M.; Bayes, B.; Llupia, A.; Almazan, C. Cross-cultural adaptation and face validity of the PREM-C9 version of the Patient Reported Experience Measure in patients with Chronic Obstructive Pulmonary Disease. XPA & Health Com 2021, 4.
Chaplin, E.; Coope, D.; Zatloukal, J.; Ward, S.; Singh, S.J. , Houchen-Wolloff, L. The minimum important difference for the PREM-9 following a course of Pulmonary Rehabilitation (PR). Euro Resp J, 2899. [Google Scholar] [CrossRef]
Tavakol, M.; Dennick, R. Making sense of Cronbach's alpha. Int J Med Educ 2011, 2, 53–55. [Google Scholar] [CrossRef]
Streiner, D.L.; Norman, G. Health measurement scales: a practical guide to their development and use. Oxford University Press, Inc., New York, 2008.
Bosworth, A.; Cox, M.; O'Brien, A.; Jones, P.; Sargeant, I.; Elliott, A.; Bukhari, M. Development and validation of a Patient Reported Experience Measure (PREM) for patients with rheumatoid arthritis (RA) and other rheumatic conditions. Current rheumatology reviews 2015, 11, 1–7. [Google Scholar] [CrossRef] [PubMed]
Moody, L.; Benn, S.; Luciano, I.; Ingale, S.; Singh, S. Validation of a real-time patient-reported experience measurement tool for cancer patients in Ontario. J Clin Oncol 2016, 34. [Google Scholar] [CrossRef]
Waldreus, N.; Jaarsma, T.; Ivarsson, B.; Stromberg, A.; Arestedt, K.; Kiellstrom, B. Development and validation of a questionnaire to measure patients experiences of health care in pulmonary arterial hypertension outpatient clinics. Heart, lung and circulation 2019, 28, 1074–1081. [CrossRef]
Mihaljevic, A.L.; Doerr-Harim, C.; Kalkum, E.; Strunk, G. Measuring patient centeredness with German language Patient-Reported Experience Measures (PREM)-A systematic review and qualitative analysis according to COSMIN. PLoS One 2022, 17, e0264045. [Google Scholar] [CrossRef]
Hinami, K.; Smith, J.; Deamant, C.D.; et al. When do patient-reported outcome measures inform readmission risk? J Hosp Med 2015, 10, 294–300. [Google Scholar] [CrossRef] [PubMed]
El Miedany, Y.; El Gaafary, M.; El Arousy, N.; et al. Arthritis education: the integration of patient-reported outcome measures and patient self-management. Clin Exp Rheumatol 2012, 30, 899–904. [Google Scholar] [PubMed]
Santana, M.J.; Feeny, D. Framework to assess the effects of using patient-reported outcome measures in chronic care management. Qual Life Res 2014, 23, 1505–1513. [Google Scholar] [CrossRef]
Greenhalgh, J.; Meadows, K. The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract 1999, 5, 401–416. [Google Scholar] [CrossRef] [PubMed]
Tsang, S.; Royse, C.F.; Terkawi, A.S. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J Anaest 2017, 11, 80–89. [Google Scholar] [CrossRef]
Aithal, A.; Aithal, P.S. Development and validation of survey questionnaire & experimental data – a systematical review-based statistical approach. Munich Personal RePEc Archive 2020, 1–19. [Google Scholar]

Figure 1. Stages of the adaptation procedure of the PREM-C9 questionnaire.

Figure 2. The Polish version of PREM-C9.

Table 1. Characteristics of the clinical validation cohort: demographic data.

Parameter	Sub parameter	Number	Percent (%)
Gender	Female	21	50.00
Gender	Male	21	50.00
Place of living	City	39	92.86
Place of living	Village	3	7.14
Education level	Primary	1	2.38
	Vocational	15	35.71
	Secondary	23	54.76
	Higher	3	7.14
Self-assessment of material status	Very good	7	16.57
	Average	25	59.52
	Poor	10	23.81
	Very poor	0	0.00
Living alone	Yes	11	26.19
Living alone	No	31	73.81

Table 2. Results according to separate questionnaires.

Questionnaire	The average of the points obtained	% of max points (showing the worst state)	Median	SD
PREM-C9	9.33	20.73	6.5	6.98
CAT	17.02	42.63	16	8.90
EQ-5D-5L	10.07	40.28	9	4.27
HADS A (anxiety)	4.09	19.48	3	4.05
HADS D (depression)	3.29	15.67	2	3.27

Table 3. The results of Bartlett and KMO tests.

Kaiser-Meyer-Olkin’s test and Bartlett’s test
KMO measure of sampling adequacy		0.675
Bartlett’s sphericity test	Approximated chi-squared	108.382
	df	36
	significance	0.000

Table 4. Spearman's correlation between PREM-C9 and EQ-5D-5L, EQ-VAS, HADS and CAT.

Pair of variables (questionnaires)	Spearman rank order correlation
Pair of variables (questionnaires)	N	Rho	p
CAT & PC-9	42	0.440098	0.003539
EQ-5D-5L & PC-9	42	0.386974	0.011352
EQ VAS & PC-9	41	-0.346096	0.026653
HADS-A & PC-9	42	0.370864	0.015612
HADS-D & PC-9	42	0.387405	0.011253

Table 5. Partial correlations between questions in PREM-C9 (significant or close to statistical significance are marked in bold and italics).

The number of a question in PREM-C9	Correlations are significant when p < 0.05000
The number of a question in PREM-C9	P-C9 1	P-C9 2	P-C9 3	P-C9 4	P-C9 5	P-C9 6	P-C9 7	P-C9 8	P-C9 9
P-C9 1	1.000000	0.509477	0.475456	0.280433	-0.067198	0.453085	0.327855	0.056390	0.254275
P-C9 2	0.509477	1.000000	0.377423	0.159650	0.147470	0.477361	0.149318	0.197162	0.159660
P-C9 3	0.475456	0.377423	1.000000	0.536886	-0.091139	0.396248	0.209143	0.251775	0.375878
P-C9 4	0.280433	0.159650	0.536886	1.000000	-0.143419	0.256114	0.081663	-0.049719	0.093140
P-C9 5	-0.067198	0.147470	-0.091139	-0.143419	1.000000	0.168737	0.080009	0.427637	0.200056
P-C9 6	0.453085	0.477361	0.396248	0.256114	0.168737	1.000000	0.527371	0.272347	0.309339
P-C9 7	0.327855	0.149318	0.209143	0.081663	0.080009	0.527371	1.000000	0.413718	0.509607
P-C9 8	0.056390	0.197162	0.251775	-0.049719	0.427637	0.272347	0.413718	1.000000	0.482369
P-C9 9	0.254275	0.159660	0.375878	0.093140	0.200056	0.309339	0.509607	0.482369	1.000000

Table 6. Correlations of individual questions with the total score obtained for the entire test.

The number of a question in PREM-C9	Correlation with the sum P-C9	p
P-C9 1	0.6	0.000
P-C9 2	0.6	0.000
P-C9 3	0.7	0.000
P-C9 4	0.4	0.000
P-C9 5	0.4	0.000
P-C9 6	0.7	0.000
P-C9 7	0.6	0.000
P-C9 8	0.6	0.000
P-C9 9	0.6	0.000

Table 7. Summary of the Spearman rank order correlation results between the Polish version of PREM-C9 and CAT and HADS with the results of the original version (Hodson 2019) and the first use in the original language version (Jones 2020).

Questionnaire	PREM-C9 total (Damps-Konstanska et al.)	PREM –C9 total (Hodson et al. 2019)	PREM –C9 total (Jones et al. 2020)
CAT	Rho= 0,44 P=0,003539	Rho= 0,42 P=0,03	Rho= 0,27 P=0,03
HADS -Anxiety	Rho= 0.370864 P=0,015612	Rho= 0,30 P<0,005	Rho= 0,4 P=0,001
HADS -Depression	Rho= 0,387405 P=0,011253	Rho= 0,41 P<0,005	Rho= 0,21 P=0,09

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer