Publish in this journal
Journal Information
Vol. 52. Issue 1.
Pages 12-16 (January 2016)
Visits
2294
Vol. 52. Issue 1.
Pages 12-16 (January 2016)
Original Article
DOI: 10.1016/j.arbr.2015.03.017
Full text access
Validation of the Spanish Version of the COPD-Q Questionnaire on COPD Knowledge
Validación de la versión en español del cuestionario COPD-Q/EPOC-Q de conocimiento de la EPOC
Visits
2294
Luis Puente-Maestua,b,c,
Corresponding author
lpuente.hgugm@salud.madrid.org

Corresponding author.
, Jorge Chancafe-Morgana,b, Myriam Calleb,d, Juan L. Rodríguez-Hermosad, Rosa Malo de Molinae, Ángel Ortega-Gonzálezf, Antonia Fusterg, Eduardo Márquez-Martính,i,j, Pedro J. Marcosk,l, Laura Ramíreza,b, Shaunta’ Raym,n, Andrea Franksm,n, en nombre del grupo GemEPOC
a Servicio de Neumología, Hospital General Universitario Gregorio Marañón, Madrid, Spain
b Instituto de Investigación Sanitaria Gregorio Marañón (IiSGM), Madrid, Spain
c Facultad de Medicina, Universidad Complutense de Madrid (UCM), Madrid, Spain
d Servicio de Neumología, Hospital Clínico San Carlos, Madrid, Spain
e Servicio de Neumología, Hospital Universitario Puerta de Hierro, Majadahonda, Madrid, Spain
f Unidad de Neumología, Hospital General Nuestra Señora del Prado, Talavera de la Reina, Toledo, Spain
g Servicio de Neumología, Hospital de Son Llàtzer, Palma de Mallorca, Spain
h Unidad Médico Quirúrgica de Enfermedades Respiratorias, Hospital Virgen del Rocío, Sevilla, Spain
i Instituto de Biomedicina de Sevilla (IBIS), Sevilla, Spain
j Facultad de Medicina, Universidad Hispalense, Sevilla, Spain
k Servicio de Neumología, Instituto de Investigación Biomédica de A Coruña (INIBIC), Complejo Hospitalario Universitario de A Coruña (CHUAC), Sergas, A Coruña, Spain
l Universidade da Coruña (UDC), A Coruña, Spain
m Department of Clinical Pharmacy, University of Tennessee College of Pharmacy, Knoxville, TN, United States
n Department of Family Medicine, University of Tennessee Graduate School of Medicine, Knoxville, TN, United States
This item has received
2294
Visits
Article information
Abstract
Full Text
Bibliography
Download PDF
Statistics
Figures (2)
Tables (3)
Table 1. General Study Sample.
Table 2. Percentage of Right Answers.
Table 3. Reproducibility of Items.
Show moreShow less
Abstract
Rationale

Although recognition of the importance of educating chronic obstructive pulmonary disease (COPD) patients has grown in recent years, their understanding of this disease is not being measured due to a lack of specific instruments. The aim of this study was to validate the COPD-Q questionnaire, a 13-item instrument for determining COPD knowledge.

Methods

The COPD-Q was translated and backtranslated, and subsequently submitted to logic and content validation by a group of COPD experts and 8 COPD patients. Reliability was studied in an independent group of 59 patients with severe COPD seen in the pulmonology ward or clinics of 6 hospitals in Spain (Andalusia, Baleares, Castilla-La Mancha, Galicia and Madrid). This sample was also used for other internal and external validations.

Results

The mean age of the group was approximately 70 years and their health literacy was low-to-medium. The number of correct answers was 8.3 (standard deviation: 1.9), median 8, range 3–13. Floor and ceiling effects were 0% and 1.5%, respectively. Internal consistency of the questionnaire was good (Cronbach's alpha=0.85) and reliability was also high, with a kappa coefficient >0.6 for all items and an intraclass correlation efficient of 0.84 for the total score.

Conclusion

The 13-item COPD-Q is a valid, applicable and reliable instrument for determining patients’ knowledge of COPD.

Keywords:
Knowledge questionnaire
Health awareness
Patient education
Chronic obstructive pulmonary disease
Resumen
Fundamentos

A pesar de que el reconocimiento de la importancia de la formación de los pacientes con EPOC ha crecido en los últimos años, no se está midiendo el grado de conocimiento de dicha enfermedad por falta de instrumentos específicos. El objetivo de este estudio es validar el cuestionario de conocimiento de la EPOC (EPOC-Q) de 13 ítems.

Métodos

Tras la doble traducción del EPOC-Q se llevó a cabo la validación lógica y de contenido por un grupo de neumólogos expertos en EPOC y 8 pacientes con la enfermedad. La fiabilidad se estudió en un grupo independiente de 59 pacientes con EPOC grave vistos en planta o en consultas de neumología de 6 centros de varias regiones de España (Andalucía, Baleares, Castilla-La Mancha, Galicia y Madrid). Esta muestra también se usó para otras validaciones internas y externas.

Resultados

El grupo tenía una media de edad de aproximadamente 70 años y una alfabetización en salud media baja. El número de respuestas acertadas fue de 8,3 (DE: 1,9), con una mediana de 8 y un rango entre 3 y 13. Los efectos suelo y techo fueron 0 y 1,5%, respectivamente. La consistencia interna del cuestionario es buena (alfa de Cronbach de 0,85) y la fiabilidad también alta, siendo el coeficiente kappa >0,6 en todos los ítems y el coeficiente de correlación intraclase de la puntuación total de 0,84.

Conclusión

El cuestionario EPOC-Q de 13 ítems es un instrumento válido, aplicable y fiable para evaluar el conocimiento de la EPOC.

Palabras clave:
Cuestionario de conocimiento
Alfabetización en salud
Educación de pacientes
Enfermedad pulmonar obstructiva crónica
Full Text
Introduction

In recent years, healthcare providers have become more aware of the importance of educating patients with chronic diseases,1 both because of increasing interest from the patients themselves in learning about their condition, and because agents have gradually recognized that standards of treatment improve among chronic patients who are able to participate in their own care.2,3 Moreover, the availability of the Internet and smartphones have greatly increased patients’ access to health information4,5 and allowed the introduction of a new more participative and personalized care model.6

Studies have shown that individuals with chronic obstructive pulmonary disease (COPD) who receive education have a better health-related quality of life and a lower rate of hospitalization.7 Yet, surprisingly, most publications measure health outcomes in terms of hospitalizations, exacerbations, and quality of life, while the patient's knowledge of their disease is not measured either before or after participating in training programs.7,8

Several English language questionnaires have been developed that use both open-ended and leading questions to determine COPD education among patients,8–13 but until now the Spanish translations of these instruments have not been validated, nor have equivalent tools been published.

In order to provide an instrument for measuring COPD education among Spanish-speaking patients, we aimed to validate the Chronic Obstructive Pulmonary Disease Knowledge Questionnaire (COPD-Q)9 in a group of both hospitalized patients and outpatients attending hospitals in Spain. COPD-Q was designed for patients with a low general level of health literacy, a situation applicable to 50% of both the general population and COPD patients.14,15 While health literacy may affect the patient's knowledge of their diseases,13 the 13 items of the COPD-Q refer to basic knowledge that should, in principle, be routinely transmitted by healthcare professionals, and should not depend on the ability of the individual to obtain information from other sources. For this reason, we also examined differences in knowledge depending on the level of health literacy.

MethodsValidation of the Logic and Content of the Translation

Logic validity is the degree to which a questionnaire seems to measure what we want it to measure. The content is considered valid if it covers all or at least a sufficient number of the issues related with the subject of the questionnaire, which in this case was COPD knowledge. The first step in this subjective process was to obtain a double translation by 2 bilingual physicians. A group of 14 Spanish-speaking COPD experts then assessed if the translation of these items was applicable and comprehensible. They were asked to indicate any inconsistency or potential difficulty that might affect the clarity or conciseness of the items. Discrepancies were resolved by consensus. Logic was validated by administering the questionnaire to 8 Spanish-speaking patients with a low level of health literacy. Health literacy was defined as high or low according to the Short Assessment of Health Literacy for Spanish-speaking Adults (SAHLSA) questionnaire.16 This instrument consists of 50 medical terms presented along with 2 more words, 1 associated with the term and 1 not associated with it. The interviewee has to read the medical term and link it with the right word. The SAHLSA questionnaire has been validated in several Spanish dialects.16 A cut-off point of 37 right answers was used as a criterion for high or low literacy, as proposed by the authors of the questionnaire.16

After participants had given their informed consent in writing, they were read each item from the translated COPD-Q, which we called the EPOC-Q (EPOC is the abbreviation of COPD in Spanish), and asked to explain the meaning, in their own words, to check that they had clearly understood the questions.

Reliability

To evaluate the reliability of the EPOC-Q, 59 Spanish-speaking patients with no vision loss or cognitive impairment, whose COPD was previously diagnosed by spirometry,17 were recruited in 6 hospitals in Spain: Coruña (Complejo Hospitalario Universitario de A Coruña), Madrid (the Gregorio Marañón and Clínico San Carlos universitary hospitals), Majadahonda (Hospital Universitario Puerta de Hierro), Palma de Mallorca (Hospital de Son Llàtzer), Seville (Hospital Universitario Virgen del Rocío), and Talavera de la Reina (Hospital Nuestra Señora del Prado). On the basis of the original COPD-Q validation study,9 a sample of 50 patients was estimated to be sufficient to detect discrepancies in a question (of 13) with a precision of 5% and a confidence of 95%.

All patients signed informed consent forms, and the protocol was evaluated by the clinical research ethics committee of each hospital.

Concordance between the results of each of the items from the 2 administrations of the EPOC-Q was evaluated using the kappa index and its corresponding confidence interval and with intraclass correlation coefficient for the overall score.

Internal Consistency

Internal consistency is the correlation of the different items within a single questionnaire, and was measured using Cronbach's alpha. To determine the weight that each item added to the questionnaire, the Spearman correlation between items and total score, excluding the item itself, was calculated.18,19

Statistical Distribution of Answers

The statistical distribution of the answers was evaluated using graphic analysis and the Kolmogorov–Smirnov/Lilliefors test.

Validity of Criterion and Intercultural Metric

The validity of a criterion is measured comparing the tool (EPOC-Q in this case) with a known measurement standard from the same construct (a construct is something that is known to exist, but which is controversial or difficult to define, for example, intelligence, personality, and creativity; the construct in this case is knowledge of COPD). As no comparative method was available for evaluating knowledge of COPD, we used the descriptive analysis of the proportion of patients who answered each question correctly, with the corresponding standard deviation (SD), distribution, range of scores, and floor and ceiling effects (percentage of individuals with extremely high or extremely low scores), which had to be less than 15%.18 The transcultural metric consists of determining if the measurement scale has the same value in both cultures, i.e., if a COPD-Q/EPOC-Q score implies the same level of knowledge in 2 different cultures. As no reference pattern was available, we evaluated this using the Mann–Whitney U test, comparing the scores obtained in this score with those of the original study.9

Results

A total of 59 patients were enrolled, of whom 30 had low health literacy. Mean level of obstruction among the patients was severe (FEV1 43.6±18%) and the mean age was about 70 years (Table 1). Women accounted for 20% of the population (Table 1). No significant differences were observed in the general characteristics of the low and high literacy groups (Table 1), but those with lower literacy tended to be younger.

Table 1.

General Study Sample.

  All (n=59)Low Literacy (n=30)High Literacy (n=29)P 
Age (years)  70.2  (9.4)  72.6  (8.4)  68.1  (9.0)  .074 
FEV1 (%)  43.6  (14.0)  48.6  (13.1)  41.1  (14.2)  .226 
FVC (%)  71.8  (14.1)  75.8  (9.4)  69.7  (15.9)  .334 
FEV1/FVC (%)  47.3  (15.2)  50.1  (18.6)  45.9  (13.7)  .537 
COPD-Q  8.4  (1.9)  8.3  (2.0)  8.4  (1.7)  .550 
SALHSA  35.1  (9.1)  28.5  (6.2)  43.5  (3.4)   
Years since diagnosis  6.6  (3.4)  7.5  (2.8)  6.4  (3.9)  .320 
Sex (M/F)  12/47  20/80%  6/24  20/80%  6/23  21/79%  .526 
Smoker (yes/no)  37/22  62/38%  22/18  74/36%  15/14  51/49%  .041 
Consultation room/hospitalization  26/33  44/56%  13/17  43/57%  13/16  45/55%  .268 

COPD-Q: Questionnaire COPD-Q score9; SAHLSA: Short Assessment of Health Literacy for Spanish-speaking Adults (SALHSA) questionnaire.16

No differences were observed in EPOC-Q scores between the 2 groups (Table 1). The overall mean number of correct answers was 8.3 (SD 0.77) and the median was 8 (interquartile range of 2) (Fig. 1). None of the patients answered all items incorrectly – the minimum number of right answers was 3 – and 1 patient (1.5%) answered all 13 questions correctly (Fig. 2). The distribution of the number of correct answers was normal (P=.491) (Fig. 2). Almost all respondents (93%) knew that “people with COPD may feel short of breath”, and that “people with COPD should have a flu shot every year” (91%); in contrast, more than half believed that “people can stop taking their long-acting breathing medications (inhaler) when their COPD symptoms get better” and that “COPD can be reversed (it can go away or be cured)” (Table 2).

Fig. 1.

Box diagrams of COPD-Q scores of health literacy measured using the SAHLSA questionnaire and the cut-off point of 37 proposed by the authors.16

(0.05MB).
Fig. 2.

Distribution of absolute frequency of right answers for each item.

(0.06MB).
Table 2.

Percentage of Right Answers.

Question  Right Answer (nS 
People with COPD should get a pneumonia shot  42  71.2  0.325 
Using oxygen at home can help people with COPD live longer  38  64.4  0.308 
COPD medicines keep the disease from getting worse.  40  67.8  0.278 
COPD can be prevented  36  61.0  0.314 
People can stop taking their long-acting breathing medications (inhaler) when their COPD symptoms get better  23  39.0  0.546 
People with COPD often have a cough that won’t go away  41  69.5  0.361 
Stopping smoking will keep COPD from getting worse  47  79.7  0.373 
Cigarette smoking or second hand smoke causes most COPD  41  69.5  0.478 
People with COPD may feel short of breath  54  91.5  0.384 
A salbutamol inhaler (medicine) can be used anytime you are short of breath.  38  64.4  0.335 
People with COPD should have a flu shot every year  53  93.2  0.370 
People should only use their COPD inhalers (medicines) when they can’t breathe  27  45.8  0.474 
COPD can be reversed (it can go away or be cured)  24  40.7  0.585 

S: Spearman correlation with total score without the corresponding item.

Women answered a mean of 9.2 (2.0) EPOC-Q questions correctly and men 8.1 (1.8) (P=.629); hospitalized patients answered 8.5 (1.7) questions correctly, and outpatients 8.2 (2.1) (P=.38); smokers answered 8.0 (1.9) questions correctly and non-smokers 8.9 (1.8) (P=.801). There was no significant correlation between EPOC-Q score and age, hospital center, or years since COPD diagnosis.

The internal consistency of EPOC-Q is high (Cronbach's alpha=0.825). In general, the reproducibility of each item, measured using the kappa index, was good (between 0.61 and 0.8) and in some cases excellent (between 0.81 and 1) (Table 3). Intraclass correlation coefficient between the first and second total scores was very high: 0.842 (0.745–0.904), P<.001.

Table 3.

Reproducibility of Items.

Question  Kappa  SD  P 
People with COPD should get a pneumonia shot  0.788  0.101  .000 
Using oxygen at home can help people with COPD live longer  0.695  0.102  .000 
COPD medicines keep the disease from getting worse  0.664  0.116  .000 
COPD can be prevented  0.766  0.088  .000 
People can stop taking their long-acting breathing medications (inhaler) when their COPD symptoms get better  0.684  0.096  .000 
People with COPD often have a cough that won’t go away  0.833  0.093  .000 
Stopping smoking will keep COPD from getting worse.  0.625  0.154  .000 
Cigarette smoking or second hand smoke causes most COPD  0.970  0.028  .000 
People with COPD may feel short of breath  1.000  0.000  .000 
The medicine salbuterol (inhaler) can be used anytime you are short of breath  0.681  0.111  .000 
People with COPD should have a flu shot every year  0.793  0.201  .000 
People should only use their COPD inhalers (medicines) when they can’t breathe  0.703  0.091  .000 
COPD can be reversed (it can go away or be cured)  0.768  0.074  .000 

SD: asymptotic standard deviation.

Discussion

The translation of the COPD-Q9 to Spanish, the EPOC-Q, is easy to administer and, in the opinion of experts and patients, all questions are easily understood. High internal consistency (Cronbach's alpha=0.825) was observed and reproducibility was good (intraclass correlation coefficient=0.842).

Although the COPD-Q was originally designed to evaluate patients with low health literacy, the EPOC-Q appears to function equally well in patients with higher health literacy, probably because, as we supposed, understanding of the items comes from information regularly imparted by healthcare professionals to all their patients.

There was only a small percentage of extreme scores (floor and ceiling effects), although most of the participants answered 2 questions correctly. One of them, “people with COPD may feel short of breath”, forms part of the daily experience of the severe COPD patient. Even so, this item should not be excluded, as it may be relevant if the questionnaire is administered to patients with milder forms of the disease, with other diseases, or healthy subjects. The other questionnaire item correctly answered by nearly all patients (“People with COPD should have a flu shot every year”) indicates a high awareness regarding influenza vaccination, which contrasts with the lack of knowledge about the benefit of maintenance therapy.

Internal consistency and reliability are similar to those of the original validation in English (Cronbach's alpha for the original: 0.72; and intraclass correlation coefficient: 0.9 [0.76–0.96]9). However, mean scores were slightly, but significantly higher (P<.001) (mean difference of 0.8 and pooled standard deviation of 1.5 points).

This study has some limitations. As no reference model for assessing COPD knowledge is available, the metrics of the English and Spanish versions cannot be compared. We believe that the differences found represent real differences in knowledge between our sample and that of the original validation, but transcultural differences cannot be fully ruled out. This uncertainty is inevitable when subjective variables are compared between 2 cultures, but this is the case for the majority of commonly used questionnaires translated from English and validated in Spanish. The EPOC-Q does not require the patient to make quantitative evaluations, since the items are answered Yes, No, or Don’t know, thus reducing cultural subjectivity. The questions are simple and were considered appropriately phrased and understandable by a panel of native-speaking experts and patients in both languages.9 It was also impossible to verify the validity of the criterion, i.e., the correlation with other ways of measuring the same construct, since no other scales of COPD knowledge have been validated in Spanish. However, the floor and ceiling effects are low, and it is our impression, based on our daily practice, that the percentage of correct answers to each question reflects what patients know and do not know about their disease. The sensitivity of the questionnaire to intervention was not evaluated, but this validation is generally performed once the questionnaire is considered acceptable from a psychometric point of view, and begins to be used by other researchers in their studies.

The practical implications of having a validated translation of the COPD-Q are clear, since this tool can be used to obtain an objective and comparable measurement of the knowledge of COPD and the impact of specific patient education. The EPOC-Q may also be useful for designing training programs tailored to address gaps in the knowledge of individual patients, taking into account that, to our knowledge, no other similar evaluation instrument has been published.

In short, the 13-item EPOC-Q questionnaire is a valid instrument that is useful and reliable for measuring COPD knowledge among patients with any level of health literacy.

Conflict of Interests

The authors declare that they have no conflict of interest with the contents of this article.

References
[1]
M. Kamisaka.
Public health activities in improvement of citizens’ health. 25 years leading up to establishment of a health improvement center.
Hokenfu Zasshi, 35 (1979), pp. 118-129
[2]
T. Effing, E.M. Monninkhof, P.D. van der Valk, J. van der Palen, C.L. van Herwaarden, M.R. Partidge, et al.
Self-management education for patients with chronic obstructive pulmonary disease.
Cochrane Database Syst Rev, (2007),
[3]
A. Tsiachristas, J.M. Cramm, A.P. Nieboer, M.P. Rutten-van Molken.
Changes in costs and effects after the implementation of disease management programs in the Netherlands: variability and determinants.
Cost Eff Resour Alloc, 12 (2014), pp. 17
[4]
N.K. Leidy, L.T. Murray, B.U. Monz, L. Nelsen, M. Goldman, P.W. Jones, et al.
Measuring respiratory symptoms of COPD: performance of the EXACT-Respiratory Symptoms Tool (E-RS) in three clinical trials.
Respir Res, 15 (2014), pp. 124
[5]
M. Tabak, M. Brusse-Keizer, P. van der Valk, H. Hermens, M. Vollenbroek-Hutten.
A telehealth program for self-management of COPD exacerbations and promotion of an active lifestyle: a pilot randomized controlled trial.
Int J Chron Obstruct Pulmon Dis, 9 (2014), pp. 935-944
[6]
L. Hood.
Systems biology and p4 medicine: past, present, and future.
Rambam Maimonides Med J, 4 (2013), pp. e0012
[7]
I. Peytremann-Bridevaux, P. Staeger, P.O. Bridevaux, W.A. Ghali, B. Burnand.
Effectiveness of chronic obstructive pulmonary disease-management programs: systematic review and meta-analysis.
Am J Med, 121 (2008), pp. 433-443
[8]
R. White, P. Walker, S. Roberts, S. Kalisky, P. White.
Bristol COPD Knowledge Questionnaire (BCKQ): testing what we teach patients about COPD.
Chron Respir Dis, 3 (2006), pp. 123-131
[9]
P. Maples, A. Franks, S. Ray, A.B. Stevens, L.S. Wallace.
Development and validation of a low-literacy Chronic Obstructive Pulmonary Disease knowledge Questionnaire (COPD-Q).
Patient Educ Couns, 81 (2010), pp. 19-22
[10]
B. O’Neill, D. Cosgrove, J. MacMahon, E. McCrum-Gardner, J.M. Bradley.
Assessing education in pulmonary rehabilitation: the Understanding COPD (UCOPD) questionnaire.
[11]
J.W. Hopp, J.W. Lee, R. Hills.
Development and validation of a pulmonary rehabilitation knowledge test.
J Cardiopulm Rehabil, 9 (1989), pp. 273-280
[12]
N. Netzer, P. Werner, W. Petro, H. Matthys.
Open-word questions: an effective tool in gauging education of patients with COPD.
Monaldi Arch Chest Dis, 51 (1996), pp. 74-76
[13]
L.F. Black, M.M. Mitchell.
Evaluation of a patient education program for chronic obstructive pulmonary disease.
Mayo Clin Proc, 52 (1977), pp. 106-111
[14]
M.R. Andrus, M.T. Roth.
Health literacy: a review.
Pharmacotherapy, 22 (2002), pp. 282-302
[15]
J.L. Rodriguez Hermosa, M. Calle Rubio, M.J. Campuzano GArcía, M.L. Pérez Gutiérez, L. Puente Maestú.
Impacto del alfabetismo en la EPOC. Estudio ALFAEPOC.
Arch Bronconeumol, 50 (2014), pp. 97
[16]
S.Y. Lee, D.E. Bender, R.E. Ruiz, Y.I. Cho.
Development of an easy-to-use Spanish Health Literacy test.
Health Serv Res, 41 (2006), pp. 1392-1412
[17]
Global Initiative for Chronic Obstructive Lung Disease (GOLD); 2014 [publicación electrónica] [consultado 23 Dic 2014]. Disponible en: http://www.goldcopd.org/uploads/users/files/GOLD_Report2014_Feb07.pdf.
[18]
C.A. McHorney, A.R. Tarlov.
Individual-patient monitoring in clinical practice: are available health status surveys adequate.
Qual Life Res, 4 (1995), pp. 293-307
[19]
J. Rattray, M.C. Jones.
Essential elements of questionnaire design and development.
J Clin Nurs, 16 (2007), pp. 234-243

Please cite this article as: Puente-Maestu L, Chancafe-Morgan J, Calle M, Rodríguez-Hermosa JL, Malo de Molina R, Ortega-González Á, et al. Validación de la versión en español del cuestionario COPD-Q/EPOC-Q de conocimiento de la EPOC. Arch Bronconeumol. 2016;52:12–16.

Copyright © 2014. SEPAR
Idiomas
Archivos de Bronconeumología (English Edition)

Subscribe to our newsletter

Article options
Tools
es en

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

es en
Política de cookies Cookies policy
Utilizamos cookies propias y de terceros para mejorar nuestros servicios y mostrarle publicidad relacionada con sus preferencias mediante el análisis de sus hábitos de navegación. Si continua navegando, consideramos que acepta su uso. Puede cambiar la configuración u obtener más información aquí. To improve our services and products, we use "cookies" (own or third parties authorized) to show advertising related to client preferences through the analyses of navigation customer behavior. Continuing navigation will be considered as acceptance of this use. You can change the settings or obtain more information by clicking here.