Application of Rasch analysis to the parent adherence report questionnaire in juvenile idiopathic arthritis
© The Author(s). 2016
Received: 19 April 2016
Accepted: 19 July 2016
Published: 28 July 2016
Adherence to treatment in children with juvenile idiopathic arthritis (JIA) is associated with better outcomes. Assessing patient adherence in JIA, as well as attitudes and beliefs about prescribed treatments, is important for the clinician in order to optimize patient management. The objective of the current study was to evaluate the psychometric properties of the Parent (proxy-report) Adherence Report Questionnaires (PARQ), which assesses beliefs and behaviors related to adherence to treatments prescribed for JIA.
A Rasch analysis was conducted on data collected with parents of children with JIA from two studies in which the PARQ was used as a measure of adherence.
The PARQ showed preliminary evidence of multidimensionality with two factors, accounting for 38 % and 27 % of the variance respectively. The PARQ in its original version does not adhere to expectations of the Rasch model. A transformed version of the PARQ obtained by deletion of the general adherence scale and modification of visual analog scales into 5-point likert scales improved fit to the model and showed preliminary evidence of unidimensionality.
The PARQ was transformed based on the results of the Rasch analysis. The transformed version of the PARQ shows preliminary evidence of unidimensionality and may allow computation of a total score, although further testing is needed to verify these findings.
KeywordsJuvenile arthritis Patient compliance Questionnaires Psychometrics
Higher levels of adherence are associated with better health outcomes among children with chronic diseases  including juvenile idiopathic arthritis (JIA) [2, 3]. Valid, reliable, and easy to administer measures assessing patient adherence, as well as factors associated with adherence to prescribed treatments, must be available for use in clinical practice in order to optimize patient management . In JIA, adherence may involve gaining access to the medications, taking the right dosage at the appropriate times, as well as performing prescribed exercises and wearing splints according to the instructions of their therapist. Although there is no gold standard to assess adherence , using patient self-report questionnaires is the most common and practical way to evaluate adherence . A few questionnaires have been developed to assess adherence in pediatric chronic diseases such as JIA but none assesses adherence to the various modalities used in JIA in a valid manner .
Our research team developed the Parent Adherence Report Questionnaire (PARQ) aimed at measuring patient and parent beliefs and behaviors related to adherence to various treatments prescribed for JIA in order to elucidate adherence issues and act upon them [4, 5]. The PARQ has shown satisfactory construct validity and test-retest reliability using Classical Test Theory . However, the assumption of unidimensionality, imperative for construct validation and proof of interval-level measurement permitting the summation of individual items into a total score, have not been verified. Also, other important psychometric properties such as the usefulness of the response categories for each item and differential item function (item bias) have not been investigated. At this time, the interpretation of the PARQ remains mostly qualitative, as no overall score can be derived from the individual items. Rasch analysis, a modern model for evaluating psychometric properties of self-report measures, can provide this valuable information and informs on how well the items contribute to defining the construct of the questionnaire: adherence to treatment in children . The aim of the present study is to evaluate the PARQ using Rasch analysis.
The development of the PARQ was guided by the World Health Organization (WHO) conceptual framework , as well as a literature review of studies on adherence in general pediatrics and JIA . The WHO model is comprised of five dimensions of factors associated with adherence: 1) social and economic factors, 2) health care team and system-related factors, 3) condition-related factors, 4) therapy-related factors, and 5) patient-related factors . Several consultations with a team of pediatric rheumatology health professionals were performed to identify elements that should be included in the questionnaire . Once the PARQ was developed, this team was consulted to determine its face and content validity, and its feasibility .
The PARQ was tested for construct validity and test-retest reliability, which were shown to be satisfactory when medication and exercise scores were compared with the General Adherence Scale and to diary reports of medication and exercise-related behaviors . The PARQ was pilot tested for ease of use in five English-speaking caregivers of youth with JIA, and translated into French by a bilingual professional translator . The PARQ has since been used in various research projects [3, 5, 8] and a child-report version (i.e., Child Adherence Report Questionnaire (CARQ)) has been developed and pilot-tested among youth with JIA [3, 5].
The PARQ focuses on the domain of patient-related factors (e.g., knowledge, attitudes, beliefs, perceptions, and expectations) of the WHO model. It assesses which member of the family is responsible for making sure that the child adheres to treatment. The following items are scored on a 100 mm visual analog scale (VAS): frequency with which children follow their prescribed treatments (medication, exercise regimen and splints), difficulties experienced in following the various forms of treatment, the frequency of negative reactions associated with following the various forms of treatment, and the degree to which the treatments are perceived as helpful. The Morisky scale, an index addressing barriers to medication adherence  is also included in the PARQ. The Morisky scale is comprised of four yes/no questions related to forgetting to take medication, neglecting to take medication, stopping the medication when the child felt better or worse than before, and choosing a type of medication (4-point scale). Finally, parent and child treatment preferences and perceptions regarding treatment helpfulness and also dissatisfaction with care are included as these may influence adherence.
Rasch analyses were performed on data collected from two studies conducted in Vancouver and Montreal (Canada) in which the PARQ was used as a measure of adherence. The first study was a longitudinal survey assessing the adherence to treatment and its associated factors (e.g., child’s disease severity, child’s age, caregivers’ perceived helpfulness of treatments, use of complementary and alternative health care) in a sample of parents of 180 children with JIA [2, 8]. The second was a cross-sectional survey of 55 children with JIA and their parents to compare their respective perceptions of treatment adherence and quality of life and to determine the association between adherence and health outcomes . In both studies, we also assessed health related quality of life (HRQOL) using the Juvenile Arthritis Quality of Life Questionnaire (JAQQ) and extracted information from medical charts on disease severity (using the active joint count, representing the number of joints with active inflammation as rated by the rheumatologist) and disease duration. Thus, from the two studies, we had a total of 235 PARQ baseline questionnaires for this study.
First, a principal component analysis (PCA) was performed to determine the factor structure and the dimensionality of the PARQ questionnaire using SAS (version 9.0). Rasch analyses were then conducted to assess how well the data fit the Rasch partial credit model. Content validity, construct validity, as well as other psychometric properties, such as the appropriateness of response categories, floor and ceiling effects, item bias and reliability were evaluated. The RUMM 2030  computer software program was used for Rasch analyses .
Principal component analysis
We analyzed the PARQ baseline data to perform a preliminary examination of its dimensionality and factor structure using PCA .
Ideally, for items to “fit” the model and for the questionnaire to have adequate content validity, items should be spread evenly on the continuum of difficulty level and have a wide range (from at least −3 to +3 logits) (see Fig. 1).
Construct validity is attained when persons and items have adequate fit statistics (e.g., item and person standardized fit residuals between ±2.5 with a mean of 0 and non-significant chi-squares and F statistics). These methods and their criteria are fully described elsewhere [14–17]. Construct validity is also attained when none of the items display differential item functioning (DIF) or item bias . Items are considered biased if they change their level of difficulty depending on the group of persons being assessed (e.g., boy vs. girl), which violates the expectations of the Rasch model. DIF was deemed to be present if analyses of variance were significant (Bonferroni-corrected p value of 0.001389).
The usefulness of the response categories was assessed for adequacy of their response options. For polytomous items, responses should be adequately distributed across the response categories, and this is usually indicated as a minimum of 10 observations in each rating scale category . The reliability of the questionnaire was assessed by the person separation index, which is interpreted as a Cronbach’s α . It indicates how well the items discriminate persons into different ability levels.
Demographic and disease related characteristics of the families included in the analyses for the PARQ
n = 235 parents of children with JIAa
Children’s sex, female, n (%)
Children’s age, years, mean (SD)
Disease duration, years, mean (SD)
Disease severity, AJC, mean (SD)
HRQOL, mean (SD)b
Pain, mean (SD)c
Principal component analyses
While the type of data for some of the items (ordinal) along with missing data precluded definite conclusions about the factor structure revealed by the PCA, it helped us identify and understand the number of dimensions within the construct and to identify items unrelated to a one-dimensional concept of adherence. The PCA indicated the presence of two main factors that explained 38 and 27 % of the variance respectively. Most of the 17 items loaded on the first factor. However, three items did not load on the first factor: Forgetting to take medication during the last 3 months, neglecting to take medication during the last 3 months and difficulty in doing prescribed exercises. These items appear to form a second factor and may contribute to the multi-dimensionality of the scale. All items were retained for the Rasch analyses of the PARQ questionnaire.
The fit of the baseline data when all 17 items of the PARQ were considered produced a significant item-trait interaction (chi-square = 244.76; p < 0.05). This is an indication that the data did not fit the Rasch model. All items originally scored as a visual analog scale displayed disordered thresholds meaning that the scoring options (0–100 mm) were not adequate. Most response options were massively underutilized. The fit of the individual items revealed that 5 out of 17 items did not fit the model.
The remaining PARQ items and their psychometric properties by location order once rescored and misfitting items removed
Difficulty in taking prescribed medication
Negative reaction to taking prescribed medication
Negative reaction to doing prescribed exercises
Negative reaction to wearing splint
Difficulty in exercises
Difficulty in wearing splint
Adherence to taking prescribed medication
Usefulness of taking prescribed medication
Usefulness of doing prescribed exercises
Adherence to doing prescribed exercises
Usefulness of wearing splint
Adherence to wearing splint
As for content validity, it was assessed by examination of the item-person map (Fig. 1), which shows the distributions of persons (top) and items (bottom) for the transformed PARQ. The item thresholds spread from approximately −10 to 13 logits, which is adequate.
Evidence of construct validity was ascertained as all items and persons had adequate fit statistics and none displayed DIF. The usefulness of the response categories was ascertained as all were found to have adequate fit statistics. Responses were also adequately distributed across the response categories, as there were more than 10 observations in each rating scale category.
Figure 1 is also useful for evaluating ceiling and floor effects. As no persons are located to the right or the left of the outermost items at the right and left of the graph, no ceiling or floor effects are present. The reliability index was 0.69 indicating that the transformed PARQ can reasonably discriminate persons into different ability levels. The new version of the PARQ is shown in the Appendix (https://www.dropbox.com/sh/tcdckuc6kfyd0vn/AAAu3yYHbD2uWOgCg9Lu3N3ha?dl=0).
To assess whether the transformed PARQ is unidimensional, we performed a post-hoc test of unidimensionality. According to the Smith’s t-tests , 6 out of 211 t-tests (2.84 %) showed significant differences in the estimates generated. Because less than 10 % of the t tests are significant, the transformed PARQ is considered unidimensional .
The aim of the present study was to evaluate the psychometric properties of the PARQ. Results of the PCA showed preliminary evidence of the multidimensionality of the original version of the PARQ and did not meet the expectations of the Rasch model. The questionnaire was transformed to determine if we could achieve a better fit to the Rasch model. These transformations include modifying the scales of some of the items (from VAS to five point Likert scale) and removing the Morisky scale items. After transformation, the items met the expectations of the Rasch model. This means that the PARQ assesses parent beliefs and behaviors related to adherence to various treatments prescribed for JIA in a cohesive manner (i.e., unidimensional). This will also allow for the computation of a total score, thus facilitating the use of the questionnaire and the conduct of quantitative analysis. Such a score is important to ensure the interpretability of the measure  and to facilitate communication amongst researchers and health professionals.
To our knowledge, the PARQ is the first proxy-report questionnaire assessing adherence to various JIA treatments which has undergone validity testing using Rasch analysis. Furthermore, this questionnaire is the only one to evaluate adherence to JIA treatments, as well as both parents’ attitudes and beliefs about these treatments. Assessing not only adherence behaviors but also related beliefs is important to optimize adherence, as they are the most important predictors of reported adherence .
The Rasch-validated version of the PARQ is a first step towards establishing a total score indicator of adherence. The transformed PARQ shows preliminary evidence of unidimensionality and may allow computation of a total score. Limitations of the current study include the type of data for some of the items (ordinal) along with missing data, which precluded definite conclusions about the factor structure revealed by the PCA. Although missing data lowered the validity of the fit statistics in the Rasch analysis, the estimates of the fit statistics should not be biased  as the sample size was sufficient .
Further testing is needed to ensure the unidimensionality of the transformed version of the PARQ and to ensure its psychometric characteristics. Such endeavour is underway in a different sample of parents of youth with JIA. Using the PARQ will help to document parents’ beliefs and behaviors related to adherence to various treatments prescribed for JIA, to identify adherence issues and to put in place measures to ensure optimal adherence and health outcomes. The PARQ can play an important role in research to help assess whether adherence is optimal in intervention studies in order to judge the effectiveness of a treatment. This questionnaire can also play an important role in clinical practice to help start a discussion to better understand parents’ attitudes and beliefs about JIA treatments in order to address barriers to adherence.
Future studies will also aim at validating the child-report version (CARQ), which would allow for assessing children’s report of adherence to JIA treatments, as well as their attitudes and beliefs about these treatments. Assessing children’s and parents’ perceptions about treatments is particularly important since they have been shown to differ . Using the PARQ and CARQ in combination could help health professionals to thoroughly assess family’s perceptions and behaviours about treatments, communicate information about the various treatments, and tailor treatments to both children’s and parents’ needs.
The current research represents an important step in evaluating the psychometric properties of the PARQ, a questionnaire that assesses beliefs and behaviors related to adherence to treatments prescribed for JIA. The Rasch analysis led to the transformation of the PARQ showing preliminary evidence of unidimensionality. Future work will verify these findings and also validate a child-report version of this questionnaire to thoroughly assess treatment adherence and address its barriers, both in research and in clinical practice.
This study was funded by the Canadian Institutes of Health Research (CIHR). CIHR had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.
Availability of data and materials
The dataset supporting the conclusions of this article is not publicly available because of a stipulation in the consent forms.
All authors participated in the concept and design; analysis and interpretation of data; drafting or revising of the manuscript. KTA conceptualized and designed the study, completed data collection, analysis and interpretation; and drafted, reviewed and approved the final manuscript. JH conceptualized and designed the study, completed data collection, analysis and interpretation; and drafted, reviewed and approved the final manuscript. DEF conceptualized and designed the study, completed data collection, analysis and interpretation; and drafted, reviewed and approved the final manuscript. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
The study was approved by the Research Ethics Board of the Montreal Children’s Hospital and the British Columbia Children’s Hospital in Vancouver. Participants signed a consent form in order to participate.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Quittner AL, Modi AC, Lemanek KL, Ievers-Landis CE, Rapoff MA. Evidence-based assessment of adherence to medical treatments in pediatric psychology. J Pediatr Psychol. 2008;33(9):916–36.View ArticlePubMedGoogle Scholar
- Feldman DE, De Civita M, Dobkin PL, Malleson P, Meshefedjian G, Duffy CM. Perceived adherence to prescribed treatment in juvenile idiopathic arthritis (JIA) over a one-year period. Arthritis Rheum. 2007;57(2):226–33.View ArticlePubMedGoogle Scholar
- April KT, Feldman DE, Zunzunegui MV, Duffy CM. Association between perceived treatment adherence and health-related quality of life in children with juvenile idiopathic arthritis: perspectives of both parents and children. Patient Prefer Adherence. 2008;2:121–8.PubMedPubMed CentralGoogle Scholar
- De Civita M, Dobkin P, Feldman D, Karp I, Duffy C. Development and preliminary reproducibility and validity of the Parent Adherence Report Questionnaire:A measure of adherence in Juvenile Idiopathic Arthritis. J Clin Psychol Med Settings. 2005;12(1):1–12.View ArticleGoogle Scholar
- Toupin April K, Ehrmann Feldman D, Platt RW, Duffy CM. Comparison between children with juvenile idiopathic arthritis (JIA) and their parents concerning perceived treatment adherence. Arthritis Rheum. 2006;55(4):558–63.View ArticleGoogle Scholar
- Surges D. Rasch analysis: an introduction to objective measurement laboratory medicine. Lab Med. 2000;31:272–4.View ArticleGoogle Scholar
- World Health Organization. Adherence to long-term therapies: evidence for action. 2003.Google Scholar
- Toupin-April K, Feldman DE, Zunzunegui MV, Descarreaux M, Malleson P, Duffy CM. Is complementary and alternative healthcare use associated with better outcomes in children with juvenile idiopathic arthritis? J Rheumatol. 2009;36(10):2302–7.View ArticlePubMedGoogle Scholar
- Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self-reported measure of medication adherence. Med Care. 1986;24(1):67–74.View ArticlePubMedGoogle Scholar
- Andrich D, Sheridan B, Luo G. RUMM2030: Rasch unidimensional models for measurement. Perth: RUMM Laboratory; 2009.Google Scholar
- Zwinderman AH. Pairwise estimation in the Rasch models. Appl Psychol Meas. 1995;19(4):369–75.View ArticleGoogle Scholar
- Tabachnick BG, Fidell LS. Using multivariate statistics. Boston: Allyn and Bacon; 2012.Google Scholar
- Andrich D. A general form of Rasch’s extended logistic model for partial credit scoring. Applied Measurement in Education. 1988;1:363–78.View ArticleGoogle Scholar
- Forget NJ, Jerosch-Herold C, Shepstone L, Higgins J. Psychometric evaluation of the Disabilities of the Arm, Shoulder and Hand (DASH) with Dupuytren’s contracture: validity evidence using Rasch modeling. BMC Musculoskelet Disord. 2014;15:361.View ArticlePubMedPubMed CentralGoogle Scholar
- Wright BD, Masters GN. Rating scale analysis: Rasch measurement. 1st ed. Chicago: Mesa Press, Pluribus Pr; 1982.Google Scholar
- Andrich D. A rating formulation for ordered response categories. Psychometrika. 1978;43:561–73.View ArticleGoogle Scholar
- Bond TG, Fox C. Applying the Rasch model. Mahwah: Lawrence Erlbaum Associates; 2001.Google Scholar
- Lim SM, Rodger S, Brown T. Using Rasch Analysis for establishing the construct validity of rehabilitation assessment tools. Int J Ther Rehabil. 2009;16(5):251–60.View ArticleGoogle Scholar
- Linacre JM. Optimizing rating scale category effectiveness. J Appl Meas. 2002;3(1):85–106.PubMedGoogle Scholar
- Bond TG, Fox CM. Applying the Rasch model: fundamental measurement in the human sciences. 1st ed. Mahwah: Lawrence Erlbaum Associates; Psychology Press; 2001.Google Scholar
- Smith Jr EV. Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. J Appl Meas. 2002;3:205–31.PubMedGoogle Scholar
- Tennant A, Pallant JF. Unidimensionality matters! (A tale of two Smiths?). Rasch Meas Trans. 2006;20:1048–51.Google Scholar
- Wolfe EW, Smith Jr EV. Instrument development tools and activities for measure validation using Rasch models: part II--validation activities. J Appl Meas. 2007;8(2):204–34.PubMedGoogle Scholar
- Winsteps Help for Rasch Analysis. http://www.winsteps.com/winman/index.htm?estimation.htm. 2015 [cited 2013 Aug].
- Linacre JM. Sample size and item calibration stability. Rasch Meas Trans. 1994;7:328.Google Scholar