Endpoints and outcomes for localized scleroderma/morphea: a scoping literature review

Abstract

Background

Current treatment for localized scleroderma (LS) has been shown to halt disease activity, but little is still known about patient experiences with these treatments, nor is there consensus about optimal measurement strategies for future clinical trials.

Objective

Conduct a scoping review of the literature for the types of outcomes and measures (i.e. clinician-, patient-, and caregiver-reported) utilized in published treatment studies of LS.

Methods

Online databases were searched for articles related to the evaluation of treatment efficacy in LS with a special focus on pediatrics.

Results

Of the 168 studies, the most common outcomes used were cutaneous disease activity and damage measured via clinician-reported assessments. The most frequently cited measure was the Localized Scleroderma Cutaneous Assessment Tool (LoSCAT). Few patient-reported outcome measures (PROMs) were used.

Limitations

Some studies only vaguely reported the measures utilized, and the review yielded a low number of clinical trials.

Conclusion

In addition to evaluating disease activity with clinician-reported measures, the field could obtain critical knowledge on the patient experience by including high-quality PROMs of symptoms and functioning. More clinical trials using a variety of outcomes and measures are necessary to determine the most suitable course of treatment for LS patients.

Capsule summary

Existing treatments for localized scleroderma halt disease activity but little is known about patient experiences with these treatments.

Clinician-reported measures are over represented in published trials. Better integration of patient-reported measures, alongside clinical outcomes, would more completely evaluate treatment efficacy in future trials by incorporating the patient perspective.

Localized scleroderma (LS) or morphea is a rare rheumatic condition that causes atrophy, fibrosis, and sclerosis of the skin and underlying tissues [1]. Morbidity is common for individuals with LS, who can be adversely impacted via skin damage, physical deformity and dysfunction (especially during childhood onset), permanent cosmetic issues, and extracutaneous manifestations [2, 3].

The conduct of clinical trials for LS encounters a number of challenges common for rare diseases, including hard to reach patient populations, low sample sizes, and a lack of consensus regarding clinical outcome assessments (COAs). Currently, there are no FDA approved treatments for LS, although consensus-based groups have started compiling evidence across clinical sites. Current regimens have been shown to be efficacious at halting disease activity, [4] but recent qualitative work has shed light on the negative experiences of patients under treatment, including side effects and overall burden [5, 6]. Further, recent attempts at meta-analyses have been unsuccessful due to the wide variation in how successful treatment efficacy has been defined across studies, clinics, and registries [7].

Much needed clinical trials to find efficacious treatments or comparable treatments with lower burden cannot occur without the identification of relevant patient-centered outcomes and high-quality clinical outcome assessments (COAs; i.e. the measures themselves). This study aims to explore and catalogue the current and past outcomes and COAs used to evaluate treatment efficacy in children and adults diagnosed with LS, with particular focus on pediatric patients who have a higher cumulative disease burden, [8] and the use of patient-reported outcomes (PROs). We expect this review to spur important clinical research into this population and identify new areas for measurement developers to pursue. We also expect to be able to discuss and provide recommendations for researchers on best practices for citing quality evidence supporting outcome/endpoint choices.

Methods

Design

This work is a scoping review. It was carried out using the Cochrane Handbook for Systematic Reviews of Interventions [9] and was reported following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement: an updated guideline for reporting systematic reviews [10]. The search was developed and conducted by a professional medical librarian in consultation with the author team and included a mix of keywords and subject headings representing localized scleroderma, morphea, and pediatrics. Search hedges or database filters were used to remove publication types such as editorials, case reports, comments and animal-only studies as was appropriate for each database. The databases searched included MEDLINE (PubMed), Embase (Elsevier), and Web of Science (Clarivate). The search was conducted on September 19, 2019 and updated on April 20, 2022 and found 3,212 citations. Complete reproducible search strategies, including date ranges and search filters are detailed in the Supplementary Materials.

Eligibility criteria

To be included in the review, studies had to meet all inclusion criteria as follows (1) the inclusion of human subjects, (2) at least some participants were diagnosed with LS, (3) reported on the treatment of patients, (4) reported at least one outcome that was linked to treatment, and (5) were available in English. Studies also were required to be primary data sources. Eligible studies included observational cohorts, case studies with 3 or more individuals, randomized clinical trials, and reports on large registries. The focus of the study was on juvenile LS, with the expectation that most studies would be have at least some pediatric patients, however, studies that included only adults with LS were not excluded to maximize the number of relevant articles.

Study selection

After the search, all identified studies were uploaded into Covidence (Veritas Health Innovation, Melbourne, Australia). The duplicates were removed by the software (n = 1,190) and a final set of 2,022 citations were left to be screened in the title/abstract phase. Study selection was carried out independently by two authors. The article selection is presented by flowchart as per PRISMA guidelines (Figure 1). For the full-text screening stage, papers were also reviewed in detail by two independent reviewers and were excluded if they did not meet the inclusion criteria. At any stage, all disagreements were resolved by discussion, consensus, and later on by the study investigator.

Fig. 1

PRISMA Flow Diagram for the scoping review detailing the number of studies screened, reviewed, excluded, and the number of full text articles included in data extraction

Data extraction

Data elements of interest included general study information (i.e. year of publication, type of study), quality of study, sample size/number of LS patients, age of patients, types and subtypes of morphea/LS, treatment, and information on outcomes and COAs. As per the National Institutes of Health Biomarkers, Endpoints, and other Tools (BEST) glossary, [11] an ‘outcome’ was defined as the domains of symptoms and functioning defined by the authors of the included articles (e.g. ‘disease activity’), while the COA was defined as the overall system of measurement including the survey/questions, the method for obtaining measurement, and the method of interpretation. Of particular interest, the type of COA was recorded and results are reported separately for different types of clinical outcome measures as defined in Table 1.

Table 1 Clinical outcome assessment (COA) types and definitions

Quality assessment

To assess the methodological quality of the cited evidence for the COAs being utilized, we intended to take a modified COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) approach [12, 13] However, due to general lack of details included in the articles, the quality assessment of the validity evidence cited was binary (yes - any reliability or validity information was cited in the methods to support use of the measure, or no – no support was cited). Further, quality at the study level was evaluated using the following criteria: (1) the research question was clearly stated, (2) the sample size was justified, (3) the treatment protocol was standardized, (4) there was a control/placebo group, and (5) blinding of the treatment condition was present.

Synthesis methods

Specific variables were catalogued across studies using pre-specified definitions. Additionally, for each outcome and COA, specific characteristics were extracted and tabulated for each category. For example, studies could and often did include multiple outcomes, meaning that the frequencies and percentages for reported outcome categories add up to more than 100%. Due to inconsistencies across articles in how outcomes and COAs were reported, we also include descriptive narratives of our findings for each type of COA.

Results

Study characteristics and quality assessment

The 168 studies in the final evaluation sample were conducted in a total of 35 countries (~ 30% of studies conducted in the United States). Almost half (78/168) of studies were retrospective, i.e. medical chart reviews. As such, in terms of study quality, 90% of articles stated the research question, but the majority of articles did not include a sample size justification (93%), a control group (92%), or any masking of the treatment effect (76%). Further, only 46% of articles reported on efficacy of a standardized treatment regimen. Further detail on study characteristics can be found in Supplemental Materials.

Patients included in the studies comprised all subtypes, with craniofacial scleroderma and generalized morphea being the most commonly reported (Table 2). The majority of treatments evaluated in the studies were systemic medications, followed by phototherapy/UV therapy, topical creams, or reconstructive surgery (Table 2). More than half of studies (56%) reported on multiple treatments.

Table 2 Frequency and percentages across the 168 included articles for reported subtypes of LS and evaluated treatments

Outcomes and endpoints

Treatment outcomes

The most common outcome reported across all articles was “disease activity” (111/168), although the operationalization of this term varied across studies (Table 3). Other outcomes names by authors to determine treatment efficacy included “disease damage”, “skin thickness”, “lesion size”, “surgical outcomes”, and “clinician satisfaction”. Outcomes utilized to evaluate other aspects of treatment from the patient perspective commonly included “health-related quality of life” and “patient satisfaction/success” (Table 3).

Table 3 Treatment outcomes as categorized by authors of included articles

Types of COAs

The most common way treatment outcomes were measured were via clinical assessments that were both informal and formal/standardized (59%) along with other measures of health (40%; blood tests, imaging; Table 1). Patient- or observer-reported measures (like those completed by a parent) were used more infrequently (26%).

Clinical assessment of treatment outcomes. Almost 59% percent of articles (99/168) included some type of clinical assessment of treatment benefit that was commonly conceptualized as improvement, progression, and/or change in disease status (e.g. activity or damage). Disease progression typically included the appearance of new lesions or the expansion of pre-existing lesions over a specific period of time. Disease improvement typically was defined as lesions getting smaller or disappearing. Change in disease status was often conceptualized as a “halt” in disease activity or progression.

The clinical assessments within this category included clinical evaluations that both did and did not explicitly utilize a COA. Some clinical assessments were described as being less formal in nature and did not utilize specific criteria or time frame when reporting on the clinical evaluation.

Other clinical assessments for disease activity were more formal and did include specific criteria that was utilized to categorize patients. For example, one study [14] evaluated defined clinical improvement in activity as the absence of the criteria suggested by Careta and Romiti: appearance of new lesions in the last 3 months, expansion of the pre-existing lesion in the last 3 months, moderate or severe erythema or skin lesions with erythematous borders, violaceous lesion or lesion border, increased induration of the lesion border, and worsening of the hair loss on the scalp, eyebrows or eyelashes. Other clinical assessments described in the articles included the evaluation of surgical outcomes or general clinician satisfaction/success by assessing facial symmetry, complications, and need for surgical revisions.

Clinician-reported outcome measures. The most frequently named ClinRO(s) across studies was the Localized Scleroderma Assessment Tool (LoSCAT) [15, 16] and/or its components (18.5%; 31/168), the modified Localized Skin Severity Index (mLoSSI), which measures disease activity, and the Localized Scleroderma Damage Index (LoSDI), which measures disease damage. All studies that utilized the LoSCAT or its components cited validity and/or reliability information in the article, including general PRO measure development, content validity, reliability, and responsiveness. The LoSCAT was most often used in association with disease activity.

Physician-global assessments (PGAs) [15, 16] were utilized in 10.7% of studies (18/168), and often together with the LoSCAT, focused on disease activity. Other named ClinROs included skin scores (7.1%; 12/168), such as the modified Rodnan Skin Score (mRSS), most often utilized to assess skin thickness, and other clinical activity scores (e.g. ‘CAS’ or LS Cutaneous Activity Measure; 1.8%; 3/168). Other named ClinROs that were utilized infrequently were the Quantitative facial symmetry score and the Derriford Appearance Scale (both n = 1) which were linked to surgical outcomes.

Patient-reported/Observer-reported outcome measures. Patient-reported outcomes (PROs) and/or observer-reported outcomes (ObsROs; typically a child’s caregiver) were included in about 26% of the manuscripts (43/168). PROs were most often used to evaluate treatment outcomes described as ‘health-related quality of life’ (HRQOL) and ‘patient satisfaction/success’. Some of the PROs/ObsROs utilized to evaluate HRQOL were listed as the World Health Organization Quality of Life (WHOQOL-110) [17], the Children’s Dermatology Life Quality Index (CDLQI), [18] the Scleroderma Health Assessment Questionnaire (SHAQ), [19] Child Health Assessment Questionnaire (CHAQ), [20] Pediatric Quality of Life (family impact, generic, rheumatology module), [21, 22] and visual analog scales (VAS) for symptoms.

Many of the patient satisfaction outcomes were linked to patient satisfaction with surgery and did not utilize specific PROs, but instead noted if the patient was satisfied. For example, one study had patients rate their satisfaction using [1] very good/very satisfied, [2] good/satisfied, [3] bad/not satisfied, and [4] very bad/very dissatisfied [23]. Some studies used unnamed ObsROs and PROs to assess if the patient (or caregiver) noticed an improvement, no change, or worsening of the disease.

Performance-based outcome measures. Performance-based outcome measures (PerfOs) were infrequently utilized to evaluate treatment benefit (5.4%; 9/168), and when included, typically were related to measuring joint mobility/range of motion before and after treatment.

Other measures of health

Other measures of health were included in about 40% of articles (67/168). The most frequent external equipment used to assess outcomes were magnetic resonance imaging (MRI), ultrasound, thermography, and photographs. In one study [24], MRI was used before and after 6 months of treatment to evaluate the depth and thickness of the soft-tissue structures and the degree of inflammation and edema. The MRI images were graded from 0 to 10 by an experienced and blinded radiologist. The MRI scores were also compared and scored from 0 (no improvement) to 10 (total healing) by investigators. These outcome measures were often used as a supplement to other COAs. Additionally, authors commonly reported using photographs when evaluating surgical outcomes.

Discussion

While the field of LS research has been historically limited to case studies and retrospective chart reviews, with the development of large, multi-site research registries and coordination among international clinical experts, our understanding of the molecular basis of LS/morphea and the efficacy of treatments has grown. The more recent studies have started to recognize the importance of switching our focus from only inclusion of traditional clinical outcomes (i.e. disease activity) to clinical outcomes alongside complementary and critical patient experience data. As stated in the U.S. Food & Drug Administration’s new Patient-Focused Drug Development Guidances, patient-reported outcome measures are “useful for assessment of symptoms, functioning, events, or other aspects of health from the patient’s perspective” [25, 26]. As our knowledge of LS continues to be refined, we hope that additional patient data will also be captured including tolerability of novel and legacy treatments and treatment impact on the presence and severity of extracutaneous manifestations.

Further, as new research studies are planned and executed, the field can improve in terms of the published justifications to support use of a specific COA. Quite a few published studies in this review did not report use of existing COAs, and thus, there is little record of their thought process when conceptualizing these important outcomes. Additionally, the field is starting to gather information on the reliability and validity of outcome measures used in this population, with the LoSCAT and the Localized Scleroderma Quality of Life Instrument (LoSQI) [27,28,29] being two measures with support specifically gathered from patients with LS.

While PerfOs were the least utilized COA, a large number of studies used other measures of health as part of their evaluation. Although the intention of using equipment is to provide a more objective evaluation of treatment, some of these measures (e.g. MRIs and photographs) required the output to be graded by a rater, perhaps giving the illusion of objectivity, but having the same possibility for error as other types of subjective measures.

One limitation of this review is publication bias, in which investigators might have used certain outcome measures, but chose not to report on them. We also did not include articles published in languages other than English, which may have contained useful information. Some manuscripts in this review were vague and could not be reproduced due to lack of reporting on how exactly they assessed treatment. Most importantly, our review did not yield many randomized clinical trials, which supports the low utilization of control groups and blinding. This reveals more clinical trials evaluating treatment efficacy for this population are needed.

Conclusion

This review demonstrates a critical need for high quality, well-designed randomized clinical trials and comparative effectiveness studies to evaluate treatment efficacy of new and legacy treatments in patients with LS. Past and current studies have focused primarily on secondary data collection, and when prospective studies are conducted, patient-reported outcomes or other measures collected from patients and families are less of a focus, although important strides have been made recently. As the field moves forward, it is imperative to use validated and high quality COAs to evaluate meaningful aspects of treatment and for publishing authors to include stronger justifications and support for their choice of measures. The inclusion of complementary ClinROs, PROs, and ObsROs will go a long way to ensure future research and clinical care is patient-centered, and that we are able to compare treatments on their ability to control disease activity but also to improve (and not worsen) patient symptoms and negative side effects.