Modified Delphi study to identify priority clinical questions for the Australian living guidelines for the management of Juvenile Idiopathic Arthritis
Pediatric Rheumatology volume 20, Article number: 52 (2022)
Juvenile Idiopathic Arthritis (JIA) is the most common rheumatic inflammatory disease in childhood. Optimal management requires clinicians to be up to date with the rapidly evolving evidence base. ‘Living’ evidence-based clinical practice guidelines, which integrate new evidence as soon as it is available, are a novel method to enhance the translation of research into practice. To determine the most relevant questions that should be prioritised in national Australian JIA living guidelines, we invited Australian and New Zealand paediatric rheumatologists and other relevant health professionals to identify and rank their most important questions in order of priority.
All 47 members of the Australian Paediatric Rheumatology Group (APRG) were invited to participate in a modified Delphi study comprising two rounds. The first round identified demographic information of respondents, current attitudes to guideline use and invited submission of priority management questions. The second round asked respondents to rank 27 collated and refined questions identified in round one in order of priority.
There were 29 (62%) and 28 (60%) responses to the first and second survey rounds respectively. About two thirds were rheumatologists or trainees (66, 68%), nearly half had more than 10 years of experience (45, 46%) and practice setting was largely hospital (79, 86%) and urban (86, 75%). Most respondents used clinical guidelines in their practice (72% sometimes, 24% often), most frequently American College of Rheumatology (ACR) (66%) and European Alliance of Associations for Rheumatology (EULAR) (59%) guidelines. Reported barriers to guideline use included that they are not up to date and access difficulties. Most respondents (83%) considered Australian guidelines were necessary and two-thirds indicated they would use them if integrated into practice software. The highest ranked topics were down-titration and discontinuation of disease modifying anti-rheumatic drugs (ranked first), best outcome measures (second) and treatment targets in JIA (third).
There is strong clinician support for the development of Australian living guidelines for JIA. Consensus was reached on the ten top-ranked priority questions. Our guidelines will develop evidence-based recommendations for these high priority questions that will be updated in real time as needed to facilitate rapid translation of evidence into clinical practice.
To effectively practice evidence-based medicine, clinicians need to keep abreast of the latest evidence in their field, an increasing challenge as the volume of new evidence rapidly expands . Systematic reviews are a valuable tool for synthesising evidence but are only useful if they are up to date. ‘Living’ systematic reviews are a new approach that use novel methods such as regular automated searches to continuously update reviews as new evidence becomes available .
Living reviews can also be incorporated into ‘living’ clinical practice guidelines by updating individual recommendations within a dynamic, web-hosted guideline structure as soon as relevant new evidence emerges . A further advantage of living guidelines is that they can be developed in a stepwise format, in which additional living recommendations can be added over time as resources permit.
To optimise immediate effect, topics for inclusion can be prioritised using a systematic approach. Such a framework has been used by the 3e (Evidence, Expertise, Exchange) Initiative that has developed international clinical practice guidelines on topics including undifferentiated peripheral inflammatory arthritis , pain management in inflammatory arthritis  and gout . Using this method, guidance on the most relevant clinical questions was developed.
To our knowledge, this process has not been performed in the field of paediatric rheumatology. Juvenile idiopathic arthritis (JIA) is the most common chronic inflammatory rheumatic disease in childhood and could be well suited to a living guideline format given the expanding range of available treatments. The most recent Australian guideline for the management of JIA was published in 2009  and many of its recommendations are outdated. The Australia and New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network, in partnership with the Australian Paediatric Rheumatology Group (APRG) and Australian Rheumatology Association (ARA), is developing Australian living guidelines for the management of JIA in parallel with the development of living guidelines for adult rheumatic diseases .
Our process of living guideline development provides a unique training opportunity to members of the multidisciplinary guideline panels in evidence-based practice, living evidence and guideline development methods. Involvement of clinicians early in guideline development also improves the relevance of the recommendations and may increase guideline uptake to hasten translation of evidence into practice [5, 9]. To facilitate a living guideline that is immediately useful, we performed a modified Delphi study to reach consensus on the most clinically relevant questions, ranked in order of priority, that APRG clinicians would like addressed in the Australian JIA living guidelines.
Design and participants
We used a modified Delphi technique including two survey rounds . The invitation to the first and second surveys were sent 31 January and 9 March 2021 respectively.
All members of the APRG (N = 47) were invited to participate by email. This group includes all currently practising paediatric rheumatologists in Australia and New Zealand, all advanced trainees in paediatric rheumatology, adult rheumatologists with an interest in paediatric rheumatology, and rheumatology health professionals, including physiotherapists, occupational therapists, research assistants and nurses.
To maximise participation, a follow-up email was sent 2 weeks after the initial invitation for each Delphi round. Each survey link was closed 6 weeks after the first email. Individual responses were anonymised. All APRG members were invited to participate in the second round, regardless of their participation status in the first round.
Survey tool and procedure
An online platform (SurveyMonkey, Momentive Inc. San Mateo, California, USA) was used to collect the data. Surveys from the first and second rounds are appended (Additional files 1 and 2 respectively).
The first round included an introduction to the concept of living guidelines, an overview of the ANZMUSC JIA living guidelines project and described the aim of the study. Participants were invited to submit at least 3 and up to 10 questions they thought were most relevant to the management of JIA via free-text entry. Example questions were provided for guidance. We also collected demographic information and asked respondents about current use of JIA guidelines and beliefs about the need for local guidelines (Additional file 1).
The complete list of questions identified in the first round was refined and duplicate questions were removed. The refined list of questions was provided to respondents in random order in the second round. Respondents were asked to indicate the ten questions they considered most important and then to rank them from most to least important.
Using results from the second round, each question was assigned a score with 10 points for questions that were ranked first to 1 point for questions that were ranked tenth. A priority list was created according to the sum of ranking scores. The final ranked order of questions, their score, and the absolute number of respondents who voted for a particular question are reported.
There were 29 (62%) and 28 (60%) responses to the first and second rounds respectively. The characteristics of respondents in comparison to the APRG membership as a whole is summarised in Table 1. The gender and discipline breakdown of respondents reflected that of the APRG membership. Most respondents in both rounds were paediatric rheumatologists (first round n = 15, 52%; second round n = 16, 57%) or trainees (n = 4, 14%; n = 3, 11%). Thirteen respondents in both survey rounds (45 and 46%) had more than 10 years’ experience in paediatric rheumatology, and the majority work in urban (n = 25, 86%; n = 21, 75%) and hospital (n = 23, 79%; n = 24, 86%) settings.
Most respondents indicated they sometimes use guidelines in their usual practice (n = 21, 72%), while almost a quarter (n = 7, 24%) indicated they used them often. The guidelines most commonly used by respondents were from the American College of Rheumatology (ACR) (n = 19, 66%) and European Alliance of Associations for Rheumatology (EULAR) (n = 17, 59%).
Several barriers to using existing guidelines were reported. Thirteen respondents (45%) indicated guidelines were not up to date and 10 (34%) indicated they had difficulties accessing a guideline. Most respondents (n = 24, 83%), indicated that Australian paediatric rheumatology guidelines are necessary and 19 (66%) would use them if integrated into practice software.
There were 134 questions proposed in the first round. The most common questions related to tapering of disease-modifying antirheumatic drugs (DMARDs) identified by 12 (41%) respondents, the most appropriate outcomes to measure in clinical practice identified by seven (24%) respondents and immunisations identified by six (21%) respondents. After refinement and duplicate removal, there were 27 questions presented in the second round.
Table 2 shows the most important questions and their rankings as identified in round two. The highest ranked question concerned tapering and discontinuation of DMARDs in patients with JIA who have responded well to treatment (score = 140). It was ranked in the top 10 by 21 of 24 (87%) respondents. Other highly ranked issues related to outcome measures / treatment targets (score = 90, 11 respondents), use of oral glucocorticoids (score = 81, 14 respondents), use of methotrexate (score = 78, 13 respondents), and DMARD strategy based on JIA subtype (score = 78, 12 respondents).
Most health professionals involved in the specialist care of Australian children with JIA and who responded to our survey use guidelines as part of routine clinical care. They were also strongly supportive of locally produced Australian JIA guidelines despite the existence of guidelines for JIA developed elsewhere. Australian living guidelines would ensure recommendations are relevant to the local context. In addition, respondents reported concerns about guidelines often being out of date. A single repository for JIA management recommendations, that are updated as new evidence emerges, and available and accessible at the point of care, would address these concerns.
It has been noted that needs raised by clinicians and consumers are not always reflected in the research that is performed . Topics without clear evidence to guide practice were understandably highly prioritised as being important to respondents. For example, there is a paucity of high quality research investigating down-titration and discontinuation of DMARDs in JIA patients with inactive disease and a large variation in clinical practice currently exists . While recently published guidelines have not addressed this issue, this was our top ranked priority. It has also been identified as a priority topic for other guidelines [13, 14].
The second ranked question, concerning the best outcome measures and treatment targets in JIA, is also not addressed in existing guidelines. Incorporating standardised outcome measures into JIA guidelines would support a ‘treat to target’ approach which an international taskforce has determined to be the accepted strategy in the management of JIA . The taskforce did not specify which instrument should be used to measure disease activity, instead leaving this decision to individual clinicians. Further research is required to address such questions.
While some of the priority questions identified would be applicable to both adult and childhood onset inflammatory arthritis, others are more particular to paediatrics. The management of JIA-associated temporomandibular joint arthritis, for example, is notoriously difficult and evidence-based treatment guidelines are limited . Drug monitoring using antibodies is also a growing area of interest in the field of paediatric rheumatology and is largely not addressed in current guidelines .
A recent Dutch nationwide survey of paediatric rheumatology clinicians yielded similar results to our study . This group identified the management of pain and fatigue, medication tapering, uveitis management, individualised treatment and patient self-management skills as high priority topics for clinicians.
There are several strengths to our study. The Delphi method is a robust way of reaching consensus without the agenda being dominated by opinion leaders . The response rates to the first (46%) and second (44%) rounds compare well to other similar surveys [9, 19, 20] and respondents appeared representative of the APRG as a whole based upon the limited variables we could assess. The diversity of experience and professional disciplines among respondents suggests that guidelines will be broadly relevant to health care professionals involved in the multidisciplinary management of JIA. The questions identified in this study pertain to an Australian context however the findings are likely to be generalisable and useful in other similar settings. This study outlines an efficient way to prioritise clinical questions and could easily be replicated in another area of paediatric rheumatology or different healthcare settings. Limitations include the lack of inclusion of other stakeholders including consumers, industry representatives and policy makers. As observed in other studies, inclusion of these groups may affect these priorities . Given the anonymous nature of the data collection, we were unable to determine if there were important differences in clinician priorities based on practice settings, be that urban or regional.
Through a Delphi method we have derived clearly defined questions that clinicians who manage JIA would like answered as part of a living JIA guideline. Since the establishment of this list, an ANZMUSC JIA living guideline multidisciplinary panel comprising representatives from medical, nursing, allied health and consumer backgrounds have made their first living guideline recommendation addressing a question in the top ranked list . The panel made a conditional recommendation for the use of adalimumab for patients with JIA-associated uveitis who have not responded to methotrexate and the findings have been published in an open access format (available at https://app.magicapp.org/#/guideline/5847). Evidence syntheses to inform further recommendations addressing other priority questions are in progress and relevant recommendations will be maintained in living mode and updated over time as new evidence emerges.
There is strong support among APRG members for the development of Australian living guidelines for JIA and our study has identified the most important clinical questions from a clinician perspective. The results will be used to inform development of our guidelines, and these will be updated as new evidence emerges. This world-first living guideline for JIA represents a significant advance in guideline development and optimisation of care and outcomes of people living with JIA.
Availability of data and materials
The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
- ACR :
American College of Rheumatology
- ANZMUSC :
Australia and New Zealand Musculoskeletal Trials Network
- APRG :
Australian Paediatric Rheumatology Group
- CARRA :
Childhood Arthritis and Rheumatology Research Alliance
Disease Modifying Anti-Rheumatic Drug
- EULAR :
European Alliance of Associations for Rheumatology
- GKJR :
German Society for Paediatric Rheumatology
- JIA :
Juvenile Idiopathic Arthritis
- RACGP :
Royal Australian College of General Practitioners
- SHARE :
Single Hub and Access point for paediatric Rheumatology in Europe
Thomas J, Noel-Storr A, Marshall I, Wallace B, McDonald S, Mavergames C, et al. Living systematic reviews: 2. Combining human and machine effort. J Clin Epidemiol. 2017;91:31–7.
Elliott JH, Turner T, Clavisi O, Thomas J, Higgins JP, Mavergames C, et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med. 2014;11(2):e1001603.
Akl EA, Meerpohl JJ, Elliott J, Kahale LA, Schunemann HJ. Living systematic review N. living systematic reviews: 4. Living guideline recommendations. J Clin Epidemiol. 2017;91:47–53.
Bombardier C, van der Heijde DM. How to investigate and follow up undifferentiated peripheral inflammatory arthritis? 3e initiative 2008-2009: systematic reviews and clinical algorithm. J Rheumatol Suppl. 2011;87:1–2.
Whittle SL, Colebatch AN, Buchbinder R, Edwards CJ, Adams K, Englbrecht M, et al. Multinational evidence-based recommendations for pain management by pharmacotherapy in inflammatory arthritis: integrating systematic literature research and expert opinion of a broad panel of rheumatologists in the 3e initiative. Rheumatology (Oxford). 2012;51(8):1416–25.
Richards BL, Whittle S, Buchbinder R, Barrett C, Lynch N, Major G, et al. Australian and New Zealand evidence-based recommendations for pain management by pharmacotherapy in adult patients with inflammatory arthritis. Int J Rheum Dis. 2014;17(7):738–48.
Munro J, Murray K, Boros C, Chaitow J, Allen RC, Akikusa J, et al. Australian Paediatric rheumatology group standards of care for the management of juvenile idiopathic arthritis. J Paediatr Child Health. 2014;50(9):663–6.
Australia and New Zealand Musculoskeletal Clinical Trials Network. An Australian Living Guideline for the Pharmacological management of Inflammatory Arthritis 2022 [version 1.15] [Available from: https://mskguidelines.org.
Nast A, Gaskins M, Eisert L, Werner RN, Borradori L, Marinovic B, et al. Prioritizing topics in guideline development: results of a two-phase online survey of dermatologist members of the EADV. J Eur Acad Dermatol Venereol. 2019;33(1):227–33.
Dalkey N. An experimental study of group opinion: The Delphi method. Futures. 1969;1(5):408-26.
Verwoerd A, Armbrust W, Cowan K, van den Berg L, de Boer J, Bookelman S, et al. Dutch patients, caregivers and healthcare professionals generate first nationwide research agenda for juvenile idiopathic arthritis. Pediatr Rheumatol Online J. 2021;19(1):52.
Renton WD, Tiller G, Munro J, Tan J, Johnston RV, Avery JC, et al. Dose reduction and discontinuation of disease-modifying anti-rheumatic drugs (DMARDs) for juvenile idiopathic arthritis. Cochrane Database Syst Rev. 2022;2022(1). https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD014961/full.
Ringold S, Angeles-Han ST, Beukelman T, Lovell D, Cuello CA, Becker ML, et al. 2019 American College of Rheumatology/Arthritis Foundation guideline for the treatment of juvenile idiopathic arthritis: therapeutic approaches for non-systemic polyarthritis, Sacroiliitis, and Enthesitis. Arthritis Rheumatol. 2019;71(6):846–63.
Horneff G, Klein A, Ganser G, Sailer-Hock M, Gunther A, Foeldvari I, et al. Protocols on classification, monitoring and therapy in children's rheumatology (PRO-KIND): results of the working group Polyarticular juvenile idiopathic arthritis. Pediatr Rheumatol Online J. 2017;15(1):78.
Ravelli A, Consolaro A, Horneff G, Laxer RM, Lovell DJ, Wulffraat NM, et al. Treating juvenile idiopathic arthritis to target: recommendations of an international task force. Ann Rheum Dis. 2018;77(6):819–28.
Stoll ML, Kau CH, Waite PD, Cron RQ. Temporomandibular joint arthritis in juvenile idiopathic arthritis, now what? Pediatr Rheumatol Online J. 2018;16(1):32.
Verstegen RHJ, McMillan R, Feldman BM, Ito S, Laxer RM. Towards therapeutic drug monitoring of TNF inhibitors for children with juvenile idiopathic arthritis: a scoping review. Rheumatology (Oxford). 2020;59(2):386–97.
Chia-Chien H, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12:10.
Proudfoot LE, Powell AM, Ayis S, Barbarot S, Baselga Torres E, Deleuran M, et al. The European TREatment of severe atopic eczema in children taskforce (TREAT) survey. Br J Dermatol. 2013;169(4):901–9.
Wall N, De'Ambrosis B, Muir J. The management of dysplastic naevi: a survey of Australian dermatologists. Australas J Dermatol. 2017;58(4):304–7.
Verwoerd A, Armbrust W, Cowan K, Berg Lvd, Boer Jd, Bookelman S, et al. Dutch patients, caregivers and healthcare professionals generate rst nationwide research agenda for juvenile idiopathic arthritis. 2020.
Australia and New Zealand Musculoskeletal Clinical Trials Network. An Australian Living Guideline for the Management of Juvenile Idiopathic Arthritis: Benefits and harms of using bDMARD or tsDMARDs in patients with JIA associated uveitis who have not responded to methotrexate 2022 [version 0.2] [Available from: https://app.magicapp.org/#/guideline/nyxpZL/section/L6mAxr.
SW is supported by an Australia and New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Practitioner Fellowship and RB is supported by a National Health and Medical Research Council (NHMRC) Investigator Fellowship (APP1194483).
The authors would like to thank the Australian Paediatric Rheumatology Group (APRG) for their contributions to the Delphi process.
This study was supported by an Australian Department of Health grant: Arthritis-Health Professional Education and Training.
Ethics approval and consent to participate
Approved by the Central Adelaide Local Health Network Human Research Ethics Committee (CALHN reference number 12728).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Tiller, G., Renton, W.D., Tan, J. et al. Modified Delphi study to identify priority clinical questions for the Australian living guidelines for the management of Juvenile Idiopathic Arthritis. Pediatr Rheumatol 20, 52 (2022). https://doi.org/10.1186/s12969-022-00710-w