Skip to main content

Modified Delphi study to identify priority clinical questions for the Australian living guidelines for the management of Juvenile Idiopathic Arthritis



Juvenile Idiopathic Arthritis (JIA) is the most common rheumatic inflammatory disease in childhood. Optimal management requires clinicians to be up to date with the rapidly evolving evidence base. ‘Living’ evidence-based clinical practice guidelines, which integrate new evidence as soon as it is available, are a novel method to enhance the translation of research into practice. To determine the most relevant questions that should be prioritised in national Australian JIA living guidelines, we invited Australian and New Zealand paediatric rheumatologists and other relevant health professionals to identify and rank their most important questions in order of priority.


All 47 members of the Australian Paediatric Rheumatology Group (APRG) were invited to participate in a modified Delphi study comprising two rounds. The first round identified demographic information of respondents, current attitudes to guideline use and invited submission of priority management questions. The second round asked respondents to rank 27 collated and refined questions identified in round one in order of priority.


There were 29 (62%) and 28 (60%) responses to the first and second survey rounds respectively. About two thirds were rheumatologists or trainees (66, 68%), nearly half had more than 10 years of experience (45, 46%) and practice setting was largely hospital (79, 86%) and urban (86, 75%). Most respondents used clinical guidelines in their practice (72% sometimes, 24% often), most frequently American College of Rheumatology (ACR) (66%) and European Alliance of Associations for Rheumatology (EULAR) (59%) guidelines. Reported barriers to guideline use included that they are not up to date and access difficulties. Most respondents (83%) considered Australian guidelines were necessary and two-thirds indicated they would use them if integrated into practice software. The highest ranked topics were down-titration and discontinuation of disease modifying anti-rheumatic drugs (ranked first), best outcome measures (second) and treatment targets in JIA (third).


There is strong clinician support for the development of Australian living guidelines for JIA. Consensus was reached on the ten top-ranked priority questions. Our guidelines will develop evidence-based recommendations for these high priority questions that will be updated in real time as needed to facilitate rapid translation of evidence into clinical practice.


To effectively practice evidence-based medicine, clinicians need to keep abreast of the latest evidence in their field, an increasing challenge as the volume of new evidence rapidly expands [1]. Systematic reviews are a valuable tool for synthesising evidence but are only useful if they are up to date. ‘Living’ systematic reviews are a new approach that use novel methods such as regular automated searches to continuously update reviews as new evidence becomes available [2].

Living reviews can also be incorporated into ‘living’ clinical practice guidelines by updating individual recommendations within a dynamic, web-hosted guideline structure as soon as relevant new evidence emerges [3]. A further advantage of living guidelines is that they can be developed in a stepwise format, in which additional living recommendations can be added over time as resources permit.

To optimise immediate effect, topics for inclusion can be prioritised using a systematic approach. Such a framework has been used by the 3e (Evidence, Expertise, Exchange) Initiative that has developed international clinical practice guidelines on topics including undifferentiated peripheral inflammatory arthritis [4], pain management in inflammatory arthritis [5] and gout [6]. Using this method, guidance on the most relevant clinical questions was developed.

To our knowledge, this process has not been performed in the field of paediatric rheumatology. Juvenile idiopathic arthritis (JIA) is the most common chronic inflammatory rheumatic disease in childhood and could be well suited to a living guideline format given the expanding range of available treatments. The most recent Australian guideline for the management of JIA was published in 2009 [7] and many of its recommendations are outdated. The Australia and New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network, in partnership with the Australian Paediatric Rheumatology Group (APRG) and Australian Rheumatology Association (ARA), is developing Australian living guidelines for the management of JIA in parallel with the development of living guidelines for adult rheumatic diseases [8].

Our process of living guideline development provides a unique training opportunity to members of the multidisciplinary guideline panels in evidence-based practice, living evidence and guideline development methods. Involvement of clinicians early in guideline development also improves the relevance of the recommendations and may increase guideline uptake to hasten translation of evidence into practice [59]. To facilitate a living guideline that is immediately useful, we performed a modified Delphi study to reach consensus on the most clinically relevant questions, ranked in order of priority, that APRG clinicians would like addressed in the Australian JIA living guidelines.


Design and participants

We used a modified Delphi technique including two survey rounds [10]. The invitation to the first and second surveys were sent 31 January and 9 March 2021 respectively.

All members of the APRG (N = 47) were invited to participate by email. This group includes all currently practising paediatric rheumatologists in Australia and New Zealand, all advanced trainees in paediatric rheumatology, adult rheumatologists with an interest in paediatric rheumatology, and rheumatology health professionals, including physiotherapists, occupational therapists, research assistants and nurses.

To maximise participation, a follow-up email was sent 2 weeks after the initial invitation for each Delphi round. Each survey link was closed 6 weeks after the first email. Individual responses were anonymised. All APRG members were invited to participate in the second round, regardless of their participation status in the first round.

Survey tool and procedure

An online platform (SurveyMonkey, Momentive Inc. San Mateo, California, USA) was used to collect the data. Surveys from the first and second rounds are appended (Additional files 1 and 2 respectively).

The first round included an introduction to the concept of living guidelines, an overview of the ANZMUSC JIA living guidelines project and described the aim of the study. Participants were invited to submit at least 3 and up to 10 questions they thought were most relevant to the management of JIA via free-text entry. Example questions were provided for guidance. We also collected demographic information and asked respondents about current use of JIA guidelines and beliefs about the need for local guidelines (Additional file 1).

The complete list of questions identified in the first round was refined and duplicate questions were removed. The refined list of questions was provided to respondents in random order in the second round. Respondents were asked to indicate the ten questions they considered most important and then to rank them from most to least important.


Using results from the second round, each question was assigned a score with 10 points for questions that were ranked first to 1 point for questions that were ranked tenth. A priority list was created according to the sum of ranking scores. The final ranked order of questions, their score, and the absolute number of respondents who voted for a particular question are reported.


There were 29 (62%) and 28 (60%) responses to the first and second rounds respectively. The characteristics of respondents in comparison to the APRG membership as a whole is summarised in Table 1. The gender and discipline breakdown of respondents reflected that of the APRG membership. Most respondents in both rounds were paediatric rheumatologists (first round n = 15, 52%; second round n = 16, 57%) or trainees (n = 4, 14%; n = 3, 11%). Thirteen respondents in both survey rounds (45 and 46%) had more than 10 years’ experience in paediatric rheumatology, and the majority work in urban (n = 25, 86%; n = 21, 75%) and hospital (n = 23, 79%; n = 24, 86%) settings.

Table 1 Characteristics of respondents and their guideline use

Most respondents indicated they sometimes use guidelines in their usual practice (n = 21, 72%), while almost a quarter (n = 7, 24%) indicated they used them often. The guidelines most commonly used by respondents were from the American College of Rheumatology (ACR) (n = 19, 66%) and European Alliance of Associations for Rheumatology (EULAR) (n = 17, 59%).

Several barriers to using existing guidelines were reported. Thirteen respondents (45%) indicated guidelines were not up to date and 10 (34%) indicated they had difficulties accessing a guideline. Most respondents (n = 24, 83%), indicated that Australian paediatric rheumatology guidelines are necessary and 19 (66%) would use them if integrated into practice software.

There were 134 questions proposed in the first round. The most common questions related to tapering of disease-modifying antirheumatic drugs (DMARDs) identified by 12 (41%) respondents, the most appropriate outcomes to measure in clinical practice identified by seven (24%) respondents and immunisations identified by six (21%) respondents. After refinement and duplicate removal, there were 27 questions presented in the second round.

Table 2 shows the most important questions and their rankings as identified in round two. The highest ranked question concerned tapering and discontinuation of DMARDs in patients with JIA who have responded well to treatment (score = 140). It was ranked in the top 10 by 21 of 24 (87%) respondents. Other highly ranked issues related to outcome measures / treatment targets (score = 90, 11 respondents), use of oral glucocorticoids (score = 81, 14 respondents), use of methotrexate (score = 78, 13 respondents), and DMARD strategy based on JIA subtype (score = 78, 12 respondents).

Table 2 Top important questions, ranked score and number of respondents who voted for each question in Round 2 (N = 24)


Most health professionals involved in the specialist care of Australian children with JIA and who responded to our survey use guidelines as part of routine clinical care. They were also strongly supportive of locally produced Australian JIA guidelines despite the existence of guidelines for JIA developed elsewhere. Australian living guidelines would ensure recommendations are relevant to the local context. In addition, respondents reported concerns about guidelines often being out of date. A single repository for JIA management recommendations, that are updated as new evidence emerges, and available and accessible at the point of care, would address these concerns.

It has been noted that needs raised by clinicians and consumers are not always reflected in the research that is performed [11]. Topics without clear evidence to guide practice were understandably highly prioritised as being important to respondents. For example, there is a paucity of high quality research investigating down-titration and discontinuation of DMARDs in JIA patients with inactive disease and a large variation in clinical practice currently exists [12]. While recently published guidelines have not addressed this issue, this was our top ranked priority. It has also been identified as a priority topic for other guidelines [13, 14].

The second ranked question, concerning the best outcome measures and treatment targets in JIA, is also not addressed in existing guidelines. Incorporating standardised outcome measures into JIA guidelines would support a ‘treat to target’ approach which an international taskforce has determined to be the accepted strategy in the management of JIA [15]. The taskforce did not specify which instrument should be used to measure disease activity, instead leaving this decision to individual clinicians. Further research is required to address such questions.

While some of the priority questions identified would be applicable to both adult and childhood onset inflammatory arthritis, others are more particular to paediatrics. The management of JIA-associated temporomandibular joint arthritis, for example, is notoriously difficult and evidence-based treatment guidelines are limited [16]. Drug monitoring using antibodies is also a growing area of interest in the field of paediatric rheumatology and is largely not addressed in current guidelines [17].

A recent Dutch nationwide survey of paediatric rheumatology clinicians yielded similar results to our study [11]. This group identified the management of pain and fatigue, medication tapering, uveitis management, individualised treatment and patient self-management skills as high priority topics for clinicians.

There are several strengths to our study. The Delphi method is a robust way of reaching consensus without the agenda being dominated by opinion leaders [18]. The response rates to the first (46%) and second (44%) rounds compare well to other similar surveys [9, 19, 20] and respondents appeared representative of the APRG as a whole based upon the limited variables we could assess. The diversity of experience and professional disciplines among respondents suggests that guidelines will be broadly relevant to health care professionals involved in the multidisciplinary management of JIA. The questions identified in this study pertain to an Australian context however the findings are likely to be generalisable and useful in other similar settings. This study outlines an efficient way to prioritise clinical questions and could easily be replicated in another area of paediatric rheumatology or different healthcare settings. Limitations include the lack of inclusion of other stakeholders including consumers, industry representatives and policy makers. As observed in other studies, inclusion of these groups may affect these priorities [21]. Given the anonymous nature of the data collection, we were unable to determine if there were important differences in clinician priorities based on practice settings, be that urban or regional.

Through a Delphi method we have derived clearly defined questions that clinicians who manage JIA would like answered as part of a living JIA guideline. Since the establishment of this list, an ANZMUSC JIA living guideline multidisciplinary panel comprising representatives from medical, nursing, allied health and consumer backgrounds have made their first living guideline recommendation addressing a question in the top ranked list [22]. The panel made a conditional recommendation for the use of adalimumab for patients with JIA-associated uveitis who have not responded to methotrexate and the findings have been published in an open access format (available at Evidence syntheses to inform further recommendations addressing other priority questions are in progress and relevant recommendations will be maintained in living mode and updated over time as new evidence emerges.


There is strong support among APRG members for the development of Australian living guidelines for JIA and our study has identified the most important clinical questions from a clinician perspective. The results will be used to inform development of our guidelines, and these will be updated as new evidence emerges. This world-first living guideline for JIA represents a significant advance in guideline development and optimisation of care and outcomes of people living with JIA.

Availability of data and materials

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.



American College of Rheumatology


Australia and New Zealand Musculoskeletal Trials Network


Australian Paediatric Rheumatology Group


Childhood Arthritis and Rheumatology Research Alliance


Disease Modifying Anti-Rheumatic Drug


European Alliance of Associations for Rheumatology


German Society for Paediatric Rheumatology


Juvenile Idiopathic Arthritis


Royal Australian College of General Practitioners


Single Hub and Access point for paediatric Rheumatology in Europe


  1. Thomas J, Noel-Storr A, Marshall I, Wallace B, McDonald S, Mavergames C, et al. Living systematic reviews: 2. Combining human and machine effort. J Clin Epidemiol. 2017;91:31–7.

    Article  Google Scholar 

  2. Elliott JH, Turner T, Clavisi O, Thomas J, Higgins JP, Mavergames C, et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med. 2014;11(2):e1001603.

    Article  Google Scholar 

  3. Akl EA, Meerpohl JJ, Elliott J, Kahale LA, Schunemann HJ. Living systematic review N. living systematic reviews: 4. Living guideline recommendations. J Clin Epidemiol. 2017;91:47–53.

    Article  Google Scholar 

  4. Bombardier C, van der Heijde DM. How to investigate and follow up undifferentiated peripheral inflammatory arthritis? 3e initiative 2008-2009: systematic reviews and clinical algorithm. J Rheumatol Suppl. 2011;87:1–2.

    Article  Google Scholar 

  5. Whittle SL, Colebatch AN, Buchbinder R, Edwards CJ, Adams K, Englbrecht M, et al. Multinational evidence-based recommendations for pain management by pharmacotherapy in inflammatory arthritis: integrating systematic literature research and expert opinion of a broad panel of rheumatologists in the 3e initiative. Rheumatology (Oxford). 2012;51(8):1416–25.

    Article  Google Scholar 

  6. Richards BL, Whittle S, Buchbinder R, Barrett C, Lynch N, Major G, et al. Australian and New Zealand evidence-based recommendations for pain management by pharmacotherapy in adult patients with inflammatory arthritis. Int J Rheum Dis. 2014;17(7):738–48.

    Article  Google Scholar 

  7. Munro J, Murray K, Boros C, Chaitow J, Allen RC, Akikusa J, et al. Australian Paediatric rheumatology group standards of care for the management of juvenile idiopathic arthritis. J Paediatr Child Health. 2014;50(9):663–6.

    Article  Google Scholar 

  8. Australia and New Zealand Musculoskeletal Clinical Trials Network. An Australian Living Guideline for the Pharmacological management of Inflammatory Arthritis 2022 [version 1.15] [Available from:

  9. Nast A, Gaskins M, Eisert L, Werner RN, Borradori L, Marinovic B, et al. Prioritizing topics in guideline development: results of a two-phase online survey of dermatologist members of the EADV. J Eur Acad Dermatol Venereol. 2019;33(1):227–33.

    Article  CAS  Google Scholar 

  10. Dalkey N. An experimental study of group opinion: The Delphi method. Futures. 1969;1(5):408-26.

  11. Verwoerd A, Armbrust W, Cowan K, van den Berg L, de Boer J, Bookelman S, et al. Dutch patients, caregivers and healthcare professionals generate first nationwide research agenda for juvenile idiopathic arthritis. Pediatr Rheumatol Online J. 2021;19(1):52.

    Article  Google Scholar 

  12. Renton WD, Tiller G, Munro J, Tan J, Johnston RV, Avery JC, et al. Dose reduction and discontinuation of disease-modifying anti-rheumatic drugs (DMARDs) for juvenile idiopathic arthritis. Cochrane Database Syst Rev. 2022;2022(1).

  13. Ringold S, Angeles-Han ST, Beukelman T, Lovell D, Cuello CA, Becker ML, et al. 2019 American College of Rheumatology/Arthritis Foundation guideline for the treatment of juvenile idiopathic arthritis: therapeutic approaches for non-systemic polyarthritis, Sacroiliitis, and Enthesitis. Arthritis Rheumatol. 2019;71(6):846–63.

    Article  Google Scholar 

  14. Horneff G, Klein A, Ganser G, Sailer-Hock M, Gunther A, Foeldvari I, et al. Protocols on classification, monitoring and therapy in children's rheumatology (PRO-KIND): results of the working group Polyarticular juvenile idiopathic arthritis. Pediatr Rheumatol Online J. 2017;15(1):78.

    Article  Google Scholar 

  15. Ravelli A, Consolaro A, Horneff G, Laxer RM, Lovell DJ, Wulffraat NM, et al. Treating juvenile idiopathic arthritis to target: recommendations of an international task force. Ann Rheum Dis. 2018;77(6):819–28.

    PubMed  Google Scholar 

  16. Stoll ML, Kau CH, Waite PD, Cron RQ. Temporomandibular joint arthritis in juvenile idiopathic arthritis, now what? Pediatr Rheumatol Online J. 2018;16(1):32.

    Article  Google Scholar 

  17. Verstegen RHJ, McMillan R, Feldman BM, Ito S, Laxer RM. Towards therapeutic drug monitoring of TNF inhibitors for children with juvenile idiopathic arthritis: a scoping review. Rheumatology (Oxford). 2020;59(2):386–97.

    Article  CAS  Google Scholar 

  18. Chia-Chien H, Sandford BA. The Delphi technique: making sense of consensus. Pract Assess Res Eval. 2007;12:10.

    Google Scholar 

  19. Proudfoot LE, Powell AM, Ayis S, Barbarot S, Baselga Torres E, Deleuran M, et al. The European TREatment of severe atopic eczema in children taskforce (TREAT) survey. Br J Dermatol. 2013;169(4):901–9.

    Article  CAS  Google Scholar 

  20. Wall N, De'Ambrosis B, Muir J. The management of dysplastic naevi: a survey of Australian dermatologists. Australas J Dermatol. 2017;58(4):304–7.

    Article  Google Scholar 

  21. Verwoerd A, Armbrust W, Cowan K, Berg Lvd, Boer Jd, Bookelman S, et al. Dutch patients, caregivers and healthcare professionals generate rst nationwide research agenda for juvenile idiopathic arthritis. 2020.

    Google Scholar 

  22. Australia and New Zealand Musculoskeletal Clinical Trials Network. An Australian Living Guideline for the Management of Juvenile Idiopathic Arthritis: Benefits and harms of using bDMARD or tsDMARDs in patients with JIA associated uveitis who have not responded to methotrexate 2022 [version 0.2] [Available from:

Download references


SW is supported by an Australia and New Zealand Musculoskeletal (ANZMUSC) Clinical Trials Network Practitioner Fellowship and RB is supported by a National Health and Medical Research Council (NHMRC) Investigator Fellowship (APP1194483).

The authors would like to thank the Australian Paediatric Rheumatology Group (APRG) for their contributions to the Delphi process.


This study was supported by an Australian Department of Health grant: Arthritis-Health Professional Education and Training.

Author information

Authors and Affiliations



RB, JM and SW conceived and obtained funding for the study. GT, WR, SW, JA, JM and RB designed and performed the study. GT collated the data and wrote the first draft of the paper. All authors contributed to drafting and reviewing subsequent drafts. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to William D. Renton.

Ethics declarations

Ethics approval and consent to participate

Approved by the Central Adelaide Local Health Network Human Research Ethics Committee (CALHN reference number 12728).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

First Survey. Description of data: First round survey as provided to participants.

Additional file 2.

Second Survey. Second round survey as provided to participants.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tiller, G., Renton, W.D., Tan, J. et al. Modified Delphi study to identify priority clinical questions for the Australian living guidelines for the management of Juvenile Idiopathic Arthritis. Pediatr Rheumatol 20, 52 (2022).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: