Skip to main content
Download PDF

A mixed methods evaluation of the Paediatric Musculoskeletal Matters (PMM) online portfolio

Pediatric Rheumatology volume 19, Article number: 85 (2021) Cite this article

Abstract

Background

The PMM Portfolio is comprised of the Paediatric Musculoskeletal Matters (PMM) website, the paediatric Gait, Arms, Legs and Spine (pGALS) app and e-learning modules (ELM). The target audiences are non-specialists in paediatric musculoskeletal medicine. Our study aimed to evaluate impact on learning and clinical practice.

Methods

Mixed methods (analytics, online survey, interviews) were used with PMM and ELM registered users and purposive sampling of users using international contacts within paediatrics and paediatric rheumatology. Data was analysed using descriptive statistics and qualitative techniques. A Paired T-Test compared self-rated confidence before and after use of the PMM Portfolio.

Results

There has been wide reach for all the e-resources; PMM website (662,827 hits, 262,476 users, 214 countries, data 31st July 2020); pGALS app (12,670 downloads, 70 countries, data 31st July 2020); ELM (150 users, 30 countries, data 30th May 2019). There were 164 responses (students, trainees and health care professionals) to the survey from 25 countries. Most responders deemed the PMM Portfolio useful / very useful for their learning with significantly increased self-rated confidence in their clinical examination and reasoning skills following access to the PMM website, p = < 0.01, pGALS app, p = < 0.01 and ELM, p = < 0.01. The most popular PMM website pages related to clinical assessment techniques (especially pGALS). There was high uptake of the pGALS app and pGALS ELM especially from trainees and allied health professionals. Many clinicians reported the PMM Portfolio to be useful when used to teach others. User feedback reported that easy navigation, open access, clinical images and cases were the most valued features. User feedback highlighted need to increase awareness of the e-resources through training programmes.

Conclusions

The PMM Portfolio was developed to aid learning for clinicians who are not specialists in paediatric MSK medicine. Our evaluation demonstrates wide international reach and positive feedback on learning. The PMM Portfolio is a highly useful e-resource for paediatric rheumatologists in their teaching of others to raise awareness, facilitate early diagnosis and referral of children with suspected disease. The wide user engagement informed future PMM Portfolio development and the mixed method of evaluation is transferable to other e-resources.

Background

Musculoskeletal (MSK) presentations in children and young people are common with a reported prevalence of 1 in 8 [1] and a frequent cause of health care consultations increasing with age (6% of 7 year olds to 16% of 22 year olds in a cohort study from the UK) [2]. Delay in diagnosis and access to specialist care is a priority to address amongst parents [3]; unfortunately delay is often reported in conditions that present with MSK features [4,5,6,7,8,9] with adverse impact on clinical outcomes [4, 5, 10]. The reasons for delay are multifactorial [9, 11] including complex care pathways involving community and various hospital care specialties (such as general paediatrics or emergency care) [8, 11] where self-rated lack of confidence in paediatric MSK clinical skills is reported [12]. Clinicians working in the community play a crucial role to suspect MSK disease and instigate specialist referral [2, 7, 8] and family medicine practitioners are often the ‘gatekeepers’ to specialist services [13]. Unfortunately many training schemes for family medicine do not include paediatrics [14], or MSK medicine [15] despite learning needs being known [16]. Other clinicians including nurses and allied health professionals (AHP) may encounter children in their clinical practice in the community, are important in the initial care pathways and yet also have unmet needs about MSK problems [17, 18].

With these challenges in mind our group developed a portfolio of e-resources (called the PMM Portfolio), to aid learning and give support to all clinicians who play an integral role in the early recognition, diagnosis and initial care of children with MSK conditions. The target audiences of the PMM Portfolio therefore include a spectrum of clinical learners who are not ‘paediatric MSK experts’; ranging from students in medicine and nursing, trainees in family medicine and paediatrics, through to practitioners in general paediatrics, family medicine, nursing and allied health. Paediatric rheumatologists are key to the teaching of others [19,20,21] and especially teaching clinicians to whom children may initially present; such teaching is important to raise awareness, facilitate diagnosis and referral to specialist care. Paediatric rheumatologists are often the drivers of teaching paediatric MSK medicine at faculty and institutional level [19, 22, 23]; our evaluation aimed to support the PMM Portfolio as a valid tool to support teaching practice.

The PMM Portfolio encompasses the Paediatric Musculoskeletal Matters (PMM) website (launched 2014), the paediatric Gait, Arms, Legs and Spine (pGALS) app (added 2015) and e-learning modules (ELM) (added 2017) to aid signposting through the website. Figures. 1, 2 and 3 – Screenshots from the PMM Portfolio – PMM Website (Fig. 1), pGALS app (Fig. 2), ELM (Fig. 3).

Fig. 1
figure 1

Screenshots of PMM Website

Full size image
Fig. 2
figure 2

Screenshots of pGALS app

Full size image
Fig. 3
figure 3

Screenshots of E Learning Modules

Full size image

The PMM website (www.pmmonline.org) is a free and open, evidence based, peer reviewed e-resource with user engagement to inform the content and website design functionality [24]. PMM Nursing (2017) is a further version of PMM to address the needs of the wider nursing community [17]. The PMM website includes an open, anonymised online survey and since 2014, 84 responses from 19 countries (data not shown but available on request), has informed PMM Portfolio development with requests for more global content and signposting. PMM International was hence developed and replaced the original PMM website in September 2018 following collaboration with paediatric rheumatologists in 11 countries (within Asia, Africa, Americas, Australasia and Europe) who developed additional content reflecting case mix and the health care in their country [25]. New content focused on infections and infection-related disease with MSK features, or differential diagnoses of rheumatic disease with further cases and images to reflect ethnic diversity. All contributions were subject to editorial review to ensure consistency of language and compliance with our governance framework [24, 25].

Paediatric Gait, Arms, Legs and Spine (pGALS) [26] is a simple MSK examination schedule useful in clinical practice [27,28,29,30,31,32,33] and is widely taught [34, 35]. The free pGALS app (launched in 2015) was developed with medical students at Newcastle University UK to inform format and content with exam revision notes and links to PMM website key pages [36, 37]. Language translations of pGALS produced with our PMM International collaborators (to date 20 languages) and a version for telehealth (V-pGALS) are freely available on the pGALS app and PMM website (http://www.pmmonline.org/doctor/approach-to-clinical-assessment/examination).

The PMM E-Learning Modules (ELM) aid navigation through the PMM website and were developed with our multi-professional team (including family medicine doctors, paediatricians, medical students) [35, 37]. At the time of our evaluation there were three e-modules (https://cpd.ncl.ac.uk/courses):

  1. i)

    pGALS and clinical examination skills for medical students

  2. ii)

    Assessment of childhood MSK presentations in family medicine

  3. iii)

    The paediatricians approach to a child with fever

Aims

This study focuses on evaluation of the PMM Portfolio to describe and understand reach and impact on learning and clinical practice. Furthermore the evaluation aimed to inform future development of the PMM Portfolio and strategy to optimise impact.

Methods

We adopted a mixed methods approach comprising e-resource analytics, an online survey (including self-rated confidence before and after access to the e-resources) and telephone interviews to explore themes raised by the survey.

Google Analytics described access to the PMM website (site hits, page hits, accessing countries) and resource analytics described pGALS App Store data (country and number of downloads) and ELM data (country, number of users, clinician role).

An online survey using©Survey Monkey (see Additional file 1) explored how the PMM Portfolio e-resources impact on learning and clinical practice. The survey included sections about each of the e-resources and respondents were asked to provide feedback on all the resources (including ‘non-users’ – defined as those who had accessed some but not all the available PMM Portfolio e-resources to give insight into barriers to access). The survey was piloted and included Likert Scale questions (on perceived usefulness and impact on confidence on MSK knowledge and skills), free text options and open-ended questions to encourage comment.

Use of various recruitment sources was important to include a range of target audiences and to enable recruitment of users and non-users of the PMM Portfolio. A random selection of PMM website registered users (n = 450) and all the users registered for the ELM at the time of survey recruitment (n = 142, July 2019) were invited to complete the online survey in addition to purposive sampling amongst user groups within the UK and through our PMM global partners. Selected 1–1 telephone interviews followed the online survey to explore the findings in more detail. Interview participants comprised medical students (n = 2), clinical lecturer paediatricians (n = 2) and family medicine doctors involved in teaching alongside their clinical practice (n = 2); 4/6 were from the UK.

Written consent was obtained from interview participants and survey respondents consented to participation through an online response. All participant information was anonymised. Interviews were audio-recorded and transcripts anonymised before data analysis. E-resource analytic and survey data was analysed using descriptive statistics, with free-text comments and interview transcripts analysed following standard procedures for qualitative analysis, including open and focused coding, constant comparison and deviant case analysis [38]. Reflexivity was maintained throughout the analysis and writing, by recording, discussing and challenging established assumptions. A Paired T Test (©Minitab) compared self-rated confidence scores (Range: 1 not very confident – 5 very confident) before and after use of the e-resources. The study had ethical approval from Newcastle University, UK.

Results

Resource analytics

The PMM website has had 662,827 hits and 262,476 users across 214 countries since launch (14th November 2014), until our data cut off for the purpose of this study (31st July 2020) - Fig. 4 and Table 1. Usage of the PMM website has grown since launch (November 2014) and furthermore following evolution to PMM International (launched September 2018). Most PMM users are from the US and UK although the total number of countries accessing PMM has markedly increased over time (see Additional file 2).

Table 1 Top 50 Countries Accessing the PMM Website
Full size table
Fig. 4
figure 4

Total Number of PMM website hits each month. Google Analytic Data from 14th November 2014 (go live date) to 31st July 2020

Full size image

The most popular PMM website pages (Table 2) relate to clinical assessment (pGALS, pREMS [39] normal development [gait/milestones], normal variants), frequent falls, fractures, MSK infections, limping child and guidance on when to be concerned (‘red flags’). Users spend approximately 2 min on the site (Mean: 01.58, Range: 5 s to 5 min, 53 s) and view 2 pages per session (Mean: 2.08, Range 1–24).

Table 2 Top 20 PMM Website Pages
Full size table

The pGALS app has had 12,670 downloads from 70 countries (data 31st July 2020). The iOS version had 8067 downloads from 54 countries (Top 3: China (2372), UK (1498) and US (706) (see Additional file 3). The android version had 4603 downloads from 63 countries (Top 3: UK (940), Mexico (756) and India (358)) (see Additional file 4).

The ELM data includes 150 registered users from 30 countries across Asia, Africa, America, Europe and Oceania (data 30th May 2019). Most (n = 101, 68%) are from the UK and Ireland with a range of clinical roles: nurse and AHP (n = 50), training doctors (n = 36) and clinicians (n = 35) (see Additional files 5 and 6). Most users completed one ELM (n = 128) and the remainder (n = 22) completed more than one. The ELM entitled ‘pGALS and clinical examination skills for medical students’ had the highest uptake (n = 130 users) followed by ‘Assessment of childhood MSK presentations in family medicine’ (n = 31 users), and ‘The paediatricians approach to a child with fever ‘(n = 11 users).

Survey and interview data

The response rate to the survey is not clear as the link to the survey was sent to PMM collaborators and forwarded to their students and trainees. We do however know that minimum of 592 received the invite (from our random selection of PMM website users and all the ELM registered users at that time). We received 164 completed responses to the survey, hence we can assume a maximum response rate of 28%.

Survey respondents from 25 countries (across Africa, Asia, Europe, North and South America) comprised a range of roles and levels of experience within community and hospital care (see Additional files 7 and 8). Table 3 describes feedback on all three e-resources within the PMM Portfolio (n = 120) and an additional 44 provided feedback on 2 or less (PMM website and pGALS app (n = 10), PMM website and ELM (n = 3), PMM website alone (n = 31). Some had experience using the e-resources (PMM website n = 103/164, 63%; pGALS app n = 48/131, 37% ELM n = 50/123, 41%) and others did not have experience of one or more of the e-resources (PMM website n = 61/164, 37%; pGALS app n = 83/131, 63%; ELM n = 73/123, 59%). Those who had not experienced one of the e-resources were termed non-users for the purpose of our analysis but notably they all had experience of at least one of the e-resources. Users and non-users held comparable job profiles and were from a similar varied mix of countries (see Additional files 7 and 8).

Table 3 Use of E-Resources
Full size table

Users of the PMM Portfolio e-resources who completed the survey comprised mainly AHP (PMM website n = 49/103, 48%; pGALS app n = 17/48, 35%; ELM n = 33/50, 66%), followed by general paediatricians (PMM website n = 13/103, 13%; pGALS app n = 7/48, 15%), paediatric rheumatologists (PMM website n = 16/103, 16%; pGALS app n = 11/48, 23%) and medical students (n = 4/50, 8%) for the ELM. They resided across 24 countries with highest survey uptake in India (PMM website n = 38/98, 39%; pGALS app n = 13/44, 30%; ELM n = 21/44, 48%). See Additional files 7 and 8.

Most respondents judged the e-resources to be ‘useful’ or ‘very useful’ and rated highly the fact that they could access them quickly and easily (Table 4). The main reason cited to access the PMM website and ELM were Continuing Professional Development (CPD)/Continuing Medical Education (CME), whereas supporting users to examine patients and for teaching purposes were main reasons to access the pGALS app. There was a difference by user group (see Table 5); students and trainees seeking pGALS guidance to support their clinical examination techniques and aiding revision being the main reasons to access the PMM website and pGALS app; paediatric rheumatologists cited the PMM website and pGALS app primarily for use in their teaching of others; nurses and AHP gave CPD/CME as the main reason to access the PMM website, with access to the pGALS app to support clinical examination technique and access to the ELM to aid understanding of a clinical problem.

Table 4 Resource Use and Impact on Education or Clinical Practice
Full size table
Table 5 Resource Use by User Group
Full size table

Most users reported that the PMM Portfolio had a positive impact on their current clinical practice and the learning of themselves or others (Table 4), with improved clinical skills and knowledge, aiding their teaching and increasing awareness about MSK issues amongst others cited as main benefits (Table 6).

Table 6 Impact of E-resources on Clinical Practice and Learning
Full size table

Most users reported using the PMM Portfolio within their own learning, clinical practice or teaching of others (Table 4). The PMM website content deemed most useful related to clinical assessment and examination skills (e.g. pGALS and pREMS), normal variants, red flags, limping child guidance, links to guidelines and access to videos to demonstrate clinical skills. The pGALS guidance with illustrations and language translations were highly rated in the pGALS app. The ELM users most valued the case based presentations with discussion about when to be concerned (‘red flags’).

Self-rated confidence about MSK knowledge and skills increased for all three e-resources: rated before and after using a Likert scale range - 1 (not very confident) – 5 (very confident);

  • PMM Website p = < 0.01 t (99) = − 6.59. Before: Mean score: 3.51 (S.D. 1.19). After: Mean score: 4.23 (S.D. 0.87)

  • pGALS App: p = < 0.01 t (46)= − 3.94. Before: Mean: 3.70 (S.D. 1.38). After: Mean: 4.30 (S.D. 1.02)

  • ELM: p = < 0.01 (t (43)= − 4.37. Before: Mean: 3.57 (S.D. 1.21). After: Mean: 4.16 (S.D. 0.91)

Non-users of one or more of the e-resources (PMM website n = 61, pGALS App n = 83, ELM n = 73) cited lack of awareness of their existence being the main reason but the majority reported that following study participation, they were planning to access the e-resources for their clinical practice and / or teaching purposes.

Increasing awareness of the PMM Portfolio amongst junior doctors, nurses and AHPs was suggested to increase reach of all the e-resources. Furthermore integration of the PMM Portfolio into training or CME/CPD programmes with promotion by professional organisations were proposed to increase use of the e-resources. Expanding the range of ELM topics with apps to aid use without internet access were suggestions for future development of the PMM Portfolio.

Discussion

The PMM Portfolio was developed to address the reality that many children will present to a myriad of clinicians and in most health care systems, not directly to a paediatric MSK specialist. Many of the clinicians involved in the early stages of the care pathway have only minimal, if any, experience or training in paediatric MSK medicine [12, 14, 15, 40]. As such, they may not easily recognise which cases are appropriate for prompt referral to a paediatric specialist. This delay in referral, which results in a delay in confirmatory diagnosis and initiation of effective treatment, can significantly impact clinical outcomes. With the aim of improving access to the “right care”, the PMM portfolio improves awareness, knowledge and clinical skills of a broad target audience, which includes medical and nursing students, medical trainees, nurses, AHPs and family medicine clinicians. The inclusion of medical and nursing students is important to instill essential skills and knowledge early in their career path [18, 34, 41].

Our evaluation demonstrates that we are reaching our target audiences and the most popular pages of the PMM website are clinical assessment, limping child, MSK infections and indicators of when to be concerned and refer to a paediatric specialist. These knowledge themes reflect essential learning in paediatric MSK medicine for medical students [42] and family medicine [16] and are key to raising awareness, facilitating early diagnosis and referral.

The mixed methods described impact with quantitative analytics (e.g. the numbers of hits, number of users, most accessed pages). Qualitative methods enabled us to explore the experiences of the users; e.g. how they use the PMM Portfolio, the impact on learning and to understand what they felt was useful and why. Our recruitment method allowed us to reach users who had used some but not all of the resources in the PMM Portfolio (e.g. those who had used the website but not the app or e-modules). Such feedback was helpful to explain why some resources had not been used and how we can further increase our reach. The methods also allowed feedback, which was used to inform iterative development of the e-resources to optimise content and format to meet the needs of the users.

Many PMM Portfolio users reported positive impact on their learning, their clinical practice and teaching of others. There was a significant increase in self-rated confidence in clinical skills and knowledge following access to all e-resources. Positive comments related to ease of use, open access and the content being at an appropriate level (either for their own learning needs or those that they teach). There was variation in access to the e-resources across user groups; e.g. students and trainees primarily using the PMM website and pGALS app for clinical skills guidance and academic examination revision, AHPs reporting use of the ELM for clinical practice and clinicians (including paediatric rheumatologists) using the PMM website and pGALS app for their teaching of others. Many users accessed the e-resources for CME/CPD.

The international uptake of the e-resources reflects the wide stakeholder engagement in the PMM Portfolio design and development [17]. Most users of the PMM website were initially from the UK and US but over time, uptake has markedly increased around the world, especially amongst trainees and AHPs. Our global partners ensure relevance of content to the target audiences and provide pGALS language translations. The global partners facilitated dissemination with the countries of several PMM global partners ranking highly in those accessing the PMM Portfolio. Dissemination has also been facilitated through endorsement by professional societies (e.g. Paediatric Rheumatology European Society (PReS), Royal College of Paediatrics and Child Health (RCPCH), Royal College of Nursing (RCN), India Paediatric Society and the National Institute for Health and Care Excellence (NICE)). The PMM Portfolio is embedded in PReS Basic Courses for paediatric rheumatology around the world, postgraduate paediatric rheumatology training programmes for paediatricians (e.g. India and Kenya), NICE Clinical Knowledge Summaries for family medicine about paediatric MSK development, RCPCH guidance for postgraduate paediatric examinations, the RCN Competency Framework for nurses and the ‘Call to Action’ strategy of the Paediatric Task Force for Global Musculoskeletal Health [43].

We firmly believe that user engagement is integral to iterative development of e-resources and optimising impact as a means of knowledge transfer [44]. User engagement provided ideas to further increase reach and these included; 1) Increasing awareness of the PMM Portfolio amongst students, trainees and AHPs; notably lack of awareness was the most cited reason amongst non-users. 2) Integration of the PMM Portfolio in training programmes through links with training bodies and increasing their exposure at CME/CPD events. 3) More ELM with topics relevant for the global context - most ELM registered users reside in UK/Ireland suggesting more work is needed to promote these internationally. 4) Offering additional e-formats to enable offline access would facilitate further their uptake. 5) Maintaining open and free access; all the e-resources are free of charge other than one ELM (‘The paediatricians approach to a child with fever’), which had higher development costs and the charge may have contributed to the lower uptake. With these suggestions in mind, work is underway to develop a PMM app to enable offline access to the PMM website content, the ELM portfolio now includes a module targeting physiotherapists (focus on gait) and a further module for school teachers is planned. To date all author contributions have been forthcoming without financial reimbursement and we gratefully acknowledge the valuable input from all our PMM partners. Funding is a major barrier to future PMM Portfolio development and we are actively working to secure sustainability and growth whilst maintaining the ethos of PMM being free and open to all.

Limitations of our study

Our informal approach to recruitment resulted in the survey response rate being imprecise. However, this approach to recruitment enabled reach to both users and non-users of the e-resources and gave valuable insights into barriers to use and ways to encourage uptake further. There was not an even spread amongst the numbers of responders per user group or by country and this may have introduced bias. For example, there was a high proportion of survey respondents from India and amongst AHPs (who gave very favorable feedback). Their responses were nonetheless comparable to other respondent groups, so we suggest that any effect on the overall findings is minimal. Their feedback was very valuable for future work to target clinicians, especially AHPs who are integral to paediatric MSK care in areas of the world with workforce challenges [20, 40, 43, 45]. Our methods explored reach and impact on learning, clinical practice and teaching; ideally evaluation would include influence on clinical outcomes (such as access to specialist care) but given that referrals are dependent on several variables (including local referral pathways and availability of specialists), a different evaluation approach would be needed.

Implications for research and practice

The evaluation of the PMM Portfolio is very relevant to paediatric rheumatologists who are integral to the teaching of others [19,20,21, 23] to raise awareness and facilitate diagnosis and referral. Paediatric rheumatologists are often the drivers and champions of paediatric MSK education at faculty and institutional level [19, 20, 23] and we hope that awareness of the PMM Portfolio and it’s positive evaluation will support its use by paediatric rheumatologists in their teaching. The PMM Portfolio is increasingly linked with educational activities delivered by paediatric rheumatologists such as the PReS Basic Courses around the world; PMM provides essential reading and preparation for the courses. Such educational activities facilitate growth of global paediatric rheumatology, especially as the content of PMM is written by paediatric rheumatologists around the world to maintain relevance to different health care contexts.

The use of e-resources in paediatric rheumatology is important; increasingly so during the COVID-19 pandemic with rapid escalation of e-learning in clinical education [46], innovative e-learning platforms [47] and the use of telemedicine [48] to reach many new users around the world. This is particularly relevant to paediatric rheumatology given the fact that many children with MSK conditions live in parts of the world (Asia and Africa) with little or no access to specialist care [49]. The PMM portfolio is therefore an exemplar model to facilitate e-learning and workforce capacity building to enable global paediatric rheumatology [43] and we anticipate a greater need to integrate e-learning with ‘face to face’ training schemes.

Our evaluation with mixed (quantitative and qualitative) methods is a valid approach applicable to other e-resources and will be useful to the paediatric rheumatology clinical educational community; understanding how to tailor e-resources to user needs and ways to optimise impact is of increasing importance given the investment in time and funding to set up and sustain e-learning programmes. Our approach highlights the importance of user engagement in iterative development, content being relevant and at the appropriate level and ways to optimise reach and impact.

Conclusions

The PMM Portfolio is fulfilling an important educational role reaching many target user groups across the world. The PMM Portfolio continues to grow and engagement with users will facilitate future iterations maintaining relevance for the global context.

Availability of data and materials

To maintain anonymity no additional data is available. Participants only consented to anonymised, exemplar, extracts of the data to being shared.

Abbreviations

AHP:

Allied Health Professionals

CME:

Continuing Medical Education

CPD :

Continuing Professional Development

CYP :

Children and Young People

ELM:

E Learning Module

HCP’s :

Health Care Professionals

NICE:

National Institute for Health and Care Excellence

pGALS:

Paediatric Gait, Arms, Legs and Spine

pREMS:

Paediatric Regional Examination of the Musculoskeletal System

PReS:

Paediatric Rheumatology European Society

PMM:

Paediatric Musculoskeletal Matters

RCN:

Royal College of Nursing

RCPCH:

Royal College of Pediatrics and Child Health

References

  1. Tan A, Strauss VY, Protheroe J, Dunn KM. Epidemiology of paediatric presentations with musculoskeletal problems in primary care. BMC Musculoskelet Disord. 2018;19(1):40. https://doi.org/10.1186/s12891-018-1952-7.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Prathivadi Bhayankaram N, Lacey R, Barnett L, Jordan K, Dunn K. Musculoskeletal consultations from childhood to adulthood: a longitudinal study. J Public Health. 2020;42(4):e428–e434.

  3. Douglas SL. A consumer perspective on embedding research in paediatric rheumatology: Oxford University Press; 2014.

  4. Fedorak GT, DeRosa DC, Brough AK, Miyamoto RH. Increased time between diagnosis and surgery in slipped capital femoral epiphysis results in increased radiographic deformity. J Child Orthop. 2018;12(3):232–5. https://doi.org/10.1302/1863-2548.12.170178.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. McErlane F, Foster HE, Carrasco R, Baildam EM, Chieng SE, Davidson JE, et al. Trends in paediatric rheumatology referral times and disease activity indices over a ten-year period among children and young people with juvenile idiopathic arthritis: results from the childhood arthritis prospective study. Rheumatology (Oxford, England). 2016;55(7):1225–34.

    Article  Google Scholar 

  6. Dang-Tan T, Trottier H, Mery LS, Morrison HI, Barr RD, Greenberg ML, et al. Delays in diagnosis and treatment among children and adolescents with cancer in Canada. Pediatr Blood Cancer. 2008;51(4):468–74. https://doi.org/10.1002/pbc.21600.

    Article  PubMed  Google Scholar 

  7. Foster HE, Eltringham MS, Kay LJ, Friswell M, Abinun M, Myers A. Delay in access to appropriate care for children presenting with musculoskeletal symptoms and ultimately diagnosed with juvenile idiopathic arthritis. Arthritis Rheum. 2007;57(6):921–7. https://doi.org/10.1002/art.22882.

    Article  CAS  PubMed  Google Scholar 

  8. Barber CE, Barnabe C, Benseler S, Chin R, Johnson N, Luca N, et al. Patient factors associated with waiting time to pediatric rheumatologist consultation for patients with juvenile idiopathic arthritis. Pediatr Rheumatol. 2020;18(1):1–5.

    Article  Google Scholar 

  9. Chausset A, Pereira B, Echaubard S, Merlin E, Freychet C. Access to paediatric rheumatology care in juvenile idiopathic arthritis: what do we know? A systematic review. Rheumatology. 2020;59(12):3633–44.

  10. Consolaro A, Giancane G, Alongi A, van Dijkhuizen EHP, Aggarwal A, Al-Mayouf SM, et al. Phenotypic variability and disparities in treatment and outcomes of childhood arthritis throughout the world: an observational cohort study. Lancet Child Adolesc Health. 2019;3(4):255–63. https://doi.org/10.1016/S2352-4642(19)30027-6.

    Article  PubMed  Google Scholar 

  11. Foster H, Rapley T. Access to pediatric rheumatology care -- a major challenge to improving outcome in juvenile idiopathic arthritis. J Rheumatol. 2010;37(11):2199–202. https://doi.org/10.3899/jrheum.100910.

    Article  PubMed  Google Scholar 

  12. Jandial S, Myers A, Wise E, Foster HE. Doctors likely to encounter children with musculoskeletal complaints have low confidence in their clinical skills. J Pediatr. 2009;154(2):267–71. https://doi.org/10.1016/j.jpeds.2008.08.013.

    Article  PubMed  Google Scholar 

  13. Saxena S, Francis N, Sharland M. Primary care of children: the unique role of GPs. Br J Gen Pract. 2012;62(600):340–1. https://doi.org/10.3399/bjgp12X652166.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Modi N, Simon C. Child health care: adequate training for all UK GPs is long overdue. Br J Gen Pract. 2016;66(646):228–9. https://doi.org/10.3399/bjgp16X684853.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Wise EM, Walker DJ, Coady DA. Musculoskeletal education in general practice: a questionnaire survey. Clin Rheumatol. 2014;33(7):989–94. https://doi.org/10.1007/s10067-013-2479-9.

    Article  CAS  PubMed  Google Scholar 

  16. Goff I, Boyd DJ, Wise EM, Jandial S, Foster HE. Paediatric musculoskeletal learning needs for general practice trainees: achieving an expert consensus. Educ Prim Care. 2014;25(5):249–56. https://doi.org/10.1080/14739879.2014.11494290.

    Article  PubMed  Google Scholar 

  17. Smith N, Foster H, Wyllie R, English C, Davies B, Jandial S, et al. Development of Paediatric musculoskeletal matters nursing–a free online evidence based education e-resource for nurses. Rheumatology. 2017;56(suppl 6). https://doi.org/10.1093/rheumatology/kex356.054.

  18. Chehade M, Gill T, Kopansky-Giles D, Schuwirth L, Karnon J, McLiesh P, et al. Building multidisciplinary health workforce capacity to support the implementation of integrated, people-centred models of care for musculoskeletal health. Best Pract Res Clin Rheumatol. 2016;30(3):559–84. https://doi.org/10.1016/j.berh.2016.09.005.

    Article  CAS  PubMed  Google Scholar 

  19. Mayer ML, Brogan L, Sandborg CI. Availability of pediatric rheumatology training in United States pediatric residencies. Arthritis Care Res. 2006;55(6):836–42. https://doi.org/10.1002/art.22347.

    Article  Google Scholar 

  20. Henrickson M. Policy challenges for the pediatric rheumatology workforce: part III. The international situation. Pediatr Rheumatol. 2011;9(1):26.

    Article  Google Scholar 

  21. Foster HE, Vojinovic J, Constantin T, Martini A, Dolezalova P, Uziel Y, et al. Educational initiatives and training for paediatric rheumatology in Europe. Pediatr Rheumatol J. 2018;16(1):77. https://doi.org/10.1186/s12969-018-0289-y.

    Article  Google Scholar 

  22. Patwardhan A, Henrickson M, Laskosz L, DuyenHong S, Spencer CH. Current pediatric rheumatology fellowship training in the United States: what fellows actually do. Pediatr Rheumatol. 2014;12(1):8. https://doi.org/10.1186/1546-0096-12-8.

    Article  Google Scholar 

  23. Tangcheewinsirikul S, Tang S-P, Smith N, Sukharomana M, Charuvanij S, Vilaiyuk S, et al. Delivery of paediatric rheumatology care: a survey of current clinical practice in Southeast Asia and Asia-Pacific regions. Pediatr Rheumatol. 2021;19(1):1–10.

    Article  Google Scholar 

  24. Smith N, Rapley T, Jandial S, English C, Davies B, Wyllie R, et al. Paediatric musculoskeletal matters (pmm)--collaborative development of an online evidence based interactive learning tool and information resource for education in paediatric musculoskeletal medicine. Pediatr Rheumatol J. 2016;14(1):1. https://doi.org/10.1186/s12969-015-0062-4.

    Article  Google Scholar 

  25. Smith N JS, Wyllie R, English C, Davies B, Khubchandani R, Chan MO, Munro J, Ferriani V, Saad Magalhães C, Yan J, Scott C, Charuvanij S, Khawaja K, Vojinovic J, Rapley T, Foster HE. Globalisation of Paediatric Musculoskeletal Matters’ (PMM). Paediat Rheumatol Online J. 2017;15(Suppl 2):65.

  26. Foster HE, Jandial S. pGALS - paediatric gait arms legs and spine: a simple examination of the musculoskeletal system. Pediatr Rheumatol J. 2013;11(1):44. https://doi.org/10.1186/1546-0096-11-44.

    Article  Google Scholar 

  27. Sukharomana M, Charuvanij S. The Thai translation of the pediatric gait, arms, legs, spine tool is useful for pediatric residents in detecting musculoskeletal abnormalities in children. J Clin Rheumatol. 2020. https://doi.org/10.1097/RHU.0000000000001372.

  28. Goff I, Bateman B, Myers A, Foster H. Acceptability and practicality of musculoskeletal examination in acute general pediatric assessment. J Pediatr. 2010;156(4):657–62. https://doi.org/10.1016/j.jpeds.2009.10.047.

    Article  PubMed  Google Scholar 

  29. Smith E, Molyneux E, Heikens GT, Foster H. Acceptability and practicality of pGALS in screening for rheumatic disease in Malawian children. Clin Rheumatol. 2012;31(4):647–53. https://doi.org/10.1007/s10067-011-1901-4.

    Article  PubMed  Google Scholar 

  30. Moreno-Torres LA, Hernández-Garduño AG, Arellano-Valdés CA, Salinas-Rodríguez A, Rubio-Perez N, Peláez-Ballestas I. Cross-cultural validation of the paediatric gait, arms, legs, spine (pGALS) tool for the screening of musculoskeletal disorders in Mexican children. Rheumatol Int. 2016;36(4):495–503. https://doi.org/10.1007/s00296-015-3386-y.

    Article  PubMed  Google Scholar 

  31. Chan MO, Sen ES, Hardy E, Hensman P, Wraith E, Jones S, et al. Assessment of musculoskeletal abnormalities in children with mucopolysaccharidoses using pGALS. Pediatr Rheumatol J. 2014;12(1):32. https://doi.org/10.1186/1546-0096-12-32.

    Article  Google Scholar 

  32. Abujam B, Aggarwal A. Hypermobility is related with musculoskeletal pain in Indian school-children. Clin Exp Rheumatol. 2014;32(4):610–3.

    PubMed  Google Scholar 

  33. Sabui TK, Samanta M, Mondal RK, Banerjee I, Saren A, Hazra A. Survey of musculoskeletal abnormalities in school-going children of hilly and foothill regions of eastern Himalayas using the pediatric gait, arms, legs, spine screening method. Int J Rheum Dis. 2018;21(5):1127–34. https://doi.org/10.1111/1756-185X.12897.

    Article  PubMed  Google Scholar 

  34. Baker KF, Jandial S, Thompson B, Walker D, Taylor K, Foster HE. Use of structured musculoskeletal examination routines in undergraduate medical education and postgraduate clinical practice - a UK survey. BMC Med Educ. 2016;16(1):277. https://doi.org/10.1186/s12909-016-0799-6.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Cope S, Jandial S, Foster HE. A blended learning approach to clinical skills teaching: E-learning for Paediatric gait, arms, legs and spine examination (pGALS). In: Arthritis & Rheumatology. HOBOKEN: WILEY; 2017.

    Google Scholar 

  36. Ratcliff C, Denman G, Jandial S, Foster H. Diagnosing arthritis in children. Paediatr Child Health. 2019;29(12):503–14. https://doi.org/10.1016/j.paed.2019.09.007.

    Article  Google Scholar 

  37. Cope S, Denman G, Foster H, Jandial S. Paediatric gait, arms, legs and spine examination (pGALS)–a case based discussion. Paediatr Child Health. 2018;28(2):73–83. https://doi.org/10.1016/j.paed.2017.11.002.

    Article  Google Scholar 

  38. Rapley T. Some pragmatics of data analysis. Qual Res. 2011;3:273–90.

    Google Scholar 

  39. Foster H, Kay L, May C, Rapley T. Pediatric regional examination of the musculoskeletal system: a practice-and consensus-based approach. Arthritis Care Res. 2011;63(11):1503–10. https://doi.org/10.1002/acr.20569.

    Article  Google Scholar 

  40. Scott C, Chan M, Slamang W, Okong'o L, Petty R, Laxer RM, et al. Juvenile arthritis management in less resourced countries (JAMLess): consensus recommendations from the cradle of humankind. Clin Rheumatol. 2019;38(2):563–75. https://doi.org/10.1007/s10067-018-4304-y.

    Article  PubMed  Google Scholar 

  41. Al Maini M, Al Weshahi Y, Foster HE, Chehade MJ, Gabriel SE, Al Saleh J, et al. A global perspective on the challenges and opportunities in learning about rheumatic and musculoskeletal diseases in undergraduate medical education. Clin Rheumatol. 2020;39:627–42.

  42. Jandial S, Stewart J, Foster HE. What do they need to know: achieving consensus on paediatric musculoskeletal content for medical students. BMC Med Educ. 2015;15(1):171. https://doi.org/10.1186/s12909-015-0449-4.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Foster HE, Scott C, Tiderius CJ, Dobbs MB, Ang E, Charuvanij S, et al. Improving musculoskeletal health for children and young people–a ‘call to action’. Best Pract Res Clin Rheumatol. 2020;34(5):101566.

  44. Gagnon ML. Moving knowledge to action through dissemination and exchange. J Clin Epidemiol. 2011;64(1):25–31. https://doi.org/10.1016/j.jclinepi.2009.08.013.

    Article  PubMed  Google Scholar 

  45. Foster HE, Scott C, Tiderius CJ, Dobbs MB. The paediatric global musculoskeletal task force-‘towards better MSK health for all’. Pediatr Rheumatol. 2020;18(1):1–3.

    Article  Google Scholar 

  46. Wijesooriya NR, Mishra V, Brand PL, Rubin BK. COVID-19 and telehealth, education, and research adaptations. Paediatr Respir Rev. 2020;35:38–42. https://doi.org/10.1016/j.prrv.2020.06.009.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Woodward AL, Harris ZL. Employment of a needs assessment survey to shape a novel web-based pediatric rheumatology curriculum for primary care providers. Pediatr Rheumatol. 2013;11(1):1–8. https://doi.org/10.1186/1546-0096-11-26.

    Article  Google Scholar 

  48. Tuckson RV, Edmunds M, Hodgkins ML. Telehealth. N Engl J Med. 2017;377(16):1585–92. https://doi.org/10.1056/NEJMsr1503323.

    Article  PubMed  Google Scholar 

  49. Dave M, Rankin J, Pearce M, Foster HE. Global prevalence estimates of three chronic musculoskeletal conditions: club foot, juvenile idiopathic arthritis and juvenile systemic lupus erythematosus. Pediatr Rheumatol. 2020;18(1):1–7.

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to all the participants in the study.

The development of the PMM Portfolio was funded by unrestricted educational grants from Pfizer Pharmaceuticals with subsequent additional unrestricted educational / investigator led research grants from Sobi, BioMarin, and Genzyme. Our evaluation work was funded by a grant from the JGW Patterson Foundation.

We acknowledge the considerable input of all the PMM collaborators (http://www.pmmonline.org/doctor/about-us) and medical students at Newcastle University UK in the development of our portfolio. We also acknowledge the support of Dr. Sarah Cope (family doctor), Dr. Graeme Denman (family doctor), Dr. Josh Bennett (general paediatric trainee) and Mr. Ashley Reynolds (e-technologist) in the developing of ELM.

Funding

This work was supported by funding from The JGW Patterson Foundation.

Author information

Authors and Affiliations

  1. Translational and Clinical Research Institute, Newcastle University, Newcastle upon Tyne, UK

    Nicola Smith & Sharmila Jandial

  2. Population and Health Sciences Institute, Newcastle University, Newcastle Upon Tyne, UK

    Helen E. Foster

  3. Paediatric Rheumatology, Great North Children’s Hospital, Newcastle Upon Tyne, UK

    Helen E. Foster & Sharmila Jandial

Authors
  1. Nicola Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Helen E. Foster
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sharmila Jandial
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

HF/SJ conceived the concept of the e-learning resources, led the development of the resources and contributed to study design, analysis and writing of the manuscript. NS contributed to the study design, undertook the evaluation work and contributed to analysis and writing of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Nicola Smith.

Ethics declarations

Ethics approval and consent to participate

Ethical approval obtained from Newcastle University UK Ethics Committee.

Consent for publication

Informed consent was obtained for all participants. As part of this, participants were informed that anonymised, exemplar, extracts of their data may be used in a study report.

Competing interests

The authors declare they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Blank Copy of Online Survey. Supplementary material for method.

Additional file 2.

Top 25 Countries Accessing PMM Website by Year. Supplementary Table 1 to further illustrate results.

Additional file 3.

pGALS App (iOS Version) Downloads by Country. Supplementary Table 2 to further illustrate results.

Additional file 4.

pGALS App (Android Version) Downloads by Country. Supplementary Table 3 to further illustrate results.

Additional file 5.

Job Title of Registered Users of ELM. Supplementary Table 4 to further illustrate results.

Additional file 6.

ELM Uptake by Country. Supplementary Table 5 to further illustrate results.

Additional file 7.

Survey Respondent Job Title or Course of Study. Supplementary Table 6 to further illustrate results.

Additional file 8.

Countries Survey Respondent Reside In. Supplementary Table 7 to further illustrate results.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Smith, N., Foster, H.E. & Jandial, S. A mixed methods evaluation of the Paediatric Musculoskeletal Matters (PMM) online portfolio. Pediatr Rheumatol 19, 85 (2021). https://doi.org/10.1186/s12969-021-00567-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12969-021-00567-5

Keywords

Pediatric Rheumatology

ISSN: 1546-0096