Development and validation of the Sorting non-trauMatIc adoLescent knEe pain (SMILE) tool – a development and initial validation study
Pediatric Rheumatology volume 19, Article number: 110 (2021)
Despite the commonality of adolescent knee pain, there are no tools to support medical doctors to correctly diagnose knee pain. This study aimed to develop and evaluate a support tool for diagnosing the most common types of non-traumatic adolescent knee pain.
A systematic search on Medline identified the literature on clinical tests and diagnoses of adolescent knee pain. The search was supplemented by textbooks and transformed into a diagnostic flowchart based on onset, symptoms, and pain localisation. This tool was revised based on feedback from general practitioners and experts in sports medicine. The tool was evaluated on two separate days with blinded assessors. Overall, 27 participants (aged 10–17 years) with non-traumatic knee pain were included. All participants were diagnosed by medical doctors or medical students, without and with the use of the tool. Diagnoses were compared to a gold standard (expert clinician). An interview to inform optimisations of the tool was performed with the assessors. Percentage agreement with the gold standard, and Kappa statistic for interrater reliability were calculated.
The final tool improved diagnostic agreement with the gold standard from 22.7% (95% CI 10.3–35.1) to 77.3% (95% CI 64.9–89.7). Inter-rater reliability increased from poor agreement k = − 0.04 (95% CI, − 0.12-0.0
4) to moderate agreement k = 0.56 (95% CI, 0.40–0.72).
This simple diagnostic tool is quick to use and may assist doctors in diagnosing non-traumatic knee pain in adolescents.
Annually, 7% of adolescents visit their general practitioner with musculoskeletal problems . Musculoskeletal problems are one of the most common causes of consultation in this population . Knee pain is common among adolescents  and often due to non-traumatic causes [3, 4]. Non-traumatic knee pain is most often the result of repetitive and excessive stress to musculoskeletal structures from sports participation and physical activity . ‘The most common types of non-traumatic knee pain during adolescence are Osgood-Schlatter disease and patellofemoral pain [6, 7]’. Despite being less common, Sinding-Larsen Johansson/patellar tendinopathy, Iliotibial band syndrome, and plica syndrome are important differential diagnoses for adolescents with non-traumatic knee pain . In addition to the non-traumatic knee complaints, knee pain can be related to growth also known as benign nocturnal pains of childhood (growing pain), referred pain from the hip or lumbar spine, osteosarcomas, and systemic causes which all are important differential diagnoses [5, 8].
Non-traumatic knee complaints can be long-lasting, with an impact on health-related quality of life and physical activity [9,10,11]. In light of this, the management strategy initiated by the health care practitioner becomes critical to improve long term prognosis. Furthermore, treatment may need to be managed differently related to the specific diagnosis  which can be a challenge for medical doctors, due to non-specific symptoms and clinical tests being of limited diagnostic value [2, 13]. Further, qualitative research shows the importance to adolescents of “getting a name” for their knee pain which underlines the need to support the health care practitioner in diagnosing the specific type of knee pain . The aim of this study was to develop and evaluate a support tool for diagnosing the most common types of non-traumatic adolescent knee pain.
Study design and setting
This study consists of the development and evaluation of the Sorting non-trauMatIc adoLescent knEe (SMILE) tool for non-traumatic knee pain in adolescents. The study was conducted at the Center for General Practice at Aalborg University between February 2019 – February 2020 and inspired by requests from our reference group of general practitioners (GPs).
Step one: development of the SMILE tool
The systematic literature search was conducted in Medline (via PubMed). The search strategy was developed using medical subject headings and text words related to knee, pain, and diagnosis. The search strategy is available in Additional file 1. No language restriction was applied to the search. We included papers published between 1950 and until 1st of March 2019. We also conducted a hand search including the reference lists of included studies and the authors’ personal files to make sure that all relevant material has been captured.
Eligibility criteria for including articles (both narrative and systematic) were articles describing any type of diagnostic, clinical assessment, or physical examination concerning non-traumatic knee pain. Articles on any age group were eligible, providing they described assessment/diagnosis of knee pain conditions seen in adolescents (e.g. studies on diagnosis of patellofemoral pain and ITBS in adults were eligible). Articles describing treatment only were excluded. Endnote version X9.1.1 was used to include or exclude articles.
Potentially eligible articles were independently screened by title/abstract and full text by one author (CG). Data were independently extracted and evaluated by discussion by two authors (CG and MES) into data extraction forms based on the Cochrane data extraction forms. We extracted data on study characteristics, prevalence of conditions, risk factors, clinical history and test for each diagnosis in the articles. Any discrepancies between forms were evaluated through discussion in the group.
Development process of the SMILE tool
Information from the review was then synthesised in a summary for each condition (see Additional file 2 and methods). This was supplemented by Brukner and Khan’s Clinical Sports Medicine 5TH edition, and input from international experts in sports medicine (published authors with more than five years of clinical experience), and GP’s with a special interest in sports medicine.
This information was then transformed into the SMILE tool. Table 1 shows an overview of the boxes included in the SMILE tool. Feedback on the content, layout, and text/frames used for the first version was sought from international experts and from medical doctors in a pilot test. The process from the first draft of the tool to the final version is shown in Additional file 2.
Step two: validation of the SMILE tool
After an initial pilot evaluation of the SMILE tool with one medical doctor and one medical student, we modified the layout based on feedback, and subsequently assessed and iterated the SMILE tool on two different test days. The content of the different test days and how we optimised and evaluated the SMILE tool after the test days is described in Fig. 1.
Recruitment of participants with knee pain
Inclusion criteria were adolescents (both sexes) aged 10–18 years, with a non-traumatic onset of knee pain. If adolescents had a traumatic event leading up to their onset of knee pain or had undergone surgery on the knee, they were excluded.
We recruited participants through flyers, videos on social media (Facebook and Instagram) and from local sports clubs.
Recruitment of medical doctors and medical students
The assessors were medical doctors or medical students recruited from the Center for General Practice and through our professional network. The medical students were required to be in their final year of medical educated needed to become a doctor in Denmark. We included medical students because the target end-users are inexperienced medical doctors without specific training/education in sports medicine or musculoskeletal disorders. We aimed that the SMILE tool would be feasible for medical doctors, independent of clinical experience with non-traumatic knee complaints in adolescents. Special care was taken to include assessors with varying interests in sports medicine. We included nine medical doctors and medical students (two younger medical doctors and seven medical students) as assessors in our study. Separate assessors were used at each session to ensure they were not familiar with the tool before assessments.
We recruited two specialists to serve as gold standards. The first (JLO) participated in the pilot test and test day 1, while the second (MSR) participated in the test day 2. Gold standard 1 is a specialist in rehumatology and sports medicine with 15 years of clinical and scientific experience in diagnosing and treating adolescents with knee pain . Gold standard 2 is a physiotherapist and an experienced clinical researcher with a specific clinical and research interest in adolescent knee pain .
Examination of the SMILE tool through three different test days and data collection
Assessors received no training on the SMILE tool or its content, aside from a 1-min introduction. Assessors were interviewed regarding their perception of using the SMILE tool, and any potential areas of improvement. At each consultation assessors documented baseline characteristics (name, age, months with knee pain), diagnosis without and with the SMILE tool, and time spent on the consultation (see Fig. 1). Participants completed self-report questionnaires on knee pain duration, sports participation, and the Knee injury and Osteoarthritis Outcome Score (KOOS child) which has previously been used in adolescent populations with overuse related injuries . Figure 1 gives an overview of the different test days and the data collection.
Baseline characteristics for adolescents were calculated using descriptive statistics. To test the validity of the SMILE tool we calculated the percentage agreement without and with the SMILE tool between the diagnoses given by the assessors and the gold standards diagnoses. We tested the inter-rater reliability of the SMILE tool using Fleiss kappa statistics and McNemar’s test to determine differences in proportion correct diagnoses with and without the SMILE tool. Descriptive statistics were performed using Microsoft excel version 16.34. Fleiss kappa and McNemar’s test were calculated in R version 3.5.3.
Development of the SMILE tool
The systematic search revealed a total of 13,429 articles of which 81 were screened by full text. After full-text screening 23 articles were eligible for data extraction (Fig. 2).
Based on the input from experts, the first version of the SMILE tool included the most common complaints which were growing pain, Osgood Schlatter, Sinding Larsen Johansson disease, Patellar tendinopathy, patellofemoral pain, and Iliotibial band syndrome as the main diagnoses in the tool (Fig. 3). We subsequently included the following differential diagnoses which were mentioned as important in the included articles and following feedback from experts and clinicians: Pes anserine tendinopathy, Baker’s cyst, popliteus tendinopathy, referred pain, inflammatory arthritis, osteochondritis dissecans, infection, and malignancy (Fig. 3). Figure 3 shows the final version of the SMILE tool. Detailed description of the development process is in Additional file 2 and methods section.
Characteristics of participants and assessors
In total 27 adolescents (aged 10–17) participated (see Table 2). Of the 11 participants on test day 2, eight (72.7%) participated in one of the previous test days and were assessed by the two gold standards on separate occasions.
The seven assessors (three female and four male) had 0–4 years of clinical experience (mean 2.4 ± 1.5). Six were medical students in the final year of their medical education. One was a medical doctor with authorisation to practice. One had a special interest in musculoskeletal pain. The assessors came from 2 different Universities in Denmark.
Agreement with the gold standards on test day one and test day two
Agreement with the gold standards increased with the use of the SMILE tool on both test days, with 77.3% percent agreement with the final version of the SMILE tool (Table 3).
One participant had differing diagnoses by the two assessors used as gold standard on the different test days; PFP and OSD respectively. Among the junior assessors, seven out of seven diagnosed the participant with OSD without the tool on the two test days. The participant reported pain directly at the tibial tuberosity with a bony prominence. Therefore, the participant was considered to have OSD.
Inter-rater reliability and agreement with the gold standard for test day one and test day two
Inter-rater reliability increased with the use of the SMILE tool on both test days, with a moderate agreement on the final version of the SMILE tool. On the final test day (test day 2) Fleiss’ kappa increased from poor agreement without the tool k = − 0–04 (95% CI, − 0.12 to 0.04), to moderate agreement with the SMILE tool k = 0.56 (95% CI, 0.40 to 0.72).
To our knowledge, this is the first study to develop and initially evaluate a support tool for clinical practice to improve management of adolescents with non-traumatic knee pain. The percentage of correct diagnoses and reliability reached nearly 80% using the final version of the tool. This reflected an approximate fourfold increase in junior medical doctors’ correct diagnoses, compared to without the SMILE tool. The reliability of the final version of the SMILE tool between assessors was moderate.
Strengths and limitations
Our validation is based on a relatively small sample size and external validation is needed. We purposefully recruited medical students to reflect inexperienced doctors. This choice may mean that the different in correct diagnosis may be larger among this sample compared to a sample of more experienced medical doctors. We cannot generalise our results to other professions in sports medicine, orthopaedics or other areas where this patient population consults. Further each participant’s cause of knee pain was unknown at recruitment and we were unable to recruit according to specific diagnoses and therefore certain conditions, i.e. growing pain, were not represented among the participants. However, our primary focus was on non-traumatic knee injuries, which were well represented and for which we showed high validity.
Our testing was not conducted as part of a normal clinical day of clinical practice. This tool was designed for non-specialist doctors (GPs). The assessors had 10 min for each to ensure comparability to clinical practice. The flow of patients was not identical to a normal clinical day as all patients had knee pain, which made it easier for the assessors to focus only on knee pain. Future studies may focus on the implementation of the SMILE tool in a clinical practice setting including the need for more explicit guidelines on when to refer patients for example if oncologic or inflammatory etiologies are considered. Finally, this tool was designed for GPs, and has not yet been tested with other types of medical doctors. The tool was largely tested by medical trainees in a general practice setting to specifically target junior doctors in this setting. In this study we recruited patients with non-traumatic knee pain. The patients included, had many of the diagnoses outlined in tools, as we developed this to consider the most common diagnoses. Further research is needed to evaluate whether the tool can adequately support clinicians with patients with more diverse conditions, and correctly rule out serious pathology as this has not been evaluated in this initial study.
Comparison with existing literature
Diagnosing knee pain can be challenging, especially during adolescence where growth related injuries (Osgood Schlatter, Sinding-Larsen Johansson) are common and unique to this population. It is also important to differentiate growing pains and growth related non-traumatic knee pain. In this study, the assessors only diagnosed about 20% of adolescents correctly compared to specialists (gold standards) without using the SMILE tool, with a poor reliability between assessors. Given the importance of diagnoses for treatment [12, 14], this tendency of low diagnostic accuracy is concerning.
Diagnostic clarification is important for adolescents when consulting for knee pain . However, GPs find it challenging to give the correct diagnosis [2, 13]. We know that non-traumatic knee complaints are overlooked compared to traumatic knee complaints [3, 4]. Our assessors focused on traumatic causes of knee pain when they assessed patients without our tool (data included in Additional file 3). This underlines younger GPs’ or junior medical doctors’ need for guidance when assessing adolescents with non-traumatic knee pain.
Current available clinical tests primarily focus on a single diagnosis [5, 18,19,20,21,22,23]. Our simple SMILE tool does not require any clinical tests except palpation and pain localisation, which was highlighted in the interviews by the assessors that this made the SMILE tool extremely user-friendly without any training. Furthermore, the assessors highlighted that this tool is easy to use and time efficient, which may facilitate implementation into a GP consultation. Most importantly, it does not need any pre-training or specialist introduction which may facilitate use and implementation.
We aimed to create a tool that includes the most common presentations of non-traumatic adolescent knee pain, as ease-of-use and time requirements are important factors for GPs and medical professionals [24, 25]. The high level of agreement between our SMILE tool and the gold standard suggest that despite having several diagnoses it has a high validity. We showed that our inexperienced assessors improved their diagnostic accuracy almost 4 times with the assistance of the SMILE tool. Further, their focus shifted from thinking about traumatic causes of knee pain, even in presentations without trauma, to thinking about non-traumatic causes (data shown in Additional file 3). This shift may be favourable in general practice, as non-traumatic causes of knee pain are the most prevalent in adolescents .
Implications for research and/or practice
This tool was developed for a general practice setting and therefore we developed a simple tool feasible to use in the setting of a 10-min consultation without previous knowledge of the patient. This tool is primarily made for GPs who have a wide range of different patients daily, where it is still necessary to maintain a high level of expertise in all topics to meet the patients’ expectations and provide the appropriate management. Our SMILE tool gives an easy overview of the most common non-traumatic knee complaints seen in adolescents without the need of much introduction and can thereby easily fit into a general clinical practice setting with a variety of different patients. Despite being designed for general practice setting, this tool could influence other medical specialities including orthopaedic surgery and pediatric rheumatology. Patients are often co-managed between specialities and if correct diagnosis is achieved in initial assessment in general practice, this could lead to more relevant referrals and targeted treatment.
We developed and performed initially validation of the SMILE tool for non-traumatic knee pain in adolescents. This development study was the first step in the process of developing a tool for diagnosing knee pain. The results indicate it’s potential, however, the external validity should be evaluated in clinical practice with clinicians. Our simple tool is quick to use and covers the different causes of non-traumatic knee pain in adolescents and can be used to support clinical practice.
Availability of data and materials
Data can be made available upon request to the authors.
- SMILE tool:
Sorting non-trauMatIc adoLescent knEe tool
Knee injury and Osteoarthritis Outcome Score
Tan A, Strauss VY, Protheroe J, Dunn KM. Epidemiology of paediatric presentations with musculoskeletal problems in primary care. BMC Musculoskelet Disord. 2018;19(1):40–7. https://doi.org/10.1186/s12891-018-1952-7.
Michaleff ZA, Campbell P, Protheroe J, et al. Consultation patterns of children and adolescents with knee pain in UK general practice: Analysis of medical records. BMC Musculoskelet Disord. 2017;18(1):239.
Junge T, Runge L, Juul-Kristensen B, Wedderkopp N. Risk factors for knee injuries in children 8 to 15 years: the CHAMPS study DK. Med Sci Sports Exerc. 2016;48(4):655–62. https://doi.org/10.1249/MSS.0000000000000814.
Rathleff CR, Olesen JL, Roos EM, Rasmussen S, Rathleff MS. Half of 12-15-year-olds with knee pain still have pain after one year. Dan Med J. 2013;60(11):A4725.
Patel DR, Villalobos A. Evaluation and management of knee pain in young athletes: overuse injuries of the knee. Transl Pediatr. 2017;6(3):190–8. https://doi.org/10.21037/tp.2017.04.05.
Barber Foss KD, Myer GD, Chen SS, Hewett TE. Expected prevalence from the differential diagnosis of anterior knee pain in adolescent female athletes during preparticipation screening. J Athl Train. 2012;47(5):519–24. https://doi.org/10.4085/1062-6050-47.5.01.
Stracciolini A, Casciano R, Levey Friedman H, Stein CJ, Meehan WP III, Micheli LJ. Pediatric sports injuries: a comparison of males versus females. Am J Sports Med. 2014;42(4):965–72. https://doi.org/10.1177/0363546514522393.
Uziel Y, Hashkes PJ. Growing pains in children. Pediatr Rheumatol Online J. 2007;5(1):5.
Guldhammer C, Rathleff MS, Jensen HP, Holden S. Long-term prognosis and impact of osgood-schlatter disease 4 years after diagnosis: a retrospective study. Orthop J Sports Med. 2019;7(10):2325967119878136. https://doi.org/10.1177/2325967119878136.
Rathleff MS, Rathleff CR, Olesen JL, Rasmussen S, Roos EM. Is knee pain during adolescence a self-limiting condition? Am J Sport Med. 2016;44(5):1165–71. https://doi.org/10.1177/0363546515622456.
Rathleff MS, Holden S, Straszek CL, et al. Five-year prognosis and impact of adolescent knee pain: A prospective population-based cohort study of 504 adolescents in denmark. BMJ Open. 2019;9(5):e024113.
Yen Y. Assessment and treatment of knee pain in the child and adolescent athlete. Pediatr Clin N Am. 2014;61(6):1155–73. https://doi.org/10.1016/j.pcl.2014.08.003.
Nunes GS, Stapait EL, Kirsten MH, de Noronha M, Santos GM. Clinical test for diagnosis of patellofemoral pain syndrome: systematic review with meta-analysis. Phys Ther Sport. 2013;14(1):54–9. https://doi.org/10.1016/j.ptsp.2012.11.003.
Johansen SK, Holden S, Pourbordbari N, Jensen MB, Rathleff MS. In: PAINSTORIES – exploring the challenges, barriers and self-management needs in adolescents with knee pain: A qualitative study with semi-structured interviews. ; 2018/02/01.
Research profile: Jens lykkegaard olesen [Internet]. Available from: https://vbn.aau.dk/da/persons/129269/publications/.
Research profile: Michael skovdal rathleff [Internet]. Available from: https://vbn.aau.dk/da/persons/130816/publications/.
Ortqvist M, Iversen MD, Janarv P, et al. Psychometric properties of the knee injury and osteoarthritis outcome score for children (KOOS-child) in children with knee disorders. Br J Sports Med. 2014;48(19):1437–46. https://doi.org/10.1136/bjsports-2013-093164.
Austermuehle PD. Common knee injuries in primary care. Nurse Pract. 2001;26(10):26 32-47.
Cook C, Mabry L, Reiman MP, Hegedus EJ. Best tests/clinical findings for screening and diagnosis of patellofemoral pain syndrome: a systematic review. Physiotherapy. 2012;98(2):93–100. https://doi.org/10.1016/j.physio.2011.09.001.
Décary S, Frémont P, Pelletier B, et al. Validity of combining history elements and physical examination tests to diagnose patellofemoral pain. Arch Phys Med Rehabil. 2018;99(4):607–614.e1.
Kodali P, Islam A, Andrish J. Anterior knee pain in the young athlete: diagnosis and treatment. Sports Med Arthrosc. 2011;19(1):27–33. https://doi.org/10.1097/JSA.0b013e3182045aa1.
Nijs J, Van Geel C. Van der auwera C, Van de Velde B. diagnostic value of five clinical tests in patellofemoral pain syndrome. Man Ther. 2006;11(1):69–77. https://doi.org/10.1016/j.math.2005.04.002.
Price JL. Patellofemoral syndrome: how to perform a basic knee evaluation. JAAPA. 2008;21(12):39–43. https://doi.org/10.1097/01720610-200812000-00010.
Elmore N, Burt J, Abel G, Maratos FA, Montague J, Campbell J, et al. Investigating the relationship between consultation length and patient experience: a cross-sectional study in primary care. Brit J Gen Pract. 2016;66(653):e896–903. https://doi.org/10.3399/bjgp16X687733.
Howie JG, Porter AM, Heaney DJ, et al. Long to short consultation ratio: a proxy measure of quality of care for general practice. Brit J Gen Pract. 1991;41(343):48–54.
We would like to acknowledge all volunteer assessors who participated in the study as well as the invaluable feedback from external experts in developing the tool.
This project was funded by the Lundbeck foundation. The funders had no influence on study design, data, the preparation of the manuscript, or the decision to submit for publication.
Ethics approval and consent to participate
Ethical approval was waived by the Research Ethics Committee of Northern Jutland due to the non-interventional nature of the study. Written and oral information about the participation in the study was given both to the adolescents and their parents/guardians according to the Declaration of Helsinki. Written informed parental consent was received before testing, and all participants provided assent on the day of testing.
Consent for publication
Nothing to declare.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Guldhammer, C., Holden, S., Sørensen, M.E. et al. Development and validation of the Sorting non-trauMatIc adoLescent knEe pain (SMILE) tool – a development and initial validation study. Pediatr Rheumatol 19, 110 (2021). https://doi.org/10.1186/s12969-021-00591-5