Inter -and intraobserver variation of ultrasonographic cartilage thickness assessments in small and large joints in healthy children
© Spannow et al; licensee BioMed Central Ltd. 2009
Received: 09 December 2008
Accepted: 04 June 2009
Published: 04 June 2009
There is an increasing interest among pediatric rheumatologist for using ultrasonography (US) in the daily clinical examination of children with juvenile idiopathic arthritis (JIA). Loss of joint cartilage may be an early feature of destructive disease in JIA. However, US still needs validation before it can be used as a diagnostic bedside tool in a pediatric setting. This study aims to assess the inter- and intraobserver reliability of US measurements of cartilage thickness in the joints of healthy children.
740 joints of 74 healthy Caucasian children (27 girls/47 boys), aged 11.3 (7.11 – 16) years were examined with bilateral US in 5 preselected joints to assess the interobserver variability. In 17 of these children (6 girls/11 boys), aged 10.1(7.11–11.1) years, 170 joints was examined in an intraobserver sub study, with a 2 week interval between the first and second examination.
In this study we found a good inter- and intraobserver agreement expressed as a coefficient of variation (CV) less than 10% in the knee (CV = 9.5%interobserver and 5.9%intraobservserI, 9.3%intraobserverII respectively for the two intraobserver measurements) and fairly good for the MCP joints (CV = 11.9%interobserver, 12.9%intraobserverI and 11.9%intraobsevrerII). In the ankle and PIP joints the inter- and intraobserver agreement was within an acceptable limit (CV<20%) but not for the wrist joint (CV>26%). We found no difference in cartilage thickness between the left and right extremity in the investigated joints.
We found a good inter -and intraobserver agreement when measuring cartilage thickness with US. The inter- and intraobserver variation seemed not to be related to joint size. These findings suggest that positioning of the joint and the transducer is of major importance for reproducible US measurements. We found no difference in joint cartilage thickness between the left and right extremity in any of the examined joint of the healthy children. This is an important finding giving the opportunity of using the non-affected extremity as a reference when assessing articular joint cartilage damage in JIA.
There is an increasing interest among pediatric rheumatologist for using musculoskeletal ultrasonography as an investigative tool for children with juvenile idiopathic arthritis (JIA) [1–3]. In JIA, early detection of inflammatory joint pathologies ideally would allow clinicians to initiate relevant therapies in a timely manner preventing destruction of the cartilaginous tissue and periarticular bone and subsequently improve morbidity and long-term outcome. Accumulating evidence in adult RA patients suggest that musculoskeletal ultrasonography (US) is superior in the detection and monitoring of early inflammatory and destructive joint changes [4, 5]. In patients with early, untreated oligoarthritis US revealed a high prevalence of subclinical disease.
US give the opportunity for precise and detailed pathoanatomical images. It has a number of practical advantages compared to other imaging techniques, such as magnetic resonance imaging (MRI), including being non-invasive, easily repeatable, safe, and relatively inexpensive. It allows dynamic real-time assessment in many joints from different anatomical regions. Ultrasonography requires no sedation, is easy to perform bedside and children seems to accept the examination.
However, there has been some concern regarding its reproducibility and operator dependability. While systematic studies on different aspects of validation of US in RA are now emerging [6–11]documented validity assessment has only been described in few studies in pediatric patients[2, 12, 13].
Loss of joint cartilage may be an early feature of destructive disease in JIA and when using high frequency US the cartilage, appearing anechoic, is easily visualized [3, 14–16]. In a pilot study, we found good agreement of interobserver variability in assessments of joint cartilage thickness in a small group of healthy children. Recently, we described a good level of agreement between MRI and US measurements of hyaline cartilage in healthy children .
The present study was undertaken to assess inter- and intraobserver reliability of US assessments of cartilage thickness in small and large joints from clinically dominant joint regions in a large group of healthy children. In addition we wanted to evaluate whether recommended EULAR standard scans was applicable in a pediatric setting.
Ten joints from each of 74 healthy Caucasian children (27 girls/47 boys), aged 11.3 (7.11 – 16) years were examined by two investigators. The unequal distribution between boys and girls is due to the fact that more boys than girls accepted to participate. All children were examined with bilateral US in 5 preselected joints (right and left knee, ankle, wrist, metacarpophalangeal (MCP) and proximal interphalangeal (PIP)) to assess the interobserver variability in measurements of joint cartilage thickness. All examinations were carried out within a period of two months. In seventeen of the 74 children (6 girls/11 boys), aged 10.1 (7.11–11.1) years, the 10 joints described above, were re-examined within a 2 week interval to access the intraobserver variation.
All children were free from known chronic diseases including musculoskeletal diseases. They had no history of joint trauma, swelling, tenderness or previous surgical interventions in the joints. A joint examination prior to the US examination was performed to confirm the clinical normality of the children's joints. None of the children was taking medicine influencing on growth or bone metabolism including corticosteroids. No sport activities were allowed on the examination day prior to the US investigation. The parents of all participants gave informed consent. The study was conducted in accordance with the Helsinki II declaration, and was approved by the local ethical committee of Aarhus, Denmark.
All US examinations (inter – and intraobserver study) were carried out by two observers (AHS (observer I) and MP (observer II) with experience in musculoskeletal ultrasonography. Prior to the study the two observers reached consensus on defining the bony landmarks for measurements of cartilage thickness in the five examined joints. The sonographers carried out the US examinations immediately after each other. US images were stored on DVD for later entry in a database. The US measurements of cartilage thickness from each child were performed blinded to the observers.
For the ultrasonographic examination, we used conventional B-mode obtained on a real-time Hitachi EUB-6500 CFM scanner, equipped with a linear 6–13 MHz transducer. Scanner settings were uniform for all measurements. The US image acquisition time was approximately 20 – 30 minutes for each child. The EULAR standard US scan planes were used[12, 17].
Knee and ankle joints
The ankle joint was examined with the plantar surface of the foot resting on the examination bed (knee in 90-degree flexion) and an anterior longitudinal scan between the first and second metatarsal bone was obtained . The anterior demarcation of the cartilage on the medial part of the dome of talus was identified. From this point, a distance of 5 mm in proximal direction was measured out and the cartilage thickness was measured perpendicular to the bone surface (Fig. 1B).
Wrist and finger joints
Cartilage thickness measurements of the wrist were obtained with the child in supine position, with both hands palm-side down on the examination bed and placed aside the body.
With the child in the same position as mentioned for wrist measurements cartilage thickness of the second MCP and second PIP joints was obtained from a longitudinal dorsal scan with the MCP and PIP joints in 90-degree flexion (Fig. 2B &2C).
Analysis of the interobserver agreement of cartilage thickness measurements consists of a systematic variation (whether one observer has a tendency to measure thicker or thinner cartilage thickness) and a random variation (e.g. biological variation within joints in the child). The systematic variation can be corrected if known, whereas this is not possible for the random variation. The systematic interobserver variation was described in absolute terms calculated from differences (mm) and compared by paired t-test. Significant level was set to 5% (p < 0.05) in all calculations.
The total variation between the two observers SD2total = SD2within child + SD2within joint is calculated. The variation between the two observers is the combination of both intraobserver variation and the additional interobserver variation.
With regard of intraobserver agreement, the statistical analysis was divided into a systematic intraobserver variation within an observers repeated measurements using the same US standard scans and equipment on two separate examination days, two weeks apart.
The random intraobserver variation can be described in the same terms as in the section of interobserver random variation and were expressed as the SD in mm and was divided into two parts: one part common for two equivalent joints of the same child (SD2within child) and the other part the additional variation of each joint (SD2within joint).
The standard deviations (SD) calculated from the relative differences of measurements were estimates of the coefficient of variation (CV = SD/mean × 100) and expressed the level of agreement in both inter- and intraobserver study. We considered a coefficient of variation ≤ 10% for good and ≤ 20% as acceptable . In all the analyses, systematic and random variations between the cartilage thickness on the right and on the left side were included.
The Proc Mixed procedure in the statistical package SAS 9.1 was used for the analyses. Paired t-test was used for comparing the systematic intraobserver variation. Significant level was set to 5% (p < 0.05) in all calculations.
We found a significant systematic interobserver variation for the knee, ankle, MCP and PIP joint, but not for the wrist (p = 0.172, Additional file 1). These findings express that one of the observers had a tendency to systematically measure a thicker cartilage thickness compared to the other observer, even though the observers had made detailed discussions on consensus for the anatomical landmarks of cartilage thickness measurements. The random interobserver variation was largest in the standard deviation for the "within joint" variable.
As an expression for the level of agreement between observers (total variation) the coefficient of variation was estimated as described in the statistical section. A good level of agreement between the two observers was found in the knee and MCP joints with a CV of 9.5% and 11.9% respectively. The coefficient of variation for the ankle and PIP joints where within an acceptable limit (<20%) but the CV for the wrist joint (26%) was found to be poor.
We found no difference in the analyses of the systematic and random variation between the cartilage thickness on the left and right side of the examined joints see Additional file 4.
Because our results in the study of inter- and intraobserver variation showed a tendency towards a systematic inter- and intraobserver variation, we conducted re-measurements of cartilage thickness on the stored US images. Consensuses for pointing out cartilage borders on the stored images were reassured by the observers (AHS and MP). The US images (knee, ankle, wrist, and MCP and PIP joints) obtained from 19 randomly selected children out of the 74 previously examined children in the interobserver study and from 10 of the 17 children in the intraobserver study were re-measured.
Both the systematic and random interobserver variation in the results of the remeasurements was consistent with the findings in the first measurements (Additional file 1). In the intraobserver remeasurements, we found that the coefficient of variation was reduced for all examined joints for both observers (data not shown).
To our knowledge, this is the first study that describes and quantifies inter- and intraobserver reliability of US assessments of cartilage thickness in large as well as small joints from clinically dominant joint regions in a large group of healthy children.
Prior to the implementation of US as a reliable method for detecting and monitoring the disease process in JIA patients, validity assessment is crucial. Our study contributes to the necessary description of the precision of this method (defined as how good a method reflects the closeness of agreement between different measurements of the same quantity). In addition we document that the recommended EULAR standard scan is applicable in a pediatric setting.
Our data showed a tendency to a somewhat small, but systematic interobserver variation, which was not related to joint size. This despite the use of standardized scanning settings and well-defined bony landmarks. The fact, that it is not possible for two observers to replicate the exact same location of the imaging plane could be part of the explanation. In accordance with Castriota-Scanderberg et al and our own previous study, small differences in transducer angulations can result in different measurements of the articular cartilage[12, 13, 19].
Fredberg an co-workers have documented that transversal scans increase the possibility of errors in measurement due to the fact that the transducer can be tilted in four planes (medially, laterally, proximally and distally) compared to only two planes in a longitudinal approach . They measured Achilles and patellar tendons, while in our study this could very well explain part of the systematic interobserver variation for the knee measurements.
Also difficulties in replicating the exact same positioning of the patients joint from examination to examination could contribute to the systematic difference in the interobserver data.
This is consistent with the findings of Castriota-Scanderberg and colleagues who described the importance of standardizing the positioning of the patients joints when studying inter- and intraobserver variation of cartilage thickness assessments in the hip and knee joint.
The random variation influence on the total measurement error seems to be mostly influenced by the "within joint" factor for both the inter- and intraobserver variation and there was no changes in this factors influence on the total measurement error after re-measuring.
The total variation between observers and the intraobserver variation was markedly larger when measuring the wrist compared to the other joints. As has also previously been stressed , this could be due to the complicated anatomical structure in this region and by the fact of investigating a growing joint. Differentiation between articular cartilage and immature growth cartilage is challenging leading to difficulties in establishing well-defined anatomical landmarks.
In a recent study we have tried to accede to this problem by comparing US wrist cartilage thickness to MRI measurements defined as "gold standard" . We found, that the best approach for assessing wrist cartilage thickness was to measure according to the articular surface between the distal radial head and the scaphoid bone. Important when investigating JIA patients with often restricted limitation of motion interfering with the capability of optimal joint positioning for standardized US measurements.
In the finger joints, we found clear cartilage demarcations, especially in the MCP joints which is consistent with a relatively low inter- and intraobserver variability. Karmazyn and colleagues  used the same approach for evaluation of affected MCP joints but could not demonstrate that US was more significant in evaluating cartilage thinning. These results of Karmazyn and colleagues is most probably due to missing data regarding expected normal cartilage thickness in children of different ages, data which our group is in progress describing.
In the PIP joints, the significant difference in the interobserver study could be due to often blurred cartilage demarcation to the underlying cortex. Probably, this is due to the positioning of the probe which is critical in obtaining an interpretable image and a slight alteration in the angle of the probe in relation to the skin surface or a variation in the amount of gel used can distort the image obtained and increase the occurrence of artifacts.
In this study we found no difference in joint cartilage thickness between the left and right extremity in any of the examined joint of the healthy children. This observation is of particular interest when US measurements of cartilage thickness is implemented in a JIA patient group. Thus, the possibility of using the non-affected extremity as a reference along with an age- and sex-related reference value for cartilage thickness in healthy children could be used to determine the damage of the joint cartilage.
Further studies are needed to establish standard reference values for cartilage thickness in large and small joints according to age and gender in a healthy pediatric population. Such reference values would be important in a scoring system for routine US cartilage thickness measurements in JIA patients as a part of detecting and monitoring the disease process.
US cartilage thickness measurements seems to have good inter- and intraobserver agreement in evaluation of cartilage thickness in synovial joints in children and that the recommended EULAR guidelines for standard scans of cartilage thickness is applicable in a paediatric setting.
We found that US was a well-accepted bed-side imaging tool, for evaluation of cartilage affection in synovial joints is children and very well could become the extended arm of the paediatric rheumatologist in the daily clinical examination.
The authors thank all the volunteer children, their parents and the primary school of Risskov, Aarhus Denmark, for their participation in this study. The Danish Arthritis Association and Aarhus University Clinical Institute are thanked for their financial support.
- Frosch M, Foell D, Ganser G, Roth J: Arthrosonography of hip and knee joints in the follow up of juvenile rheumatoid arthritis. Ann Rheum Dis. 2003, 62: 242-244. 10.1136/ard.62.3.242.PubMed CentralView ArticlePubMedGoogle Scholar
- Collado P, Naredo E, Calvo C, Crespo M: Assessment of the joint recesses and tendon sheaths in healthy children by high-resolution B-mode and power Doppler sonography. Clin Exp Rheumatol. 2007, 25: 915-921.PubMedGoogle Scholar
- Lamer S, Sebag GH: MRI and ultrasound in children with juvenile chronic arthritis. Eur J Radiol. 2000, 33: 85-93. 10.1016/S0720-048X(99)00158-8.View ArticlePubMedGoogle Scholar
- Bresnihan B, Kane D: Sonography and subclinical synovitis. Ann Rheum Dis. 2004, 63: 333-334. 10.1136/ard.2003.015982.PubMed CentralView ArticlePubMedGoogle Scholar
- Wakefield RJ, Green MJ, Marzo-Ortega H, Conaghan PG, Gibbon WW, McGonagle D: Should oligoarthritis be reclassified? Ultrasound reveals a high prevalence of subclinical disease. Ann Rheum Dis. 2004, 63: 382-385. 10.1136/ard.2003.007062.PubMed CentralView ArticlePubMedGoogle Scholar
- Balint PV, Sturrock RD: Intraobserver repeatability and interobserver reproducibility in musculoskeletal ultrasound imaging measurements. Clin Exp Rheumatol. 2001, 19: 89-92.PubMedGoogle Scholar
- Szkudlarek M, Court-Payen M, Jacobsen S, Klarlund M, Thomsen HS, Ostergaard M: Interobserver agreement in ultrasonography of the finger and toe joints in rheumatoid arthritis. Arthritis Rheum. 2003, 48: 955-962. 10.1002/art.10877.View ArticlePubMedGoogle Scholar
- Brown AK, O'Connor PJ, Roberts TE, Wakefield RJ, Karim Z, Emery P: Recommendations for musculoskeletal ultrasonography by rheumatologists: setting global standards for best practice by expert consensus. Arthritis Rheum. 2005, 53: 83-92. 10.1002/art.20926.View ArticlePubMedGoogle Scholar
- Scheel AK, Schmidt WA, Hermann KG, Bruyn GA, D'Agostino MA, Grassi W: Interobserver reliability of rheumatologists performing musculoskeletal ultrasonography: results from a EULAR "Train the trainers" course. Ann Rheum Dis. 2005, 64: 1043-1049. 10.1136/ard.2004.030387.PubMed CentralView ArticlePubMedGoogle Scholar
- Schmidt WA, Schmidt H, Schicke B, Gromnica-Ihle E: Standard reference values for musculoskeletal ultrasonography. Ann Rheum Dis. 2004, 63: 988-994. 10.1136/ard.2003.015081.PubMed CentralView ArticlePubMedGoogle Scholar
- Wakefield RJ, Balint PV, Szkudlarek M, Filippucci E, Backhaus M, D'Agostino MA: Musculoskeletal ultrasound including definitions for ultrasonographic pathology. J Rheumatol. 2005, 32: 2485-2487.PubMedGoogle Scholar
- Spannow AH, Stenboeg E, Pfeiffer-Jensen M, Herlin T: Ultrasound measurement of joint cartilage thickness in large and small joints in healthy children: a clinical pilot study assessing observer variability. Pediatr Rheumatol Online J. 2007, 5: 3-10.1186/1546-0096-5-3.PubMed CentralView ArticlePubMedGoogle Scholar
- Castriota-Scanderbeg A, De MV, Scarale MG, Bonetti MG, Cammisa M: Precision of sonographic measurement of articular cartilage: inter- and intraobserver analysis. Skeletal Radiol. 1996, 25: 545-549. 10.1007/s002560050132.View ArticlePubMedGoogle Scholar
- Saarakkala S, Laasanen MS, Jurvelin JS, Toyras J: Quantitative ultrasound imaging detects degenerative changes in articular cartilage surface and subchondral bone. Phys Med Biol. 2006, 51: 5333-5346. 10.1088/0031-9155/51/20/017.View ArticlePubMedGoogle Scholar
- Shahin AA, el-Mofty SA, el-Sheikh EA, Hafez HA, Ragab OM: Power Doppler sonography in the evaluation and follow-up of knee involvement in patients with juvenile idiopathic arthritis. Z Rheumatol. 2001, 60: 148-155. 10.1007/s003930170063.View ArticlePubMedGoogle Scholar
- Toyras J, Nieminen HJ, Laasanen MS, Nieminen MT, Korhonen RK, Rieppo J: Ultrasonic characterization of articular cartilage. Biorheology. 2002, 39: 161-169.PubMedGoogle Scholar
- Backhaus M, Burmester GR, Gerber T, Grassi W, Machold KP, Swen WA: Guidelines for musculoskeletal ultrasound in rheumatology. Ann Rheum Dis. 2001, 60: 641-649. 10.1136/ard.60.7.641.PubMed CentralView ArticlePubMedGoogle Scholar
- Fredberg U, Bolvig L, Andersen NT, Stengaard-Pedersen K: Ultrasonography in evaluation of Achilles and patella tendon thickness. Ultraschall Med. 2008, 29: 60-65. 10.1055/s-2007-963027.View ArticlePubMedGoogle Scholar
- Zukotynski K, Jarrin J, Babyn PS, Carcao M, Pazmino-Canizares J, Stain AM: Sonography for assessment of haemophilic arthropathy in children: a systematic protocol. Haemophilia. 2007, 13: 293-304. 10.1111/j.1365-2516.2006.01414.x.View ArticlePubMedGoogle Scholar
- Doria AS, Babyn PS, Feldman B: A critical appraisal of radiographic scoring systems for assessment of juvenile idiopathic arthritis. Pediatr Radiol. 2006, 36: 759-772. 10.1007/s00247-005-0073-2.View ArticlePubMedGoogle Scholar
- Karmazyn B, Bowyer SL, Schmidt KM, Ballinger SH, Buckwalter K, Beam TT: US findings of metacarpophalangeal joints in children with idiopathic juvenile arthritis. Pediatr Radiol. 2007, 37: 475-482. 10.1007/s00247-007-0438-9.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.