Development of a new ultrasound scoring system to evaluate glandular inflammation in Sjögren’s syndrome: an OMERACT reliability exercise

Abstract

Introduction

Primary Sjögren's syndrome (pSS) is a systemic autoimmune disease characterized by the chronic inflammation of lacrimal and salivary glands and culminating in their glandular dysfunction and damage [1]. Morphological changes of inflamed salivary glands in pSS are easily depicted by imaging, including US [2]. Based on the morphological changes of salivary glands visible on US, several studies have emphasized the potential role of US as a diagnostic tool for pSS [3]. Until very recently, a wide heterogeneity of scoring systems existed, making a comparison of the findings difficult. With the aim of creating a standardized score, the Sjögren US subgroup of the OMERACT Ultrasound working group previously developed a consensually agreed definition of B-mode ultrasonographic appearance of salivary glands in patients with pSS [4, 5].

Apart from scoring structural damage of the salivary glands in pSS, assessing disease activity is of major importance for monitoring the disease and evaluating the effect of therapeutic interventions.

Doppler US, i.e. the use of US to detect the difference between the emitted and received frequency of moving particles, is capable of assessing the vascularization of the salivary gland and theoretically represents an option to evaluate inflammatory activity in pSS [6]. To date, there is a paucity of data regarding the assessment of salivary gland vascularization in pSS [7–11].

The aim of the current study conducted by the OMERACT Ultrasound subgroup on Sjögren’s was to standardize the ultrasonographic assessment of vascularization of salivary glands, develop a consensus-based scoring system for the assessment of the vascularization of major salivary glands and test the reliability of the proposed score in salivary glands static images from pSS and subsequently in a patient-based exercise involving pSS patients.

Methods

The study followed the stepwise OMERACT methodological approach [12] and consisted of two parts.

Part 1: (i) the development of definition of the scanning protocol for the assessment of vascularization of salivary glands and the scoring system, and (ii) defining the US definition of normal vascularization of salivary glands in order to be able to separate from pathological flow. For the latter purpose, a collection of images representing the vascularization of salivary glands in healthy individuals was used.

Part 2: the validation of a scoring system for the evaluation of salivary gland vascularization, a reliability exercise on the assessment of salivary gland vascularization on static images of pSS patients to test the definition, and a live exercise on pSS patients using the proposed score to test both definitions and acquisition.

Part 1: scanning protocol and vascularization of salivary glands in healthy individuals

With the aim of reaching a broad consensus on the assessment of vascularization of salivary glands, a Delphi survey was sent to 31 members of the OMERACT Ultrasound Sjögren’s subgroup. The questionnaire consisted of 24 statements divided into four sections. The participants were asked to state their level of agreement (using a 1–5 Likert scale). Group consensus was pre-specified as total cumulative agreement >75% of Likert scores >3. Section 1 comprises 12 general statements relating to the scanning of salivary glands, Section 2 focuses on the assessment of vascularization on parotid glands (five statements) and Section 3 focuses on the assessment of vascularization in submandibular glands (three statements). Finally, Section 4 consists of four statements relating to the development of a new semiquantitative scoring system for grading the vascularization of salivary glands in pSS.

Next, nine salivary-gland US experts, all rheumatologists, were asked to record anonymized images of parotid and submandibular glands of healthy individuals in a standardized manner. Prior to the exercise, a consensually agreed scanning protocol on salivary gland vascularization was sent to all sonographers in order to receive their feedback regarding normal vascularization in salivary glands. Healthy subjects of five different age groups (20–29, 30–39, 40–49, 50–59, ≥60 years), without sicca symptoms, and without medications that might affect the saliva flow (i.e. antihistamines, diuretics, antidepressants, psychotropic medications, anxiolytics or NSAIDs) were included in the exercise. Fasting was required for at least 1 h before the US examination. In addition, the volunteers were not permitted to smoke prior to the assessment. In each subject, both parotid and submandibular glands were examined using colour Doppler US. Parotid glands were scanned in both longitudinal and transverse planes, and submandibular glands only in the longitudinal (axial) plane. Images of salivary glands were recorded before and after the meal, yielding 12 images per subject. The standardized acquisition protocol (linear probe, colour Doppler frequency up to 12 MHz, pulse repetition frequency of 500 Hz) was used to record images. Images were then sent to the study centre in Brest, France, where they were re-evaluated, and high-quality images were then selected to create an atlas of images to define normal vascularization in salivary glands.

Part 2: reliability exercises assessing the salivary gland vascularization in pSS

Static images

The members of the OMERACT Sjögren’s subgroup who participated in the Delphi were invited to participate in the static image exercise. This group included members from 14 different countries (Austria, Czech Republic, Denmark, Egypt, France, Germany, Italy, Mexico, The Netherlands, Slovenia, Spain, Switzerland, Turkey and the USA).

A total of 48 images, representing each grade of the vascularization of the parotid and submandibular glands (six images per gland and per grade), from pSS patients were collected and used for the web-based inter-reader reliability exercises and were re-sent to the participants with an interval of 4 weeks for establishing the intra-reader reliability.

Patient-based exercise

Nine sonographers (G.A.B., S.F., A.H., A.I., N.I., S.J.-J., E.N., W.A.S., L.T.) who also participated in the web-based exercise participated in the patient-based exercise, held over 2 days at the University Medical Center (UMC) Ljubljana, Slovenia. All nine sonographers evaluated all study subjects in two rounds.

This patient-based exercise was approved by the Slovenian national medical ethics committee (approval number 0120-470/2019). All patients provided informed consent prior to the participation in the study.

Patients

Nine patients with pSS recruited from the Department of Rheumatology, UMC Ljubljana were examined. All pSS patients fulfilled the ACR and/or EULAR Classification Criteria for pSS (ACR-EULAR criteria, 2016), or American–European Consensus Group Classification criteria (AECG, 2002) [13, 14]. Patient characteristics are shown in supplementary Table S1, available at Rheumatology online. Patients provided written informed consent prior to participation in the study.

US equipment and settings

In the exercise, the following nine US machines with linear array transducers were used: two Philips Epiq (5–18 MHz), one Samsung WS80 (3–12 MHz), one Samsung RS85 (4–18 MHz), one Samsung HS60 (4–18 MHz), one Samsung Hera W10 (3–12 MHz), two General Electric Logiq S7 (3–12 MHz) and one General Electric Versana Premier (3–12 MHz). The following baseline settings were applied for the examination of the parotid and submandibular glands: image depth 2.5 cm, one focus point at 1 cm below the skin’s surface, colour Doppler frequency up to 9.4 MHz (range from 5.6–9.4 MHz) and pulse repetition frequency from 400 to 800 Hz.

The colour Doppler US settings were standardized and optimized on all the machines prior to the exercise based on the examination of one female pSS patient not participating in the final exercise. The sonographers were not permitted to change these predefined settings except for adjusting the depth of the image if necessary.

Examinations were performed in patients in supine position. Patients were not permitted to eat, drink, smoke or brush their teeth at least 1 h before the exercise. Between the two rounds of the exercise, patients were not permitted to eat or drink in order to avoid the physiologic changes occurring in salivary gland blood flow after a meal. The ambient temperature was kept stable.

In addition to salivary gland vascularization, two ultrasonographers (S.J.-J. and A.H.) assessed the morphological changes on greyscale US images of both parotid glands and submandibular glands. This very preliminary research was done by two experienced ultrasonographers, who were looking for potential correlations between salivary gland morphology and vascularization. To assess the morphological changes, the previously described four-grade semiquantitative scoring system was used [4].

Statistical analysis

Intra-reader and inter-reader reliability were computed by using kappa statistics [15, 16]. Intra-reader reliability was assessed using Cohen’s kappa between two readings of each expert. Next, the mean intra-reader kappa values were determined. Inter-reader reliability was determined by calculating the weighted kappa coefficient for each pair of readers. The minimum (min) and maximum (max) kappa values were calculated and, finally, Light’s kappa was determined. Kappa values and the corresponding 95% CIs were interpreted according to Landis and Koch [16], kappa values of 0, 0.20 were considered as poor, 0.21, 0.40 as fair, 0.41, 0.6 as moderate, 0.61, 0.80 as good, and 0.81, 1.00 as excellent agreement. Statistical analyses were performed using R Statistical Software (http://www.r-project.org/). Spearman’s correlation coefficient was used to evaluate potential association between the morphological changes and vascularization of salivary glands.

Results

Part 1: consensus on scanning protocol

A consensus on the scanning protocol and the assessment of salivary gland vascularization was reached after three Delphi rounds. Of the 31 OMERACT working group members invited, 25 members responded to the first Delphi questionnaire (80.6%), and 23 and 21 members to the second and third Delphi questionnaire, respectively. Table  1 shows the agreed final statements and the percentage of agreement in detail.

Salivary gland vascularization in healthy individuals

We collected 600 images from a total of 50 healthy volunteers. Fig.  1 represents the blood flow detected in parotid and submandibular glands in healthy individuals before and after a meal. Postprandially, an increased perfusion was commonly detected by colour Doppler US. Normal salivary gland blood vessels were defined as follows: when examining parotid glands, the external carotid artery, transverse facial artery and retromandibular vein are identified as normal anatomical structures with the US aspect usually detected. The external carotid artery and the retromandibular vein run through the deep lobe of the parotid gland and can be depicted in longitudinal or transverse views. The parotid gland is vascularized by the transverse facial artery, branching from the external carotid artery. When examining the submandibular gland, the facial artery and facial vein are identified as normal anatomical structures. Branches of the facial artery vascularize the submandibular gland and run through the gland. The facial vein runs in the anterosuperior part of the submandibular gland.

Fig. 1

The vascularization of salivary glands in a healthy individual

Color Doppler US image, showing vascularization of submandibular gland parenchyma in a healthy individual before meal (left upper quadrant) and after meal (right upper quadrant); and vascularization of a parotid gland parenchyma in a healthy individual before meal (bottom left quadrant) and after meal (bottom right quadrant).

Open in new tabDownload slide

Consensus on scoring system

In Table  2, and Figs  2 and 3, the commonly agreed four-grade semiquantitative scoring system with corresponding images for the evaluation of salivary gland vascularization in pSS is presented. Grade 0 corresponds to no visible vascular signals in the gland; grade 1 corresponds to focal, dispersed vascular signals; grade 2 corresponds to diffuse vascular signals detected in <50% of the gland; and grade 3 corresponds to diffuse vascular signals detected in >50% of the gland. Flow detected from normal, anatomical vessels in salivary gland was excluded from scoring.

Fig. 2

Representative images of parotid gland vascularization according to the grades of the proposed scoring system

Color Doppler US images of parotid glands in a longitudinal view in patients with SS, showing glandular vascularization graded according to the proposed scoring system: grade 0, no visible vascular signals; grade 1, focal, dispersed vascular signals; grade 2, diffuse vascular signals detected in <50% of the gland; grade 3, diffuse vascular signals in >50% of the glandular parenchyma.

Open in new tabDownload slide

Fig. 3

Representative images of submandibular gland vascularization according to the grades of the proposed scoring system

Color Doppler US images of submandibular glands in a longitudinal view in patients with SS, showing glandular vascularization graded according to the proposed scoring system: grade 0, no visible vascular signals; grade 1, focal, dispersed vascular signals; grade 2, diffuse vascular signals detected in <50% of the gland; grade 3, diffuse vascular signals in >50% of the glandular parenchyma.

Open in new tabDownload slide

Part 2: reliability exercises assessing salivary gland vascularization

Static images

Of the 25 OMERACT working group members who participated in the Delphi, 19 responded to the invitation to the web-based exercise (76% response rate) and all completed both rounds.

For all four glands, the mean overall inter-reader agreement for assessing the vascularization of salivary glands using the proposed scoring was 0.8 for both rounds (95% CI 0.74, 0.84) (Table  3), and the overall mean intra-reader agreement for assessing salivary gland vascularization using the proposed scoring was excellent, at 0.90 (95% CI 0.87, 0.93). Results at the glandular level are presented in Table  3.

Patient-based exercise

The evening before the exercise, a final agreement between the sonographers was reached on the scanning protocol and the scoring system. The sonographers agreed to evaluate the vascularization in the reliability exercise using colour Doppler US, as it enabled assessment of both parenchymal vascularization and differentiation between anatomical vessels in salivary glands (i.e. arteries vs veins). The scanning protocol included assessment of the bilateral parotid and submandibular glands. Sublingual glands were not assessed according to the protocol. In the parotid gland, only the superficial lobe was assessed, as the deep lobe of the parotid gland cannot be entirely visualized with US. The superficial lobe was scanned in both the longitudinal and transverse planes. To remain consistent with the recent patient-based reliability exercise on morphological changes [5], submandibular glands were scanned in an axial plane (parallel to the inferior border of the mandible) only.

The subjects examined in the study were nine women with a mean age of 61.9 years (s.d., 14.7; range 31.6–79.0), with SS diagnosed [median (interquartile range)] 2.5 (1.2–6.5) years prior to the study. ANA, anti-SSA and anti-SSB antibodies were present in 8/9, 8/9 and 6/9 patients, respectively. Minor salivary gland biopsy was performed in 7/9 patients and was consistent with SS in 6 of them [the average (s.d.) focus score in the biopsies was 2.8 (2.2)]. Supplementary Table S1, available at Rheumatology online, shows the clinical characteristics of pSS patients, including OMERACT B mode US score in individual patients, and the detailed results of the reliability exercise are shown in Table  4.

Mean percentage prevalence (s.d.) of grades scored by all readers for bilateral parotid and submandibular glands in both rounds is reported in Table  4, Column C. The vascularization of salivary glands was grade 0 in 15.2% of the patients, grade 1 in 33.8% of the patients, grade 2 in 44.5% and grade 3 in 6.4% of the patients.

The mean overall inter-reader agreement in the assessment of the vascularization of salivary glands using the proposed scoring was good [weighted Light’s kappa (95% CI) of 0.7 (0.64, 0.76)] (Table  4, Column D).

The overall mean intra-reader agreement using the proposed scoring was good to excellent [weighted Light’s kappa (95% CI) of 0.84 (0.73, 0.92)] (Table  4, Column E).

Two sonographers assessed salivary gland morphological changes on greyscale US. In looking for a potential correlation between morphological changes and vascularization, Spearman’s correlation coefficient was used. A weak to moderate positive and statistically significant correlation was found between the severity of morphological changes and vascularization in parotid glands [Spearman rho: 0.44 (0.13–0.67) for the first and 0.37 (0.05–0.63) for the second round], but not in submandibular glands [Spearman rho –0.04 (–0.36 to 0.29) for the first, and –0.12 (–0.43 to 0.22) for the second round].

Discussion

In the present study, we developed and validated a Doppler US scoring system for the assessment of vascularization of the major salivary glands in patients with pSS. The results show a good inter-reader and excellent intra-reader reliability in both static images and in patients.

To the best of our knowledge, this study represents the first colour Doppler US consensus-based scoring system in which multiple sonographers/readers explored the vascularization of major salivary glands in static images of pSS patients, and in live patients with pSS. The scoring system is founded on a Doppler US acquisition protocol for the assessment of vascularization of salivary glands agreed through a Delphi questionnaire, and the assessment of vascularization in the healthy non-sicca population pre- and post-stimulation, resulting in a novel colour Doppler US atlas. When combined with our previously published OMERACT definitions and scoring of greyscale morphological lesions in major salivary glands [4], the results presented herein may form a critical step towards a reliable global outcome measure for the assessment of disease activity and structural damage in patients with pSS.

In static images, a slightly lower reliability was seen in the parotid glands than in the submandibular glands, but the opposite was seen in patients where the reliability in submandibular glands was lower than in parotid glands. A possible explanation for the lower reliability rates when assessing the vascularization of the submandibular glands might be the fact that the position of the US probe is more prone to motion artefacts compared with parotid glands. Even a slight mouth-floor movement can cause interference in the assessment of submandibular gland blood flow. Interestingly, the reliability was also higher when assessing parotid glands in a transverse plane compared with the longitudinal plane, which could similarly be explained by a more stable transducer position in a transverse parotid gland view (using the mandible and mastoid process as a landmark to position the probe). Another possible explanation could be the fact that the assessment of vascularization was limited to the superficial lobe in parotid glands, while the vascularization of the entire gland was performed in the case of submandibular glands.

The limitations of our study are the use of different US machines, although the colour Doppler US settings were standardized and optimized prior to the study. Nevertheless, this resembles daily clinical practice where various US machines are used. Evaluation of videoclips instead of static images could further improve the quality of the reliability exercise. The repetition of the examination on the same day without any break between the two rounds may lead to an overestimation of the intra-reader reliability. However, this was essential to reduce the influence of meal-induced changes (increases) on salivary gland blood flow.

Meal-induced changes of salivary gland vascularization in pSS were not evaluated in this exercise. Although these data are currently lacking, it is a challenge to do this in a real-time study with multiple sonographers participating (including standardization), as changes in salivary gland vascularization are transitional and limited in time. Indeed, reports on blood flow assessment using a spectral colour Doppler at baseline and stimulated conditions are exceedingly rare and poorly standardized [7, 8].

The limited number of patients in the study did not enable us to analyse potential correlations between pSS duration time and Doppler activity. However, in a pilot sub-study we explored the potential correlation between salivary gland morphological changes seen on greyscale US (using the OMERACT proposed definitions [4]) and the vascularization in our patient-based exercise. Our preliminary results point to a weak to moderate correlation between the severity of morphological changes and the increased vascularization in parotid glands. These preliminary findings are in line with Martinoli et al., who reported an association between a hypervascular pattern and salivary gland parenchymal heterogeneity [7]. In contrast, Lee et al. reported in their study a decrease of power Doppler signals in patients with advanced pSS compared with those without definite morphological abnormalities [9]. Due to observed inconsistencies in correlation between the salivary gland morphology and vascularization, further studies on this topic are needed. Indeed, in the study of Lee et al. the decrease of Doppler might represent the late stage of the disease, with gland fibrosis that is not inflammatory [9]. On the contrary, OMERACT grade 2 or 3 without predominant hyperechogenic bands could reflect inflammation as described by Martinoli et al. [7] In our opinion, this new colour Doppler US scoring system should be used in combination with the OMERACT greyscale scoring system. The combination of these two scoring systems, one to evaluate morphological changes and the other to assess inflammatory changes, should lead to a comprehensive global salivary gland US score. Furthermore, information on the potential correlation between morphology and/or vascularization evaluated by US on one hand and salivary gland histopathology on the other would be also valuable for the further management of pSS patients.

The strength of our study is the use of standardized salivary gland ultrasonographic assessment. Moreover, the scoring system is consensus-based and validated according to the OMERACT methodology [12]. Furthermore, our experience tells us that only a rather simple scoring system will have a chance of being used in clinical trials or in daily practice.

In conclusion, we have tested a novel Doppler US scoring system for the evaluation of the vascularization in major salivary glands, showing good to excellent inter- and intra-reader reliabilities. The proposed scoring system should be further evaluated for clinical application in a larger group of pSS patients and controls, and correlated with salivary gland morphological changes, as well as with biomarkers of systemic inflammation. Furthermore, the proposed scoring should be tested in prospective, longitudinal studies in order to estimate the sensitivity to change. This should also include prospective pharmacological studies.

Acknowledgements

We are grateful to Sara Kamp Felbo for the statistical help in the static image exercise. All authors meet the authorship requirements. All authors have read and approved the manuscript for submission. Study design was carried out by G.A.B., A.H., L.T., V.F. and S.J.-J. Data acquisition was done by A.H., G.A.B., L.T., J.J.A., D.M.C., S.C., P.C., C.D., V.F., G.F., S.F., F.G., P.H., D.H., C.H.-D., A.I., M.A.M., N.I., E.N., S.O., W.A.S., G.F., I.C.-V., A.Z. and S.J.-J. Statistical analysis was performed by N.P., A.H., L.T. and V.F. Data interpretation was carried out by A.H., G.A.B., L.T., V.F. and S.J.-J. Manuscript preparation was done by A.H., G.A.B., L.T. and S.J.-J., and manuscript revision by J.J.A., D.M.C., S.C., P.C., C.D., V.F., G.F., S.F., F.G., P.H., D.H., C.H.-D., A.I., M.-A.M., N.I., E.N., S.O., W.A.S., G.F., I.C.-V., A.Z., M.T., H.I.K. and M.-A.D’A.

Funding: No specific funding was received from any bodies in the public, commercial or not-for-profit sectors to carry out the work described in this article.

Disclosure statement: The authors have declared no conflicts of interest.

Data availability statement

The datasets analysed in the current study are available from the corresponding author on request.

Supplementary data

Supplementary data are available at Rheumatology online.

References