Development of quality indicators for pregnancy and childbirth in patients with systemic lupus erythematosus

ABSTRACT

Objectives

A quality indicator (QI) for the treatment of systemic lupus erythematosus (SLE) during pregnancy and childbirth that is useful for sharing standard treatment policies has not yet been developed. This study aimed to develop a QI for SLE associated with pregnancy and childbirth.

Methods

To identify candidate QIs, we conducted a systematic literature review on the development of QIs for SLE related to pregnancy and childbirth and on clinical practice guidelines. Candidate QI items were extracted from the final selected articles, and a first evaluation, panel meeting, and second evaluation were conducted to determine whether the candidate items were appropriate as QIs. Items for which all panel members reached a consensus were designated pregnancy and childbirth-related SLE QIs.

Results

Four articles on SLE QI development and 28 practice guidelines were listed through abstract/text screening. Based on these studies, 52 candidate QIs were extracted that were limited to items related to pregnancy and childbirth and 41 items were selected on which all panel members agreed.

Conclusion

We developed pregnancy-related SLE QIs using the RAND/UCLA method and selected 41 items, which could be used clinically.

Introduction

Systemic lupus erythematosus (SLE) is a chronic inflammatory disease of unknown aetiology that presents with various symptoms. Because the disease is common in young women, pregnancy and childbirth are important life events that must be considered in routine clinical practice. In Japan, SLE treatment guidelines [1] and ‘Treatment guidelines for pregnant and postpartum female patients with SLE, rheumatoid arthritis, juvenile idiopathic arthritis, and inflammatory bowel disease’ [2] have been developed, highlighting the importance of this topic. Daily care of patients with SLE requires a wide range of medical skills, including not only management of the underlying disease, but also advice on medication side effects, pregnancy planning, care during pregnancy and childbirth, advice on daily activities such as sun avoidance, and psychological support. In particular, for patients who want to have children, are pregnant, or have recently given birth, the quality of care is important as drug use and testing may be limited, and the health of the foetus may be affected by the severity of the underlying disease or any associated conditions. The quality of care varies with an existing gap (evidence–practice gap) between ideal care and care provided in clinical practice. Quality of care is evaluated using quality indicators (QIs), which are divided into three categories: ‘structure’, ‘process’, and ‘outcome’. To improve the desired outcome of medical care, which is the goal, the preceding ‘process’ should be improved [3, 4]. Ensuring the quality of the process is particularly important for patients with SLE who require long-term outpatient care to improve their desired outcomes. In 2009, Yazdany et al. developed a QI for SLE [5], and since then, three additional SLE QIs have been developed [6–8]. These SLE QIs include QIs related to pregnancy and childbirth; however, there is no set of QIs focused on pregnancy and childbirth. Yajima et al. developed QIs for Japanese patients with SLE, but they did not include QIs related to pregnancy or childbirth [9]. Physiological changes during pregnancy can adversely affect SLE, potentially exacerbating the condition and increasing the risks for the mother and foetus. These complexities are not adequately addressed by current comprehensive QIs, which lack specificity for managing SLE during pregnancy.

Additionally, developing pregnancy-specific QIs will not only elevate the quality of care for pregnant patients with SLE but also enhance healthcare providers’ understanding and management of SLE during pregnancy. High QI adherence has been shown to significantly reduce disease damage in patients with SLE [10] and decrease hospitalization rates in patients with connective tissue diseases [11]. This study aimed to develop a QI to quantitatively evaluate the quality of medical care related to pregnancy and childbirth in Japanese patients with SLE.

Materials and methods

We used a modified version of the RAND/UCLA Appropriateness Method to develop a set of QIs related to pregnancy and childbirth for patients with SLE by combining expert opinion and scientific evidence. The process was divided into two phases: a systematic literature review to identify QI candidates and an assessment of appropriateness by an expert panel. The study protocol was registered with the Open Science Framework (https://osf.io/ahzj5/).

Systematic literature review

We searched the literature for studies on the development of QIs and clinical practice guidelines (CPGs) related to pregnancy and childbirth in women with SLE published up to September 2021.

Electronic searches

We conducted a literature search in MEDLINE via OVID and EMBASE up to September 2021, limiting our search to articles written only in English. Supplementary Tables S1 and S2 present the search strategies for MEDLINE and EMBASE. We searched for additional potential studies by manually searching the websites of relevant organizations and reference lists of the included studies.

Types of studies

We included all published studies on QIs measuring any stage of Donabedian structure, process, and outcome measurements or CPG development that specifically addressed pregnant patients with SLE [12]. If multiple reports from a single study were identified, the most recent version was included. We excluded studies that met any of the following criteria: studies with QIs specifically related to discoid lupus erythematosus or skin lupus erythematosus, QIs that were presented as checklists without a validated development methodology, such as the Delphi method or other consensus development methods, editorials, and case reports. The definition of a CPG is ‘statements that include recommendations intended to optimize patient care. They are informed by systematic reviews of evidence and an assessment of the benefits and harms of alternative care options’ [13].

Inclusion criteria

Studies on women with SLE who (1) wish to get pregnant, (2) are pregnant, or (3) are in puerperium were included in this study.

Study selection

Pairs of four reviewers (M.H., R.Y., S.I., and Y.T.) independently screened the titles and abstracts of the records identified through the literature search. After title and abstract screening, the same pairs determined the eligibility of the studies based on the full texts. Discrepancies were resolved through discussion. When consensus could not be reached, a third author (N.Y.) acted as an arbitrator.

Extraction of QI items

Candidate QIs were extracted from the selected papers. When several articles reported QI candidates that were overlapping but slightly different from each other, two experts (M.H. and N.Y.) discussed whether they should be grouped.

Assessment of appropriateness by an expert panel

This approach comprised two rounds of anonymous ratings on a standardized scale and face-to-face discussions among panellists. We initially conducted two rounds, but the rounds and discussions continued until a consensus was reached.

Selection of the panellists

We compiled a panel of ten interdisciplinary experts comprising seven rheumatologists (T.A., Y.K., Y.T., M.K., H.K., Y.H., and A.N.), one paediatrician (T.M.), one obstetrician (D.F.), one primary care physician (T.A.), and one nephrologist (S.N.). We did not include doctors practicing at general hospitals and were limited to doctors working at university hospitals. The group convened to discuss the evidence and provide final ratings on the appropriateness of the QIs.

Assessment of appropriateness

Appropriateness was assessed in four steps (a first round of ratings, face-to-face meetings, a second round of ratings, and data analysis). Before the ratings and meetings, we developed a rating sheet (range 1–9, where 1 = not appropriate and 9 = definitely appropriate) with a column containing free-text comments. We composed documents that summarized the information (evidence grade and references) for each QI.

First round of ratings

In the first-round survey, rating sheets and documents containing all QI information were shared with the panellists via email. The panellists were asked to rate the appropriateness of each QI. The ratings were decided individually at homes or offices with no interaction among the panellists. Anonymous results comprising the median for each item and a variety of measures, comments, and suggestions were sent to each panel member after the first round. For items with low median scores and high variability and for items with comments, rounds and discussions were continued until a consensus was reached.

Face-to-face meeting

The discussion at the face-to-face meeting aimed to determine whether different ratings resulted from real clinical disagreements requiring minor revisions or whether clarifications in wording were needed. Based on the results of the first round, the panellists discussed the appropriateness of each QI (especially those that received different ratings from members) under the leadership of a moderator experienced in using this method. The QI items were revised as necessary. The moderator did not force the panellists to reach a consensus.

Second round of ratings and public comment

After the face-to-face meeting, we shared the second survey via email and the panellists re-rated the appropriateness of each item individually using the same scale with no interaction among the panellists. Items with a median score of at least 7 and no variation between the median score and the mean absolute deviation were included. Finally, we requested public comments on the QI set from the members of the Japan College of Rheumatology and revised it.

Data analysis

Each QI selected after the secondary evaluation was explained using statistics such as the median rating by the panellists. The analytical method was based on previous reports [5, 14, 15]. Based on the median appropriateness ratings of all panellists and their discrepancies, indicators were classified into three categories (‘appropriate’, ‘uncertain’, and ‘inappropriate’). Indicators with a median of 7–9 were defined as ‘appropriate’, 4–6 as ‘uncertain’, and 1–3 as ‘inappropriate’. For the agreement rating of each indicator, ‘strict agreement’ was defined as the mean (mean absolute deviation from the median) of the difference from the median not spanning an area different from the median [15, 16].

As a result, QIs with a median score of at least 7 points that met strict agreements were finally selected as QIs for pregnancy and childbirth in patients with SLE. The selected indicators were classified into three types according to Donabedian’s model as structure, process, and outcome. Structure was defined as the environment in which care is provided, qualifications of the provider, and management system; process as the care provided; and outcome as the impact on health status, including recovery, functional recovery, and survival [12].

Results

Literature selection

A literature search (MEDLINE and EMBASE) identified 4992 articles on SLE QI development and 2533 articles on CPGs. Title and abstract screening identified 118 articles on SLE QI development and 224 articles on CPGs. Full-text screening resulted in the exclusion of 109 articles on QI development that were not QI development articles, 1 that was not related to pregnancy and childbirth, and 4 whose data could not be extracted; among studies on CPG, 190 that were not CPG articles and 7 that were not related to pregnancy and childbirth were excluded. Finally, 4 articles on QI development [17–20] and 27 on CPGs [5, 7, 8, 21–42] were selected, including those related to pregnancy, childbirth, and lactation (Figure 1).

QI item selection

From the selected articles, 52 candidate QI items related to pregnancy, childbirth, and lactation were extracted. Eleven expert panel members, composed of specialists in rheumatology and collagenomics, nephrology, general internal medicine, obstetrics and gynaecology, and paediatrics, made a primary evaluation of the 52 candidate QI items. This was followed by a web-based face-to-face meeting with all panel members. During the meetings, the wording of each QI was revised to incorporate the opinions of each expert.

The 52 candidate QI items revised by the panel committee were evaluated in the second round. Consensus ratings of appropriateness were allotted using the modified Delphi method (RAND/UCLA method). The median score was divided into three categories: 1–3 as inappropriate, 4–6 as uncertain, and 7–9 as appropriate. Agreement of views was also evaluated based on variations in the median and absolute mean deviation. Items with a median score of at least 7 and no variation between the median and absolute mean deviations were included. Eleven items were excluded based on adequacy assessment. Three items were excluded for contraception, four for testing, and four for treatment. The time course is shown in Figure 2. The final QI set of 41 items is shown in Table 1 and was drawn from 4 SLE QI development articles and 27 CPGs.

Characteristics of the QI items

Table 1 presents the final set of the QIs developed in this study. There were 15 QI items related to pregnancy planning (contraception, assisted reproductive technology, acceptable pregnancy conditions, etc.), 6 related to examination during pregnancy (blood collection, urine analysis, foetal echocardiography, etc.), and 20 related to treatment before pregnancy, during pregnancy, and lactation (prednisolone immunosuppressive drugs, etc.). The wording of the QI items was changed or added in accordance with the Japanese guidelines at the panel committee meeting and panel discussions after the second rating.

Discussion

Employing a modified version of the RAND/UCLA Appropriateness Method, we successfully developed a comprehensive set of QIs for monitoring and improving care in pregnancy and childbirth-related SLE. A total of 52 candidate QI items were selected from 32 studies. Finally, 41 QI items were selected through a process of appropriateness checks; of the 41 pregnancy–delivery-related SLE QI items, 15 items were related to pregnancy planning, 6 to pregnancy testing, and 20 to treatment before pregnancy, during pregnancy, and during lactation.

On comparison with previous studies, various differences were noted. First, this was a QI study that focused on pregnancy and childbirth. In previous studies, QI items did not include those related to pregnancy planning (contraceptive methods, preconception drug adjustment, etc.) or the handling of antiphospholipid (aPL)–positive and anti-SS-A antibody–positive cases [17–20]. Clarification of the acceptable conditions for pregnancy is important for patients with SLE who wish to plan for pregnancy. Since many SLE patients are aPL-positive and anti-SS-A-positive, it is essential to check for antibodies at the time of pregnancy planning and to consider the risk of additional drug administration and foetal testing in advance for positive cases in collaboration with the obstetrician and gynaecologist. Therefore, it was considered necessary to include these items. It was also problematic that some studies did not include QI items for male patients with SLE. This is important because information on lupus paternity is limited, and they are likely to be less attentive to pregnancy than female patients with SLE. Based on the above, we developed a pregnancy- and delivery-related SLE QI that can be used from the time of pregnancy planning through lactation, including SLE-related antibody-positive and male cases. On the other hand, some items related to contraception and routine laboratory monitoring, which were adopted in previous studies, were not incorporated in this study [5–8]. These items were excluded due to the limited availability of drugs in Japan and because they were assumed to be 100% implemented. In this study, using a modified version of the RAND/UCLA Appropriateness Method, pregnancy- and childbirth-related QIs for collagen disease were extracted from SLE QI development reports and CPGs [5, 7, 8, 17–42], and QIs for antiphospholipid syndrome (APS) other than SLE and items from Japanese guidelines were also added. The QIs selected through multiple evaluations by a panel of experts constitute an evidence-based QI set. Second, the QIs developed in this study were primarily process indicators. Process indicators are likely to lead to better quality care for patients with SLE. However, their use is limited because process indicators are not necessarily associated with disease prognosis. Previous studies have reported that the SLE QI constructed from process measures is proportional to the Systemic Lupus International Collaborating Clinics damage index [10] and reduces hospitalization rates in patients with collagen disease [11]. Similarly, the focus of this study was on process indices.

This study has several clinical implications. First, QI assessments can identify specific facilities that require interventions. QI assessment can improve the quality of care, thereby allowing for more efficient education. Second, our QI set may increase the number of hospitals that can adequately manage prenatal and postpartum patients with SLE, thereby facilitating better care to patients with SLE who consider pregnancy and delivery. In Japan, there are few maternal outpatient clinics for patients with prenatal and postpartum collagen diseases, and the experiences and knowledge are highly biased among the professionals of various facilities. This QI set allows any facility to provide evidence-based care. Third, the importance of QI in the treatment of rheumatoid collagen disease in Japan is widely recognized. The challenge is that QI is less recognized in Japan than that in other countries and areas of rheumatological collagen disease, and its development is lagging. Fourth, it could be expanded to assess the quality of medical care for other chronic diseases during pregnancy. Based on the results of this study, we hope that the development of QI for pregnancy and childbirth for other chronic diseases will be promoted in the future.

This study has some limitations. First, the panel members did not include healthcare professionals other than patients and physicians (pharmacists, nurses, etc.), which may have biased the panel towards a particular viewpoint. However, the panel in this study consisted of 11 members, including specialists from related departments (nephrology, general internal medicine, obstetrics and gynaecology, and paediatrics), and efforts were made to include physicians from several departments to allow discussion from a broad perspective. Second, this study is based on a systematic literature review of studies based on articles published up to September 2021, but does not include studies published after that date.

This is because the process of QI development is time-consuming, requiring multiple discussions. Third, this QI set may not be applicable in other countries. This is because the clinical factors, including drug availability, may differ in each country. Fourth, this QI set may include QI for which 100% implementation has been achieved. In the future, we will revise this QI set in a series of ways to improve its relevance. Fifth, whether the use of this QI set improved the patient outcomes could not be assessed. This should also be assessed in future studies.

Despite these limitations, we developed the first QI set that specifically focused on pregnancy and childbirth in SLE using a rigorous and validated method that was conducted in accordance with the RAND/UCLA method, followed by a web-based face-to-face panel discussion and two rounds of ratings. We plan to use the developed QI to conduct assessments at related facilities and investigate the relationship between assessment results, pregnancy outcomes, and prognosis.

Conclusion

In the present study, we developed the first 41-item SLE QI related to pregnancy and childbirth. Evaluating the quality of care from the patient’s perspective is also important and will be the subject of future research. Additional research is required to confirm whether the measurement of this QI set improves clinically relevant outcomes.

Acknowledgements

We would like to thank all Japan College of Rheumatology (JCR) members who contributed to the public comments.

Supplementary data

Supplementary data is available at Modern Rheumatology online.

Conflict of interest

None declared.

Funding

This study was supported by JSPS KAKENHI under Grant Number 21K10358.

References