Abstract
We sought to identify potential evidence-practice gaps in palliative radiotherapy using quality indicators (QIs), previously developed using a modified Delphi method. Seven QIs were used to assess the quality of radiotherapy for bone metastases (BoM) and brain metastases (BrM). Compliance rate was calculated as the percentage of patients for whom recommended medical care was conducted. Random effects models were used to estimate the pooled compliance rates. Of the 39 invited radiation oncologists, 29 (74%) from 29 centers participated in the survey; 13 (45%) were academic and 16 (55%) were non-academic hospitals. For the QIs, except for BoM-4, the pooled compliance rates were higher than 80%; however, for at least some of the centers, the compliance rate was lower than these pooled rates. For BoM-4 regarding steroid use concurrent with radiotherapy for malignant spinal cord compression, the pooled compliance rate was as low as 32%. For BoM-1 regarding the choice of radiation schedule, the compliance rate was higher in academic hospitals than in non-academic hospitals (P = 0.021). For BrM-3 regarding the initiation of radiotherapy without delay, the compliance rate was lower in academic hospitals than in non-academic hospitals (P = 0.016). In conclusion, overall, compliance rates were high; however, for many QIs, practice remains to be improved in at least some centers. Steroids are infrequently used concurrently with radiotherapy for malignant spinal cord compression.
INTRODUCTION
Clinical practice is not always performed in accordance with the guideline recommendations. Difficulties in implementing evidence-based practices have been demonstrated in palliative radiation oncology [1] and other areas of medicine [2, 3]. Quality indicators (QIs) are valuable tools for evaluating the quality of healthcare systems. Some QIs have been developed in radiation oncology [4]; however, surveys using them seem to be limited [5]. Previously, QIs have been developed to assess the quality of radiotherapy for bone metastases (BoM) and brain metastases (BrM) [6]. Additionally, these QIs were pilot tested in five centers, and the feasibility of their measurement was confirmed [6]. Radiotherapy for BoM and BrM is the standard of care for these diseases [7, 8]; however, its quality has scarcely been surveyed. To identify potential gaps between clinical practice and evidence in palliative radiotherapy, we conducted the present survey in radiation oncology centers in Japan.
MATERIALS AND METHODS
Process QIs are widely used tools to evaluate the processes involved in health care [9]. Process QIs are presented as numerators and denominators (the percentage of patients for whom recommended medical care was conducted). In the present study, we used seven process QIs (Table 1), previously developed through a modified Delphi method [6], which is a method for determining expert consensus [10]. The QIs were developed through three online meetings and two e-mail surveys by a panel consisting of eight radiation oncologists, with expertise in palliative radiation oncology, and one expert on the Delphi methodology [6]. Of the seven QIs, four were on BoM and three were on BrM; the definitions of the denominators and numerators of the QIs are presented in Table 1. The denominator of the QI represents the number of patients for whom the QI was used to evaluate the quality of certain aspects of radiation oncology practice. The numerator represents the number of patients (among those in the denominator) for whom recommended care was provided. The compliance rate was calculated as the percentage of patients for whom practice was performed as recommended.
Quality indicators
| Quality indicators | Brief description | Definition of denominator | Definition of numerator | Total number of patients assessed | Pooled compliance rate (95% confidence interval) |
|---|---|---|---|---|---|
| BoM-1 | Choice of radiation schedules | Patients who received radiation therapy for painful BoMa | Patients who received radiation therapy in ≤10 fractions, or for whom the reason for the use of extended-fractionation was written in the medical chart | 435 | 99% (97–100%) |
| BoM-2 | Assessment of pain before radiation therapy | Patients who received radiation therapy for painful BoMa | Patients for whom some description on pain before radiation therapy was written in the medical chart | 435 | 97% (94–99%) |
| BoM-3 | Prompt initiation of radiation therapy for clinical MSCC | Patients who received radiation therapy for clinical MSCCb | Patients for whom radiation therapy was initiated on the day of referral to radiation oncology or the next day | 115 | 82% (68–93%) |
| BoM-4 | Concurrent use of steroids with radiation therapy for clinical MSCC | Patients who received radiation therapy for clinical MSCCb | Patients for whom steroids were initiated or increased concurrently with the initiation of radiation therapy | 115 | 32% (18–47%) |
| BrM-1 | Assessment of performance status before radiation therapy | Patients who received radiation therapy for BrM | Patients for whom performance status before radiation therapy was recorded by radiation oncologists in the medical chart or radiology information system | 288 | 92% (82–99%) |
| BrM-2 | Completion of planned radiation therapy | Patients who received whole-brain radiation therapy for BrM | Patients for whom the planned radiation therapy was completed | 215 | 97% (93–99%) |
| BrM-3 | Initiation of radiation therapy without delay | Patients who received whole-brain radiation therapy for BrMc | Patients for whom the radiation therapy was initiated within 10 days from referral to radiation oncology | 201 | 97% (92–99%) |
| Quality indicators | Brief description | Definition of denominator | Definition of numerator | Total number of patients assessed | Pooled compliance rate (95% confidence interval) |
|---|---|---|---|---|---|
| BoM-1 | Choice of radiation schedules | Patients who received radiation therapy for painful BoMa | Patients who received radiation therapy in ≤10 fractions, or for whom the reason for the use of extended-fractionation was written in the medical chart | 435 | 99% (97–100%) |
| BoM-2 | Assessment of pain before radiation therapy | Patients who received radiation therapy for painful BoMa | Patients for whom some description on pain before radiation therapy was written in the medical chart | 435 | 97% (94–99%) |
| BoM-3 | Prompt initiation of radiation therapy for clinical MSCC | Patients who received radiation therapy for clinical MSCCb | Patients for whom radiation therapy was initiated on the day of referral to radiation oncology or the next day | 115 | 82% (68–93%) |
| BoM-4 | Concurrent use of steroids with radiation therapy for clinical MSCC | Patients who received radiation therapy for clinical MSCCb | Patients for whom steroids were initiated or increased concurrently with the initiation of radiation therapy | 115 | 32% (18–47%) |
| BrM-1 | Assessment of performance status before radiation therapy | Patients who received radiation therapy for BrM | Patients for whom performance status before radiation therapy was recorded by radiation oncologists in the medical chart or radiology information system | 288 | 92% (82–99%) |
| BrM-2 | Completion of planned radiation therapy | Patients who received whole-brain radiation therapy for BrM | Patients for whom the planned radiation therapy was completed | 215 | 97% (93–99%) |
| BrM-3 | Initiation of radiation therapy without delay | Patients who received whole-brain radiation therapy for BrMc | Patients for whom the radiation therapy was initiated within 10 days from referral to radiation oncology | 201 | 97% (92–99%) |
BoM, bone metastases; BrM, brain metastases; MSCC, malignant spinal cord compression.
Patients with hematologic tumors should be excluded from the denominators in all the quality indicators.
aPatients who had received radiation therapy or surgery to the same bone metastases should be excluded from the denominator.
bWhen a symptom in the lower extremities, caused by spinal cord compression, was written in the medical chart or referral letter.
cPatients who received intensity modulated whole brain radiotherapy should be excluded from the denominator.
Quality indicators
| Quality indicators | Brief description | Definition of denominator | Definition of numerator | Total number of patients assessed | Pooled compliance rate (95% confidence interval) |
|---|---|---|---|---|---|
| BoM-1 | Choice of radiation schedules | Patients who received radiation therapy for painful BoMa | Patients who received radiation therapy in ≤10 fractions, or for whom the reason for the use of extended-fractionation was written in the medical chart | 435 | 99% (97–100%) |
| BoM-2 | Assessment of pain before radiation therapy | Patients who received radiation therapy for painful BoMa | Patients for whom some description on pain before radiation therapy was written in the medical chart | 435 | 97% (94–99%) |
| BoM-3 | Prompt initiation of radiation therapy for clinical MSCC | Patients who received radiation therapy for clinical MSCCb | Patients for whom radiation therapy was initiated on the day of referral to radiation oncology or the next day | 115 | 82% (68–93%) |
| BoM-4 | Concurrent use of steroids with radiation therapy for clinical MSCC | Patients who received radiation therapy for clinical MSCCb | Patients for whom steroids were initiated or increased concurrently with the initiation of radiation therapy | 115 | 32% (18–47%) |
| BrM-1 | Assessment of performance status before radiation therapy | Patients who received radiation therapy for BrM | Patients for whom performance status before radiation therapy was recorded by radiation oncologists in the medical chart or radiology information system | 288 | 92% (82–99%) |
| BrM-2 | Completion of planned radiation therapy | Patients who received whole-brain radiation therapy for BrM | Patients for whom the planned radiation therapy was completed | 215 | 97% (93–99%) |
| BrM-3 | Initiation of radiation therapy without delay | Patients who received whole-brain radiation therapy for BrMc | Patients for whom the radiation therapy was initiated within 10 days from referral to radiation oncology | 201 | 97% (92–99%) |
| Quality indicators | Brief description | Definition of denominator | Definition of numerator | Total number of patients assessed | Pooled compliance rate (95% confidence interval) |
|---|---|---|---|---|---|
| BoM-1 | Choice of radiation schedules | Patients who received radiation therapy for painful BoMa | Patients who received radiation therapy in ≤10 fractions, or for whom the reason for the use of extended-fractionation was written in the medical chart | 435 | 99% (97–100%) |
| BoM-2 | Assessment of pain before radiation therapy | Patients who received radiation therapy for painful BoMa | Patients for whom some description on pain before radiation therapy was written in the medical chart | 435 | 97% (94–99%) |
| BoM-3 | Prompt initiation of radiation therapy for clinical MSCC | Patients who received radiation therapy for clinical MSCCb | Patients for whom radiation therapy was initiated on the day of referral to radiation oncology or the next day | 115 | 82% (68–93%) |
| BoM-4 | Concurrent use of steroids with radiation therapy for clinical MSCC | Patients who received radiation therapy for clinical MSCCb | Patients for whom steroids were initiated or increased concurrently with the initiation of radiation therapy | 115 | 32% (18–47%) |
| BrM-1 | Assessment of performance status before radiation therapy | Patients who received radiation therapy for BrM | Patients for whom performance status before radiation therapy was recorded by radiation oncologists in the medical chart or radiology information system | 288 | 92% (82–99%) |
| BrM-2 | Completion of planned radiation therapy | Patients who received whole-brain radiation therapy for BrM | Patients for whom the planned radiation therapy was completed | 215 | 97% (93–99%) |
| BrM-3 | Initiation of radiation therapy without delay | Patients who received whole-brain radiation therapy for BrMc | Patients for whom the radiation therapy was initiated within 10 days from referral to radiation oncology | 201 | 97% (92–99%) |
BoM, bone metastases; BrM, brain metastases; MSCC, malignant spinal cord compression.
Patients with hematologic tumors should be excluded from the denominators in all the quality indicators.
aPatients who had received radiation therapy or surgery to the same bone metastases should be excluded from the denominator.
bWhen a symptom in the lower extremities, caused by spinal cord compression, was written in the medical chart or referral letter.
cPatients who received intensity modulated whole brain radiotherapy should be excluded from the denominator.
The present survey study was performed by members of the Japanese Society for Radiation Oncology (JASTRO) palliative radiotherapy committee and the Japanese Radiation Oncology Study Group (JROSG) palliative medicine committee. We evaluated the quality of palliative radiotherapy for BoM and BrM. For BoM, we evaluated painful BoM and clinical malignant spinal cord compression (MSCC). Patients with clinical MSCC were defined as those who were reported to have lower extremity symptoms caused by spinal cord compression in the medical chart or referral letter. Regarding BrM, we evaluated patients who received any radiation therapy for BrM and those who received whole-brain radiation therapy for BrM.
One panel member (N.S.) sent an e-mail to the members of the JASTRO palliative radiotherapy committee and the JROSG palliative medicine committee, inviting them to participate in a survey to evaluate the quality of palliative radiation oncology. Patients for whom the radiotherapy start date was between 1 January 2021 and 30 June 2021 were screened for eligibility for the study. The patients were screened consecutively from 1 January 2021. When the denominator of a QI reached 10, even if the screening period did not reach 30 June 2021, the screening of patients for the QI was allowed to be declared complete; full screening (i.e. from 1 January 2021 to 30 June 2021), while the denominator of the QI exceeded 10, was alternatively allowed. This study was approved by the Institutional Review Boards of Juntendo University (E22-0229), Seirei Mikatahara General Hospital (22-36), Shizuoka Cancer Center (T2022-40-2022-1-3), Nanbu Tokushukai Hospital (TGE02048-005), Fukuchiyama City Hospital (4-29) and Kyorin University (2039); the informed consent was waived.
Random effects models were used to estimate the pooled compliance rates. Compliance rates and 95% confidence intervals for each center were presented in a forest plot. Mixed effects models with Q tests were used to compare compliance rates between academic and non-academic centers. A P-value of <0.05 was considered statistically significant. Statistical analyses were performed using R version 4.2.2.
RESULTS
Of the 39 invited members of the JASTRO palliative radiotherapy committee and the JROSG palliative medicine committee, 29 (74%) from 29 centers participated in the survey. Of the 29 centers, 13 (45%) were academic hospitals (12 university hospitals and one cancer center) and 16 (55%) were non-academic hospitals.
The compliance rates are presented in Table 1 and Figs 1 and 2. In Fig. 1, the participating centers are shown in the order of magnitude of the estimates of the compliance rate for BoM-1; centers for which the denominator of the QI was zero are left blank. Similarly, in Fig. 2, the centers are shown in the order of the magnitude of the estimates of the compliance rate for BrM-1. The pooled compliance rates for all QIs were higher than 80%, except BoM-4, for which it was as low as 32%.
Compliance rates. Below the estimates and 95% confidence intervals of the compliance rates of the participating centers, the 95% confidence intervals of the pooled compliance rates are shown as diamonds. The two leftmost columns of numbers in each quality indicator are the numbers of patients for whom recommended medical care was performed and the total number of patients assessed in the participating hospitals. BoM, bone metastases; BrM, brain metastases.
Compliance rates. Below the estimates and 95% confidence intervals of the compliance rates of the participating centers, the 95% confidence intervals of the pooled compliance rates are shown as diamonds. The two leftmost columns of numbers in each quality indicator are the numbers of patients for whom recommended medical care was performed and the total number of patients assessed in the participating hospitals. BoM, bone metastases; BrM, brain metastases.
As shown in Fig. 3, for BoM-1, the compliance rate was higher in academic hospitals than in non-academic hospitals (P = 0.021). For the BrM-3, the compliance rate was lower in academic hospitals than in non-academic hospitals (P = 0.016, Fig. 4).
Compliance rates in academic vs non-academic centers. Below the estimates and 95% confidence intervals of the compliance rates of the participating centers, the 95% confidence intervals of the pooled compliance rates are shown as diamonds. BoM, bone metastases; BrM, brain metastases.
Compliance rates in academic vs non-academic centers. Below the estimates and 95% confidence intervals of the compliance rates of the participating centers, the 95% confidence intervals of the pooled compliance rates are shown as diamonds. BoM, bone metastases; BrM, brain metastases.
DISCUSSION
The pooled compliance rates of all the QIs were higher than 80%, except BoM-4. One reason for these high compliance rates may be that we surveyed practices in centers where members of the JASTRO palliative radiotherapy committee and the JROSG palliative medicine committee worked. The quality of palliative radiation oncology at these centers may be higher than the average quality of radiation oncology centers in Japan. This potential bias in the selection of the centers surveyed highlights the finding that steroids are infrequently used concurrently with radiotherapy for MSCC.
The use of steroids concurrent with radiotherapy for clinical MSCC is widely recommended [11–16]. A randomized trial comparing high-dose dexamethasone and no dexamethasone concurrent with radiotherapy for MSCC found significantly higher ambulation rates in patients who received high-dose dexamethasone [17]. Small studies inconclusively indicated that high-dose steroids were not different from moderate-dose steroids in enhancing ambulation but were more frequently associated with serious adverse effects than were moderate-dose steroids [18–20]. Despite the lack of high-quality evidence to determine the appropriate dose of steroids, moderate-dose steroids (e.g. dexamethasone 16 mg per day) seem to be widely used for MSCC [21, 22].
Evidence exists suggesting that steroids should not be used routinely where a patient has good motor function [23]. In this single-arm trial [23], 20 patients with no neurologic deficits or only radiculopathy received radiotherapy for MSCC without steroids; the ambulation status was good after treatment in these patients with a median survival of 14 months. Therefore, we defined the BoM-4 denominator to include only patients with ‘clinical’ MSCC, excluding those with ‘radiological’ asymptomatic MSCC.
Our pooled compliance rate of 32% for the concurrent use of steroids with radiotherapy for clinical MSCC (BoM-4) was lower than those reported in previous single-center studies [24, 25]. In a retrospective audit of clinical practice on MSCC at a UK regional cancer center, in 42% of patients, dexamethasone had been prescribed before cancer center admission, and this increased to 96% following admission [24]. In a study based on the medical records of patients who received palliative radiotherapy for MSCC at a US university hospital, 80% and 88% of patients received steroids before and after the quality improvement initiative was introduced, respectively [25]. Although these studies have reported the utilization rate of steroids in single-center settings [24, 25], the present study may be the first multicenter survey to investigate the utilization rate of steroids concurrent with radiotherapy for MSCC.
Two previous studies that investigated the appropriate selection of dose schedules in radiotherapy for BoM [26, 27] reported comparable and lower compliance rates than ours (i.e. the pooled compliance rate of 99% for the use of fractions ≤10 for painful BoM [BoM-1]). In the Michigan Radiation Oncology Quality Consortium, among patients treated in the 28 participating radiation oncology practices, 97% and 98% of them who received radiotherapy for BoM received treatment with ≤10 fractions before and after the implementation of quality improvement measures (performed in 2019), respectively [26]. In another study based on the National Cancer Database, 60.2% of the patients with metastatic thoracic non-small cell lung cancer who were diagnosed between 2004 and 2016 and received radiotherapy for BoM received treatment with one of the following schedules: 30 Gy in 10 fractions, 24 Gy in 6 fractions, 20 Gy in 5 fractions or 8 Gy in a single fraction [27].
Compliance rates for the BoM-1 were higher in academic hospitals than in non-academic hospitals. Extended fractionation (≥11 fractions) for the BoM, which is not routinely recommended [27], appears to be performed less frequently in academic centers than in non-academic centers. In academic centers, practice may be more concordant with recommendations and guidelines than in non-academic centers. Another explanation is that the number of patients receiving radiotherapy tends to be higher in academic centers, and the burden of extended fractionation tends to be unacceptable in terms of machine time and manpower.
Compliance rates for the BrM-3 were lower in academic hospitals than in non-academic hospitals. Thus, the initiation of radiotherapy for BrM is more frequently delayed in academic centers than in non-academic centers. This may again reflect the heavier workload in academic centers.
This study had some limitations. First, there may have been bias in the selection of the centers surveyed. In the surveyed centers, the quality of palliative radiotherapy may have been, on average, high. Second, since the number of patients screened for eligibility for this study was not recorded, we could not evaluate how the assessed patients were selected from the patients who received palliative radiotherapy for BoM or BrM during the study period. Third, the primary sites of BoM and BrM were not recorded and, therefore, could not be analyzed. Fourth, the survey was conducted by physicians at their own facility rather than being audited by independent experts.
CONCLUSION
In summary, we conducted a multicenter survey on the quality of palliative radiotherapy for BoM and BrM using previously developed and pilot-tested QIs. Overall, compliance rates were high; however, in many QIs, the practice remains to be improved in at least some centers. Despite supporting evidence and guideline recommendations, steroids seem to be underutilized concurrently with radiotherapy for MSCC. Approaches to improve the underutilization of steroids in the treatment of MSCC are required.
CONFLICT OF INTEREST
N.S. was a member of the advisory board and has received honorariums from Elekta K.K.
FUNDING
This study was supported by the Health Labor Sciences Research Grant from the Ministry of Health, Labor, and Welfare of Japan (JPMH21EA1010 and 23EA1012).
Footnotes
†Presentation at a conference: This study was presented at a poster session during the American Society for Radiation Oncology Annual Meeting 2023.
