Abstract

Background

Evidence for the management of inadequate clinical response to clozapine in treatment-resistant schizophrenia is sparse. Accordingly, an international initiative was undertaken with the aim of developing consensus recommendations for treatment strategies for clozapine-refractory patients with schizophrenia.

Methods

We conducted an online survey among members of the Treatment Response and Resistance in Psychosis (TRRIP) working group. An agreement threshold of ≥75% (responses “agree” + “strongly agree”) was set to define a first-round consensus. Questions achieving agreement or disagreement proportions of >50% in the first round, were re-presented to develop second-round final consensus recommendations.

Results

Forty-four (first round) and 49 (second round) of 63 TRRIP members participated. Expert recommendations at ≥75% agreement included raising clozapine plasma levels to ≥350 ng/ml for refractory positive, negative, and mixed symptoms. Where plasma level-guided dose escalation was ineffective for persistent positive symptoms, waiting for a delayed response was recommended. For clozapine-refractory positive symptoms, combination with a second antipsychotic (amisulpride and oral aripiprazole) and augmentation with ECT achieved consensus. For negative symptoms, waiting for a delayed response was recommended, and as an intervention for clozapine-refractory negative symptoms, clozapine augmentation with an antidepressant reached consensus. For clozapine-refractory suicidality, augmentation with antidepressants or mood-stabilizers, and ECT met consensus criteria. For clozapine-refractory aggression, augmentation with a mood-stabilizer or antipsychotic medication achieved consensus. Generally, cognitive-behavioral therapy and psychosocial interventions reached consensus.

Conclusions

Given the limited evidence from randomized trials of treatment strategies for clozapine-resistant schizophrenia (CRS), this consensus-based series of recommendations provides a framework for decision making to manage this challenging clinical situation.

Introduction

For approximately 20%–30% of patients with schizophrenia, the illness fails to respond to ≥2 adequate, in terms of dose and duration, trials of first-line antipsychotic medication.1 This is the clinical definition of treatment-resistant schizophrenia (TRS), which is associated with significantly lower patient quality of life and with significantly higher socioeconomic costs2 compared with non-TRS resulting in an immense individual and societal burden. Since 1988, clozapine has been recommended in all guidelines as the gold-standard treatment for TRS.1,3–8 In meta-analyses, clozapine has been found to be superior to first-generation antipsychotic medication regarding total symptoms9 and to first- and second-generation antipsychotic medication regarding total, positive, and negative symptoms.10 However, up to 60% of clozapine-treated patients do not respond adequately,11 and how to clinically manage these patients is unclear.

Clozapine-resistant schizophrenia (CRS) is defined by the Treatment Response and Resistance in Psychosis (TRRIP) Working Group as persistence of either positive, negative, or cognitive symptoms of schizophrenia of at least moderate severity after an adequate trial of clozapine.12 More specifically, persistent symptoms in the positive or negative domain are defined as ≥2 symptoms in the respective domain with at least moderate severity, or ≥1 symptom with at least a severe rating.12 Although cognitive impairment is a characteristic feature of schizophrenia, cognitive symptoms are not defined in the TRRIP publication.12

One of the most relevant questions in the clinical care of people with schizophrenia is how to treat CRS. In general, such illness shows only limited symptomatic improvement from the pre-clozapine baseline, with ongoing functional deficits and disabling symptoms. The TRRIP working group recommended that in the case of TRS, clozapine treatment should be offered for a duration of ≥3 months after reaching therapeutic plasma levels,12 but did not discuss strategies for the treatment of persistent symptoms despite adequate clozapine monotherapy. Evidence from meta-analyses indicates only marginal and/or low-quality benefits for pharmacological clozapine combination strategies after insufficient response to clozapine monotherapy.13,14

Randomized controlled trials (RCTs) investigating pharmacological treatment options for CRS have been subject to meta-analyses, but the methodological quality in these RCTs suggest considerable heterogeneity.13,15 Obtaining high-level evidence is further hampered by different or absent definitions of CRS.16 Electroconvulsive therapy (ECT) is reported to be an efficacious treatment option for clozapine-refractory positive symptoms in open-label studies,17,18 but more high-quality trials are needed before ECT can be included in treatment algorithms.17 A recent RCT (n = 487) in CRS failed to demonstrate a benefit for clozapine augmentation with cognitive-behavioral therapy (CBT).19 Despite limited evidence of effectiveness, antipsychotic polypharmacy is common among patients with schizophrenia20: clozapine may be combined with another antipsychotic medication in up to half of clozapine prescriptions.21 This relatively high prevalence of clozapine-antipsychotic cotreatment in clinical practice reflects the need for guidelines covering a hierarchy of pharmacological and nonpharmacological treatment recommendations for patients with CRS. Finally, treatment with clozapine is often delayed due to barriers related to prescribers and institutions,22 reducing the likelihood for a potential treatment response.23

As the current evidence for the management of CRS is limited by the available data sources, and as treatment of CRS represents a major challenge for clinicians and cessation of clozapine is associated with unfavorable outcomes in real-world settings,24 we decided to conduct a 2-step survey and consensus process among international experts. Such approaches have been used previously to define antipsychotic dosing25 or recovery in psychosis.26 The main purpose of this project was to outline clinically meaningful treatment options for CRS patients with persistent symptoms despite an adequate trial of clozapine monotherapy.

Methods

Participants and Survey

The project was initiated during a TRRIP meeting at the Schizophrenia International Research Society conference 2018 in Florence, Italy. The online survey was developed and revised by all authors and approved by the local data protection officer and the ethics committee (Ref. nr. 18–706 UE). The 63 members of TRRIP comprise expert researchers and clinicians, scientists from the pharmaceutical industry and other specialists with experience and expertise in the area of schizophrenia12 (supplementary paragraph 2.1). The complete survey and descriptions of possible ratings and ranks appear in the supplement.

Statistical Analyses

IBM SPSS-25 for Windows was used for statistical analyses. Descriptive statistics include frequencies, mean, standard deviation (SD), minimum, maximum, sum, median, and interquartile range (IQR).

All these parameters were applied to continuous variables, and frequencies were applied to present dichotomous variables. An agreement threshold of ≥75% (defined as sum of frequencies for “agree” and “strongly agree” in responses) in the second round for each question/statement/substance was set to define consensus in accordance with Delphi methods.27,28 Furthermore, a disagreement threshold (defined as sum of frequencies for “disagree” and “strongly disagree”) was also defined with a threshold of ≥75% in the second round. To simplify the interpretation, ranked response questions were transformed in the final analyses with higher values representing a higher ranking. In the second round, questions that previously achieved agreement or disagreement with a >50% threshold were re-presented to determine if consensus was present (defined as ≥75% in the second round28). When a specific treatment strategy was rated ≤50% in the first round, but single substances were rated >50%, single substances were presented again in the second round. Unless otherwise specified, the following results represent the second round. Due to the threshold definition, several questions were only presented in the first round and in these cases, results from the first round are displayed. More details and the complete descriptive statistics appear in the supplement (first round: Supplement-A [SA]; second round: Supplement-B [SB]).

Results

Survey

Of the 63 invited TRRIP members, 47 responded to the first-round invitation. Thirty participants fully completed the survey, while 17 provided incomplete responses. Three participants were excluded due to lack of relevant data. Altogether, 44 participants were included in the analysis (30 complete, 14 incomplete). In the second round, 49 participants responded to the invitation with 38 complete and 11 incomplete responses. Next, these findings and recommendations were reviewed by the group.

Survey Results

Description of the Participant Sample

The participants resided in 12 different countries (SA-table 1A) and on average had 26.1 (±9.3) years of clinical experience. See SA-table 1C for the overall ratings.

Experience of Severe Complications and Safety

We asked only for severe complications (SA-table 1B), but not for everyday side-effects of clozapine. At least two-thirds of survey participants had experience with ≥1 case of an ileus, myocarditis, neuroleptic malignant syndrome (NMS), or agranulocytosis (SA-table 1B). Agranulocytosis was the most frequently reported side-effect (median = 3) followed by myocarditis (median = 2), ileus (median = 1) and NMS (median = 0) (SA-table 1B). However, the overall safety of clozapine was rated to be good (median = 7, SA-table 1C).

Optimal Clozapine Treatment Definitions and Waiting for Delayed Response to Clozapine Monotherapy

Based on first-round data, median suggested maximum wait times for delayed response to clozapine were shortest for aggression and suicidality (5 and 6 weeks), followed by positive, mixed, and negative symptoms (12 weeks), and longest for cognitive symptoms (17 weeks) (table 1). Similarly, optimal clozapine trial lengths were also shorter for aggression and suicidality (8 weeks), intermediate for positive and mixed symptoms (12 weeks), and longest for negative and cognitive symptoms (16 weeks) (table 1).

Table 1.

Optimal Trial Length Definitions and Wait for Delayed Response

NMean (weeks)SDMedian (weeks)Min (weeks)Max (weeks)IQR (weeks)
Wait for delayed responsea
 Mixed symptoms3916.117.11221048
 Positive symptoms3915.017.21221046
 Negative symptoms3919.114.71205212
 Cognition3819.713.71705212
 Suicidality398.59.160528
 Aggression397.36.5512610
Optimal trial lengthb
 Mixed symptoms3915.911.21245216
 Positive symptoms3914.611.31245216
 Negative symptoms3919.613.61605212
 Cognition3918.311.61605212
 Suicidality3911.210.280528
 Aggression3910.810.480528
NMean (weeks)SDMedian (weeks)Min (weeks)Max (weeks)IQR (weeks)
Wait for delayed responsea
 Mixed symptoms3916.117.11221048
 Positive symptoms3915.017.21221046
 Negative symptoms3919.114.71205212
 Cognition3819.713.71705212
 Suicidality398.59.160528
 Aggression397.36.5512610
Optimal trial lengthb
 Mixed symptoms3915.911.21245216
 Positive symptoms3914.611.31245216
 Negative symptoms3919.613.61605212
 Cognition3918.311.61605212
 Suicidality3911.210.280528
 Aggression3910.810.480528

Note: N, number of participants; SD, standard deviation; min, minimum value; max, maximum value; IQR, interquartile range.

a  Delayed response: Participants were asked how long they would wait for a delayed response to CLZ monotherapy at an adequate dosage for the respective persistent symptom type.

b  Optimal trial length: Participants were asked for the optimal trial length of CLZ monotherapy at an adequate dosage for the respective symptom type before describing a symptom type as clozapine-refractory. These results are derived from the first round of the survey.

Table 1.

Optimal Trial Length Definitions and Wait for Delayed Response

NMean (weeks)SDMedian (weeks)Min (weeks)Max (weeks)IQR (weeks)
Wait for delayed responsea
 Mixed symptoms3916.117.11221048
 Positive symptoms3915.017.21221046
 Negative symptoms3919.114.71205212
 Cognition3819.713.71705212
 Suicidality398.59.160528
 Aggression397.36.5512610
Optimal trial lengthb
 Mixed symptoms3915.911.21245216
 Positive symptoms3914.611.31245216
 Negative symptoms3919.613.61605212
 Cognition3918.311.61605212
 Suicidality3911.210.280528
 Aggression3910.810.480528
NMean (weeks)SDMedian (weeks)Min (weeks)Max (weeks)IQR (weeks)
Wait for delayed responsea
 Mixed symptoms3916.117.11221048
 Positive symptoms3915.017.21221046
 Negative symptoms3919.114.71205212
 Cognition3819.713.71705212
 Suicidality398.59.160528
 Aggression397.36.5512610
Optimal trial lengthb
 Mixed symptoms3915.911.21245216
 Positive symptoms3914.611.31245216
 Negative symptoms3919.613.61605212
 Cognition3918.311.61605212
 Suicidality3911.210.280528
 Aggression3910.810.480528

Note: N, number of participants; SD, standard deviation; min, minimum value; max, maximum value; IQR, interquartile range.

a  Delayed response: Participants were asked how long they would wait for a delayed response to CLZ monotherapy at an adequate dosage for the respective persistent symptom type.

b  Optimal trial length: Participants were asked for the optimal trial length of CLZ monotherapy at an adequate dosage for the respective symptom type before describing a symptom type as clozapine-refractory. These results are derived from the first round of the survey.

Clozapine Combination and Augmentation Strategies and Clozapine Dosage Changes

Positive Symptoms

For the management of clozapine-refractory positive symptoms, the strategy to raise clozapine plasma levels350 ng/ml obtained the highest ranking (median = 5) (table 2 and SA-table 10). The consensus threshold was met for the strategy of combining clozapine with another antipsychotic (79.5% in the second round) (table 3).

Table 2.

Hierarchy of Top 5 Treatment Options According to the Specific Symptom Domain or Clinical Scenario (Positive, Negative, Mixed Symptoms, and Absent/Improvement or Initial Worsening) and Hierarchy of All Treatment Options for Suicidality and Aggression

Treatment OptionNMinMaxMean ScoreSDMedian
CLZ-refractory positive symptoms
 Raise CLZ plasma levels ≥350 ng/ml38154.581.135
 Combination with second AP36142.921.033
 Wait for delayed response26152.621.273
 Augmentation with non-AP psychotropic26152.621.273
 Augmentation with ECT32152.381.12
CLZ-refractory negative symptoms
 Raise CLZ plasma levels ≥350 ng/ml32154.341.265
 Augmentation with non-AP psychotropic medication34153.150.933
 Combination with second AP23153.131.143
 Wait for delayed CLZ response301531.443
 Augmentation with CBT/psychosocial interventions36152.751.252.5
CLZ-refractory mixed symptoms
 Raise CLZ plasma levels ≥350 ng/ml37154.431.285
 Combination with second AP34142.850.993
 Wait for delayed response23152.871.423
 Augmentation with non-AP psychotropic medication26142.7313
 Augmentation with CBT/psychosocial interventions28152.751.352.5
Absent improvement/worsening under CLZ
 Raise CLZ plasma levels ≥350 ng/ml34154.530.965
 Combination with second AP26153.121.113
 Wait for delayed CLZ response221531.383
 Switch to different AP19142.891.213
 Augmentation with non-AP psychotropic medication18152.670.843
CLZ-refractory suicidality
 Augmentation with an AD29132.210.822
 Augmentation with a MS29131.970.732
 Augmentation with ECT29131.830.892
CLZ-refractory aggression
 Combination with AP26143.150.833
 Augmentation with MS26243.080.853
 Augmentation with ECT26142.51.112.5
 Augmentation with AD26121.270.451
Treatment OptionNMinMaxMean ScoreSDMedian
CLZ-refractory positive symptoms
 Raise CLZ plasma levels ≥350 ng/ml38154.581.135
 Combination with second AP36142.921.033
 Wait for delayed response26152.621.273
 Augmentation with non-AP psychotropic26152.621.273
 Augmentation with ECT32152.381.12
CLZ-refractory negative symptoms
 Raise CLZ plasma levels ≥350 ng/ml32154.341.265
 Augmentation with non-AP psychotropic medication34153.150.933
 Combination with second AP23153.131.143
 Wait for delayed CLZ response301531.443
 Augmentation with CBT/psychosocial interventions36152.751.252.5
CLZ-refractory mixed symptoms
 Raise CLZ plasma levels ≥350 ng/ml37154.431.285
 Combination with second AP34142.850.993
 Wait for delayed response23152.871.423
 Augmentation with non-AP psychotropic medication26142.7313
 Augmentation with CBT/psychosocial interventions28152.751.352.5
Absent improvement/worsening under CLZ
 Raise CLZ plasma levels ≥350 ng/ml34154.530.965
 Combination with second AP26153.121.113
 Wait for delayed CLZ response221531.383
 Switch to different AP19142.891.213
 Augmentation with non-AP psychotropic medication18152.670.843
CLZ-refractory suicidality
 Augmentation with an AD29132.210.822
 Augmentation with a MS29131.970.732
 Augmentation with ECT29131.830.892
CLZ-refractory aggression
 Combination with AP26143.150.833
 Augmentation with MS26243.080.853
 Augmentation with ECT26142.51.112.5
 Augmentation with AD26121.270.451

Note: These results are derived from the first round of the survey. Participants were asked for the hierarchy of treatment options in case of clozapine-refractory symptom domains. They were asked to select the 5 most effective options from the list and rank from 1 to 5 starting with the option estimated to be most effective. For analysis, ranks were transformed (supplementary methods). Hierarchy of top 5 treatment options according to the specific symptom domain and starting with the treatment option with the highest median and then, if medians were the same between options, the highest mean; For clozapine-refractory suicidality and aggression, only participants who were at least neutral (rank ≥ 3) toward 2 of the offered combination/augmentation options for the respective symptom domain could rank responses to questions regarding clozapine-refractory suicidality and aggression. For suicidal symptoms, single substances could be rated if the participant agreed or strongly agreed to (rank ≥ 4) clozapine combination and/or augmentation for suicidal ideation symptoms. If the participant favored (rank ≥ 4) combination and/or augmentation for clozapine-refractory aggression, preferred single substance(s) could be named as free text. N, number of participants; CLZ, clozapine; AP, antipsychotic; AD, antidepressant; CBT, cognitive-behavioral therapy; ECT, electroconvulsive therapy.

Table 2.

Hierarchy of Top 5 Treatment Options According to the Specific Symptom Domain or Clinical Scenario (Positive, Negative, Mixed Symptoms, and Absent/Improvement or Initial Worsening) and Hierarchy of All Treatment Options for Suicidality and Aggression

Treatment OptionNMinMaxMean ScoreSDMedian
CLZ-refractory positive symptoms
 Raise CLZ plasma levels ≥350 ng/ml38154.581.135
 Combination with second AP36142.921.033
 Wait for delayed response26152.621.273
 Augmentation with non-AP psychotropic26152.621.273
 Augmentation with ECT32152.381.12
CLZ-refractory negative symptoms
 Raise CLZ plasma levels ≥350 ng/ml32154.341.265
 Augmentation with non-AP psychotropic medication34153.150.933
 Combination with second AP23153.131.143
 Wait for delayed CLZ response301531.443
 Augmentation with CBT/psychosocial interventions36152.751.252.5
CLZ-refractory mixed symptoms
 Raise CLZ plasma levels ≥350 ng/ml37154.431.285
 Combination with second AP34142.850.993
 Wait for delayed response23152.871.423
 Augmentation with non-AP psychotropic medication26142.7313
 Augmentation with CBT/psychosocial interventions28152.751.352.5
Absent improvement/worsening under CLZ
 Raise CLZ plasma levels ≥350 ng/ml34154.530.965
 Combination with second AP26153.121.113
 Wait for delayed CLZ response221531.383
 Switch to different AP19142.891.213
 Augmentation with non-AP psychotropic medication18152.670.843
CLZ-refractory suicidality
 Augmentation with an AD29132.210.822
 Augmentation with a MS29131.970.732
 Augmentation with ECT29131.830.892
CLZ-refractory aggression
 Combination with AP26143.150.833
 Augmentation with MS26243.080.853
 Augmentation with ECT26142.51.112.5
 Augmentation with AD26121.270.451
Treatment OptionNMinMaxMean ScoreSDMedian
CLZ-refractory positive symptoms
 Raise CLZ plasma levels ≥350 ng/ml38154.581.135
 Combination with second AP36142.921.033
 Wait for delayed response26152.621.273
 Augmentation with non-AP psychotropic26152.621.273
 Augmentation with ECT32152.381.12
CLZ-refractory negative symptoms
 Raise CLZ plasma levels ≥350 ng/ml32154.341.265
 Augmentation with non-AP psychotropic medication34153.150.933
 Combination with second AP23153.131.143
 Wait for delayed CLZ response301531.443
 Augmentation with CBT/psychosocial interventions36152.751.252.5
CLZ-refractory mixed symptoms
 Raise CLZ plasma levels ≥350 ng/ml37154.431.285
 Combination with second AP34142.850.993
 Wait for delayed response23152.871.423
 Augmentation with non-AP psychotropic medication26142.7313
 Augmentation with CBT/psychosocial interventions28152.751.352.5
Absent improvement/worsening under CLZ
 Raise CLZ plasma levels ≥350 ng/ml34154.530.965
 Combination with second AP26153.121.113
 Wait for delayed CLZ response221531.383
 Switch to different AP19142.891.213
 Augmentation with non-AP psychotropic medication18152.670.843
CLZ-refractory suicidality
 Augmentation with an AD29132.210.822
 Augmentation with a MS29131.970.732
 Augmentation with ECT29131.830.892
CLZ-refractory aggression
 Combination with AP26143.150.833
 Augmentation with MS26243.080.853
 Augmentation with ECT26142.51.112.5
 Augmentation with AD26121.270.451

Note: These results are derived from the first round of the survey. Participants were asked for the hierarchy of treatment options in case of clozapine-refractory symptom domains. They were asked to select the 5 most effective options from the list and rank from 1 to 5 starting with the option estimated to be most effective. For analysis, ranks were transformed (supplementary methods). Hierarchy of top 5 treatment options according to the specific symptom domain and starting with the treatment option with the highest median and then, if medians were the same between options, the highest mean; For clozapine-refractory suicidality and aggression, only participants who were at least neutral (rank ≥ 3) toward 2 of the offered combination/augmentation options for the respective symptom domain could rank responses to questions regarding clozapine-refractory suicidality and aggression. For suicidal symptoms, single substances could be rated if the participant agreed or strongly agreed to (rank ≥ 4) clozapine combination and/or augmentation for suicidal ideation symptoms. If the participant favored (rank ≥ 4) combination and/or augmentation for clozapine-refractory aggression, preferred single substance(s) could be named as free text. N, number of participants; CLZ, clozapine; AP, antipsychotic; AD, antidepressant; CBT, cognitive-behavioral therapy; ECT, electroconvulsive therapy.

Table 3.

Evaluation of Treatment Strategies According to Positive, Negative, Mixed Positive and Negative, Cognitive, Suicidal Ideation and Aggressive Symptoms

Symptom DomainNStrongly Disagree (%)Disagree (%)Neutral (%)Agree (%)Strongly Agree (%)Sum Disagree + Strongly Disagree (%)Sum Agree + Strongly Agree (%)
CLZ-refractory positive symptoms
 Combine with another AP 444.513.62.338.640.918.179.5
 Augment with an AD (other than fluvoxamine)4151.229.37.312.2080.512.2
 Augment with fluvoxamine 4022.540.017.517.52.562.520.0
 Augment with a MS 39015.423.148.712.815.461.5
 Augment with a drug from a different class 3920.546.220.57.75.166.712.8
 Augment with ECT385.32.6044.747.47.992.1
 ECT maintenance in case of symptomatic improvement 382.62.623.763.27.95.271.1
 Augment with rTMSa306.736.746.610043.410
 Augment with CBT372.78.18.162.218.910.881.1
 Augment with psychosocial interventions 372.702.743.251.42.794.6
CLZ-refractory negative symptoms
 Combine with another AP 4413.631.822.722.79.145.431.8
Augment with an AD (other than fluvoxamine) 402.52.517.557.520.05.077.5
Augment with fluvoxamine 3920.538.520.520.5059.020.5
Augment with a MS 395.112.848.728.25.117.933.3
Augment with a drug from a different class 3923.135.923.112.85.159.017.9
Augment with ECT 3716.240.521.616.25.456.821.6
ECT maintenance in case of symptomatic improvement 336.13.024.263.63.09.140.8
Augment with rTMS 3315.245.530.39.1060.79.1
Augment with CBT 375.42.716.254.121.68.175.7
Augment with psychosocial interventions 372.72.72.737.854.15.491.9
CLZ-refractory mixed symptoms
 Combine with another AP444.59.14.556.825.013.681.8
 Augment with an AD (other than fluvoxamine)a339.127.345.515.2336.418.2
 Augment with fluvoxamine 4017.545.012.520.05.062.525.0
 Augment with a MS 39012.823.148.715.412.864.1
 Augment with a drug from a different class 3920.546.217.910.35.166.715.4
 Augment with ECT 385.32.62.652.636.87.989.4
 ECT maintenance in case of symptomatic improvement 382.67.928.952.67.910.560.5
 Augment with rTMSa301036.743.310046.710
 Augment with CBT 372.75.410.864.916.28.181.1
 Augment with psychosocial interventions 372.702.743.251.42.794.6
CLZ-refractory cognitive deficits
 Augment with a procognitive drug 387.934.234.221.12.642.123.7
CLZ-refractory suicidal ideation
 Augment with a MS 38005.371.123.7094.8
 Augment with an AD 3805.3050.044.75.394.7
 Augment with ECT 382.62.65.347.442.15.289.5
CLZ-refractory aggression
 Combine with another AP 382.62.610.544.739.55.284.2
 Augment with a MS382.62.67.942.144.75.286.8
 Augment with an AD 3826.347.418.47.9073.77.9
 Augment with ECT 372.7024.356.816.22.773
Symptom DomainNStrongly Disagree (%)Disagree (%)Neutral (%)Agree (%)Strongly Agree (%)Sum Disagree + Strongly Disagree (%)Sum Agree + Strongly Agree (%)
CLZ-refractory positive symptoms
 Combine with another AP 444.513.62.338.640.918.179.5
 Augment with an AD (other than fluvoxamine)4151.229.37.312.2080.512.2
 Augment with fluvoxamine 4022.540.017.517.52.562.520.0
 Augment with a MS 39015.423.148.712.815.461.5
 Augment with a drug from a different class 3920.546.220.57.75.166.712.8
 Augment with ECT385.32.6044.747.47.992.1
 ECT maintenance in case of symptomatic improvement 382.62.623.763.27.95.271.1
 Augment with rTMSa306.736.746.610043.410
 Augment with CBT372.78.18.162.218.910.881.1
 Augment with psychosocial interventions 372.702.743.251.42.794.6
CLZ-refractory negative symptoms
 Combine with another AP 4413.631.822.722.79.145.431.8
Augment with an AD (other than fluvoxamine) 402.52.517.557.520.05.077.5
Augment with fluvoxamine 3920.538.520.520.5059.020.5
Augment with a MS 395.112.848.728.25.117.933.3
Augment with a drug from a different class 3923.135.923.112.85.159.017.9
Augment with ECT 3716.240.521.616.25.456.821.6
ECT maintenance in case of symptomatic improvement 336.13.024.263.63.09.140.8
Augment with rTMS 3315.245.530.39.1060.79.1
Augment with CBT 375.42.716.254.121.68.175.7
Augment with psychosocial interventions 372.72.72.737.854.15.491.9
CLZ-refractory mixed symptoms
 Combine with another AP444.59.14.556.825.013.681.8
 Augment with an AD (other than fluvoxamine)a339.127.345.515.2336.418.2
 Augment with fluvoxamine 4017.545.012.520.05.062.525.0
 Augment with a MS 39012.823.148.715.412.864.1
 Augment with a drug from a different class 3920.546.217.910.35.166.715.4
 Augment with ECT 385.32.62.652.636.87.989.4
 ECT maintenance in case of symptomatic improvement 382.67.928.952.67.910.560.5
 Augment with rTMSa301036.743.310046.710
 Augment with CBT 372.75.410.864.916.28.181.1
 Augment with psychosocial interventions 372.702.743.251.42.794.6
CLZ-refractory cognitive deficits
 Augment with a procognitive drug 387.934.234.221.12.642.123.7
CLZ-refractory suicidal ideation
 Augment with a MS 38005.371.123.7094.8
 Augment with an AD 3805.3050.044.75.394.7
 Augment with ECT 382.62.65.347.442.15.289.5
CLZ-refractory aggression
 Combine with another AP 382.62.610.544.739.55.284.2
 Augment with a MS382.62.67.942.144.75.286.8
 Augment with an AD 3826.347.418.47.9073.77.9
 Augment with ECT 372.7024.356.816.22.773

Note: In the first round, participants were asked to evaluate treatment options according to CLZ-refractory symptom domains on a Likert-scale from 1 to 5. In the second round, participants were asked again to rate treatment strategies meeting the threshold of >50% agreement or disagreement in the first round. These treatment strategies and their respective ratings from the second round are displayed in the table. Agreement (positive evaluation, sum of “agree” and “strongly agree” at least 75%) and disagreement (negative evaluation, sum of “disagree” and “strongly disagree” are highlighted in bold. AD, antidepressant; AP, antipsychotic; CBT, cognitive-behavioral therapy; CLZ, clozapine; ECT, electroconvulsive therapy; MS, mood-stabilizer; rTMS, repetitive transcranial magnetic stimulation.

aTreatment strategy not meeting the threshold of >50% agreement or disagreement in the first round (cf. SA-table 24) and was not assessed in the second round of the consensus process, thus results from the first round are displayed here.

Table 3.

Evaluation of Treatment Strategies According to Positive, Negative, Mixed Positive and Negative, Cognitive, Suicidal Ideation and Aggressive Symptoms

Symptom DomainNStrongly Disagree (%)Disagree (%)Neutral (%)Agree (%)Strongly Agree (%)Sum Disagree + Strongly Disagree (%)Sum Agree + Strongly Agree (%)
CLZ-refractory positive symptoms
 Combine with another AP 444.513.62.338.640.918.179.5
 Augment with an AD (other than fluvoxamine)4151.229.37.312.2080.512.2
 Augment with fluvoxamine 4022.540.017.517.52.562.520.0
 Augment with a MS 39015.423.148.712.815.461.5
 Augment with a drug from a different class 3920.546.220.57.75.166.712.8
 Augment with ECT385.32.6044.747.47.992.1
 ECT maintenance in case of symptomatic improvement 382.62.623.763.27.95.271.1
 Augment with rTMSa306.736.746.610043.410
 Augment with CBT372.78.18.162.218.910.881.1
 Augment with psychosocial interventions 372.702.743.251.42.794.6
CLZ-refractory negative symptoms
 Combine with another AP 4413.631.822.722.79.145.431.8
Augment with an AD (other than fluvoxamine) 402.52.517.557.520.05.077.5
Augment with fluvoxamine 3920.538.520.520.5059.020.5
Augment with a MS 395.112.848.728.25.117.933.3
Augment with a drug from a different class 3923.135.923.112.85.159.017.9
Augment with ECT 3716.240.521.616.25.456.821.6
ECT maintenance in case of symptomatic improvement 336.13.024.263.63.09.140.8
Augment with rTMS 3315.245.530.39.1060.79.1
Augment with CBT 375.42.716.254.121.68.175.7
Augment with psychosocial interventions 372.72.72.737.854.15.491.9
CLZ-refractory mixed symptoms
 Combine with another AP444.59.14.556.825.013.681.8
 Augment with an AD (other than fluvoxamine)a339.127.345.515.2336.418.2
 Augment with fluvoxamine 4017.545.012.520.05.062.525.0
 Augment with a MS 39012.823.148.715.412.864.1
 Augment with a drug from a different class 3920.546.217.910.35.166.715.4
 Augment with ECT 385.32.62.652.636.87.989.4
 ECT maintenance in case of symptomatic improvement 382.67.928.952.67.910.560.5
 Augment with rTMSa301036.743.310046.710
 Augment with CBT 372.75.410.864.916.28.181.1
 Augment with psychosocial interventions 372.702.743.251.42.794.6
CLZ-refractory cognitive deficits
 Augment with a procognitive drug 387.934.234.221.12.642.123.7
CLZ-refractory suicidal ideation
 Augment with a MS 38005.371.123.7094.8
 Augment with an AD 3805.3050.044.75.394.7
 Augment with ECT 382.62.65.347.442.15.289.5
CLZ-refractory aggression
 Combine with another AP 382.62.610.544.739.55.284.2
 Augment with a MS382.62.67.942.144.75.286.8
 Augment with an AD 3826.347.418.47.9073.77.9
 Augment with ECT 372.7024.356.816.22.773
Symptom DomainNStrongly Disagree (%)Disagree (%)Neutral (%)Agree (%)Strongly Agree (%)Sum Disagree + Strongly Disagree (%)Sum Agree + Strongly Agree (%)
CLZ-refractory positive symptoms
 Combine with another AP 444.513.62.338.640.918.179.5
 Augment with an AD (other than fluvoxamine)4151.229.37.312.2080.512.2
 Augment with fluvoxamine 4022.540.017.517.52.562.520.0
 Augment with a MS 39015.423.148.712.815.461.5
 Augment with a drug from a different class 3920.546.220.57.75.166.712.8
 Augment with ECT385.32.6044.747.47.992.1
 ECT maintenance in case of symptomatic improvement 382.62.623.763.27.95.271.1
 Augment with rTMSa306.736.746.610043.410
 Augment with CBT372.78.18.162.218.910.881.1
 Augment with psychosocial interventions 372.702.743.251.42.794.6
CLZ-refractory negative symptoms
 Combine with another AP 4413.631.822.722.79.145.431.8
Augment with an AD (other than fluvoxamine) 402.52.517.557.520.05.077.5
Augment with fluvoxamine 3920.538.520.520.5059.020.5
Augment with a MS 395.112.848.728.25.117.933.3
Augment with a drug from a different class 3923.135.923.112.85.159.017.9
Augment with ECT 3716.240.521.616.25.456.821.6
ECT maintenance in case of symptomatic improvement 336.13.024.263.63.09.140.8
Augment with rTMS 3315.245.530.39.1060.79.1
Augment with CBT 375.42.716.254.121.68.175.7
Augment with psychosocial interventions 372.72.72.737.854.15.491.9
CLZ-refractory mixed symptoms
 Combine with another AP444.59.14.556.825.013.681.8
 Augment with an AD (other than fluvoxamine)a339.127.345.515.2336.418.2
 Augment with fluvoxamine 4017.545.012.520.05.062.525.0
 Augment with a MS 39012.823.148.715.412.864.1
 Augment with a drug from a different class 3920.546.217.910.35.166.715.4
 Augment with ECT 385.32.62.652.636.87.989.4
 ECT maintenance in case of symptomatic improvement 382.67.928.952.67.910.560.5
 Augment with rTMSa301036.743.310046.710
 Augment with CBT 372.75.410.864.916.28.181.1
 Augment with psychosocial interventions 372.702.743.251.42.794.6
CLZ-refractory cognitive deficits
 Augment with a procognitive drug 387.934.234.221.12.642.123.7
CLZ-refractory suicidal ideation
 Augment with a MS 38005.371.123.7094.8
 Augment with an AD 3805.3050.044.75.394.7
 Augment with ECT 382.62.65.347.442.15.289.5
CLZ-refractory aggression
 Combine with another AP 382.62.610.544.739.55.284.2
 Augment with a MS382.62.67.942.144.75.286.8
 Augment with an AD 3826.347.418.47.9073.77.9
 Augment with ECT 372.7024.356.816.22.773

Note: In the first round, participants were asked to evaluate treatment options according to CLZ-refractory symptom domains on a Likert-scale from 1 to 5. In the second round, participants were asked again to rate treatment strategies meeting the threshold of >50% agreement or disagreement in the first round. These treatment strategies and their respective ratings from the second round are displayed in the table. Agreement (positive evaluation, sum of “agree” and “strongly agree” at least 75%) and disagreement (negative evaluation, sum of “disagree” and “strongly disagree” are highlighted in bold. AD, antidepressant; AP, antipsychotic; CBT, cognitive-behavioral therapy; CLZ, clozapine; ECT, electroconvulsive therapy; MS, mood-stabilizer; rTMS, repetitive transcranial magnetic stimulation.

aTreatment strategy not meeting the threshold of >50% agreement or disagreement in the first round (cf. SA-table 24) and was not assessed in the second round of the consensus process, thus results from the first round are displayed here.

Consensus was achieved for amisulpride and oral aripiprazole as combination agents with clozapine (78.8% and 76.9% in the second round) (figure 1 and SB-table 3). Notably, augmenting clozapine with an antidepressant was the only option meeting the threshold for disagreement by 80.5% (table 3). Augmentation with other compounds was in general not advised (table 3) as detailed in SA-table 3 and SB-table 3. Furthermore, amisulpride and aripiprazole were the only compounds of any class meeting the positive evaluation by ≥75% threshold (figure 1 and SA-table 3 and SB-table 3 for a complete list with the ratings). Since fluvoxamine affects clozapine plasma levels, augmentation with fluvoxamine was assessed separately. Altering clozapine plasma levels with fluvoxamine augmentation was advised by a minority of 20.5% (table 3). Consensus was reached for clozapine augmentation with ECT (92.1% in the second round) (table 3). In the case of symptomatic improvement following an acute course of ECT, 71.1% of the survey participants suggested ECT maintenance treatment (table 3). The median number of ECT was recommended to be 12 (SA-table 4), and consensus was reached for a frequency of 3 ECT sessions per week (76.5% in the second round) (SB-table 11). Augmentation with rTMS was suggested by 10% of the participants (table 3). Augmenting clozapine treatment with CBT reached consensus (81.2% in the second round) (table 3) and 15.5 (median) sessions were recommended (SA-table 4). Furthermore, consensus was reached (94.6% in the second round) to augment clozapine with psychosocial interventions (table 3). Survey respondents reached consensus to leave the clozapine dose the same as during clozapine monotherapy (SB-tables 2, 6–9, 12–14) if any combination strategy was chosen.

Evaluation of single substances for persistent positive, negative, and mixed symptoms. Participants were asked to rate single substances on a list for CLZ-refractory positive, negative and mixed symptoms on a Likert-scale from 1 to 5. Dots represent the observed proportion of agreement (sum of “agree” and “strongly agree”). Error bars represent 95% CI derived from the observed proportion (%) and the individual sample size. Single substances marked with * were not asked in the second round of the consensus process, since agreement or disagreement (“disagree” and “strongly disagree”) in the first round was not >50%. Further substances that were named as free text by the experts are listed in the legend of SA-table 3. N, number of participants; CLZ, clozapine.
Fig. 1.

Evaluation of single substances for persistent positive, negative, and mixed symptoms. Participants were asked to rate single substances on a list for CLZ-refractory positive, negative and mixed symptoms on a Likert-scale from 1 to 5. Dots represent the observed proportion of agreement (sum of “agree” and “strongly agree”). Error bars represent 95% CI derived from the observed proportion (%) and the individual sample size. Single substances marked with * were not asked in the second round of the consensus process, since agreement or disagreement (“disagree” and “strongly disagree”) in the first round was not >50%. Further substances that were named as free text by the experts are listed in the legend of SA-table 3. N, number of participants; CLZ, clozapine.

Negative Symptoms

For the management of clozapine-refractory negative symptoms, the treatment option to raise clozapine plasma levels350 ng/ml obtained the highest ranking (median = 5). Further highly ranked treatment options were waiting for a delayed response or clozapine augmentation with a nonantipsychotic psychotropic agent, or combination with a second antipsychotic (median 3 each) (table 2 and SA-table 12). Nearly a third of the participants (31.8%) advised combining clozapine with a different antipsychotic (table 3, for single substances see figure 1, SB-table 5). Consensus was reached to augment clozapine with an antidepressant (77.5% in the second round) (table 3). Escitalopram/citalopram was most commonly suggested as the antidepressant agent (positive evaluation of 67.5%) (figure 1, SB-table 5). Altogether, 33.3% of the participants favored augmenting clozapine with a mood-stabilizer (table 3). Augmentation with a different psychotropic drug was suggested by 17.9% (table 3, figure 1). The add-on use of fluvoxamine was suggested by 20.5% (table 3). Augmentation with ECT was advised by 21.6% (table 3). In the case of symptomatic improvement, 40.8% of the surveyed experts suggested ECT maintenance treatment (table 3). The median number of ECT sessions was defined as 12 (SA-table 4) and the majority of participants favored a frequency of 3 ECT sessions per week (56.5%) (SB-table 11). Augmentation with rTMS was advised by 9.1% of the surveyed experts (table 3). Consensus was reached to augment clozapine with CBT (75.7% in the second round) (table 3) and 17.5 (median) sessions were recommended (SA-table 4). Similar as for positive symptoms, 91.8% (second round) of the participants reached consensus to augment clozapine with psychosocial interventions (table 3). Consensus was reached to leave the clozapine dose the same when deciding to add combination or augmentation strategies (SB-tables 2, 6–9, 12–14).

Mixed (Both Positive and Negative) Symptoms

For mixed symptoms, results are comparable to those for positive symptoms (tables 23 and supplement).

Cognitive Symptoms

For the management of clozapine-refractory cognitive symptoms, the use of so-called procognitive drugs was suggested only by 23.7% (table 3, details in SB-table 10).

Persistent Suicidal Ideation Symptoms

For the management of clozapine-refractory suicidal ideation symptoms, the treatment option to augment clozapine with an antidepressant obtained the highest ranking (median = 2) (table 2). For suicidal ideation symptoms, consensus was reached for all offered augmentation strategies (mood-stabilizer, antidepressants, and ECT) (table 3). Citalopram/escitalopram and fluoxetine achieved consensus as recommended antidepressants (84.2% and 75.7% in the second round, respectively). Consensus was reached for lithium and lamotrigine as mood-stabilizing agents (92.1% and 79% in the second round, respectively). No other substances met our cutoff criteria (SB-table 15).

Persistent Aggression

For the management of clozapine-refractory aggressive symptoms, the treatment option to combine clozapine with an antipsychotic and with a mood-stabilizer obtained the highest ranking (median = 3 each) (table 2). For clozapine-refractory aggressive symptoms, consensus was achieved to augment clozapine with a mood-stabilizer (86.8% in the second round) or with an antipsychotic (84.2% in the second round). A majority suggested augmentation with ECT (73%), and a minority favored augmentation with an antidepressant (7.9%) (table 3).

Absent Improvement/Worsening After Clozapine Monotherapy

For the management of absent improvement in positive, negative, and mixed symptoms and functioning or even worsening after clozapine monotherapy, the treatment option to raise clozapine plasma levels350 ng/ml was ranked first (median = 5) (table 2).

Discontinuation of Clozapine

Overall, for clozapine-refractory positive symptoms, consensus was reached (76.6% in the second round) on disagreement with the strategy of tapering off and discontinuing clozapine and using alternative pharmacological, neurostimulation (including but not restricted to ECT) or complementary treatment options. Results were similar for clozapine-refractory negative symptoms (SB-table 1).

TRRIP Consensus Recommendations for the Management of CRS

Consensus recommendations were developed from the aforementioned results achieving ≥75% agreement (“agree” or “strongly agree”) or achieving ≥75% disagreement in the second round. Recommendations were developed from the aforementioned results achieving ≥75% agreement in the first round and questions could not re-presented again in the second round due to the question structure (eg, hierarchy questions) (table 4).

Table 4.

TRRIP Consensus Recommendations for the Management of Clozapine-Resistant Schizophrenia (CRS)

Consensus Recommendations from the TRRIP Working Group
1. General recommendations
For clozapine-refractory positive, negative, or mixed symptoms, raise clozapine levels ≥350 ng/ml (recommendation ≥ 75%; 1st round)a
For clozapine-refractory positive, negative, or mixed symptoms, augment clozapine with CBT and psychosocial interventions
For all combination or augmentation strategies, leave the clozapine dose the same as during clozapine monotherapy
Not discontinue clozapine and not switch to alternative pharmacological, neurostimulation (including but not restricted to ECT) or complementary treatment options for clozapine-refractory positive or negative symptoms
2. Clozapine-refractory positive symptoms
Combine clozapine with another antipsychotic, namely amisulpride or oral aripiprazole
Augment clozapine with ECT
3. Clozapine-refractory negative symptoms
Augment clozapine with an antidepressant
4. Clozapine-refractory mixed symptoms
Combine clozapine with another antipsychotic, namely oral aripiprazole or amisulpride
Augment clozapine with ECT
5. Clozapine-refractory suicidal ideation symptoms
Augment clozapine with a mood-stabilizer (namely lithium, lamotrigine) or antidepressants (namely citalopram/escitalopram or fluoxetine) or ECT
6. Clozapine-refractory aggressionb
Augment clozapine with a mood-stabiliser or combine clozapine with an antipsychotic
Consensus Recommendations from the TRRIP Working Group
1. General recommendations
For clozapine-refractory positive, negative, or mixed symptoms, raise clozapine levels ≥350 ng/ml (recommendation ≥ 75%; 1st round)a
For clozapine-refractory positive, negative, or mixed symptoms, augment clozapine with CBT and psychosocial interventions
For all combination or augmentation strategies, leave the clozapine dose the same as during clozapine monotherapy
Not discontinue clozapine and not switch to alternative pharmacological, neurostimulation (including but not restricted to ECT) or complementary treatment options for clozapine-refractory positive or negative symptoms
2. Clozapine-refractory positive symptoms
Combine clozapine with another antipsychotic, namely amisulpride or oral aripiprazole
Augment clozapine with ECT
3. Clozapine-refractory negative symptoms
Augment clozapine with an antidepressant
4. Clozapine-refractory mixed symptoms
Combine clozapine with another antipsychotic, namely oral aripiprazole or amisulpride
Augment clozapine with ECT
5. Clozapine-refractory suicidal ideation symptoms
Augment clozapine with a mood-stabilizer (namely lithium, lamotrigine) or antidepressants (namely citalopram/escitalopram or fluoxetine) or ECT
6. Clozapine-refractory aggressionb
Augment clozapine with a mood-stabiliser or combine clozapine with an antipsychotic

Note: CBT, cognitive-behavioral therapy; ECT, electroconvulsive therapy.

aRecommendation derived from first-round findings.

bSubstances were not systematically specified in the survey, see legend SA-table 9 for free-text suggestions of the experts; treatment options were all consented, but for the specific domains positive, negative, mixed, suicidal, and aggressive symptoms, a hierarchy can only be defined as presented in SA-tables 10–17, since hierarchy itself was not consented in a 2-step Delphi-process due to the question structure (ranking of options).

Table 4.

TRRIP Consensus Recommendations for the Management of Clozapine-Resistant Schizophrenia (CRS)

Consensus Recommendations from the TRRIP Working Group
1. General recommendations
For clozapine-refractory positive, negative, or mixed symptoms, raise clozapine levels ≥350 ng/ml (recommendation ≥ 75%; 1st round)a
For clozapine-refractory positive, negative, or mixed symptoms, augment clozapine with CBT and psychosocial interventions
For all combination or augmentation strategies, leave the clozapine dose the same as during clozapine monotherapy
Not discontinue clozapine and not switch to alternative pharmacological, neurostimulation (including but not restricted to ECT) or complementary treatment options for clozapine-refractory positive or negative symptoms
2. Clozapine-refractory positive symptoms
Combine clozapine with another antipsychotic, namely amisulpride or oral aripiprazole
Augment clozapine with ECT
3. Clozapine-refractory negative symptoms
Augment clozapine with an antidepressant
4. Clozapine-refractory mixed symptoms
Combine clozapine with another antipsychotic, namely oral aripiprazole or amisulpride
Augment clozapine with ECT
5. Clozapine-refractory suicidal ideation symptoms
Augment clozapine with a mood-stabilizer (namely lithium, lamotrigine) or antidepressants (namely citalopram/escitalopram or fluoxetine) or ECT
6. Clozapine-refractory aggressionb
Augment clozapine with a mood-stabiliser or combine clozapine with an antipsychotic
Consensus Recommendations from the TRRIP Working Group
1. General recommendations
For clozapine-refractory positive, negative, or mixed symptoms, raise clozapine levels ≥350 ng/ml (recommendation ≥ 75%; 1st round)a
For clozapine-refractory positive, negative, or mixed symptoms, augment clozapine with CBT and psychosocial interventions
For all combination or augmentation strategies, leave the clozapine dose the same as during clozapine monotherapy
Not discontinue clozapine and not switch to alternative pharmacological, neurostimulation (including but not restricted to ECT) or complementary treatment options for clozapine-refractory positive or negative symptoms
2. Clozapine-refractory positive symptoms
Combine clozapine with another antipsychotic, namely amisulpride or oral aripiprazole
Augment clozapine with ECT
3. Clozapine-refractory negative symptoms
Augment clozapine with an antidepressant
4. Clozapine-refractory mixed symptoms
Combine clozapine with another antipsychotic, namely oral aripiprazole or amisulpride
Augment clozapine with ECT
5. Clozapine-refractory suicidal ideation symptoms
Augment clozapine with a mood-stabilizer (namely lithium, lamotrigine) or antidepressants (namely citalopram/escitalopram or fluoxetine) or ECT
6. Clozapine-refractory aggressionb
Augment clozapine with a mood-stabiliser or combine clozapine with an antipsychotic

Note: CBT, cognitive-behavioral therapy; ECT, electroconvulsive therapy.

aRecommendation derived from first-round findings.

bSubstances were not systematically specified in the survey, see legend SA-table 9 for free-text suggestions of the experts; treatment options were all consented, but for the specific domains positive, negative, mixed, suicidal, and aggressive symptoms, a hierarchy can only be defined as presented in SA-tables 10–17, since hierarchy itself was not consented in a 2-step Delphi-process due to the question structure (ranking of options).

Discussion

General Findings

Our aim was to establish the first international survey and expert-based consensus process for the management of CRS. According to TRRIP guidelines, achieving plasma clozapine levels of ≥350 ng/ml constitutes a minimum threshold requirement for establishing clozapine nonresponse.12 There was consensus that clozapine-refractory symptoms in either of the 3 domains, should prompt dosage increase to raise the plasma clozapine levels to ≥350 ng/ml before using any pharmacological combination or augmentation strategies. In our previous TRRIP publication, we recommended that clozapine monotherapy should be prescribed for at least ≥3 months after therapeutic plasma levels have been attained12; however, we did not address the possibility of different durations for specific symptom domains. Because there is evidence that clozapine has anti-suicidal29 and anti-aggressive effects,30 these domains were included, too. We now consider that the duration of an optimal trial to establish CRS should be longer for negative symptoms (16 weeks) and cognitive symptoms (16 weeks) than for positive and mixed symptoms (both 12 weeks). Regarding the concept of delayed response to clozapine monotherapy, the experts estimated a median of 12 weeks of (acute) treatment to be sufficient for positive, negative and mixed symptoms alike, which is consistent with the previous TRRIP publication.12 The judgment that the optimal duration for a clozapine trial for negative, suicidal, and aggressive symptoms should be longer than the time to wait for a delayed clozapine response for these symptoms may seem odd. However, this may reflect the view that for suicidality and aggression longer inadequate trials in terms of dosage or plasma levels would be taking an unacceptable risk of harm to self or others.

Hierarchy of Treatment Options and Recommendations

For clozapine-refractory positive symptoms, the next therapeutic intervention—after raising plasma clozapine levels—was the combination with a second antipsychotic, namely amisulpride or oral aripiprazole. In a recent meta-analysis, adding sulpiride/amisulpride showed no beneficial effects on overall symptoms, and sulpiride showed no effects on positive symptoms.14 Similarly, oral aripiprazole was significantly superior to placebo with regard to overall, but not positive symptoms.14 Thus, the empirical evidence supporting the combination of clozapine with amisulpride and oral aripiprazole for positive symptoms remains sparse, although both combinations reached the consensus threshold. Indirectly, the potential benefit of oral aripiprazole for positive symptoms align with results from a recent naturalistic study showing that the combination of clozapine and aripiprazole was associated with better real-world outcomes compared to clozapine monotherapy.31

For clozapine-refractory negative symptoms, the next therapeutic intervention, after raising clozapine plasma levels, was augmentation with a nonantipsychotic psychotropic medication, with escitalopram/citalopram as the preferred agent. This agent has not yet been specifically investigated for clozapine-refractory negative symptoms. Nonetheless, it was suggested that adjunctive antidepressants may have small, beneficial effects on negative symptoms.32,33 For clozapine-refractory cognitive deficits, augmentation of clozapine with a precognitive drugs or alternative compounds was not recommended, consistent with meta-analytic evidence,13 even though there is some evidence for raloxifene addition, especially in women.34

The majority of experts recommended ECT for clozapine-refractory positive symptoms and suggested that subsequent ECT maintenance treatment should be offered for positive symptoms, whereas there was a low rate of approval for ECT for clozapine-refractory negative symptoms. In accordance with meta-analytic data, ECT augmentation can be considered to be effective in open-label studies,17,18 even though one small sham-controlled pilot study was negative.35 However, more high-quality trials for clozapine augmentation with ECT are needed to further strengthen the evidence for efficacy and safety of this treatment option.

The application of adjunctive fluvoxamine as CYP1A2-inhibitor altering plasma clozapine levels and/or clozapine:norclozapine ratio was generally not recommended for positive or negative symptoms. A systematic review found evidence graded as level A for the effects of fluvoxamine to increase plasma clozapine levels, but the effects of fluvoxamine on clozapine-refractory positive symptoms were only assessed in 2 small studies.36 For clozapine-refractory negative symptoms, no reliable data are available, since the one small relevant study did not report plasma clozapine levels or distinguish between primary and secondary negative symptoms.36

Only one large-scale RCT has examined CBT specifically for CRS.19 Our expert recommendation to offer CBT for different clozapine-refractory symptom domains is consistent with this trial showing that despite average symptom improvements being small and not seen at 1-year follow-up, some patients might benefit from CBT.19 Nonetheless, this trial’s overall negative findings are consistent with our experts suggesting that CBT should not be offered prior to pharmacological augmentation strategies. Of note, more CBT sessions were recommended for negative than for positive symptoms, whereas the median number of sessions for both types of symptoms were within the range recommended by NICE guidelines.8

The negative recommendation for rTMS for clozapine augmentation in our survey is consistent with results from meta-analyses where no superiority was found for rTMS augmentation in CRS.14 Most studies assessing rTMS for auditory hallucinations included CRS patients and generally found small effect sizes for auditory hallucinations, with insignificant effects on delusions.37

According to the experts, initial worsening or a lack of evident improvement after an adequate clozapine trial should not lead to discontinuation of clozapine, as worsening may be observed in some patients.38 Instead, the experts suggested raising plasma clozapine plasma to ≥350 ng/ml and, if symptoms persist, combining clozapine with a second antipsychotic. However, empirical evidence for this scenario or guidelines beyond clozapine are lacking.

For clozapine-refractory suicidality, the experts suggested that augmentation with an antidepressant (recommending escitalopram/citalopram and fluoxetine) or with a mood-stabilizer (recommending lamotrigine and lithium) should precede ECT augmentation. For clozapine-refractory aggression, the experts recommended combination with an antipsychotic or with a mood-stabilizer with both options reaching the ≥75% threshold, before escalating treatment with ECT augmentation. Amisulpride and valproate were most commonly named as preferred agents. Nonetheless, available data on the efficacy of valproate augmentation have been disappointing.13

Side-Effects

Agranulocytosis and myocarditis were reported to be the most potentially life-threatening side-effects of clozapine. These results are consistent with the literature.39 NMS was not that frequently reported in our survey but this is consistent with a recent systematic review on NMS on clozapine which identified only 12 cases of NMS on clozapine in the world literature, most of whom were successfully rechallenged with clozapine.40 Differences between our results and prior publications as well as differences between participants may be due to variation in treatment settings, population characteristics, and regulations of clozapine use.41 For clinical practice, it is important that most clozapine-related side-effects should not result in clozapine discontinuation.42 Despite acknowledging that clozapine can have many side-effects, the experts still provided a good overall rating for clozapine’s safety, allowing for speculation that experienced clozapine experts base their decision on a strict risk-benefit evaluation with particular attention to side-effects associated with clozapine augmentation (eg, weight gain, QTc-prolongation).

Limitations

First, experts were selected and may not be representative, and patient/caregiver, payer and policy maker stakeholder were missing from the group, yet, since we focused on specific prescriber action, we felt that those stakeholders would not be appropriate for this particular survey. Second, the quality of evidence for expert opinions is described as very low (recommendation grade D) according to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system.43 Third, clinicians and other users of this consensus statement should be aware that the composition of our group was not pluralistic—eg, patients or their relatives were not involved. Even though the composition of the TRRIP Working Group was similar to the previous publication12 with a majority of experienced clinicians in the field of psychiatry including experts involved in national and international guideline development, the type of consensus reached here should be defined as nonrepresentative. However, in a clinical situation where there is a paucity of individual or aggregated evidence, our approach may help to reduce the clinical uncertainty and foster the conduct of urgently needed clinical trials. Given the limitations of an overall low grade of evidence of our survey and given the fact that some of our pharmacological augmentation recommendations are off-label and call for future RCTs clinicians should carefully, on a case-by-case basis, consider every recommended augmentation strategy, and discontinue any augmentation trial in case of inadequate response of target symptoms, relevant intolerability, or any safety concerns. Finally, the important management of clozapine-related side-effects and complications was not the scope of our work and should be covered in future projects.

Conclusion

The clinical situation of a patient not responding to clozapine is still one of the biggest challenges in psychiatry and more research regarding this topic is urgently needed. Our results provide insights into the complex clinical scenario of CRS, will inform clinicians who are responsible for the management of this difficult situation, provide a new consensus-based source of evidence for guideline developers and will foster future research.

Acknowledgments

We thank all TRRIP members and acknowledge their contribution: Ofer Agid, Lone Baandrup, Thomas R. Barnes, Andrea de Bartolomeis, Nico J. M. van Beveren, Michael L. Birnbaum, Michael A. P. Bloomfield, Rodrigo A. Bressan, Robert W. Buchanan, William T. Carpenter, David J. Castle, Eric Chen, Leslie Citrome, Christoph U. Correll, Zafiris J. Daskalakis, Michael Davidson, Richard J. Drake, Serdar Dursun, Bjørn H. Ebdrup, Helio Elkis, Peter Falkai, Emilio Fernandez-Egea, W. Wolfgang Fleischhacker, Oliver Freudenreich, Ary Gadelha, Fiona Gaughran, Birte Y. Glenthøj, Donald C. Goff, Ariel Graff-Guerrero, Jaime E. C. Hallak, Alkomiet Hasan, William G. Honer, Oliver Howes, John Kane, James Kennedy, Bruce J. Kinon, Stephen M. Lawrie, Jimmy Lee, F. Markus Leweke, James H. MacCabe, Stephen R. Marder, Carolyn B.McNabb, Rob McCutcheon, Herbert Meltzer, Hans-Jürgen Möller, Shinchiro Nakajima, Jimmi Nielsen, Christos Pantelis, Tiago Reis Marques, Gary Remington, Susan L. Rossell, Bruce R. Russell, Tian-Mei Si, Dan Siskind, Cynthia O. Siu, Takefumi Suzuki, Iris E. Sommer, David Taylor, Neil Thomas, Hiroyuki Uchida, Alp Üçok, Daniel Umbricht, James T. R. Walters.

TRRIP working group: Lone Baandrup, Thomas R. E. Barnes, Andrea de Bartolomeis, Nico J. M. van Beveren, Michael L. Birnbaum, Michael A. P. Bloomfield, Robert W. Buchanan, William T. Carpenter, David J. Castle, Leslie Citrome, Zafiris J. Daskalakis, Michael Davidson, Serdar Dursun, Bjørn H. Ebdrup, Emilio Fernandez-Egea, Helio Elkis, Ary Gadelha, Fiona Gaughran, Birte Y. Glenthøj, Ariel Graff-Guerrero, Jaime E. C. Hallak, James Kennedy, Bruce J. Kinon, James H. MacCabe, Stephen R. Marder, Rob McCutcheon, Christos Pantelis, Tiago Reis Marques, Gary Remington, Susan L. Rossell, Cynthia O. Siu, Takefumi Suzuki, Iris E. Sommer, David Taylor, Neil Thomas, Hiroyuki Uchida, Alp Üçok, James T. R. Walters.

Conflicts of Interest

E.W., D.S., T.S.A., and J.L. report no conflicts of interest. W.G.H. is an unpaid member of the Advisory Board of In Silico Biosciences, and a paid consultant to Otsuka/Lundbeck, Newron, Translational Life Sciences, AlphaSights, Guidepoint, and the Canadian Agency for Drugs and Technology in Health. P.F. was honorary speaker for Janssen-Cilag, Astra-Zeneca, Eli Lilly, Bristol Myers-Squibb, Lundbeck, Pfizer, Bayer Vital, SmithKline Beecham, Wyeth, and Essex. During the last 5 years he was a member of the advisory boards of Janssen-Cilag, Astra-Zeneca, Eli Lilly, and Lundbeck. Presently, he is a member of the advisory boards of Richter Pharma, Abbot and Otsuka. J.M.K. has been a consultant for or received honoraria from Alkermes, Dainippon Sumitomo, Eli Lilly, Forum, Allergan, Genentech, H. Lundbeck. Intracellular Therapies, Janssen Pharmaceutica, Johnson and Johnson, LB Pharmaceuticals, Merck, Minerva, Neurocrine, Otsuka, Pierre Fabre, Reviva, Roche, Sunovion, Takeda, and Teva. J.M.K. has received grant support from Otsuka, Lundbeck, and Janssen. J.M.K. has participated in Advisory Boards for Alkermes, Dainippon Sumitomo, Intracellular Therapies, Lundbeck, Neurocrine, Otsuka, Pierre Fabre, Takeda, and Teva. J.M.K. is a Shareholder in Vanguard Research Group and LB Pharmaceuticals, Inc. C.U.C. has been a consultant and/or advisor to or has received honoraria from Alkermes, Allergan, Angelini, Boehringer-Ingelheim, Gedeon Richter, Gerson Lehrman Group, Indivior, IntraCellular Therapies, Janssen/J&J, LB Pharma, Lundbeck, MedAvante-ProPhase, Medscape, Merck, Neurocrine, Noven, Otsuka, Pfizer, Recordati, Rovi, Servier, Sumitomo Dainippon, Sunovion, Supernus, Takeda, and Teva. He has provided expert testimony for Bristol-Myers Squibb, Janssen, and Otsuka. He served on a Data Safety Monitoring Board for Boehringer-Ingelheim, Lundbeck, Rovi, Supernus, and Teva. He received royalties from UpToDate and grant support from Janssen and Takeda. He is also a shareholder of LB Pharma. O.H. has received investigator-initiated research funding from and/or participated in advisory/speaker meetings organized by Angellini, Astra-Zeneca, Autifony, Biogen, BMS, Eli Lilly, Heptares, Jansen, Lundbeck, Lyden-Delta, Otsuka, Servier, Sunovion, Rand, Recordati and Roche. Neither O.H. or his family have been employed by or have holdings/ a financial stake in any pharmaceutical company. A.H. has been invited to scientific meetings by Lundbeck, Janssen, and Pfizer, and he received paid speakerships from Desitin, Janssen, Otsuka, and Lundbeck. He was member of Roche, Otsuka, Lundbeck, and Janssen advisory boards.

References

1.

Hasan
 
A
,
Falkai
P
,
Wobrock
T
, et al. ;
World Federation of Societies of Biological Psychiatry (WFSBP) Task Force on Treatment Guidelines for Schizophrenia
.
World Federation of Societies of Biological Psychiatry (WFSBP) Guidelines for Biological Treatment of Schizophrenia, part 1: update 2012 on the acute treatment of schizophrenia and the management of treatment resistance
.
World J Biol Psychiatry
.
2012
;
13
(
5
):
318
378
.

2.

Kennedy
 
JL
,
Altar
CA
,
Taylor
DL
,
Degtiar
I
,
Hornberger
JC
.
The social and economic burden of treatment-resistant schizophrenia: a systematic literature review
.
Int Clin Psychopharmacol
.
2014
;
29
(
2
):
63
76
.

3.

Kane
 
J
,
Honigfeld
G
,
Singer
J
,
Meltzer
H
.
Clozapine for the treatment-resistant schizophrenic. A double-blind comparison with chlorpromazine
.
Arch Gen Psychiatry
.
1988
;
45
(
9
):
789
796
.

4.

Galletly
 
C
,
Castle
D
,
Dark
F
, et al.  
Royal Australian and New Zealand College of Psychiatrists clinical practice guidelines for the management of schizophrenia and related disorders
.
Aust N Z J Psychiatry
.
2016
;
50
(
5
):
410
472
.

5.

Buchanan
 
RW
,
Kreyenbuhl
J
,
Kelly
DL
, et al. ;
Schizophrenia Patient Outcomes Research Team (PORT)
.
The 2009 schizophrenia PORT psychopharmacological treatment recommendations and summary statements
.
Schizophr Bull
.
2010
;
36
(
1
):
71
93
.

6.

Scottish Intercollegiate Guidelines Network (SIGN)
.
Management of Schizophrenia
. Edinburgh: SIGN; 2013. (SIGN publication no. 131). March
2013
. Available from: http://www.sign.ac.uk

7.

APA
.
American Psychiatric Association—Practice Guideline for the Treatment of Patients With Schizophrenia
. 2nd ed.
American Psychiatric, Association Publishing
;
2010
.

8.

NICE
.
Psychosis and Schizophrenia in Adults—The NICE Guideline on Treatment and Management
. Updated edition
2014
.
National Clinical Guideline Number 178
.
National Collaborating Centre for Mental Health commissioned by the National Institute for Health and Care Excellence
.

9.

Samara
 
MT
,
Dold
M
,
Gianatsi
M
, et al.  
Efficacy, acceptability, and tolerability of antipsychotics in treatment-resistant schizophrenia: a network meta-analysis
.
JAMA Psychiatry
.
2016
;
73
(
3
):
199
210
.

10.

Siskind
 
D
,
McCartney
L
,
Goldschlager
R
,
Kisely
S
.
Clozapine v. first- and second-generation antipsychotics in treatment-refractory schizophrenia: systematic review and meta-analysis
.
Br J Psychiatry
.
2016
;
209
(
5
):
385
392
.

11.

Siskind
 
D
,
Siskind
V
,
Kisely
S
.
Clozapine response rates among people with treatment-resistant schizophrenia: data from a systematic review and meta-analysis
.
Can J Psychiatry
.
2017
;
62
(
11
):
772
777
.

12.

Howes
 
OD
,
McCutcheon
R
,
Agid
O
, et al.  
Treatment-resistant schizophrenia: Treatment Response and Resistance in Psychosis (TRRIP) Working Group consensus guidelines on diagnosis and terminology
.
Am J Psychiatry
.
2017
;
174
(
3
):
216
229
.

13.

Correll
 
CU
,
Rubio
JM
,
Inczedy-Farkas
G
,
Birnbaum
ML
,
Kane
JM
,
Leucht
S
.
Efficacy of 42 pharmacologic cotreatment strategies added to antipsychotic monotherapy in schizophrenia: systematic overview and quality appraisal of the meta-analytic evidence
.
JAMA Psychiatry
.
2017
;
74
(
7
):
675
684
.

14.

Siskind
 
DJ
,
Lee
M
,
Ravindran
A
, et al.  
Augmentation strategies for clozapine refractory schizophrenia: a systematic review and meta-analysis
.
Aust N Z J Psychiatry
.
2018
;
52
(
8
):
751
767
.

15.

Kane
 
JM
,
Correll
CU
.
The role of clozapine in treatment-resistant schizophrenia
.
JAMA Psychiatry
.
2016
;
73
(
3
):
187
188
.

16.

Wagner
 
E
,
Löhrs
L
,
Siskind
D
,
Honer
WG
,
Falkai
P
,
Hasan
A
.
Clozapine augmentation strategies—a systematic meta-review of available evidence. Treatment options for clozapine resistance
.
J Psychopharmacology
.
2019
;
33
(
4
):
423
435
.

17.

Lally
 
J
,
Tully
J
,
Robertson
D
,
Stubbs
B
,
Gaughran
F
,
MacCabe
JH
.
Augmentation of clozapine with electroconvulsive therapy in treatment resistant schizophrenia: a systematic review and meta-analysis
.
Schizophr Res
.
2016
;
171
(
1–3
):
215
224
.

18.

Wang
 
G
,
Zheng
W
,
Li
XB
, et al.  
ECT augmentation of clozapine for clozapine-resistant schizophrenia: a meta-analysis of randomized controlled trials
.
J Psychiatr Res
.
2018
;
105
:
23
32
.

19.

Morrison
 
AP
,
Pyle
M
,
Gumley
A
, et al. ;
FOCUS trial group
.
Cognitive behavioural therapy in clozapine-resistant schizophrenia (FOCUS): an assessor-blinded, randomised controlled trial
.
Lancet Psychiatry
.
2018
;
5
(
8
):
633
643
.

20.

Gallego
 
JA
,
Bonetti
J
,
Zhang
J
,
Kane
JM
,
Correll
CU
.
Prevalence and correlates of antipsychotic polypharmacy: a systematic review and meta-regression of global and regional trends from the 1970s to 2009
.
Schizophr Res.
2012
;
138
(
1
):
18
28
.

21.

Paton
 
C
,
Lelliott
P
,
Harrington
M
,
Okocha
C
,
Sensky
T
,
Duffett
R
.
Patterns of antipsychotic and anticholinergic prescribing for hospital inpatients
.
J Psychopharmacol
.
2003
;
17
(
2
):
223
229
.

22.

Verdoux
 
H
,
Quiles
C
,
Bachmann
CJ
,
Siskind
D
.
Prescriber and institutional barriers and facilitators of clozapine use: a systematic review
.
Schizophr Res
.
2018
;
201
:
10
19
.

23.

Üçok
 
A
,
Çikrikçili
U
,
Karabulut
S
, et al.  
Delayed initiation of clozapine may be related to poor response in treatment-resistant schizophrenia
.
Int Clin Psychopharmacol
.
2015
;
30
(
5
):
290
295
.

24.

Luykx
 
JJ
,
Stam
N
,
Tanskanen
A
,
Tiihonen
J
,
Taipale
H
.
In the aftermath of clozapine discontinuation: comparative effectiveness and safety of antipsychotics in patients with schizophrenia who discontinue clozapine
.
Br J Psychiatry.
January 8
,
2020
:
1
8
.

25.

Gardner
 
DM
,
Murphy
AL
,
O’Donnell
H
,
Centorrino
F
,
Baldessarini
RJ
.
International consensus study of antipsychotic dosing
.
Am J Psychiatry
.
2010
;
167
(
6
):
686
693
.

26.

Law
 
H
,
Morrison
AP
.
Recovery in psychosis: a Delphi study with experts by experience
.
Schizophr Bull
.
2014
;
40
(
6
):
1347
1355
.

27.

Murphy
 
MK
,
Black
NA
,
Lamping
DL
,
McKee
CM
,
Sanderson
CF
,
Askham
J
,
Marteau
T
.
Consensus development methods, and their use in clinical guideline development
.
Health Technol Assess.
1998
;
2
(
3
):
i
iv
, 1–88.

28.

Diamond
 
IR
,
Grant
RC
,
Feldman
BM
, et al.  
Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies
.
J Clin Epidemiol
.
2014
;
67
(
4
):
401
409
.

29.

Meltzer
 
HY
,
Alphs
L
,
Green
AI
, et al. ;
International Suicide Prevention Trial Study Group
.
Clozapine treatment for suicidality in schizophrenia: international Suicide Prevention Trial (InterSePT)
.
Arch Gen Psychiatry
.
2003
;
60
(
1
):
82
91
.

30.

Frogley
 
C
,
Taylor
D
,
Dickens
G
,
Picchioni
M
.
A systematic review of the evidence of clozapine’s anti-aggressive effects
.
Int J Neuropsychopharmacol
.
2012
;
15
(
9
):
1351
1371
.

31.

Tiihonen
 
J
,
Taipale
H
,
Mehtala
J
,
Vattulainen
P
,
Correll
CU
,
Tanskanen
A
.
Association of antipsychotic polypharmacy vs monotherapy with psychiatric rehospitalization among adults with schizophrenia
.
JAMA Psychiatry.
2019
.

32.

Galling
 
B
,
Vernon
JA
,
Pagsberg
AK
, et al.  
Efficacy and safety of antidepressant augmentation of continued antipsychotic treatment in patients with schizophrenia
.
Acta Psychiatr Scand
.
2018
;
137
(
3
):
187
205
.

33.

Helfer
 
B
,
Samara
MT
,
Huhn
M
, et al.  
Efficacy and safety of antidepressants added to antipsychotics for schizophrenia: a systematic review and meta-analysis
.
Am J Psychiatry
.
2016
;
173
(
9
):
876
886
.

34.

de Boer
 
J
,
Prikken
M
,
Lei
WU
,
Begemann
M
,
Sommer
I
.
The effect of raloxifene augmentation in men and women with a schizophrenia spectrum disorder: a systematic review and meta-analysis
.
NPJ Schizophr
.
2018
;
4
(
1
):
1
.

35.

Melzer-Ribeiro
 
DL
,
Rigonatti
SP
,
Kayo
M
, et al.  
Efficacy of electroconvulsive therapy augmentation for partial response to clozapine: a pilot randomized ECT—sham controlled trial
.
Arch Clin Psychiatry
2017
;
44
(
2
):
45
50
.

36.

Polcwiartek
 
C
,
Nielsen
J
.
The clinical potentials of adjunctive fluvoxamine to clozapine treatment: a systematic review
.
Psychopharmacology (Berl)
.
2016
;
233
(
5
):
741
750
.

37.

He
 
H
,
Lu
J
,
Yang
L
, et al.  
Repetitive transcranial magnetic stimulation for treating the symptoms of schizophrenia: a PRISMA compliant meta-analysis
.
Clin Neurophysiol
.
2017
;
128
(
5
):
716
724
.

38.

Tollefson
 
GD
,
Dellva
MA
,
Mattler
CA
,
Kane
JM
,
Wirshing
DA
,
Kinon
BJ
.
Controlled, double-blind investigation of the clozapine discontinuation symptoms with conversion to either olanzapine or placebo. The Collaborative Crossover Study Group
.
J Clin Psychopharmacol
.
1999
;
19
(
5
):
435
443
.

39.

Manu
 
P
,
Sarpal
D
,
Muir
O
,
Kane
JM
,
Correll
CU
.
When can patients with potentially life-threatening adverse effects be rechallenged with clozapine? A systematic review of the published literature
.
Schizophr Res
.
2012
;
134
(
2–3
):
180
186
.

40.

Lally
 
J
,
McCaffrey
C
,
OʼMurchu
C
, et al.  
Clozapine rechallenge following neuroleptic malignant syndrome: a systematic review
.
J Clin Psychopharmacol
.
2019
;
39
(
4
):
372
379
.

41.

Nielsen
 
J
,
Young
C
,
Ifteni
P
, et al.  
Worldwide differences in regulations of clozapine use
.
CNS Drugs
.
2016
;
30
(
2
):
149
161
.

42.

Nielsen
 
J
,
Correll
CU
,
Manu
P
,
Kane
JM
.
Termination of clozapine treatment due to medical reasons: when is it warranted and how can it be avoided?
J Clin Psychiatry.
2013
;
74
(
6
):
603
613
; quiz 613.

43.

GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group
 
2015
. Available from gradeworkinggroup.org.

This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://dbpia.nl.go.kr/journals/pages/open_access/funder_policies/chorus/standard_publication_model)