The Causal Impact of Socio-Emotional Skills Training on Educational Success

Abstract

We study the long-term effects of a randomized intervention targeting children's socio-emotional skills. The classroom-based intervention for primary school children has positive impacts that persist for over a decade. Treated children become more likely to complete academic high school and enrol in university. Two mechanisms drive these results. Treated children show fewer attention deficit/hyperactivity disorder symptoms: they are less impulsive and less disruptive. They also attain higher grades, but they do not score higher on standardized tests. The long-term effects on educational attainment thus appear to be driven by changes in socio-emotional skills rather than cognitive skills.

1. INTRODUCTION

Socio-emotional skills are predictive of major life outcomes like educational attainment, employment, earnings, health, and participation in crime (Heckman and Rubinstein, 2001; Roberts et al., 2007; Almlund et al., 2011). While the predictive power of socio-emotional skills has been established, there is an ongoing debate about how malleable these skills are. If these skills are indeed malleable, interventions targeting children's socio-emotional skills may change the trajectory of a life and lead to lasting changes in educational attainment and labour market outcomes.

In this paper, we study how a randomized intervention among eight-year-old children in Switzerland affects tracking, high school completion, and university enrolment. The promoting alternative thinking strategies (PATHS) intervention is a classroom-based socio-emotional learning (SEL) programme for elementary school students that aim to reduce behavioural problems (Greenberg et al., 1995). The intervention consists of weekly lessons and homework assignments embedded in the school curriculum. PATHS lasts for up to two years and is designed to foster self-control, patience, social problem-solving skills, self-esteem, emotional intelligence, and academic engagement.

PATHS teaches children to think twice and to look ahead. For example, in one classroom exercise, children learn to make less impulsive choices in difficult situations with the three-part “stoplight approach”. First, on the red light, children slow down, take a few deep breaths, and explain the problem they face. Next, on the yellow light, children think about solution options and the consequences of their actions, and they plan a solution to the problem. Finally, on the green light, children execute their plan and evaluate whether it worked. Teachers support children in applying the stoplight approach in role-play and real-life situations occurring in class such as a conflict with peers. Children also practice this approach in homework assignments: they describe a school-related social or academic problem, explain solution strategies to parents or classmates, and collect feedback on their solution strategies. PATHS includes elements of cognitive behavioural therapy and targets problem-solving and regulatory skills that have been associated with improved externalizing behaviour conducive to learning, achievement, and future school success (Izard et al., 2004; Fantuzzo et al., 2007; Roberts et al., 2007; Raver et al., 2011; Deming, 2017).

The PATHS intervention was implemented in 2005 in Zurich, Switzerland. Its main goal was to reduce disruptive and aggressive behaviour by improving children's socio-emotional skills (Eisner et al., 2012a).¹ PATHS was introduced in twenty-eight out of fifty-six randomly selected public primary schools. Randomization took place at the school level and was stratified within-school districts. The intervention was supposed to last for one school year in second grade; however, the programme was so popular that over 70% of schools accepted the offer to continue with the programme for a second year. The experimental design also included the Triple P parenting training programme, which was implemented in half of the schools in the PATHS treatment schools and in half of the control group schools. The Triple P intervention, in contrast to PATHS, was less intensive and parents received less than 2 h of intervention time on average. Triple P did not affect educational outcomes. In this paper, we focus on the PATHS intervention and, for completeness, discuss the additional intervention and evaluation results in Supplementary Appendix D.

To evaluate the long-term effects of the PATHS intervention, we follow the treatment and control groups over seventeen years using the Zurich Project on Social Development from Childhood to Adulthood (z-proso) panel. This panel surveyed children, teachers, and primary caregivers annually or biannually from late 2004 until early 2022, with the last wave interviewing children at age 24. The data include baseline and follow-up measures of children's socio-emotional skills, parenting practices, and family and household characteristics, as well as administrative and self-reported educational outcomes. The combination of multi-respondent survey data matched to administrative education records allows us to provide detailed evidence on how treatment effects evolve over time and what skills the intervention affects.

PATHS has lasting effects on educational careers. At age 13, four years after the intervention, treated children become 4.4 percentage points more likely to get tracked into academic high school (Gymnasium).² The treatment effect persists, and treated children become 7.1 percentage points more likely to complete academic high school. This effect is economically significant. It represents a 23% increase relative to the mean of the control group. At age 24, fifteen years after the end of the intervention, the treatment group is 6.5 percentage points more likely to attend or have completed university, which is a 26% increase relative to the control group.

How does PATHS's effectiveness compare to other childhood intervention programmes? The size of the PATHS treatment effect is one-sixth of the treatment effect of the Abecedarian programme for college attendance (Campbell et al., 2002) and one-third of the size of the Perry Preschool Program for high school completion (Schweinhart, 1993; Heckman et al., 2010a). Our effect size on academic high school attendance is very similar to the treatment effect of the Baloo and You mentoring programme (Falk et al., 2020).

To investigate how the PATHS intervention affected children's educational attainment, we study four potential mechanisms. We evaluate changes in (1) grades and test scores, (2) socio-emotional skills, (3) children's classroom behaviour, and (4) parenting practices. We find evidence for the first three mechanisms. First, we find suggestive evidence that the intervention increases students’ teacher-assessed grades, but it has no impact on academic high school admission test scores. As grades are more influenced by socio-emotional skills and classroom behaviour than achievement tests (Borghans et al., 2016), our results suggest that the treatment effect is more likely to operate through changes in socio-emotional skills rather than through improved cognitive skills. Second, treated children display less attention deficit/hyperactivity disorder (ADHD) symptoms: they are less impulsive and less disruptive. Treated children also display less opposition, defiance, and non-aggressive conduct disorders (NACD). Children's anxiety, physical aggression, and prosociality are not affected by the intervention. Third, treated children are less likely to disturb lessons and more likely to focus on the teaching content in class. We do not find that treated children become more likely to complete their homework assignments, which suggests that the treatment mostly affects engagement and attention in the classroom. Fourth, we find no treatment effects on parenting practices.

Taken together, our analysis of the underlying mechanisms paints a consistent picture. The PATHS programme reduces children's impulsiveness and fosters their decision-making process. These behavioural changes improve classroom behaviour, which is rewarded by higher grades. In the long run, these improvements in grades lead students to enter the academic high school track and ultimately, university.

Our paper makes a series of novel contributions. Existing work typically studies treatment effects on socio-emotional skills or academic outcomes in isolation. Although a number of existing studies hypothesize that the long-term impact of early childhood interventions is driven by changes in socio-emotional skills, the direct empirical evidence for this link is limited. This paper fills that gap by studying both types of outcomes in a common framework and estimating to which degree changes in academic careers are mediated by changes in socio-emotional skills.

Existing childhood intervention studies either suffer from small sample sizes or are not able to follow individuals over an extended time period.³ Our study addresses both shortcomings. In contrast to some iconic childhood intervention studies, we have a substantially larger sample size and low attrition rates.⁴ Our ability to document how treatment effects evolve over a seventeen-year period distinguishes our work from the literature.

Our results add to recent evidence that challenge the view that effective interventions need to take place before age 6. We highlight that adding socio-emotional skills training to the primary school curriculum at ages 8–9 has meaningful long-run impacts on educational careers. Our paper thereby relates to a growing literature on other classroom-based primary school interventions targeting socio-emotional development. Alan et al. (2019) show that an intervention targeting grit increases students’ perseverance and math test scores two years after the intervention. Alan and Ertac (2018) show that an intervention targeting patience improves self-control and the ability to imagine future selves. These effects lead students to make more-patient intertemporal choices and persist over a three-year period. Schunk et al. (2022) show that training in self-regulation improves both cognitive and non-cognitive skills and makes students’ more likely to attend the academic high school track three years after the intervention. Berger et al. (2020) show that an intervention targeting students’ working memory improves both cognitive and socio-emotional skills with effects measured up to four years after the intervention. Brown et al. (2022) show that training in cognitive endurance improves student performance by 0.09 standard deviations. Cipriano et al. (2023) conduct a meta-analysis of school-based SEL programmes outside the economics literature and conclude that SEL programmes are generally effective. While Cipriano et al. (2023) find that most interventions lead to improved school climates and student behaviour, evidence on long-run effects remains extremely scarce. Hart et al. (2020) conduct a meta-analysis of education interventions targeting either cognitive or socio-emotional skills and conclude that most intervention effects appear to fade out within a few years. In contrast to existing papers on primary school interventions and the work summarized by these two meta-analyses, we have a much longer follow-up period and are the first to look at university enrolment as an outcome.

Our paper is not the first evaluation of the PATHS programme. Over the past two decades, PATHS has become increasingly popular and has been used in over thirty-six countries. Similar to the broader literature on SEL programmes, existing evaluations of PATHS focus on short- or medium-term behavioural changes in primary school (Malti et al., 2012; Crean and Johnson, 2013; Averdijk et al., 2016; Humphrey et al., 2016).⁵ Most of these evaluations were not designed to provide evidence of long-run effectiveness. In contrast to these studies, we follow students over a substantially longer time horizon and do not limit the analysis to survey-based behavioural measures. By looking at how the intervention affects university enrolment and graduation, we provide unique evidence on the long-term effectiveness of one increasingly popular SEL programme used in a variety of countries. In contrast to previous evaluations, our study is the first to establish a causal link between the PATHS programme, tracking, and participation in higher education.

2. THE PATHS TRAINING PROGRAMME

PATHS is a teacher-led programme for primary school children that was developed by Mark T. Greenberg and Carol A. Kusché at the University of Washington for the U.S. context (Kusché and Greenberg, 1994). The programme teaches systematic coping and decision-making strategies with the aim of fostering children's self-control, emotional understanding, and social problem-solving skills (Greenberg et al., 1995).

PATHS focuses on regulatory skills; it aims to foster social skills and improve externalizing behaviour (Greenberg et al., 1995, 1998). These behaviour changes should improve educational participation, reduce disruptive and aggressive behaviour in the classroom, and, ultimately, reduce violence, delinquency, and crime. Supplementary Table C1 provides an overview of the PATHS curriculum, which targets the following competencies:⁶

• (1)

  Self-control and patience

• (2)

  Decision-making strategies and social problem-solving

• (3)

  Self-esteem

• (4)

  Emotional intelligence

• (5)

  Fairness and rules

(1) Self-control and patience: PATHS targets self-control and patience through several exercises. Children learn to calm down in stressful situations using breathing techniques. They learn that it is their own responsibility to avoid exploding in anger and losing self-control through the analogy of a balloon that can burst. They role-play situations in which they practice ignoring, interpreting, and handling teasing of other children. They listen to a story of a girl who learned how to control herself by calming down and recognizing her emotions. The children complete some of these exercises at home. For example, children interview their parents about situations in which they had to calm down and write a summary of how their parents managed the situation.

(2) Decision-making strategies and social problem-solving: PATHS targets decision-making strategies and social problem-solving based on the stoplight approach described in the introduction. Supplementary Figure C1 shows a poster used to explain the stoplight approach. After introducing the method in class, the teacher discusses concrete situations in which children can use the approach. Children then apply the stoplight approach in repeated role-play exercises that simulate everyday situations. These exercises involve conflict situations with peers, parents, or teachers, or problems with school assignments. In homework assignments, children describe their problem-solving approach to a specific situation. They also practice the approach at home and explain the three steps to their parents, who receive a separate information leaflet about the benefits of the stoplight approach (Supplementary Figure C5).

(3) Self-esteem: PATHS aims to increase children's self-esteem by making them aware of their strengths and skills. In one of the lessons, children learn to give and accept compliments from peers and teachers. The teacher explains the concepts of compliments and respect as well as how to express compliments. Children then practice how to give compliments to each other in the classroom. In one homework assignment, children exchange compliments with parents and other family members at home.

In another exercise, the “child of the week” receives special privileges and duties for one week. As part of this exercise, the child acts as the teacher's assistant. At the end of the week, the teacher and classmates prepare a special child-of-the-week certificate with a picture of the child and a series of compliments and anecdotes describing what attributes classmates value in the child. While this activity is supposed to foster self-esteem, it also teaches children that privilege comes with responsibilities. They are supposed to learn that being valued by others also requires contributing to the common good.

(4) Emotional intelligence: PATHS targets emotional intelligence by fostering the understanding and expression of feelings. In one lesson, the teacher reads stories and children guess what feelings the protagonist felt. In one homework assignment, children describe their feelings during a recent emotional situation and discuss with their parents how they dealt with their emotions. With this exercise, children learn about themselves and become more aware of how their behaviour affects the feelings and perceptions of peers, parents, and teachers. To facilitate the recognition and expression of feelings, children receive “feelings cards”. These cards show children expressing different emotions such as happiness, excitement, anger, surprise, sadness, and worry (see Supplementary Figure C2). Children first colour these cards and then use them to express their current emotional state by placing the corresponding card on their table. In a final step, children reflect on how to demonstrate an emotion. For example, they have to find appropriate verbal responses to feelings like anger or sadness.

(5) Fairness and rules: Starting with the first PATHS lesson, children discuss the importance of having rules and manners. They discuss with their teachers in class and parents at home which rules should be established in the classroom, at home, and in everyday life. PATHS also tries to foster children's understanding of fairness by introducing children to principles of fair behaviour. In one lesson, children have to identify fair and unfair behaviour in different situations. In another lesson, the teacher reads a story and the children discuss whether the protagonists’ behaviour is fair or unfair.

3. DATA AND INSTITUTIONAL BACKGROUND

This section provides the institutional background of this study. First, we introduce the Zurich Project on Social Development from Childhood to Adulthood (z-proso data collection). Second, we illustrate the main characteristics of the education system in the Canton of Zurich.

3.1 The z-proso study

The data we analyse in the paper come from the z-proso panel study (Malti et al., 2011; Eisner et al., 2012b; Averdijk et al., 2016). Ribeaud et al. (2022) provide a detailed description of the Zurich Project on the Social Development from Childhood to Adulthood (z-proso). The study surveys students, teachers, and primary caregivers⁷ to investigate the life course of 1,675 children starting primary school in 2004 in Zurich, which is the largest city in Switzerland. Supplementary Table B1 provides an overview of the timing of the surveys, the respondents, and the response rate in nine different waves that took place between 2004 and 2022. By 2022, the study had followed children over a seventeen-year period until they were twenty-four-years old. Throughout the nine interview waves, response rates remained high. At age 24, for example, about 70% of the original sample responded to the survey.

The z-proso data include pre-intervention (baseline) and follow-up measures of children's socio-emotional skills, parenting practices, family and household characteristics, and administrative and self-reported educational outcomes. Supplementary Appendix B describes the data collection, informed consent, and ethics approval in greater detail.

Two early prevention programmes were implemented as part of the z-proso study. The first intervention was PATHS—the school-based social and emotional learning-based programme we focus on in this paper (see Section 2). The second intervention was the “Positive Parenting Program” (Triple P). Triple P encourages “positive parenting” by teaching techniques that support desired child behaviour, routines that avoid parent–child conflicts, and techniques that help the child plan (Sanders, 1999). In this paper, we focus on the PATHS intervention. For completeness, we provide more details on the Triple P intervention in Section 4 and show its treatment effects in Sections 6 and Supplementary Appendix D.

3.2 Education and tracking system

Figure 1 illustrates the school system and educational transitions in the canton of Zurich. Children start primary school at age 7. At age 12, after six years of primary school, children are tracked into different secondary schools.

The highest school track is academic high school (Gymnasium). Students attend this school for six years and typically graduate when they are eighteen-years old. It prepares students for university education and allows them to obtain the Matura degree required to enrol in university.⁸ Tracking is determined by grades and an admission test. Parents cannot choose the secondary school track and have no direct influence on the tracking outcome.

Students can obtain the Matura degree from an academic high school either through attending long-term academic high school or short-term academic high school. Tracking into long-term academic high school takes place after Grade 6. Tracking into short-term academic high school takes place after Grade 8 or 9. Later transitions are possible if a student has sufficiently high grades and passes the standardized admission test. During the first two probation years, some students initially tracked into academic high school fail to meet performance standards and move to a lower track. Additionally, a substantial number of students from the lower track move into the academic high school at different points in time. As a result, the share of students in the highest track increases by 25% during the first three years of secondary school.

4. EXPERIMENTAL DESIGN

4.1 Selection of schools, randomization, and definition of treatment group

Selection of participating schools: Zurich has seven school districts and a total of ninety primary schools. In each school district, eight schools were randomly selected to participate in the experiment. All fifty-six selected schools complied with the request of the City of Zurich's School and Sports Department to participate in the study.

Stratification and randomization: The fifty-six participating schools were assigned to fourteen strata cells. These cells were constructed by dividing each of the seven school districts into two groups consisting of four similar-sized schools. Within each strata, each school was randomly assigned to one of four treatment groups using a random number generated in Microsoft Excel. Schools with the largest random number in each strata were assigned to the PATHS programme (PATHS only). Schools with the second-largest number were assigned to the Triple P programme (Triple P only). Schools with the third-largest number were assigned both to the PATHS and Triple P programmes (PATHS and Triple P). Finally, schools with the lowest number received neither the PATHS nor the Triple P intervention. These schools are the pure control group.

Definition of treatment and control groups: In this paper, we focus on the PATHS intervention and define the treatment group as the group of schools assigned to one of the two PATHS treatment arms—either PATHS only or PATHS and Triple P combined. The control group consists of the pure control group and the Triple P-only group. Based on this definition, we have twenty-eight treated and twenty-eight control schools.

We include Triple P schools in the control group because this programme had no impact on children's educational careers (Supplementary Table D1). Triple P also had no impact on children's problematic behaviour or educational outcomes (Malti et al., 2011; Eisner et al., 2012b). Triple P has been shown to be effective for younger children (Doyle, 2020). In our setting, however, participation rates were low: only 27% of parents assigned to Triple P enroled in the programme and attended at least one session. Less than 19% of parents assigned to Triple P completed all four course units. Triple P parents received, on average, less than 2 h of intervention time. Eisner et al. (2011) show that parents who decided to attend courses were more likely to come from families with a high socio-economic background and be of Swiss origin.

Given that there are four treatment arms of the original experimental design, we could also estimate effects for each of the three treatment groups separately. In Section 6.3, we show that this approach leads to similar results. Alternatively, we could drop all students that received the Triple P treatment and compare only the pure PATHS with the pure control group. We provide results based on this alternative sample definition in Section 6.3. Although we lose about half of our observations with this definition, results remain very similar.

4.2 Implementation of the intervention

In the 2005/2006 school year, PATHS was implemented in twenty-eight primary schools in cooperation with the Department of School and Sports of the City of Zurich. Prior to the implementation, the original PATHS material was translated and adjusted to the Swiss context by Rahel Jünger in collaboration with the U.S. developers (Eisner et al., 2007). Rahel Jünger also implemented the programme and conducted the teacher training and supervision. This implementation was done independently from the evaluation.

In the selected schools, all Grade 2 classes were treated. Parents were not aware of the school's treatment status or the implementation of the programme in Grade 2 when enroling their children in primary schools. The level of compliance was remarkably high, with less than 6% of children changing schools between Grade 1, when baseline characteristics were collected, and Grade 2, when the PATHS programme was implemented in the treated schools. We do not find that school changes were related to the treatment status.⁹

To prepare schools for delivering the PATHS intervention, all teachers in charge of running PATHS lessons participated in a three-day workshop with a PATHS coach. There was usually one teacher per treated class that received the PATHS training. Teachers not delivering PATHS were not trained. During this workshop, the PATHS coach gave teachers an overview of the key concepts, classroom activities, posters, toys, and over 400 pages of materials. During the first year of the programme, teachers regularly met their PATHS coach, who gave them feedback and support. PATHS coaches also monitored the implementation and observed six PATHS lessons for each participating class. After each of these observations, the coach provided suggestions for improvements and graded the quality of the implementation.

The 45-min PATHS lessons typically took place twice per week. Treated children received PATHS lessons throughout the entire year of Grade 2. PATHS lessons replaced the class “Humans and Environment” (Mensch und Umwelt), which teaches children about the environment and organization of Swiss society. To reinforce the practice of PATHS methods, teachers also applied PATHS strategies in lessons not explicitly dedicated to the PATHS curriculum itself. Over the course of Grade 2, children received about 45 h of PATHS lessons and about 20 h of PATHS homework assignments (Eisner et al., 2007). Because the majority of teachers, parents, and children highly appreciated PATHS, over 70% of schools continued using the programme for a second year in Grade 3. The programme ended for all children at the end of Grade 3 when classes were reshuffled, and students received a new teacher.

4.3 Outcome variables and descriptive statistics

Outcome variables: We evaluate the long-term effects of the PATHS intervention on educational outcomes. The key outcomes of interest are whether individuals attend and complete the academic high school track (Gymnasium), whether they obtain the Matura degree, which allows them to enrol in any university, and whether they are enroled in or graduated from university at age 24.

We observe students’ secondary school tracks at ages 13, 15, and 17 from administrative school data provided by the Department of Education of the Canton of Zurich. Some children leave the canton of Zurich and therefore disappear from the administrative data. We therefore complement the administrative records with self-reported tracking outcomes based on the z-proso survey.¹⁰ We observe whether students complete academic high school and enrol in university or graduate from university in the wave 9 z-proso survey administered at age 24.

Table 1 shows that 16% of the participants attend academic high school at age 13, right after tracking has taken place. This number increases to 20% at age 15 and 26% at age 17.¹¹ Twenty-seven percent of children complete academic high school, and 17% are enroled in university at age 20. At age 24, 22% have graduated from university or are still enroled in university.

Baseline measures: Table 1 shows characteristics of children and parents measured at the baseline, that is, in the year before the start of the intervention. At this time, children are, on average, seven-years old. Forty-eight percent are girls. Our sample comes from a diverse population: only 60% are Swiss, 90% were born in Switzerland, and only 49% of mothers are Swiss. Seventeen percent of households are single-parent households. About 39% of mothers have completed at least academic high school (Gymnasium), and 16% hold a university degree. Fathers are slightly more educated than mothers, with 52% having completed Gymnasium or other types of higher education and 25% holding a university degree. The average family household income is USD 86,000 per year; 38% of families are entitled to state funded financial aid, and 18% report financial problems at the baseline. Our data also contain detailed baseline measures of child behaviour assessed through the Social Behavior Questionnaire (SBQ; Tremblay et al., 1991; Murray et al., 2019) and the Alabama Parenting Questionnaire (APQ; Shelton et al., 1996).

5. EMPIRICAL STRATEGY

5.1 Empirical model

We aim to estimate the treatment effect of the PATHS intervention on educational outcomes. Equation (1) shows our main empirical model:

where $Y_{is}$ is the outcome of interest (initial tracking into academic high school at age 13, academic high school completion, and university enrolment or graduation at age 24) of individual i in school s. $\text{PATH}{\text{S}}_s$ is an indicator showing whether the school s was randomly assigned to participate in the PATHS programme. ${\beta }_1$ is the parameter of interest. It captures the treatment effect of participating in the PATHS programme.

Vector $X_{is}$ contains baseline control variables. These differ depending on the specification. In our most complete specification, we include pre-treatment measures of child characteristics, household characteristics, and child socio-emotional skills. Child characteristics include age, gender, and Swiss citizenship. Household characteristics include household income, mother's age at the baseline, mother's education, father's education, and indicator variables for whether the mother was born in Switzerland and whether she has Swiss citizenship, whether the household is single-headed, whether the household reports receiving financial aid, and whether the household reports financial problems. For a child's socio-emotional skills, we rely on SBQ measures reported by the child, the teacher, and the primary caregiver. These include ADHD symptoms (disruptiveness and impulsiveness), anxiety and depressivity, aggression, prosociality, non-aggressive externalizing problem behaviour, non-aggressive conduct disorder, and opposition and defiance. The model always includes strata fixed effects ${\theta }_j$⁠, with $j=1,\dots ,14$⁠, for the level at which randomization took place. ${\epsilon }_{is}$ represents the error term of the model.

We estimate equation (1) using linear probability models and cluster standard errors at the school level. We additionally provide p-values based on randomization inference with 10,000 repetitions following Young (2019).

5.2 Balancing tests

The identifying assumption of our empirical strategy relies on the random assignment of children to the treatment status. To verify this assumption, we test whether baseline characteristics predict treatment status. In particular, we regress treatment status on each of the pre-treatment characteristics separately. We use all available characteristics on child and family demographics and measures for socio-emotional skills, and we estimate a total of fifty-six regressions.

Table 2 summarizes the balancing tests. Column (1) shows the number of statistically significant coefficients we obtain when regressing the indicator for treatment status (⁠$\text{PATHS}$⁠) on baseline characteristics. Column (2) shows the number of coefficients we would expect to find statistically significant due to chance variation. Overall, Table 2 suggests that the randomization was successfully implemented: the number of significant coefficients is similar to the expected number of significant coefficients under random assignment.

Table 3 provides a closer look at unbalanced variables by reporting point estimates from all fifty-six balancing regressions. The analysis reveals a substantial and significant imbalance (p < 0.01) in fathers’ education levels between the treatment and the control group. Given that parental education is a key determinant of children's educational outcomes, this imbalance deserves careful consideration. Children receiving the PATHS intervention come from families with, on average, less-educated parents. Treated children are about 10 percentage points less likely to have a father that holds at least an academic high school degree. This imbalance in fathers’ education levels will make it harder for us to identify effects of the intervention if the treatment affects children's educational outcomes positively. Without accounting for this imbalance, we would underestimate treatment effects of the PATHS programme.

To provide a better understanding of the initial imbalance and its consequences for our results, we conduct a series of exercises. In Section 6.2, we show how different sets of control variables affect our results. We highlight that we need to control for unbalanced parental education to obtain unbiased treatment effects. In Section 6.2, we further conduct a permutation exercise that highlights that creating a balanced estimation sample is an alternative way to obtain unbiased estimates that does not require the inclusion of control variables. Overall, our analyses highlight that it is essential to account for the initial imbalance in the sample to obtain unbiased results.

6. RESULTS

In this section, we provide the main results of our analysis on the impact of the PATHS curriculum on educational careers. We also provide a series of sensitivity analyses and test whether results are driven by selective attrition and estimate treatment effects for different subgroups.

6.1 Main results

Table 4 shows estimates of the PATHS treatment effect on education trajectories from equation (1). The dependent variable in column (1) is an indicator for academic high school attendance at age 13, immediately after tracking has occurred. The dependent variable in column (2) is an indicator for academic high school completion at age 20. In column (3), the dependent variable is an indicator for university enrolment or graduation at age 24.

Table 4 shows positive and statistically significant treatment effects across all educational outcomes. The PATHS programme leads to a 4.4 percentage point increase in the likelihood of children attending an academic high school at age 13, immediately after tracking. This effect is economically significant, as it corresponds to a 22% increase compared to the control group mean. The positive impact persists over time. By age 20, attending PATHS increases the likelihood of completing academic high school by 7.1 percentage points, representing a 23% increase over the completion rate of the control group. Furthermore, the treatment effect of PATHS remains visible for higher education. At age 24, PATHS increases the likelihood of attending university or having already graduated from university by 6.5 percentage points (26%).¹²

The results presented in Table 4 suggest that the PATHS treatment effect is fairly persistent over time. Many students, however, move between tracks during their career. Supplementary Figure A1 shows a flowchart that illustrates how students switch between school tracks over time, which highlights substantial mobility. About 35% of students who graduate from academic high school were initially not admitted to this track at age 13. Similarly, 38% of students that are enroled in or have graduated from university at age 24 did not attend the academic high school track at age 13. This evidence calls for a closer examination of how treatment effects evolve over time.

Figure 2 shows how the PATHS treatment effect evolves over time using data from all available waves. The figure displays the treatment effects on academic high school enrolment at ages 13, 15, and 17, as well as high school completion and university enrolment at age 20, and university enrolment or graduation at age 24. All the point estimates in Figure 2 are positive and statistically significant. The PATHS treatment effect slightly increases over time. Notably, the mean of the dependent variables also increases over time with the relative treatment effect being fairly constant over time.^13,14

Overall, we observe large and economically significant effects. In the average class in our sample, five out of twenty-eight children attend university at age 20. The size of the treatment effect implies that one additional child—six instead of five—will attend university due to the intervention. We discuss our effect sizes and how they compare to other childhood interventions in Section 9.

6.2 Sensitivity analysis—imbalanced treatment and control groups

One potentially important concern for the interpretation of our results comes from the imbalance in fathers’ education between the treatment and control groups. To assess how much this initial imbalance affects our estimates, we estimate treatment effects with and without controls for the initial imbalance in Table 5. Panel A reports results without any control variables except the randomization strata. Panel B reports results with controls for parental education to account for the imbalance between the treatment and control groups at the baseline. Panel C reports results from the model that additionally include the full set of baseline control variables.

Panel A of Table 5 shows that, despite having less-educated parents, children who attend the PATHS programme seem to do slightly better than children in the control group. For example, treated children are 2.3 percentage points more likely to have completed academic high school at age 20. While the point estimates are positive for all outcomes, none of the estimates in Panel A are statistically significant at conventional levels. Panel B accounts for the imbalance in parental education at the baseline by including control variables for parental education. By only adding controls for parental education, we find positive and statistically significant treatment effects for all educational outcomes. Panel C of Table 5 replicates the analysis in Table 4 and confirms that the results are robust to including this large set of additional control variables. Despite the lower number of observations due to missing values in our control variables, point estimates in the model with controls for parental education (Panel B) and the full set of controls (Panel C) are not statistically different from each other.¹⁵

Taken together, Table 5 highlights that our results depend on whether we control for parental education. In a randomized control trial (RCT) with perfect randomization, we would not expect estimates to change when including controls. However, in our case, it is essential to account for the initial imbalance to obtain unbiased estimates. This imbalance—having fathers with less education in the treatment group—leads to downward bias and makes it harder for us to identify positive treatment effects.¹⁶ We conduct a series of sensitivity analyses to support this intuition.

To understand the consequences of the imbalance, we conduct a permutation exercise that first creates a balanced estimation sample from the full sample and then estimates treatment effects using the model without controls (Table 5, Panel A). To obtain a balanced estimation sample, we exclude classes with highly imbalanced shares of fathers who completed academic high school. Because the treatment group has a lower share of fathers who completed academic high school, we sequentially drop classes from the treatment (control) group with the lowest (highest) share of parents who completed academic high school. We start with the full sample and then symmetrically trim the sample by gradually dropping the same number of classes from both the treatment and control groups. We start by eliminating the class in the treatment group with the lowest share of parents who completed academic high school and the class in the control group with the highest share. By dropping imbalanced classes with unusually high or low levels of fathers’ education, we can investigate how treatment effects evolve in an increasingly balanced sample.

Figure 3 shows the impact of the sample restriction procedure on the imbalance in fathers’ education. The x-axis shows the number of classes we drop. The y-axis shows the “balancing coefficient”—the correlation between treatment status and having a father who completed academic high school. We begin with the entire sample and then gradually exclude classes. Figure 3 shows that the gradual exclusion of classes restores balance between the treatment and the control groups. The exclusion of fourteen out of 129 classes results in an imbalance that is no longer statistically significant at conventional levels, although the point estimates remain sizeable (−4.14 percentage points). When excluding twenty-four classes, the difference between the treatment and the control groups is closest to zero. We will use the subsample that minimizes the imbalance as our benchmark model and mark corresponding estimates with a blue diamond. This subsample consists of 1,329 observations corresponding to 80% of the sample.

Figure 4 shows how treatment effects evolve depending on the number of classes dropped. Panel (a) shows treatment effects on academic high school completion, and Panel (b) shows treatment effects on university enrolment or graduation at age 24. Treatment effects are estimated using the model without controls (Table 5, Panel A). The figure highlights that removing imbalanced classes increases the treatment effects. Once the sample is sufficiently balanced, the effect becomes statistically significant. In our benchmark model that minimizes the imbalance, we observe a highly significant treatment effect of 7.8 percentage points (p-value = 0.012). This treatment effect is not distinguishable from the treatment effect we observe in the full sample model that includes controls (Table 5, Panel C). Results in Panel (b) show that treatment effects evolve very similarly when we examine university enrolment or graduation at age 24 as an outcome.¹⁷

Table 6 shows treatment effects on education outcomes for the restricted sample (twenty-four classes excluded) using the specification without controls (Panel A) and the specification with controls for parental education (Panel B). Table 6 provides the same conclusion as Figure 4 and highlights two important findings. First, the treatment effect of PATHS in the model without control variables is consistently positive and significant for all education outcomes. Second, the results in the restricted sample without controls are remarkably similar to the estimates in the full sample that account for the initial imbalance by controlling for fathers’ education level. This similarity confirms that, as expected in an RCT, adding controls has no impact on the estimates once the sample is balanced. Additionally, it confirms that the model with no controls in the entire sample underestimates the treatment effect of PATHS and that controlling for parental education is a necessity that corrects for this bias.

6.3 Additional robustness tests

This section provides an additional set of robustness tests for our results. First, we test whether our conclusions remain the same if we compute p-values based on randomization inference. Second, we estimate specifications using an alternative treatment group definition. Third, we test whether selective attrition drives our results. Fourth, we discuss possible concerns of experimenter demand effects.

Randomization inference: In the main analysis, we cluster standard errors at the school level, resulting in fifty-six clusters. As a robustness test, we compute p-values based on randomization inference using 10,000 random permutations following Young (2019). With this procedure, we account for possible bias in standard errors due to a small number of clusters. Table 4 shows that p-values based on randomization inference lead to the same overall conclusions.

Alternative treatment group definitions: Our baseline analysis compares individuals exposed to PATHS (treatment) to individuals who were not exposed to PATHS (control). However, some individuals in the treatment and control groups were also exposed to the Triple P programme. In Supplementary Table D1, Panel B, we show estimates comparing individuals who were assigned to the Triple P programme versus individuals who were not exposed to Triple P. This table shows that the Triple P intervention has no significant effect on educational outcomes. The lack of effects for Triple P is consistent with Eisner et al. (2012b), who show that the intervention had no short-term effects on either parenting practices or child problem behaviour.

An alternative evaluation strategy is to drop all students that received the Triple P treatment and compare only the pure PATHS with the pure control group. In Supplementary Table A8, we show that, although we lose about half of our observations, results remain very similar.

Alternative treatment group definitions—full 2 × 2 design: Supplementary Table D1 provides additional evidence on the robustness of our results by comparing all treatment arms of the original experiment. Panels A and B confirm that PATHS, in contrast to Triple P, is effective. Panel C in Supplementary Table D1 reports estimates for a model that includes an interaction effect between the two interventions, PATHS and Triple P. The PATHS treatment effect in Panel C is similar to Panel A and confirms that children exposed to PATHS are more likely to complete academic high school. As in Panel B, children exposed to the Triple P programme are not significantly affected. The interaction effect between both interventions (PATHS * Triple P) never reaches statistical significance in any of the columns (1)–(3). These results suggest that there is no additional benefit from exposure to both programmes.

Selective attrition: We observe initial tracking from administrative data for 95% of the initial sample. We observe university enrolment/graduation for 69% of the initial sample. To test for selective attrition, we estimate the effect of attending the PATHS programme on the probability of observing an individual in our estimation sample at five different points in time: at ages 13, 15, 17, 20, and 24.¹⁸ More specifically, we regress an indicator showing whether we observe the individual in our sample at a given time on a PATHS treatment indicator. Supplementary Table A6, Panel A, shows that the treatment does not affect the probability of being observed in the sample at different points in time. The PATHS coefficients are small and not statistically significant in all specifications. Selective attrition does not appear to drive our results.¹⁹

Demand effects: As in any social experiment, our results raise the question of whether knowledge of treatment or experimenter demand effects could drive treatment effects. Two reasons speak against this. First, tracking in Switzerland is determined by two objective student performance measures: (1) grades in core subjects in the last grade of primary school and (2) standardized externally evaluated admission test scores. Therefore, tracking is not a choice variable, and it is not determined by subjective teacher recommendations. Second, those teachers involved in the programme in Grade 2 have no direct or indirect influence on the tracking decision that takes place four years later. At the end of Grade 3, after the intervention is completed, children are reassigned to new teachers uninvolved in the intervention. It is therefore not plausible that these new teachers—who did not implement the programme—manipulated treated students’ grades four years after the end of the programme to push them into academic high school. Third, the competitive tracking system and the fact that the treatment effect persists over time reject the interpretation that demand effects drive our results. If the intervention motivated teachers to inflate treated students’ grades and pushed unqualified students into academic high school, they would not have survived in this competitive track. During the first two probation years, students initially tracked into academic high school who fail to meet performance standards are moved to the lower track. At the same time, a substantial number of students from the lower track move up into the academic high school at different times. If initial treatment effects had been driven by teacher-inflated grades, these students would not have survived the competitive environment of academic high schools. The fact that students move substantially between tracks in combination with the lasting treatment effect for academic high school completion rejects the idea that demand effects or knowledge of the treatment status could be driving our main results.

6.4 Heterogeneous treatment effects

Figure 5 investigates potential heterogeneous treatment effects for the impact of PATHS. Panel (a) displays the effects on initial tracking at age 13, Panel (b) shows the effects for academic high school completion, and Panel (c) illustrates the effect on university enrolment or graduation at age 24. We estimate heterogeneous effects by family income, parental education, socio-economic status (SES), and the child's gender and age. We also investigate heterogeneity in treatment effects by the main socio-emotional skills targeted by PATHS including ADHD symptoms, opposition and defiance, and non-aggressive conduct disorder. To estimate heterogeneous effects, we augment equation (1) with interaction terms between the treatment variable and indicators for different subgroups. All subgroups are defined based on pre-treatment characteristics. Figure 5 shows the estimation results with the respective subgroup shown on the y-axis. Overall, the analysis remains suggestive and mainly shows that we lack the statistical power to detect systematic differences between subgroups.

7. MECHANISMS

In this section, we study four possible mechanisms for the effect of the PATHS programme on educational trajectories. First, we analyse whether PATHS affected the two elements that determine the tracking outcome: primary school grades and academic high school admission test scores. Second, we study whether PATHS affected children's socio-emotional development—the main target of the intervention. Third, as some of the PATHS activities involve parent–child interactions, we test whether the intervention affected parenting practices. Fourth, we investigate whether PATHS affected school-related behaviour like classroom disruption and homework completion.²⁰

7.1 Effects on grades and admission test scores

Primary school grades are given on a scale of 1–6 and are based on tests and the subjective assessments of the primary school teacher. The standardized high school admission test is graded on that same 1–6 scale and covers mathematics, reading comprehension, and writing. The test is evaluated by an external high school teacher who typically does not know the child. Students’ tracking outcomes are determined by their average primary school grades and their admission test scores. Both performance measures have equal weight, and students with a minimum of 4.5 out of 6 are admitted to academic high school.²¹

We estimate the effect of PATHS on grades and admission test scores using the specification with the full set of controls. To simplify the interpretation of the results, we standardized both outcome variables to have means of zero and standard deviations of one. Figure 6 provides suggestive evidence that the PATHS programme increases children's grades. Due to the lower sample size for this outcome, effects are less precisely estimated. Figure 6 suggests that PATHS increases grades by 20–25% of a standard deviation.²² Figure 6 also shows the treatment effect on the admission test scores. Point estimates on test scores are close to zero. While this coefficient is imprecisely estimated and not statistically significant, we cannot fully rule out that the treatment had some positive impact on the standardized admission test.

Taken together, our results suggest that the intervention raises grades but has only a limited impact on admission test scores. On the one hand, test scores mainly capture dimensions of children's cognitive skills. On the other hand, grades are likely to also reflect differences in classroom behaviour, aptitude, and engagement.²³ One plausible interpretation for the effects is that treated children display better classroom behaviour that is rewarded with higher grades by the teacher. Our results suggest that long-term intervention effects are more likely to operate through changes in socio-emotional skills rather than cognitive skills.

7.2 Effects on socio-emotional skills

We investigate changes in children's socio-emotional development as possible mechanisms. We measure this development with the SBQ, which teachers and parents’ answer. This questionnaire includes the following six domains: (1) ADHD symptoms (disruptiveness and impulsiveness), (2) opposition and defiance, (3) non-aggressive conduct disorder, (4) anxiety and depressivity, (5) aggression, and (6) prosociality. Each of these domains is measured with up to ten subitems that ask about the prevalence of a specific behaviour.²⁴ For every survey wave, we combine all available responses from the primary caregiver and the teacher. We do this by computing the sum of answers to each subitem domain, then take the average and standardize for the primary caregiver and teacher, then compute the average of teacher and primary caregiver reports and standardize again to obtain measures with a mean of zero and a standard deviation of one.

Figure 7 shows the PATHS treatment effect on ADHD symptoms and opposition and defiance. Panel (a) of Figure 7 shows the evolution of the PATHS treatment effect on ADHD symptoms (disruptiveness and impulsiveness) over time. PATHS causes children to become more impulsive and disruptive during the intervention period and persistently less impulsive and disruptive after the intervention is completed.²⁵

Seeing more behavioural problems during the intervention is, at first sight, surprising. This effect goes against the aim of the intervention. One explanation is that the intervention made teachers and parents more aware of what appropriate child behaviour should look like. This possible increased awareness may have made them more critical in the short run. Consistent with this interpretation, the PATHS developers provide anecdotal evidence showing that teachers raise their expectations about children's appropriate behaviour during the intervention.

After the intervention, starting from age ten, we see that PATHS reduces ADHD symptoms by making children less disruptive and impulsive. At age ten, children were also reassigned to new classes and new teachers who were not involved in the intervention. From this age, our measures therefore likely reflect child behaviour and development more objectively. The treatment effect persists until primary school completion, when children are twelve years old, and remains visible at ages 13 and 15.²⁶

Panel (b) of Figure 7 shows the PATHS treatment effect on opposition and defiance. Opposition and defiance capture behaviours like telling lies, cheating, or ignoring teachers’ instructions. The overall picture is similar to the treatment effect for ADHD symptoms. PATHS increases opposition and defiance during the intervention and decreases those behaviours after the intervention is completed. The treatment effects fade out after children transition to secondary school.

Figure 8 shows the effects of PATHS on non-aggressive conduct behaviour (Panel a), anxiety and depressivity (Panel b), aggression (Panel c), and prosociality (Panel d). PATHS reduces children's non-aggressive conduct disorders such as lying, stealing, or destroying other children's belongings after the intervention. This effect remains visible until age 11 and fades out afterward. Anxiety, aggression, and prosociality do not appear to be systematically affected by the intervention.

7.3 Effects on parenting practices

We investigate changes in parent–child interactions and parenting practices.²⁷ We analyse parenting practices using the APQ, which captures the following five domains: (1) corporal punishment, (2) parental control and supervision, (3) inconsistent discipline, (4) parental involvement, and (5) positive parenting. Each domain is measured with up to ten questions answered by the primary caregiver on a 5-point Likert scale ranging from “never” to “always”.²⁸ To facilitate comparisons, we standardize each subdomain to have a mean of zero and a standard deviation of one.

Figure 9 shows the evolution of the PATHS treatment effect on parenting practices over time. Taken together, our analysis suggests that the intervention had no systematic impact on parenting practices. Our analysis is, however, limited by the type of parenting practices we observe in the data. It remains possible that parents, who we directly and indirectly targeted by several activities of the PATHS programme, changed their behaviour in domains that we do not observe in the data. However, we believe these changes are likely minor in our setting.

7.4 Effect on behaviour in class

In this section, we look at possible intervention effects on school-related behaviour. We have measures on school-related behaviour for four different domains: (1) disturbing lessons, (2) being busy with other things during classes, (3) displaying impertinent school behaviour, and (4) neglecting homework. We observe these outcomes starting from Grade 4, after children are reassigned to new classes and evaluated by a new teacher. Each domain is measured through a 5-point Likert scale ranging from “never” to “very often”. To facilitate comparisons, we standardize each subdomain to have a mean of zero and a standard deviation of one.

Figure 10 shows results for school-related behaviour. PATHS reduces children's likelihood of disturbing lessons by 13% of a standard deviation at age 10. The effect persists throughout secondary education. We see a similar pattern for children's ability to focus. Treatment effects are largest immediately after the intervention at age 10 with an effect equivalent to a reduction of about 20% of a standard deviation. We find no significant treatment effects for impertinent conduct at school or neglecting homework.

7.5 Multiple hypothesis testing

We estimate treatment effects for a substantial number of outcomes. This implies that some statistically significant effects might simply represent chance findings. We address this concern by: (1) testing which estimates remain significant after grouping outcomes, (2) correcting for multiple hypothesis testing, and (3) creating an overall index for children's socio-emotional and cognitive development measured post-treatment.

Heckman et al. (2010a) point out that there is some arbitrariness in defining the blocks of hypotheses to be jointly tested in a multiple hypothesis testing procedure. We apply a simple and conservative criterion for our analysis: drawing on the fact that PATHS mainly targets children's socio-emotional, behavioural, and cognitive development, we include all child outcomes analysed as mechanisms and our latest educational outcome.

Socio-emotional skills and classroom behaviour are measured at multiple points in time. We can substantially reduce the number of hypotheses tested by creating a post-intervention index for a given skill. We can then test whether the intervention affected average post-intervention measures of a skill domain. To aggregate a given domain, we first standardize each period-specific measure. We then calculate the mean over all post-treatment periods and standardize again. This results in ten distinct socio-emotional and two cognitive skills measures.

Table 7 shows treatment effects for these ten socio-emotional skill measures plus university enrolment or graduation at age 24, grades, and admission test scores. Results highlight that the post-treatment indexes for ADHD symptoms (column 4), opposition and defiance (column 5), non-aggressive conduct disorder (column 6), disturbing lessons (column 10), and being busy with other things (column 11) are significantly affected by the intervention. In other words, children exposed to the PATHS programme significantly improve their post-treatment behaviour in these realms.

Given that some of these significant effects may still represent chance findings, we next apply a Bonferroni correction. Table 7 reports the Bonferroni (Abdi, 2007) and Bonferroni–Holm (Holm, 1979; Jones et al., 2019) corrected p-values for the thirteen socio-emotional and cognitive measures we investigate as candidate mechanisms. The positive treatment effect for university enrolment or graduation at age 24 remains significant as well as reductions in ADHD symptoms, opposition and defiance, and improvements in two measures of classroom behaviour. The marginally significant treatment effect for non-aggressive conduct disorder does not survive the correction.

Finally, instead of considering thirteen different post-treatment outcomes, we construct one overall index for children's socio-emotional and cognitive development. This index is obtained by combining the thirteen post-intervention measures shown in Table 7 in the following way: first, we negate the sign on all “negative outcomes”, that is, ADHD symptoms, opposition and defiance, non-aggressive conduct disorder, anxiety and depressivity, aggression, disturbs lessons, busy with other things in class, impertinent conduct at school, and neglects homework to align their interpretation with the other positive socio-emotional outcomes. Second, we average across these thirteen standardized measures. Third, we standardize the resulting super-index to obtain a measure capturing productive child development.²⁹ Table 8 shows that the PATHS treatment significantly increases the post-intervention child development super-index (p-value = 0.008). Taken together, our results remain robust with respect to multiple hypothesis testing.

8. MEDIATION ANALYSIS

We perform a mediation analysis in the spirit of Heckman and Pinto (2015) and Gelbach (2016) to quantify the proportion of the treatment effect mediated by our proposed mechanisms and to separate the contribution of each single mechanism to the estimated treatment effect.

The results of the mediation analysis should be interpreted with caution. Imai et al. (2010) show that to be able to interpret this type of analysis causally, one needs to make strong assumptions about the source of variation of the mediators. As we lack exogenous variation in specific channels and have to rely on a single source of exogenous variation (one randomization), this analysis can only provide suggestive evidence on the importance of different mediators in explaining treatment effects.

We perform the mediation analysis for the following education outcomes: attendance of academic high school at ages 13, 15, and 17, academic high school completion, university enrolment at age 20, and university enrolment or graduation at age 24. As possible mediators, we focus on socio-emotional skills, parenting practices, and classroom behaviour. The set of mediators includes all variables analysed as potential mechanisms in Section 7 except grades and test scores because these variables are only available for a subsample of students that sit for the standardized academic high school admission test. We aggregate all candidate mechanisms into three domains: (1) socio-emotional skills, (2) parenting practices, and (3) classroom behaviour. Given the longitudinal nature of our data, we only consider measures obtained after the intervention and before the educational outcome is measured. In cases in which we have multiple observations for the same mediator, we construct a summary index using the covariance weighting procedure discussed in Anderson (2008). We assume that the PATHS treatment has both direct and indirect effects on education outcomes. The indirect effects run through treatment effects of the intervention on socio-emotional skills, parenting practices, and classroom behaviour. The results of the mediation analysis will give us an estimate of the importance of these indirect effects.

Panel (a) of Figure 11 shows the results of the mediation analysis. Each horizontal bar represents a specific outcome of interest.³⁰ Coloured areas within the bars illustrate the contribution of each mediator to the overall treatment effect. The grey area stands for the unexplained share of the treatment effect. The mediation analysis highlights that our candidate mechanisms explain about 20–39% of the treatment effect. Among the mechanisms we study, socio-emotional skills appear as the most important mediator of the PATHS treatment effect. For example, socio-emotional skills explain about 25% of the PATHS treatment effect on university attendance at age 20 and about 39% for university enrolment or graduation at age 24. The contribution of parenting practices and classroom behaviour are smaller and less stable across different outcomes, suggesting that these are less important mechanisms.

Given their important mediating role, in Panel (b) of Figure 11, we investigate the contribution of each socio-emotional skill. The main mediator is reduction of ADHD symptoms. The relative importance of its mediating role is similar across outcomes and does not depend on the children's age. Non-aggressive conduct disorders and opposition and defiance are also relevant mediators, but quantitatively less important. Their mediating role is also less stable over time. The remaining socio-emotional skills seem to have a negligible role as mediators (prosociality) or have a negative load as mediators (anxiety and aggression).

Taken together, the mediation analysis described in this section suggests that the PATHS treatment effect on educational outcomes is driven by treatment-induced improvements in children's socio-emotional skills, in particular, by reductions in ADHD symptoms—impulsiveness and disruptiveness.

9. COMPARISON OF COSTS, BENEFITS, AND PREVIOUS EVALUATIONS

In this section, we contextualize the main results of this study. We start with the discussion of other RCTs that evaluated the PATHS programme. We then compare the size of the treatment effects and the cost of PATHS to other (iconic) childhood intervention studies.

9.1 Previous evaluations of PATHS

This is not the first evaluation of the PATHS programme. Over the past two decades, PATHS has become increasingly popular and has been used in at least thirty-six countries. Supplementary Table A1 provides an overview of all studies that provide causal evidence on PATHS effectiveness. Crean and Johnson (2013) examine the effect of PATHS on U.S. elementary school students’ aggressive behaviour and find lower levels of aggressive behaviour for treated students. The effect persists over two years after the intervention. Greenberg et al. (1995) show that PATHS increases vocabulary and emotional intelligence of second and third grade children in the U.S. Schonfeld et al. (2015) find similar results and show that PATHS improves reading and math proficiency in primary school. This effect, however, disappears two years after the intervention. Kam et al. (2004) evaluate PATHS in a sample of children with special needs living in the U.S. They find positive effects on externalizing and internalizing behaviour and reduced self-reported depressivity three years after the intervention. Riggs et al. (2006) show that PATHS fosters inhibitory control and leads to less disruptive behaviour. While many of the results of previous evaluations are consistent with our evidence on underlying mechanisms, we find no evidence that the intervention reduced physical aggression in our setting.

Supplementary Table A1 highlights that previous evaluations of PATHS focus on short-term behavioural changes in primary school and were not designed to provide evidence of long-run effectiveness. In contrast to these studies, we follow students over a substantially longer time horizon and do not limit the analysis to survey-based behavioural measures. By looking at how the intervention affects school careers as well as university enrolment and graduation, we provide unique evidence on the long-term effectiveness of one increasingly popular SEL programme. Our study is the first to establish a causal link between the PATHS programme and participation in higher education.

9.2 Other intervention studies and comparison of effect sizes and costs

In this section, we benchmark our intervention to similar interventions affecting educational outcomes and targeting child development. Figure 12 summarizes related intervention studies and our contribution to this literature. Panel (a) shows childhood intervention programmes with long-term evaluations: Campbell et al. (2002) evaluate the Abecedarian preschool programme, one of the oldest early childhood interventions, and show that the intervention improved IQ, achievement, and college enrolment. Heckman et al. (2010a) and Schweinhart (1993) evaluate the Perry Preschool Program, which aimed to foster the development of disadvantaged children, and show that programme participants obtained more schooling, had higher earnings, and committed fewer crimes.³¹ Gertler et al. (2014) analyse long-term effects of the Jamaican Study that contained an intervention aimed at improving mother–child interactions through home visits. They find increases of 25% in earnings twenty years after the intervention. Algan et al. (2022) use data from the Montreal Longitudinal Experimental Study, which aimed to improve socio-emotional skills in boys with after-school training sessions. This intervention increased self-control and trust during adolescence and increased educational achievements in early adulthood.

Panel (b) in Figure 12 summarizes interventions with relatively short follow-ups. These interventions explicitly targeted socio-emotional skills in children. Compared to these studies, we study treatment effects over a longer time horizon.

Alan et al. (2019) show that an intervention targeting grit increases students’ perseverance and subsequent math test scores two years after the intervention. Alan and Ertac (2018) show that an intervention targeting patience improves self-control and the ability to imagine future selves. These effects lead to more patient intertemporal choices and persist over a three-year period. Cappelen et al. (2020) show that early childhood education affects children's social preferences for fairness and the importance children place on efficiency relative to fairness. Oreopoulos et al. (2017) evaluate a mentoring and tutoring programme and find that the programme increases high school completion by 35% and postsecondary enrolment by more than 60%. Kosse et al. (2020) study a mentoring programme for primary school children and show that the programme persistently increases prosociality. Falk et al. (2020) follow these children over time and show that the programme also increases the probability of being assigned to the academic high school track. Heller et al. (2017) evaluate the “Becoming a Man” (BAM) intervention in Chicago and find that the programme increases high school completion rates and reduces delinquent behaviour.

Comparison of effect sizes: Figure 13 illustrates differences in effect sizes across studies. In our setting, PATHS increases children's probability of completing academic high school by 23%. This effect size is comparable to effects of other interventions. The Montreal Longitudinal Study social skills training programme increases the probability of completing high school by 45% (Boisjoli et al., 2007; Algan et al., 2022). The BAM intervention forecasts treatment effects of 12–19% on high school completion (Heller et al., 2017). The Pathways mentoring and tutoring programme increases high school completion by 35% (Oreopoulos et al., 2017). The Working Memory Training programme increases the probability of getting tracked into academic high school by 16% (Berger et al., 2020). The Baloo and You mentoring programme increases the probability of getting tracked into academic high school by 20% (Falk et al., 2020). While Baloo and You and PATHS differ in their content, both interventions are similarly long (as measured in contact hours), target similarly aged children, and have almost identical treatment effects.

The effect of PATHS is substantially smaller than effects of U.S. preschool programmes. The PATHS effect is about one-third of the effect size of the Perry Preschool Program on high school completion (Barnett, 1995; Heckman et al., 2010a) and about one-sixth of the effect of the Abecedarian programme on college attendance (Campbell et al., 2002). These studies might find larger effects because they are more time- and resource-intensive and target disadvantaged populations.

Comparison of costs: We complement our effect size comparison with a comparison of costs. This comparison is difficult because information on costs is sometimes missing and sometimes, like in the case of teacher salaries, very context dependent. Therefore, the following analysis should be interpreted with caution.

Figure 14 shows the costs of interventions for which this information is available. The total intervention cost per child refers to all costs over the intervention period, excluding evaluation costs. These costs are in nominal USD. The implementation of PATHS in Zurich cost USD 1,540 per class and USD 67 per child. The main cost of implementing PATHS stems from the teachers’ training and the material for PATHS activities, for example, teaching folders, posters, books, and feeling cards. The Baloo and You intervention costs USD 1,266 per child (Péron and Baldauf, 2015). The BAM intervention costs USD 1,475 per child (Heller et al., 2017). The socio-emotional skills and parenting training implemented as part of the Montreal Longitudinal Study costs USD 4,038 per child (Algan et al., 2022). The Pathways mentoring and tutoring programme costs USD 10,100 per child (Oreopoulos et al., 2017). In light of their substantial treatment effects, all these interventions seem cost-effective. However, PATHS stands out as remarkably low-cost.

PATHS is also substantially less expensive than early childhood education programmes like the Perry Preschool Program or the Abecedarian project. The Perry Preschool Program costs USD 10,000 per child (Web Appendix of Heckman et al., 2010b). The Abecedarian programme costs USD 13,400 per child (Campbell et al., 2014). These striking cost differences reflect that the Perry Preschool Program and the Abecedarian programme are high-intensity interventions targeted at particularly disadvantaged populations.

10. THE VOLTAGE EFFECT—SCALABILITY OF THE PATHS INTERVENTION

In this section, we discuss the scalability of the PATHS programme, following List's (2022) five-step discussion guideline that addresses key concerns for scaling.

False positives: A general concern regarding scalability is that an effect may disappear once a programme is rolled out because it was a false positive—a chance finding. This concern also applies to our study because we examine multiple outcomes, including education outcomes over time and mechanisms like changes in socio-emotional skills. However, the broad range of analysed outcomes makes it difficult to attribute our findings solely to chance. Additionally, we pre-registered (AEARCTR-0003496) our intention to analyse academic high school attendance, grades, and socio-emotional skills when applying for data access. Nevertheless, to address this concern, we conducted multiple hypothesis testing in Section 7.5. Our results remain robust with respect to multiple hypothesis testing, indicating that false positives do not pose a threat to the scalability of the PATHS intervention.

Audience representativeness: Scalability is also challenged if the initial population that is exposed to the intervention differs from the population that receives the intervention when the programme is scaled up. Our intervention sample consists of fifty-six Swiss primary schools in Zurich. These schools were randomly selected and mandated to participate in the field experiment and data collection. This sampling procedure suggests that our sample is representative of all schools and students in Zurich. As there was no selection into the evaluation sample at the student, teacher, or school level, we believe that concerns about audience representativeness are minimal.

At a broader level, it is reasonable to ask how representative the student population in Zurich is for Switzerland or Europe. Table 1 highlights that our sample is quite diverse. A significant percentage of mothers (57.7%) were not born in Switzerland, and 38.5% come from another European country. Treatment effects are visible for both native and foreign children. This evidence mitigates concerns regarding audience representativeness.

One additional potential concern is that the tracking-based education system in Switzerland is distinct, and we may not expect similar intervention effects in other countries. However, Figure 15, Panel (a), illustrates that other European countries have similar school tracking systems. Several countries including Austria, Germany, and the Netherlands use tracking systems akin to the Swiss one. Hence, we believe that our results are applicable to various settings outside Switzerland. Moreover, in addition to treatment effects for tracking, we find significant reductions in ADHD symptoms—even before tracking occurred. These results indicate that the intervention has positive effects independent of school tracking.

Unscalable ingredients: Some interventions include elements that cannot easily be replicated. The primary ingredient for the PATHS intervention is teacher training and the introduction of a new school subject. According to the coordinators of the Swiss programme, PATHS has been successfully scaled up. In recent years, the PATHS programme has been rolled out in Switzerland. To date, around 40,000 children in Switzerland have participated in the programme since the initial experiment that we evaluate. Therefore, we believe that PATHS does not include unscalable components.

Have similar programmes been scaled in other settings? According to the 2018 European Union Report on SEL by Cefai et al. (2018), most education systems do not have a dedicated subject devoted to socio-emotional skill development. According to Cefai et al. (2018), Ireland and Malta are the only countries in Europe with a country-wide distinct SEL subject. In many other European countries, SEL is not a distinct subject but rather is included in other subjects such as citizenship, health and physical education, prevention of violence and bullying, moral/religious education, and art and crafts (OECD, 2015; Torrente et al., 2015). In Finland, for example, “Growth as a Person” is a cross-curricular theme dedicated to social and emotional education and applied in all subjects.

In which settings has the PATHS programme been implemented? Panel (b) in Figure 15 illustrates that the PATHS programme has been implemented in primary schools in various countries. In sum, given that the PATHS intervention was adapted from the U.S. context and is currently widely used worldwide, we do not believe that it contains unscalable components.

Cost traps: As discussed in Section 9.2, the PATHS programme is relatively cost-effective. While scaling a programme can sometimes result in unforeseen costs that diminish the attractiveness of the intervention, our framework suggests that this risk is minor. In fact, according to the developers, implementation costs for the PATHS intervention decreased after the initial experiment. Following the initial costs of adopting the material to the Swiss context, costs have remained relatively stable over the past decade. Thus, we do not believe that cost traps represent a threat to the scalability of the PATHS programme.

Negative spillovers: Negative spillovers introduced by the rollout of interventions can also hinder scalability. Our findings raise questions about whether we would observe similar tracking effects if every child in Switzerland participated in the programme. These general equilibrium effects are ex ante unclear and depend on whether academic high schools would admit more students if the programme made all students more qualified. While academic high schools do not have explicit capacity constraints or quotas, there is a strong belief in Switzerland that these schools should remain selective.

To determine whether we would observe the same treatment effects on tracking if the entire population were treated, we can examine the year-to-year variation in the number of students admitted to academic high school.³² Figure 16 shows the year-to-year percentage change in the number of admitted students at the school level, revealing substantial variation in student admissions. The average year-to-year deviation in the school-level number of admitted students is 20 percentage points. Considering that we find treatment effects of 20–25% on the control group mean of 20%, it is feasible that academic high schools could admit all the additional students “pushed” by the intervention. Therefore, it is unlikely that large parts of the treatment effects would be absorbed by the general equilibrium effects of a nationwide rollout.

While the ultimate general equilibrium effects for tracking remain somewhat ambiguous, the reduction in ADHD symptoms, improvements in classroom behaviour, and enhancement of other socio-emotional skills represent important treatment effects. These changes benefit students, parents, and teachers, regardless of whether they lead students to a more prestigious secondary school track.³³

11. CONCLUSION

This paper provides experimental evidence that fostering socio-emotional skills in primary school children has persistent positive effects on educational careers. We provide evidence on the PATHS programme, a teacher-run intervention that lasts for up to two years in primary school. The intervention increases the probability of completing academic high school and enroling or graduating from university seventeen years after the intervention.

Our results on underlying mechanisms suggest that the PATHS treatment effect is mainly driven by changes in some of the socio-emotional skills targeted by the intervention. Treated children become less impulsive, less disruptive, and display less opposition to teachers and parents. In class, treated children become less likely to disturb lessons and more likely to focus on the teaching content. Although we find suggestive evidence that treated children have better grades, we find no evidence that standardized test scores are affected by the intervention. Long-term effects thus seem more likely to operate through changes in socio-emotional skills rather than cognitive skills.

Taken together, the results of this study raise an interesting and policy-relevant question. Would it be possible to teach children socio-emotional skills with a subject that is explicitly dedicated to it, similar to the way math and reading are taught? While it has been shown that teachers have lasting impacts on behaviour (Chetty et al., 2011; Jackson, 2018), there is no school subject explicitly designed to foster socio-emotional skills. The results of this study suggest that primary schools are a promising place to institutionalize socio-emotional skills training.

Acknowledgments

We thank the editor Katrine V. Løken and four anonymous referees for their guidance and suggestions. We also thank Caroline Chuard, Jan Feld, Mark T. Greenberg, Jonathan Guryan, James Heckman, Rahel Jünger, Nicolás Salamanca, and Ursina Schaede for helpful discussions and comments. Matthew Bonci, Maximilian Mähr, Albert Thieme, and Jeffrey Yusof provided outstanding research assistance. We gratefully acknowledge financial support from the Swiss National Science Foundation (Grants 405240-69025, 100013_116829, 100014_132124, 100014_149979, 100014_149979, 10FI14_170409, 10FI14_198052), the Jacobs Foundation (Grants 2010-888, 2013-1081-1), the Jacobs Center for Productive Youth Development, the Swiss Federal Office of Public Health (Grants 2.001391, 8.000665), the Canton of Zurich's Department of Education, the Swiss Federal Commission on Migration (Grants 03-901 (IMES), E-05-1076), the Julius Baer Foundation, and the Visana Foundation.

Manuel Eisner and Denis Ribeaud designed the z-proso RCT study, organized all data collection, and provided feedback on the manuscript. Giuseppe Sorrenti and Ulf Zölitz analysed the data, collected additional data, and prepared the manuscript. The author order of Sorrenti and Zölitz has been randomized following Ray $◯\mathrm{r}$  Robson (2018).

Supplementary Data

Supplementary data are available at Review of Economic Studies online.

Data Availability Statement

The data and code underlying this research are available on Zenodo at https://doi.org/10.5281/zenodo.10797136

REFERENCES

Footnotes

Author notes