Assessing the Pragmatic Effectiveness and Implementation of Click City: Tobacco: A School-Based Prevention Program Targeting Youth Cigarette and E-cigarette Use

Abstract

Introduction

Given the rapid increase in the prevalence of e-cigarette use among youth, we updated Click City: Tobacco, an existing, efficacious, online tobacco prevention program for 5th graders with a 6th-grade booster, to also target e-cigarette use.

Methods

To evaluate the effectiveness of the updated 5th-grade program within a “real world” setting, we conducted a pragmatic randomized trial with 5th-grade students in 43 schools across Arizona and Oregon, assessing change in intentions and willingness to use e-cigarettes/cigarettes, from baseline to one-week post-intervention. Students in intervention schools (n = 1327) received the updated version of Click City: Tobacco; students in control schools (n = 1346) received their usual tobacco prevention curriculum.

Results

Students in intervention schools significantly decreased their intentions and willingness to use e-cigarettes and cigarettes, compared to students in control schools. The intervention also significantly changed all targeted etiological mechanisms predictive of intentions and willingness. The intervention was more effective for at-risk students, as defined by student’s previous tobacco use, current family use of tobacco, and/or high in sensation-seeking. The effects of the intervention on all outcomes were similar as a function of state, gender, ethnicity, and historical timing (before COVID-19 pandemic school closures vs. after schools re-opened). Close to 90% of the students in the intervention condition completed the entire program, supporting fidelity of implementation, and teachers reported satisfaction with the program.

Conclusions

Findings suggest that Click City: Tobacco is effective for all 5th-grade students and can be delivered with fidelity across school settings.

Implications

The results of a randomized pragmatic effectiveness trial showed that the updated Click City: Tobacco program decreased intentions and willingness to both vape e-cigarettes and smoke cigarettes among 5th-grade students, particularly for those at high risk. Program effectiveness and lack of differences because of factors such as state, gender, ethnicity, and historical timing have universal implications, suggesting that all students can benefit from this program. Click City: Tobacco received high ratings of program satisfaction from teachers and was implemented with fidelity. Findings suggest that Click City: Tobacco is effective and can be easily implemented in schools.

Introduction

In 2014, the prevalence of e-cigarette use (e-cigs, vapes) among youth surpassed that of other forms of nicotine use and has continued to increase rapidly.^1–3 Prior to this increase, we developed and evaluated an online, school-based cigarette and smokeless tobacco prevention program for 5th graders, with a booster in 6th grade (CA098555; 2004-2011).^4,5 A randomized efficacy trial showed that Click City: Tobacco was efficacious in changing behavioral intentions and willingness⁵ to use conventional and smokeless tobacco products as well as all in changing targeted etiological factors, both in the short-term (one-week post-intervention) and long-term (two years post-intervention).^4,5

The increasing prevalence of e-cigarettes among youth suggests an urgent need to focus on e-cigarette use, in addition to other tobacco products, in tobacco prevention programs.³ Thus, we updated the content of both the 5th-grade program and 6th-grade booster of Click City: Tobacco to include e-cigarette prevention. The goal of the updated program was to decrease intentions and willingness to use e-cigarettes among youth and to change etiological mechanisms predictive of this use. In this paper, we present the findings from the evaluation of the updated 5th-grade program of Click City: Tobacco using a pragmatic effectiveness trial design.

Harm Associated With E-Cigarette Use

An increasing body of evidence shows both the acute⁶ and more long-term health consequences of e-cigarette use,^7,8 particularly among adolescent users, as nicotine use during this period has been associated with neurological effects on the developing brain.⁹ Further, adolescents become addicted to nicotine much more quickly than adults.¹⁰ However, among those who use e-cigarettes, e-cigarette products are viewed as less addictive than cigarettes.¹¹ In addition, several studies suggest that among never-smokers, e-cigarette use increases the probability of subsequent cigarette smoking.¹²

Timing of the Delivery of Tobacco Prevention Programs

Delivering a tobacco prevention program prior to tobacco use onset is essential for reducing the use of these products. Findings from epidemiologic studies show that the use of tobacco increases dramatically between elementary (ages 11 or 12) and middle school (ages 12 or 13)^13,14 and a significant proportion of adolescents initiate e-cigarette use by 8th grade (age 14).¹⁵ Delaying onset is imperative as the age of nicotine initiation is a significant predictor of continued use and dependence.¹⁶ Further, use in youth is related to the extent of use in adulthood.¹⁷ Based on this evidence, Click City: Tobacco was designed for delivery in schools in the 5th grade, with a booster in the 6th grade.

Precursors of Tobacco Initiation by Youth

The primary goal of both the original and updated version of Click City: Tobacco is to decrease behavioral intentions and willingness to use tobacco. It is theorized that behavioral intentions are the first step in the initiation of a tobacco product,¹⁸ and changing intentions lead to changing behavior.¹⁹Willingness measures openness to risk-associated opportunities,²⁰ and is less planful than intentions. Both intentions and willingness to use a nicotine product increase developmentally across adolescence and are predictive of subsequent nicotine use.²¹

Activities for both the original and updated programs were designed to target specific etiological mechanisms, or risk factors, that are theoretically and empirically predictive of intentions, willingness, and subsequent use of a nicotine product. These include social images and subjective norms, as well as risk perceptions of both health consequences and addiction. Social images (prototypes; favorable images) and subjective norms (normative social images)²² are integral to the “prototype/willingness” model²³ and the Theories of Planned Behavior and Reasoned Action²⁴ and are empirically associated with intentions, willingness, and subsequent use of both conventional cigarettes and e-cigarettes.^2,21,22 Social images of cigarette/e-cigarette users are perceptions of youth of a similar age who use cigarettes/e-cigarettes. Examples of favorable images are “cool” or “popular.” Favorable normative social images (eg, “cool”) are beliefs that peers have favorable images of youth who use cigarettes/e-cigarettes. Mechanisms based on risk perception and decision research theories include risk of short- and long-term health consequences, risk of exposure to second-hand smoke or vapor, and risk of addiction. Activities targeting risk perceptions were guided by the “affect heuristic” proposed by Slovic et al.^25,26 which is based on an inverse relation between the perception of risk and the positive effect toward a product. Since e-cigarettes are commonly viewed as not harmful⁶ and less addictive than conventional tobacco,¹¹ activities targeting the risk of health consequences and addiction are essential in an e-cigarette prevention program.

Development of an Evaluation of the Original Version of Click City: Tobacco

The development of each activity in the original program was iterative, consisting of feedback from 5th and 6th graders via both focus and user groups, followed by empirical evaluation of components in the lab (one or two activities).⁴ Only components that significantly changed the targeted mechanisms^4,27 were included in the final program.

To rigorously evaluate the original program, we conducted a randomized efficacy trial with schools randomized to two arms, In the intervention group, 1168 students within 13 middle schools and 23 feeder elementary schools received both the 5th and 6th-grade programs. In the control group, 1154 students within 13 middle schools and 24 feeder elementary schools received their usual tobacco prevention curriculum. Students in intervention schools completed three assessments: baseline, 5th-grade follow-up (one week following the intervention), and 7th-grade follow-up (one year following the 6th-grade booster). Students in control schools were yoked with students in intervention schools in terms of timing of assessments, to control for history. Students in intervention schools reported significantly lower intentions and willingness to smoke conventional cigarettes at both the short-term (5th grade)⁴ and long-term (7th grade)⁵ follow-ups than did students in control schools. In addition, all etiological mechanisms changed in the expected direction, and changes were maintained across two years for students in the intervention group, in contrast to the control group. To ensure that the original version was effective with “high-risk” students, we examined the moderating effect of risk factors on outcomes.²⁸ “High-risk” students had family members who smoked,²⁹ had previously experimented with cigarettes,³⁰ and/or were high in sensation-seeking.³¹ Our results suggested that “high-risk” students were more responsive to the Click City: Tobacco intervention compared to students less at risk.

Pragmatic Effectiveness Trial

To evaluate the updated 5th grade Click City: Tobacco program, we conducted a pragmatic effectiveness trial in a “real world” school setting. We chose this design over one that focused primarily on the systematic comparative evaluation of varying methods of implementation, such as an effectiveness implementation design.^32,33 An effectiveness implementation design requires a controlled environment, rather than a “real world” setting, with outcomes focused on the different methods of implementation rather than intervention effectiveness. Several researchers have recommended a pragmatic effectiveness trial design, which includes an evaluation of implementation, as well as effectiveness, as a next step following an efficacy trial,³⁴ and prior to program dissemination.³⁴ Therefore, Click City: Tobacco was evaluated in schools, using school personnel, rather than research staff, to implement the program and facilitate data collection.

Methods

Description of the Updated Click City: Tobacco Program

We updated the 5th-grade program through (1) developing two new activities (components) that targeted only e-cigarettes; (2) modifying three activities to target e-cigarette use instead of conventional tobacco use (ie, cigarettes or smokeless tobacco); and (3) modifying 11 activities to target e-cigarettes in addition to conventional tobacco. Updates to the 6th-grade booster consisted of modifying activities to target e-cigarettes in addition to conventional tobacco. Unique samples of grade-appropriate students participated in focus and user groups to review all new and updated activities, and appropriate modifications were made. As in our prior research,⁴ new 5th-grade activities were empirically evaluated in the laboratory with new samples of 5th graders and only activities that changed targeted mechanisms were included in the final program. Tables S1 and S2 describe the activities and delineate the modifications made to the 5th-grade program and 6th grade booster.

Design of the Pragmatic Effectiveness Trial

Because of time constraints, only the updated 5th grade program was evaluated in schools. In the fall of 2019, we recruited 44 schools, stratified them by state (Oregon, Arizona) and size (small, medium, large), yoked them according to their desired start dates, and randomized them to intervention (received the Click City: Tobacco program) or control (implemented their standard curriculum). These two states are diverse in terms of race and ethnicity. According to the 2020 census, 60.4% of the population of Arizona were “white alone” and 30.7% were Hispanic. In Oregon, 74.8% were “white alone” and 13.9% were Hispanic. The trial began in January of 2020. By March of 2020, 26 schools had completed their participation. At that time, schools closed because of the COVID-19 pandemic and the trial was suspended. When schools re-opened in 2022, we re-recruited nine schools from the original sample as well as eight new schools and used the same method as described above to randomize schools to condition. A total of 17 schools completed the trial between January and June of 2022. Of the 44 anticipated schools, 43 completed the trial, with 22 schools assigned to intervention and 21 to control.

Procedures

Oregon Research Institute’s Institutional Review Board (IRB) approved all research activities before the start of the trial (IRB Registration No. 00000278). A passive consent procedure was used to recruit students within schools as personal data identifying students was not collected. Parents of 5th graders were sent a letter and a form with a prepaid return envelope if they did not want their child to participate in data collection. The decline rate was 3.8%.

A volunteer school “liaison” (eg, a teacher) facilitated the trial and assessments in each school. A project Research Assistant (RA) provided an overview of the program and offered assistance should any issues arise. The RA sent a link to the program and teacher instructions to intervention schools one week prior to the start of the program. The 5th-grade program consisted of eight 20- to 25-minute lessons, designed to be delivered over four weeks, with two lessons per week. Each student worked on the online program individually, with students progressing through each lesson and the entire program in a systematic, sequential fashion. The days of the week and time of day that students worked on the program varied by school. No information was given to control schools regarding the provision of tobacco education. All assessments were completed during the school day on school computers. Students in the intervention schools completed a baseline survey one week prior to starting the program and a follow-up survey one week following completion of the program. Students in yoked control schools completed assessments at the same time as students in intervention schools.

Measures

A detailed description of each measure is in Table S3. Outcomes included the behavioral intention to smoke cigarettes (r = 0.68) and vape e-cigarettes (r = .74) and willingness to smoke (α = 0.86) or vape (α = 0.92) if the opportunity presented itself. Etiological mechanisms included social images, normative beliefs (normative social images and friends’ approval), risk perceptions of health consequences, and risk perceptions of addiction. To assess social images of youth who smoke (α = 0.78) or vape (α = 0.81), students indicated what they “think kids . . . are like”³⁵ and to assess normative social images of youth who smoke (α = 0.79) or vape (α = 0.83) students indicated their perceptions of what other 5th graders think. Friend’s approval of the student smoking cigarettes (α = 0.76) or vaping e-cigarette (α = 0.81) is an indicator of subjective norms. To assess the risk of health consequences, students rated their risk of exposure to second-hand smoke (α = 0.84) and vapor (α = 0.88), their risk of the cumulative consequences over time from smoking (α = 0.93) and vaping (α = 0.93), and their risk of smoking each cigarette (r = 0.15) and vaping each e-cigarette (r = 0.17). To assess the risk of addiction, students rated their personal risk of addiction from smoking (α = 0.93) and vaping (α = 0.93), their perception of personal control over quitting smoking (α = 0.89) and vaping (α = 0.89), and their personal risk of difficulty of quitting smoking (α = 0.90) and vaping (α = 0.90).

Moderators included sensation-seeking, family smoking, or vaping and prior experimentation with smoking conventional cigarettes or vaping. Sensation-seeking was measured using the Brief Sensation-seeking Scale³⁶ (α = 0.75). Family smoking or vaping was measured by student’s report of family members’ smoking conventional cigarettes or vaping (stability = 0.81). Prior experimentation with smoking conventional cigarettes or vaping was measured by student self-report (stability = .75).

Program Evaluation and Implementation

Teachers rated their own and their students’ satisfaction with the 5th grade program. They also rated the ease/difficulty of using the program, how likely they were to use it again, and their likelihood of recommending it to others, using a 5-point scale. The program tracked when students logged onto the program and the number of activities completed within each session.

Analysis Strategy

To maximize the sample, study enrollment consisted of students who completed the baseline and/or the follow-up survey in 5th grade. Missing data was estimated using full information maximum likelihood methods. To assess the effectiveness of the intervention in changing the outcomes and etiological mechanisms in the hypothesized direction, the data were analyzed using a general linear mixed (GLM) model with students nested within schools and schools nested within conditions, using SPSS, Version 24. Since schools were randomized to condition, both the student and the school were random effects. To control for the significance difference between conditions as a function of students prior experimentation with smoking conventional cigarettes or vaping, we used analysis of covariance within GLM with prior experimentation as the covariate. To assess if the intervention was effective across demographic groups and as a function of whether the student was “high-risk,” we examined the significance of the two-way interaction between condition and each demographic variable and risk factor. Significant interactions were decomposed using simple main effects for each condition. To assess the teacher’s evaluation of the program and the extent of the program completed by students, we used descriptive statistics. We examined the dose/response relation between program completion and outcomes by comparing those who completed the entire program with those who did not using GLM with students nested within schools.

Results

Participants

Results are based on the evaluation of the 5^th-grade program. Study enrollment (N = 2673) was defined as completing only the baseline survey (n = 332), only the follow-up survey (n = 218), or both surveys (n = 2123). Among those enrolled, 1327 students were in the intervention condition and 1346 were in the control condition.

Differences in demographics between conditions are based on the baseline sample of 2448 students (see Table 1). There were no significant differences between conditions in gender, ethnicity, race, sensation-seeking, or family use. However, significantly fewer students from Arizona were in the Control Condition than in the Intervention Condition. Additionally, significantly more students in the control condition reported prior experimentation with conventional cigarettes or e-cigarettes compared to those in the intervention condition.

Differences Between Conditions in Outcomes and Etiological Mechanism

Outcomes: Intentions and Willingness

Students in the intervention schools significantly decreased their intentions and willingness to use cigarettes and e-cigarettes in the future (see Table 2). In contrast, both intentions and willingness increased for students in control schools. Effect sizes for change in intentions and willingness were small,³⁷ possibly because of a floor effect, restricting range. Controlling for student prior experimentation with conventional cigarettes or e-cigarettes, the effects of condition remained significant (willingness to smoke, F(2,90.7) = 5.68, p < .01; willingness to vape, F(2,91.6) = 4.95, p < .01; intention to smoke, F(2,81.2) = 7.19, p < .001; intention to vape, F(2,94.8) = 5.00, p < .01).

Etiological Mechanisms

All etiological mechanisms targeted in the program changed in the hypothesized direction (see Table 2). Favorable social images, normative social images, and perception of friend’s approval significantly decreased for students in intervention schools as compared to students in control schools. Perceived risks of health consequences and addiction significantly increased for students in the intervention schools as compared to those in the control schools. Perceived control over smoking and vaping decreased in intervention schools as compared to control schools. For most risk factors, effect sizes were moderate to large.

Moderators

To assess if the effect of the intervention varied as a function of demographics (ie, gender, ethnicity, state of residence), historical timing, or as of function of each at-risk variable (ie, family tobacco use, students’ prior experimentation with smoking conventional cigarettes or vaping e-cigarettes, and sensation-seeking), we examined the effect of the interaction of condition with each of these variables on major outcomes. For this analysis, we trichotomized sensation-seeking into three groups: (1) high sensation seekers (ie, +1 SD above the M; 12.0); (2) low sensation seekers (−1 SD below the M; ≤ 10.01); or (3) moderate sensation seekers (between + 1 and − 1 SDs from the M; >10.01 and <12).

The interactions between conditions and demographics and historical timing were all non-significant. We also did not find significant interaction effects between each at-risk moderator and condition for willingness to smoke or intentions to vape or smoke. However, for willingness to vape, the interactions between conditions and each of the three moderators were significant: family used tobacco/nicotine, F(1,220) = 6.39; p < .05; student prior experimentation, F(1,1143) = 8.81, p < .01; and sensation-seeking, F(2,303) = 3.22, p < .05). We decomposed these significant interactions using simple main effects for each condition. We used orthogonal analyses to decompose the interaction with sensation-seeking: (1) students who were high in sensation-seeking versus those who were low or moderate; and (2) students who were low in sensation-seeking versus those who were moderate (see Table 3). “High-risk” students had a significantly higher willingness to vape at baseline across conditions. Willingness to vape decreased significantly for these “high-risk” students in the intervention condition, but the difference was not significant for students in the control condition.

Teacher Evaluation and Program Implementation

Teacher Evaluation

Of the 53 out of 61 teachers who completed the evaluation, 75% (n = 40) were either moderately or extremely satisfied with the program, and 87% (n = 46) indicated that their students were moderately or extremely satisfied with the program. Teachers stated that the program was engaging and that students enjoyed what they learned. Most (83%; n = 44) rated the program as moderately or extremely easy to implement. Finally, 78% (n = 49) would be likely to use it again and would recommend it to other teachers and/or schools. Comments related to teacher dissatisfaction were primarily related to technological issues (which were subsequently resolved) and time constraints.

Program Implementation

Among the 1327 participating students in the intervention schools, 1305 students (98.3%) logged into the program, and 1139 (85.8%) completed all 23 program activities. Within the intervention condition, we linked program completion with assessment data. Only data for students for whom we had matching ID numbers for both the surveys and the program were included in the analysis (n = 1172 of 1305 students; 89.8%). Among the 1172 students, 1032 (88.1%) completed all 23 activities, and 140 (11.9%) completed at least one, but fewer than 23, activities. These 140 students completed an average of 16 out of 23 activities, and 88% completed more than half of the program.

We compared the students in the intervention condition who completed all 23 activities with the students who completed less than 23 activities on outcomes and etiological mechanisms. As shown in Table 4, although differences in major outcomes were in the expected direction, they were not significant. However, we did find several differences in etiological mechanisms. Among those who completed the entire program, favorable social images and normative social marginally decreased, as compared to those who did not complete the program. In addition, among those who completed the entire program, as compared to those who did not, the perceived risk of smoking/vaping each cigarette/e-cigarette significantly increased and perceived difficulty in quitting smoking (but not vaping) marginally increased.

Discussion

The results of the randomized pragmatic effectiveness trial of the updated version of the 5th grade of Click City: Tobacco program showed that this program is effective, as 5th-grade students who received the program decreased their intentions and willingness to smoke and vape. In addition, this program significantly modified all etiological mechanisms targeting both smoking and vaping in the expected direction, with moderate to large effect sizes.³⁷ A recent review of school-based preventive interventions targeting e-cigarette use³⁸ shows limited evidence indicative of vaping prevention. However, several well-designed trials of theory-driven e-cigarette prevention programs that meet the criteria for “evidence based”³⁹ are in different stages of development and evaluation.^40–42 Thus, Click City: Tobacco is one of the first school-based conventional cigarette and e-cigarette prevention programs that has been shown to be effective using a rigorous experimental design and reproducible methods.

Findings support Click City: Tobacco as a universal prevention program. We found no differences in outcomes as a function of state of residence, gender, ethnicity, and historical timing (2020 vs. 2022). The program was universally effective for both “high-risk” students and those not at “high-risk” with the exception of one outcome, willingness to use e-cigarettes. Following the intervention, “high-risk” students were less willing to vape than other students. One possible explanation for this finding is based on the student's implicit attitudes toward vaping. In laboratory studies, we found that “high-risk” students had more favorable implicit attitudes toward smoking than did low-risk students.⁴³ We also found that several activities within the original version of Click City: Tobacco, particularly those targeting risk perceptions, in addition to increasing student's risk perceptons decreased their implicit attitudes toward smoking.⁴³ These activities in the updated version could potentially have decreased “high-risk” students’ implicit attitudes toward vaping in addition to increasing their perceived risk perceptions and decreasing their willingness to vape.

The fidelity of program implementation of Click City: Tobacco was high; with 85% of the students completing the entire program. This far exceeds the fidelity of implementation of other school-based prevention programs.^44,45 For example, an evaluation of the implementation of the Life Skills Training Program showed that 60% of the students did not receive all components of the program,⁴⁴ and after one year of training, only 25% of teachers maintained fidelity.⁴⁶ In another school-based pragmatic trial, only 12 out of 29 of the intervention schools implemented a substance use prevention program.⁴⁵ Several other pragmatic school-based trials of substance use programs have been conducted, but. to our knowledge, no information was provided regarding the level of implementation,⁴⁷ or evaluations were not based on objective criteria.⁴⁸ For Click City: Tobacco, the online method of program delivery, student engagement with the program, and ease of implantation by school staff may all be responsible for successful implementation in a real-world setting.

Limitations

Despite the strengths of the pragmatic effectiveness trial, there were limitations. First, participation in the trial was limited to Arizona and Oregon, limiting the generalizability of results. Although the inclusion of these two states contributed to the diversity of students, particularly the relative proportion of Hispanic/Latino students, the proportion of Black students across the two states was low. Thus, our findings may not generalize to a large segment of American youth. Second, the trial did not assess the effect of the 6th grade booster. However, the 6th-grade booster updates were formatively evaluated by focus and user groups. Third, we did not measure student conventional tobacco or e-cigarette use at follow-up. In the present study, this would require a measure of current use, such as use in the last 7 days, which in 5th grade has an extremely low prevalence resulting in a restriction of range. Since the ultimate outcome is prevention of use, it is imperative that a future evaluation includes a longer follow-up period to evaluate the effect of the program on use. Fourth, students did not report on their own satisfaction with the program; student satisfaction was reported by teachers. Fifth, we did not collect data regarding existing tobacco prevention curriculum delivery within the control schools. However, informally we found that the tobacco education curriculum was not consistently delivered or evidence-based.

Additional Strengths of the Program

Click City: Tobacco has advantages over more traditional school-based prevention programs. First, it translates theory into practice, as the targeted mechanisms are theoretically based and empirically supported. Second, using effective experimental procedures,⁴ each component was empirically evaluated as effective at changing the targeted mechanism(s) in the laboratory before inclusion in the final program. Including only effective components may improve the effectiveness of the overall program.⁴ Third, the interactive design of the program encouraged active student engagement. In a meta-analysis of 120 school-based drug prevention programs, Tobler and Stratton⁴⁹ identified interactive (vs. didactic) teaching formats as being the most effective. Click City: Tobacco also provides an ideal environment for students to engage in “experiential learning,” an important method of behavior change.²⁵

Implications of Pragmatic Effectiveness

As noted by Marchand et al.,³⁴ pragmatic effectiveness is necessary for dissemination. Also supportive of dissemination was the satisfaction with the program reported by teachers. Further, the proportion of students who completed the program was exemplary as it exceeded that of most school-based prevention programs implemented in a real-world setting. These factors portend well for the successful dissemination of the updated Click City: Tobacco program. Results of this trial suggest that Click City: Tobacco may be superior to existing tobacco prevention curricula targeting e-cigarettes as well as conventional tobacco.

Future Development and Evaluation Plans

We plan to develop a culturally adapted version of Click City: Tobacco in Spanish for students for whom Spanish is their primary language. Following the development of the Spanish adaptation, we plan to conduct a pragmatic effectiveness trial of both the English and Spanish versions of the program, assessing both short- and long-term outcomes. Plans are to assess long-term outcomes to attain an estimate of changes in e-cigarette and conventional tobacco use as a function of the intervention. This future trial will also include a systematic assessment of the tobacco prevention curriculum in control schools. We plan to implement this trial in several states, further expanding the diversity of the sample and the generalizability of our findings.

Supplementary Material

Supplementary material is available at Nicotine and Tobacco Research online.

Acknowledgments

The authors wish to thank Emilia Yessenya Barraza Perez in Arizona and Jessica Liu in Oregon for assisting with the trial, answering school staff questions, and assuring that student questionnaires and teacher surveys were administered on time, and Linda Jackson for helping with school recruitment in Oregon after schools re-opened. We also thank Missy Peterson for data organization and analyses. A special thanks goes to the staff at InterVision, for developing the original version of Click City: Tobacco, with special thanks to Steve Christianson and Tom Jacobs, for directing and producing the original version and portions of the revised version of Click City: Tobacco. Finally, we greatly appreciate the work done on the revised version of Click City: Tobacco by Twenty Ideas and Influents Innovations. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Cancer, the National Institute of Drug Abuse, or the National Institutes of Health.

Author Contributions

Judy Andrews (Conceptualization [Lead], Funding acquisition [Equal], Investigation [Lead], Methodology [Lead], Project administration [Lead], Supervision [Equal], Writing—original draft [Lead]), Judith Gordon (Conceptualization [Supporting], Funding acquisition [Supporting], Investigation [Supporting], Project administration [Equal], Supervision [Equal], Writing—review & editing [Lead]), Erika Westling (Conceptualization [Supporting], Funding acquisition [Supporting], Investigation [Supporting], Writing—review & editing [Supporting]), and David Smith (Funding acquisition [Equal], Project administration [Supporting], Resources [Supporting], Writing—review & editing [Supporting])

Declaration of Interests

None declared.

Funding

This work was supported by a grant from the National Cancer Institute at the National Institutes of Health (R01 CA98555), and by a grant from National Institute of Drug abuse at the National Institutes of Health (R44 DA04402).

Data Availability

The data underlying this article will be shared on reasonable request to the corresponding author.

References