Evaluation of the Psychometric Properties of Dependence Measures for Exclusive Electronic Cigarette Users

Abstract

Introduction

Extant electronic cigarette (ECIG) dependence measures are largely adapted from those designed for cigarette smoking, though few have been evaluated for their psychometric properties.

Aims and Methods

Never-smoking ECIG users (N = 134) participating in an online survey completed four dependence measures: Penn state electronic cigarette dependence index (PSECDI), e-cigarette dependence scale (EDS-4), diagnostic and statistical manual for tobacco use disorder (DSM-5), and Glover Nilsson behavioral questionnaire (GNBQ). They also reported on their ECIG use characteristics (eg, behaviors and reasons).

Results

Internal consistency was highest for the EDS-4 (Cronbach’s α = 0.88) followed by the GNBQ (α = 0.75), PSECDI (α = 0.72), and DSM (α = 0.71). Confirmatory factor analyses revealed a single-factor structure for the PSECDI, EDS-4, and GNBQ. For the DSM-5, however, two items did not load significantly (ECIG use interferes with responsibilities; reduce/give up activities because of ECIG use). Significant correlations were observed between all measures and the number of ECIG use days/week and/or years using ECIGs, as well as between DSM-5 scores and the number of ECIG quit attempts and initiation age. Endorsement of using ECIGs because “I like flavors” was correlated positively with DSM-5 and GNBQ scores.

Conclusions

All dependence measures evaluated herein demonstrated adequate reliability and construct validity. Future work should focus on determining which aspects of dependence are those that are unique to ECIG use, and subsequently developing a more comprehensive measure of ECIG dependence.

Implications

The measures assessed herein—PSECDI, EDS-4, DSM-5, and GNBQ—demonstrated adequate to good reliability and construct validity among a sample of never-smoking ECIG users. The dependence domains covered across measures were related yet distinct. Findings demonstrate the need for future evaluation of these different domains to determine which are the most salient characteristics of ECIG dependence.

Introduction

Electronic cigarette (ECIG) design has advanced such that many devices have shown capable of delivering nicotine at levels similar to those for cigarette smoking, including some vape pen, mod, and pod-style ECIGs.^1–4 Devices that are efficient at delivering nicotine might be expected to produce dependence, though existing work suggests that ECIG use leads to lower levels of dependence than cigarette smoking.^5–7 Yet the large majority of sampled ECIG users have been former or current cigarette smokers,^6,8–10 and thus preexisting nicotine dependence confounds interpretation of these findings. Another limitation of extant work involves the use of ECIG dependence measures that were adapted from those designed for measuring cigarette dependence.^8,9 The psychometric properties for only a few of these adapted measures have been evaluated, again mostly using samples of ECIG users who also smoke cigarettes.^6,11,12 More work is needed to evaluate the psychometrics of measures used to evaluate ECIG dependence, particularly among ECIG users with minimal history of cigarette smoking.

In dual cigarette-ECIG users, several measures have demonstrated both internal consistency and validity, with corresponding ECIG dependence scores significantly related to ECIG use frequency (amount per day, days/week), time to first vape, use of ECIGs that contain (vs. do not) nicotine, and/or ECIG quit attempts.^6,11,12 These measures include ECIG-adapted versions of the Fagerström Test of Nicotine Dependence (e-FTND), the Wisconsin Inventory of Smoking Dependence Motives (e-WISDM), and the Nicotine Dependence Syndrome Scale (NDSS), as well as the Penn State Electronic Cigarette Dependence Index (PSECDI) and the Electronic Cigarette Dependence Scale (EDS). In the abovementioned work by Morean and colleagues,¹¹ the psychometric properties of the EDS also were examined among a subset of participants who were exclusive ECIG users (but most of whom were former smokers). The pattern of results was similar to that reported for the dual cigarette-ECIG users, with the EDS observed to be consistent and valid (eg, associated with ECIG use frequency and time to first vape).¹¹ The only other work to examine the psychometrics of an ECIG dependence measure among exclusive ECIG users (no current use of other products; former use of other products not reported) focused on an instrument used in the national-level Population Assessment of Tobacco and Health (PATH) study that is an assembly of items selected from four separate dependence scales.^7,13 This measure demonstrated concurrent validity in that ECIG dependence scores were associated with increased frequency of ECIG use (daily vs. non-daily),⁷ but did not demonstrate longitudinal predictive validity when dependence scores were compared to ECIG frequency (daily vs. non-daily), quantity (average number of ECIG uses per day), and quit attempts approximately 1 year later.¹³

A few of these studies also evaluated whether items within a given ECIG dependence measure represent a common underlying construct of nicotine dependence, and cross-study comparisons show that results are mixed. Among dual cigarette-ECIG users, a single-factor structure has been observed for some ECIG dependence measures (e-FTND; e-WISDM; EDS) but not others (NDSS; PSECDI).^12,13 Among exclusive ECIG users, a single-factor structure was observed for the EDS and the PATH-based dependence measure.^7,11 Moreover, the PATH-based items were shown to be a valid measure of nicotine dependence across a range of tobacco products that included cigarettes and ECIGs.⁷ This finding contrasts with that of the NDSS when examined among dual cigarette-ECIG users, with cigarette and ECIG dependence representing distinct factors.⁶ Of course, in addition to differences in the tobacco use histories of the ECIG users included across studies, the measures also may differ in terms of the aspects of dependence that are being evaluated (eg, physical and behavioral).

The current study examined the psychometric properties of four dependence measures among a sample of never-smoking ECIG users. The measures chosen included the (1) PSECDI and, (2) EDS, both of which were developed specifically for measuring ECIG dependence. Also included were two measures that we adapted for ECIG use:, (3) Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5), a more global measure of a tobacco use disorder, and (4) Glover Nilsson Smoking Behavioral Questionnaire (GNBQ), a measure designed to capture behavioral aspects of dependence.

Methods

Participants

Participants were recruited online (eg, Amazon Mechanical Turk, Reddit, University listservs) from November 2020 to May 2021. Those eligible for the study were English speakers, residents of the United States, at least 18 years old, and users of a nicotine-containing ECIG for at least 4 days a week for at least 3 months. Individuals were excluded if they reported smoking more than 100 cigarettes in their lifetime. Of those who met these eligibility criteria, 507 individuals provided informed consent and continued on the survey. Responses from 134 participants were deemed valid (as described below), and thus included in current analyses.

Procedures

All study procedures described below were acknowledged by the West Virginia University Institutional Review Board. Interested individuals were directed to an online screening questionnaire, and those deemed eligible then provided informed consent. Next, they completed a ~30-minute survey via Qualtrics; questions probed ECIG device/liquid characteristics, use patterns, reasons for use, and dependence level. Participants were instructed to answer these questions with regard to their most preferred ECIG device/liquid. At the end of the survey, they were required to upload a clear picture of this preferred device/liquid along with a unique identifier (eg, “ECIG89G” written on a piece of paper) to ensure image authenticity (eg, not copied from the internet). Other data validity checks included the Completely Automatic Public Turing test to tell Computers and Humans Apart (CAPTCHA), a question presented in the form of a picture (“Dog is to puppy as cat is to …”), and two attention check questions (eg, “Please select ‘yes’ for this question”). Participants who completed the survey and passed these checks were compensated ($4 via MTurk or Amazon gift card) and included in the final sample (N = 134).

Measures

Demographic Characteristics and Drug Use History

Participants provided information about their demographic characteristics (eg, age and gender), as well as their ECIG product features (eg, nicotine concentration and adjustable power) and use behaviors (eg, years vaping and vaping days/week) for their preferred device. In addition, participants were asked to “select all that apply” among a list of potential reasons for vaping: Friends/family use, curious, enhanced social status, more acceptable to nonsmokers, less harmful than cigarettes, more affordable, appealing flavors, vape tricks, and other (range = 0–9).¹²

ECIG Dependence Measures

Penn State Electronic Cigarette Dependence Index.

The PSECDI⁹ consists of 10 items that sum to create a total score that ranges from 0 to 20. Higher scores indicate higher levels of dependence (0–3 not dependent, 4–8 low dependence, 9–12 medium dependence, and >13 high dependence).

Electronic Cigarette Dependence Scale (EDS-4).

The EDS-4 consists of 4 items that derive from the Patient-Reported Outcomes Measurement Information System Nicotine Dependence Item Bank for E-Cigarettes (PROMIS-E), which have been validated previously using adult¹¹ and youth¹⁴ samples. All items are presented on a Likert scale that ranges from 0 (never) to 4 (almost always). Items are averaged to create a total score, with higher scores indicating higher dependence.

Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5).

The DSM-5 criteria for tobacco use disorder¹⁵ were adapted to assess ECIG use disorder (ie, the term “tobacco” was replaced with “vape/e-cigarette”). Scores from 11 items are summed to create a total score between 0 and 11; higher scores indicate stronger dependence (0–1 no dependence, 2–3 mild, 4–5 moderate, >6 severe).

Glover Nilsson Behavioral Questionnaire.

The GNBQ¹⁶ is an 11-item scale that assesses the behavioral components (eg, rituals, reactivity to cues) of cigarette dependence. All items were adapted for ECIG use such that terms were replaced as needed (eg, “cigarette” to “vape/e-cigarette”). Items were scored using a Likert scale that ranged from 0 (not at all/never) to 4 (extremely so/always). Higher scores indicated higher dependence levels (<12 mild, 12–22 moderate, 23–33 strong, >33 very strong).

Analytic Plan

Internal consistency and reliability (inter-item and item-total) for each measure were evaluated using Cronbach’s α and Pearson’s bivariate correlations, respectively. Confirmatory factor analysis was conducted for each dependence measure using MPlus V8.6. Model fit was determined using recommended cutoffs for the comparative fit index (CFI; ≥0.90), the Tucker Lewis Index (TLI; ≥0.95), the root mean square error of approximation (RMSEA; <0.08), and the standardized root mean square residual (SRMR; <0.08).^11,12,17 Items were retained if they met statistical significance (p < .05). Finally, Pearson’s correlations assessed construct validity by comparing scores on each dependence measure with (1) scores on the other dependence measures, (2) ECIG use characteristics (eg, vape days/week), and (3) reasons for ECIG use (as in¹²). Statistical significance was set at p < .05 for all analyses.

Results

Participants

Of 507 individuals who were eligible, provided informed consent, and began the survey, 277 (54.6%) uploaded a picture as requested. However, that picture was considered valid for only 136 (26.8%) respondents in that an ECIG was depicted, the ECIG appeared to be a picture of the respondents’ device (vs., for example, a stock photo), and the unique survey code was presented. An additional 2 participants were excluded because they failed to complete all survey items, resulting in a final sample of 134 respondents.

As shown in Table 1, participants consisted of young adults (range 18–44 years) who were primarily female (vs. 38.1% male, 3.7% other, or 1.5% declined to answer) and White (vs. 6.7% more than one race, 5.2% Asian, 4.5% Black, and 0.7% Puerto Rican). They had smoked, on average, one pack of cigarettes in their lifetime and reported near-daily use of ECIGs for the past few years. Thirty-nine (29.1%) participants reported use of another tobacco product in the past month: Cigarillos (12.7%), cigars (5.2%), smokeless tobacco (5.2%), and/or water pipe (3.0%). Most self-reported use of a pod-style device (eg, JUUL and PuffBar) followed by a mod (eg, Smok) or vape pen (eg, Airis). Scores on each dependence scale were indicative of moderate levels of ECIG dependence. Additionally, the most commonly reported reasons for using ECIGs included: family or friends using ECIGs, out of curiosity, available in appealing flavors, and being less harmful than cigarettes. Additional ECIG use characteristics (eg, initiation age, quit attempts) can be found in Table 1.

Internal Consistency and Reliability

Internal consistency was satisfactory for the PSECDI (α = 0.72), DSM-5 (α = 0.71), and GNBQ (α = 0.75) and good for the EDS-4 (α = 0.88) (Nunnally & Bernstein, 1994). The majority of inter-item correlations were significant for all measures except the DSM-5. Specifically, the proportion of significant correlations (p’s < .05) observed was 77.8% for the PSECDI (mean r = 0.27; range = 0.01 to 0.73), 100% for the EDS-4 (mean r = 0.51; range = 0.41 to.66), and 85.5% for the GNBQ (mean r = 0.34; range = 0.04 to 0.59). In contrast, less than half (42.0%) of inter-item correlations were significant for the DSM-5 (mean r = 0.14; range = 0.00 to 0.40). All item-total correlations were statistically significant for the PSECDI (mean r = 0.56; range 0.39 to 0.72), the EDS-4 (mean r = 0.79; range 0.76 to 0.85), and the GNBQ (mean r = 0.62; range 0.51 to 0.75). For the DSM-5, correlations for all but one item (“give up important social, occupational, or recreational activities”) were significant (mean r = 0.45; range 0.21 to 0.60).

Construct Validity

Results for the confirmatory factor analyses are presented in Table 2. For the EDS-4, CFI (0.95) and SRMR (0.04) indicated adequate model fit whereas TLI (0.85) and RMSEA (.18) indicated poor model fit. Fit indices were adequate for all other measures: PSECDI (CFI = 0.98; TLI = 0.97; RMSEA = 0.04; SRMR = 0.13), GNBQ (CFI = 0.94); TLI = 0.90; RMSEA = 0.08; SRMR = 0.07), and DSM-5 (CFI = 1.0, TLI = 0.93; RMSEA = 0.04; SRMR = 0.11). Factor loadings were significant (p’s < .05) for all items on the PSECDI, EDS-4, and GNBQ, and suggested a single-factor structure for each. For the DSM-5, however, two items did not display significant factor loadings: “Does your ECIG use interfere with your ability to fulfill major role obligations at work, school, or home?” and “Do you reduce or give up important social, occupational, or recreational activities because of ECIG use?.” Also, the model fit did not significantly improve when these two items were removed (CFI = 0.98; TLI = 0.97; RMSEA = 0.03; SRMR = 0.10). A 2-factor confirmatory factor analysis was conducted to examine whether these items loaded onto a separate factor (CFI = 0.95; TLI = 0.93; RMSEA = 0.03; SRMR = 0.11); however, neither loaded significantly (loadings were 0.38, p = .31 and .69, p = .29, respectively).

Table 3 shows intercorrelations between dependence measures, all of which were significant. The strongest correlation was observed between the EDS-4 and the GNBQ (r = 0.73), while the weakest correlation was observed between the EDS-4 and the DSM-5 (r = 0.51). Shown in this same table are correlations between dependence measure scores and ECIG use behaviors. All dependence scores were correlated positively with the number of ECIG use days per week (p’s < .01), and scores for the PSECDI and the EDS-4 were correlated positively with the number of years vaping (p’s < .01). Also, DSM-5 scores were correlated positively with the age of ECIG initiation and the number of ECIG quit attempts (p’s < .01). Finally, Table 3 shows correlations between dependence scores and reasons for vaping. Significant correlations were observed only for one item, “I like flavors”; this item was correlated positively with DSM-5 and GNBQ scores (p’s < .05).

Discussion

Existing measures of ECIG dependence are adapted from those created for cigarette dependence, and work evaluating their psychometric properties has relied almost exclusively on samples of ECIG users who are former and/or current cigarette smokers.^6,11,12 The current study examined the properties of four dependence measures—PSECDI, EDS-4, DSM-5, and GNBQ—in a sample of ECIG users with minimal cigarette smoking experience. Internal consistency and reliability were strongest for the EDS-4 and weakest for the DSM-5, though at least adequate for all measures. Also, all measures revealed a single-factor structure, with the caveat that two DSM-5 items were revealed to be poor indicators of ECIG dependence. Moreover, scores on some or all measures were correlated with only a few reasons for (ie, appealing flavors), and/or behaviors associated with (eg, frequency and duration of use), ECIG use.

The finding that a single-factor structure was observed for all measures parallels that reported elsewhere for several dependence measures,^7,12 including the EDS-4 among both never-smoking and current-smoking ECIG users.¹¹ In contrast, previous work with dual cigarette-ECIG users suggests that the PSECDI items do not represent a common underlying construct of dependence.¹² Future work is needed to reconcile these differences with regard to the structure of the PSECDI. Findings also revealed that the DSM-5 was the only measure with items that did not load significantly onto the single factor, specifically one item related to ECIG use interfering with major obligations (eg, work and school) and another related to reducing or giving up important activities (eg, social and recreational) because of ECIG use. However, these same items have been challenged for their relevance to cigarette smoking dependence,¹⁸ as the answers are influenced by numerous personal factors such as financial capacity, workplace/home restrictions, and health status. These items may be even less relevant to ECIG dependence; indoor smoking restrictions do not apply uniformly to ECIG use,¹⁹ and the concealable nature of ECIG devices facilitates their use in situations where restricted. ECIGs may go unnoticed because of their appearance (eg, the shape of a USB flash drive) and/or the fact that they do not produce smoke.²⁰

All measures correlated significantly with items that reflect frequency/duration of ECIG use.^6,7,11,12 The positive correlation of dependence level with years of vaping specifically may be because of increases in the quantity of ECIG use and/or changes in the types of devices being used (eg, those more efficient at delivering nicotine) over time. Another possibility is that duration of product use accounted for these findings, such that this sample of ECIG users were in the earlier stages of product use. Our findings also showed that the DSM-5 was correlated with number of ECIG quit attempts and age of initiation, contrasting with work by Morean and colleagues¹¹ who observed EDS-4 scores to be associated significantly with quit attempts among both ECIG users who do and do not also smoke cigarettes. Also, results showed that dependence scores were related to only one reason for ECIG use; the endorsement of “I like flavors” was associated positively with DSM-5 andGNBQ scores, but not with PSECDI and EDS-4 scores. Piper and colleagues¹² found that scores on several dependence measures (PSECDI, e-FTCD, e-WISDM) correlated significantly not only with liking flavors but also with other reasons for use. Although, their sample consisted of dual cigarette-ECIG users and the majority of reasons they assessed are not applicable to never-smokers (eg, “use e-cigarettes when I can’t smoke,” “may help me quit smoking,” “help with nicotine withdrawal,” and “avoid going outside to smoke”). More work is needed to identify dependence-related constructs that are unique to ECIGs, such as product characteristics (eg, vape clouds, device appearance, and modifiable parts) or the contexts in which ECIGs are used (eg, stealth vaping).

Constructs that have been proposed for the measurement of ECIG dependence include quantity and frequency of use, tolerance, perceived benefits, withdrawal symptoms, craving/urge to use, use despite harm, impaired control, automaticity, preferred over competing rewards, and sensory dependence²¹ (also see²²). None of the dependence measures included herein covered all of these recommended constructs. The PSEDCI, which is arguably the most widely used measure of ECIG dependence to date, overlaps most with the DSM-5 in terms of the number of constructs assessed: withdrawal symptoms (irritability and anxiety during cessation), craving/urges to use, impaired control (unsuccessful quit attempts; vape now because really hard to quit), and preferred over competing rewards (difficulty abstaining when not permitted; spend a lot of time obtaining ECIGs). Still, the PSECDI is the only included measure to cover quantity/frequency of use (uses per day), while the DSM-5 is the only measure to cover tolerance (vape more over time to achieve the same effect) and use despite harm (use despite social or interpersonal problems; use in hazardous situations) constructs. Some constructs covered by the EDS-5 overlap with those from the PSECDI and the DSM-5 (craving and preferred over competing rewards); however, the EDS-5 also clearly addresses automaticity (reaching for ECIG without thinking about it). The GBNQ, developed specifically to measure the behavioral aspects of dependence, primarily taps into automaticity (find oneself vaping routinely; enjoyment of ritual) and sensory dependence (handle/manipulate ECIG). These observations, combined with intercorrelations between measures that were moderate to high, suggest that these measures are tapping into different yet related dependence constructs. Thus, future work should focus on the development of a single comprehensive measure of ECIG dependence.

Cross-study differences may be due, not only to the different dependence measures administered and the varied cigarette smoking histories of the sampled ECIG users, but also to the type of ECIG devices represented. Users’ ECIG device characteristics were reported only by Rest and colleagues,⁶ with the majority being users of “rechargeable” and “refillable” devices. However, the period of data collection reported in most of these studies (ie, 2013–2016^7,12,13) precedes the emergence of pod-style and modern disposable devices (eg, JUUL and PuffBar²³). These latter device types are those preferred by over three-fourths of ECIG users in our sample. Some work suggests that dependence levels may be higher among users of pod-style (vs. non-pod styles) ECIGs, though such work is limited to adolescent samples.^24,25 In our own work with adults, ECIG dependence has shown to be dependent on²⁶ and independent of²⁷ device type using two different samples of never-smoking ECIG users. Unfortunately, we were unable to consider device type in the present analyses because of small subsample sizes. Next steps should include a more detailed evaluation of individual device characteristics and related behaviors (eg, modifiable features; ease of stealth vaping) to determine their role in the assessment of ECIG dependence.

Another potential study limitation is the online method of data collection; only 48.7% (ie, 134/275) of those who completed the survey provided a picture and were included in the final analyses. This number approximates that reported in a recent study that also required respondents to upload a picture of their device (ie, 1209/2813 = 42.9%) as part of an online survey of ECIG use.²⁸ It is also worth mentioning that our sample of ECIG users were not entirely naïve to other tobacco products; some participants used cigarettes in their lifetime and nearly 30% reported use of at least one other tobacco product in the past month. However, because most ECIG users report concurrent use of other products (56%–87.2%),^29,30 the number who have never used any tobacco is likely negligible. Additionally, our sample consisted primarily of pod-style users and those who identified as non-Hispanic White and female. Among sampled ECIG users, most report use of a pod-style device^23,31 as well as being non-Hispanic White and male.^32,33 Finally, we were unable to examine other types of reliability and validity that are important for measurement evaluation. For existing ECIG dependence scales, test-retest reliability has not been examined in any published work and predictive validity has been examined in only a couple of studies with mixed findings for different measures.^7,34 These other measurement properties should be the focus of future work, as well as the recruitment of a more diverse sample of ECIG users in terms of their demographic characteristics and preferred device types.

In conclusion, the evaluation of ECIG dependence must account for factors that are product-specific.³⁵ Such factors include those related to nicotine (eg, primary reinforcement) and non-nicotine (eg, secondary reinforcement) components. It is equally important to assess ECIG dependence among users who are not already dependent on nicotine via other tobacco products like cigarettes. In recent years, the number of ECIG users who are never smokers grew faster than those who are former or current smokers.³⁶ This pattern may have been facilitated by the development of devices that are more efficient at delivering nicotine than earlier models.³⁷ Future work should focus on this population of ECIG users, ensuring that ECIG dependence measures accurately reflect dependence on these particular products.

Supplementary Material

A Contributorship Form detailing each author’s specific involvement with this content, as well as any supplementary data, are available online at https://dbpia.nl.go.kr/ntr.

Funding

Funding was provided by the West Virginia University Eberly College of Arts & Sciences Department of Psychology Thesis Research Award, as well as the National Institute of General Medical Sciences (NIGMS) predoctoral training grant (T32 GM132494).

Declaration of Interests

All authors have no conflicts to disclose.

Acknowledgment

These data were presented at the 28th Annual Meeting of the Society for Research on Nicotine and Tobacco (March 2022, Baltimore MD).

Data Availability

The data underlying this article will be shared upon reasonable request to the corresponding author.

References