Examining COVID-19 vaccine attitude using SEM-Artificial Neural Networks approach: a case from Reddit community

Summary

As new coronavirus variants continue to emerge, in order to better address vaccine-related concerns and promote vaccine uptake in the next few years, the role played by online communities in shaping individuals’ vaccine attitudes has become an important lesson for public health practitioners and policymakers to learn. Examining the mechanism that underpins the impact of participating in online communities on the attitude toward COVID-19 vaccines, this study adopted a two-stage hybrid structural equation modeling (SEM)-artificial neural networks (ANN) approach to analyze the survey responses from 1037 Reddit community members. Findings from SEM demonstrated that in leading up to positive COVID-19 vaccine attitudes, sense of online community mediates the positive effects of perceived emotional support and social media usage, and perceived social norm mediates the positive effect of sense of online community as well as the negative effect of political conservatism. Health self-efficacy plays a moderating role between perceived emotional support and perceived social norm of COVID-19 vaccination. Results from the ANN model showed that online community members’ perceived social norm of COVID-19 vaccination acts as the most important predictor of positive COVID-19 vaccine attitudes. This study highlights the importance of harnessing online communities in designing COVID-related public health interventions and accelerating normative change in relation to vaccination.

INTRODUCTION

As new COVID variants continue to emerge, vaccines have been considered as the most effective approach to containing the coronavirus and achieving herd immunity (Omer et al., 2020; Largent and Miller, 2021; Sanderson, 2021). However, recent research revealed that vaccine hesitancy has emerged as a barrier against full inoculation (Dror et al., 2020), and thus many schemes have been proposed to overcome these barriers and increase vaccination rates (Largent and Miller, 2021). Burgess et al. (Burgess et al. 2021), for example, suggested that policymakers should leverage online communities to ‘gain deeper understandings of intersecting local challenges and opportunities, while establishing trust with communities and building effective communication and public health messaging’ (p. 10).

As one of the most important sources from which individuals acquire COVID-related information, social media-based virtual communities has indeed played a critical role in shaping and altering people’s perceptions and behaviors during the pandemic (Al-Dmour et al., 2020; Cinelli et al., 2020; Cuello-Garcia et al., 2020; Low et al., 2020). Our present study seeks to contribute to the scholarship by examining the impact of participating in online communities on people’s attitudes toward COVID vaccines. Specifically, we ask the following research question: what are the major factors affecting online community members’ attitudes toward COVID vaccines, and to what extent?

Informed by theoretical perspectives of social support, sense of community, social norms, self-efficacy, social media usage and political ideology, we conducted a behavioral survey study to examine and describe the mechanism through which online community members’ attitudes toward COVID vaccines are influenced by their perceived social support, sense of online community, health self-efficacy, social norms, social media usage and political ideologies. We drew on health promotion research to build our conceptual model. Research has demonstrated that perceived social support plays a crucial role in leading to health-promoting behaviors (Caprara et al., 2006; Karademas, 2006; Anderson et al., 2007; Peterson et al., 2008) as it alleviates emotional and physical hardship (Kelly et al., 1991; Berkman, 1995). It also enhances individuals’ recognition of the community and sense of community (Koh and Kim, 2003; Blanchard and Markus, 2004; Jason et al., 2015). A high-level sense of community can facilitate within-group communication, foster the construction of common values, as well as give rise to shared social norms or expectations within the community, thus precipitates members’ adherence to the behavioral tendencies that the group chooses to adopt (Goldstein et al., 2007; Fiesler and Bruckman, 2019). Individuals’ health behaviors are also found to be influenced by their perceived social norms, self-efficacy, social media usage and political ideology. Perceived social norms positively relate to health-promoting behaviors such as vaccine uptake and trust toward vaccines (Freimuth et al., 2017; Agranov et al., 2021; Latkin et al., 2021; Quinn et al., 2017a, b). Self-efficacy has been demonstrated as positively predicting health behaviors, life expectancy, quality of life and well-being status (McMillan, 1996; Bandura, 1997; Pretty et al., 2006; Ernsting, et al., 2015; Stout et al., 2020). The use of social media in health promotion has been valued for its potential to engage with the audience for enhanced communication and improved capacity to promote health programs or services as well as foster the development of healthy minds and attitudes (Bender et al., 2011; Bonnevie et al., 2020, 2021; Choi and Noh, 2020). Finally, in the context of the COVID-19 pandemic, recent studies have suggested the potential impacts of individuals’ political ideologies on their attitudes toward vaccines (Featherstone et al., 2019; Zuk and Zuk, 2020; Agarwal et al., 2021; Hornsey et al., 2021; Milligan et al., 2021), further complicating the relationship between virtual community participation and COVID-19 vaccine acceptance.

METHODS

Data collection

This study draws upon a cross-sectional survey and is approved by the institutional review boards of the organization (University of South Florida IRB Approval No. 002479) with which the research team is affiliated. Reddit community was chosen because it reflects the formation and evolution of COVID-related public opinion by allowing its members to engage in conversations and interact with one another (Chen and Tomblin, 2021). With >1.5 billion registered users and >430 million monthly active users (https://en.­wikipedia.org/wiki/Reddit#:~:text=The%20name%20%22Reddit%22%20is%20a,are%20known%20as%20%22redditors%22), Reddit is considered as one of the biggest online community and has been widely examined in social scientific research on virtual communities (Hwang and Foote, 2021). Public opinion is manifested, deliberated and aggregated as the members freely and anonymously post in relevant subreddits as well as receive feedback from the community through votes, comments and awards. As a niche social media platform, Reddit does not typically require the disclosure of individual demographic information and thus acts as an ideal channel for researching online community members’ engagement and communication at the ‘most raw and richest level’ (Silberman and Record, 2021, p. 381). Respondents were provided the consent form before they were presented with the questions. A total of 1037 US citizens were recruited via Qualtrics panel service during 4 June–21 June 2021. The eligibility criterion was whether the respondent was a registered user on Reddit when filling out the questionnaire. A preliminary data examination was conducted, and no missing value was detected.

Measures

Dependent variable

Attitude toward COVID vaccines.

Our dependent variable is the attitude toward COVID vaccines (M = 3.896, SD = 1.032, Cronbach’s α = 0.905). We adopted the measurement scale from the literature (Quinn et al., 2017a, b) and asked respondents ‘How much do you trust COVID vaccines’, ‘How much do you favor COVID vaccines’, and ‘Would you recommend to your family and friends to get COVID vaccines’ (1 not at all–5 a great deal).

Independent variables

Perceived emotional support.

To test perceived emotional support, we followed the literature (Oh et al., 2014) and asked respondents to score (1 strongly disagree–5 strongly agree) the statements such as, ‘In this community, I can find someone who understands my problems’, ‘In this community, I can find someone that I can count on to listen to me when I need talk’, ‘In this community, I can find someone to confide in or talk to about myself’ and ‘In this community, I can find someone to share my most private worries and fears with’ (M = 3.277, SD = 1.057, Cronbach’s α = 0.891).

Health self-efficacy.

The measurement scale of health self-efficacy was adopted from previous research (Lee et al., 2008) and asked for respondents’ scores (1 strongly disagree–5 strongly agree) on items such as, ‘I am confident I can have a positive effect on my health’, ‘I have set some definite goals to improve my health’, ‘I have been able to meet the goals I set for myself to improve my health’, ‘I am actively working to improve my health’ (M = 3.911, SD = 0.781, Cronbach’s α = 0.835).

Sense of online community.

To measure Reddit users’ sense of community, we adopted the scale from the literature (Batterham and Calear, 2017) to ask respondents’ opinions (1 strongly disagree–5 strongly agree) on the extent to which participating in Reddit community can provide information about COVID-related mental health issues, as well as strategies for dealing with COVID-related unhealthy activities and negative feelings and for relaxation (M = 3.136, SD = 1.047, Cronbach’s α = 0.914).

Perceived social norm.

We asked about Reddit community members’ perceived social norm about COVID vaccines in our survey. Two of the original items (Quinn et al., 2017a, b) that we adopted displayed an inconsistency with other items; after excluding the inconsistent items, our final question asked about respondents’ evaluations (1 none/not at all–5 a great deal) on the following items, ‘Of the people close to you, what proportion want you to get the COVID vaccine’, ‘It is my moral obligation to other people to get the COVID vaccine’ and ‘What is the expectation at your workplace when it comes to the COVID vaccine’ (M = 3.657, SD = 0.866, Cronbach’s α = 0.728).

Social media usage.

To measure the usage of Reddit, we asked respondents to evaluate how frequently they use Reddit in daily life and then make a selection on a scale (1–6) with the items being, ‘about once per week or less’, ‘about 2 to 3 times per week’, ‘about 4 to 5 times per week’, ‘about once a day’, ‘about 2 to 4 times a day’ and ‘about at least or more than 5 times a day’ (M = 2.90, SD = 1.637).

Political ideology.

We asked for respondents’ evaluations on a scale (1–5) about the description of their political ideologies, ‘strongly liberal’, ‘liberal’, ‘neutral’, ‘conservative’ and ‘strongly conservative’ (M = 2.60, SD = 1.027).

We also included several demographic variables such as gender (45.1% female, 51.8% male, 2.4% non-binary or third gender, 0.3% not sure and 0.4% prefer not to say), education level (3.6% less than high school, 20.8% high school, 34.3% some college and professional degree, 25.3% college graduate or bachelor’s degree, 2.8% some advanced graduate study, 11.2% graduate degree or master’s degree and 2% doctorate) and marital status (48.5% married or domestic partnership, 41.5% single and never married, 10% divorced, separated or widowed) to control for their potential effects.

Statistical models

This study used a hybrid approach that consists of structural equation modeling (SEM) and artificial neural network (ANN) analysis as the two methods complement each other in delivering a clear understanding of the mechanisms. In particular, the SEM analysis was employed to test the relationships among constructs, and the ANN analysis was used to determine the relative importance of the predictors.

SEM

We applied SEM to detect relationships among our constructs. We first conducted a confirmatory factor analysis (CFA) where all the constructs were allowed to correlate freely with each other. The CFA provided a test of convergent validity for each of the latent variables that contribute to the observed constructs (Maruyama, 1997). After performing confirmatory analysis, we then specified the structural model and tested the main, moderation, and mediation effects. A bootstrapping technique with 5000 subsamples was used to estimate the 95% bias-corrected confidence interval of the indirect effect (Hair et al., 2017).

ANN analysis

Machine learning-based ANN models are increasingly used in many social behavioral research areas (Chowdhury and Samuel, 2014; Tan et al., 2014; Jabłońska and Zajdel, 2020). This analytical method is defined as ‘a massively parallel distributed processor consisting of simple processing units that have a natural propensity to store experimental information and make it available for use’ (Sharma et al., 2021, p. 9). As an artificial intelligence tool, ANN models do not require multivariate assumptions like normality or homoscedasticity, and thus are proficient in handling both linear and non-linear relations (Lek et al., 1996; Leong et al., 2013). ANN allows the evaluation of the relative significance of multiple explanatory constructs by taking into account all possible connections that could potentially have impacts on the outcome constructs (Shmueli and Koppius, 2011), and therefore is powerful in overcoming the limits of linear statistical models (Paruelo and Tomasel, 1997; Ramos-Nino et al., 1997; Starrett et al., 1997; Manel et al., 1999). Moreover, this method outperforms traditional regression techniques by incorporating hidden neurons and providing higher prediction accuracy, and thus complements traditional statistical modeling approaches such as SEM (Scott and Walczak, 2009; Shmueli and Koppius, 2011).

In this study, SPSS 27.0 was used to construct the neural networks model, which employed a multi-layer perceptron with Feedforward-Back Propagation algorithm to identify the models. To specify the model, all the independent variables and control variables from the SEM analysis were included as part of the input layer (neurons), and the attitude toward COVID vaccines was specified as the output layer. A 10-fold cross validation with an 80:20 training-testing data partition was applied. Both the hidden and output layers were produced using Hyperbolic Tangent function, and the importance of the predictors from the input layer of the neural network was thus determined by the number of non-zero synaptic weights connected to the hidden layer (Tan et al., 2014). After identifying the ANN architecture, a sensitivity analysis was conducted to identify the relative importance of each independent variable. Sensitivity analysis in the ANN model presents the variations in the dependent variable by virtue of changes in the independent variables associated with it (Nourani and Fard, 2012; Weber et al., 2018).

RESULTS

Results from SEM are summarized for detecting relationships among constructs, and results from ANN analysis are used to evaluate the relative importance of all the predictive constructs.

Results from SEM

All path estimates from the CFA (Table 1) were significant at 0.001 level and the loadings were >0.90, indicating adequate levels of convergent validity (Barki and Hartwick, 2001). Regarding the structural model, the model fit indices exceed the recommended thresholds (χ²/df = 2.659, RMSEA = 0.040, CFI = 0.996, GFI = 0.994, AGFI = 0.970, NFI = 0.994, IFI = 0.996, TLI = 0.984), confirming that our model fits very well with the collected data (Hu and Bentler, 1999; Prajogo and Hong, 2008). Specifically, our model shows a significant positive direct effect of perceived emotional support on sense of online community, and a significant positive direct effect of sense of online community on perceived social norm of COVID vaccination. Reddit community members’ sense of online community significantly mediates the effects of perceived emotional support on perceived social norm of COVID vaccination. Although the effect of health self-efficacy on perceived social norm is found to be insignificant, health self-efficacy significantly interacts with perceived emotional support in affecting perceived social norm. Meanwhile, Reddit community members’ perceived social norm has a significantly positive impact on their attitudes toward COVID vaccines as well as positively mediates the effect of health self-efficacy on COVID vaccine attitude, without mediating the effect of sense of community on attitude, thus resulting in an insignificant serial mediation effect among perceived emotional support, sense of community, perceived social norm and COVID vaccine attitude. The community members’ frequent usage of Reddit significantly leads up to an increase of sense of community but not a higher level of perceived social norm and attitude in relation to COVID vaccination. Reddit community members’ sense of community plays a significant mediating role between their usage of the community and perceived social norm of COVID vaccination, however, the mediating role played by perceived social norm between Reddit usage and COVID vaccine attitude was found to be insignificant. In addition, Reddit community members’ political conservatism has significant negative impacts on their perceived social norm of COVID vaccination and attitudes toward COVID vaccines. As such, virtual community members’ perceived social norm of COVID vaccination significantly mediates the relation between their political ideology and attitudes toward COVID vaccines. Finally, our structural model demonstrated that no statistically significant relationship was detected between perceived emotional support and perceived social norm of COVID vaccination based on analyzing Reddit users’ responses. All the path coefficients of direct and indirect effects are summarized in Table 2 and Figure 1.

Results from ANN analysis

To specify the ANN models, the independent variables (Perceived Emotional Support, Sense of Community, Health Self-Efficacy, Perceived Social Norm, Social Media Usage, Political Ideology and control variables) were considered as the input for sensitivity analysis, and the dependent variable (Attitude toward COVID Vaccines) was tested as the output (Leong et al., 2015). Figure 2 shows the identified neural network architecture. Root Mean Square Error (RMSE) was used to predict the accuracy of the neural network models (Chiang et al., 2006; Chong, 2013; Tan et al., 2014; Liebana-Cabanillas, et al., 2017; Priyadarshinee et al., 2017; Sharma et al., 2021). The RMSE values (Table 3) ranged from 0.226 to 0.252 for the outcome variable, Attitude toward COVID Vaccine, and the average difference between training and testing RMSE values in the model was small (M = 0.012, SD = 0.006). Consistent with prior research, it can be summarized that the results of the ANN analysis are reliable (Chong, 2013; Liebana-Cabanillas et al., 2017; Sharma et al., 2017, 2019). In addition, results from the sensitivity analysis showed that among the independent variables, perceived social norm has the highest normalized importance ratio and thus acts as an essential and most influential factor that leads to the variance of the attitude toward COVID vaccines, followed by gender, perceived emotional support, education, Reddit usage, health self-efficacy, sense of online community, marital status and political ideology (Table 4).

DISCUSSION

Findings and implications

Our study contributes to the burgeoning COVID-related literature by examining the mechanism that underpins the impact of participating in online communities on the attitude toward COVID vaccines. With new coronavirus variants continuing to emerge and substantial vaccination levels needed to achieve herd immunity, it is imperative for healthcare professionals and health officials to clearly understand public attitudes toward COVID vaccination in the context of social media, in order to harness online communities to best address concerns and promote vaccine uptake. In the present study, we employed an integrative SEM-ANN method to test the effects of perceived emotional social support, sense of online community, health self-efficacy, perceived social norm of COVID vaccination, social media usage and political ideology on COVID vaccine attitude.

In the first place, our study demonstrates that Reddit community members’ perceived emotional support positively associates with their sense of online community, consistent with other findings in the literature (Koh and Kim, 2003; Blanchard and Markus, 2004; Vieno et al., 2007; Blanchard, 2008; Chiessi et al., 2010; Jason et al., 2015). This implies that when individuals receive more social support that can help them maintain a positive emotional status, they are more likely to develop a high-level sense of community in the virtual world. Our study also confirms the positive relation between Reddit community members’ sense of online community and perceived social norm of COVID vaccination (Goldstein et al., 2007; Esham and Garforth, 2013; van der Linden, 2015; Fiesler and Bruckman, 2019; Yu et al., 2019), meaning that those who have positive feelings about participating in the Reddit community are more likely to perceive the social norm of COVID vaccination uptake. Further, in alignment with previous vaccine research (Marchand et al., 2012; Freimuth et al., 2017; Quinn et al., 2017a, b), such a high-level perceived social norm results in positive attitudes toward COVID vaccines among Reddit community members. Additionally, health self-efficacy plays a moderating role between perceived emotional support and perceived social norm of COVID vaccination, even though health self-efficacy is unable to solely influence perceived social norm. This finding echoes previous research that considered self-efficacy and social norm as independent constructs not directly related to each other (de Vries et al., 1988; Vries et al., 1995; Sheeran et al., 2016) and demonstrates that health self-efficacy improves perceived social norm when it is enhanced by emotional social support. In other words, when individuals receive positive feedback from the social group, they become more confident about the normative behaviors that they have recognized.

Our research also studied the impacts of Reddit usage and the community members’ political ideologies. Regarding Reddit usage, aligned with previous research findings, our study revealed its significant positive impact on the members’ sense of community (Paton and Irons, 2016; Gatti and Procentese, 2021). However, our results did not support the positive effects of Reddit use frequency on attitudes and perceived social norms about COVID vaccination. Possible reasons could be that social media such as Reddit can facilitate the spread of both useful information and misinformation about COVID vaccines (Ferrara et al., 2020; Tasnim et al., 2020), and hence future research should distinguish various kinds of purposes of online community members’ social media use in order to accurately test the effects of social media use on health-related social norms and attitudes. Additionally, Reddit community members’ political ideologies have a significant relationship with their perceived social norm of COVID vaccination as well as attitudes toward vaccines. In agreement with prior research (Featherstone et al., 2019; Zuk and Zuk, 2020; Agarwal et al., 2021; Hornsey et al., 2021; Hwang et al., 2022; Milligan et al., 2021), our study showed that those who are politically conservative are less likely to recognize the social norm of COVID vaccination and tend to generate negative vaccination attitudes.

The mediation analysis built in our model has identified sense of online community and perceived social norm as significant mediators. For the relations among perceived emotional support, sense of online community and perceived social norm, our results showed that the effect of perceived emotional support on perceived social norm was transmitted via Reddit community members’ sense of the community. In other words, the emotional social support that Reddit community members receive can increase the extent to which they perceive the social norm of COVID vaccination through an enhanced sense of online community. Moreover, the positive effect of Reddit members’ sense of community on their attitudes toward COVID vaccines was transmitted through perceived social norm, meaning that participating in the online community reinforces the members’ perceptions of the commonly acceptable vaccination behavior and thus promotes attitude change regarding COVID vaccines within the Reddit community. Reddit members’ sense of community also plays a mediating role between their social media usage and perceived social norm of COVID vaccination, suggesting that when Reddit members participate more in the community, they are more likely to be part of helpful and meaningful interactions and hence are more likely to perceive the social norm of COVID vaccination. Regarding Reddit members’ vaccine attitude, it is worth noting that perceived social norm mediates the effect of political ideology but not the effect of frequent Reddit usage. Possible reasons could be that compared to social media usage, political stances are more closely associated with the cognitive processing and the reflection of normative behaviors that occur within one’s social groups, and thus are more likely to lead to attitudinal change (Citrin and Muste, 1999; Levi and Stoker, 2000; Bartlett and Miller, 2010; Jolley and Douglas, 2014).

There are several implications put forth by this study. Theoretically, our study confirms the roles played by sense of online community and perceived social norm in leading up to health-promoting attitudes. According to the results from our ANN analysis, perceived social norm acts as the most influential factor in generating positive attitudes toward COVID vaccination; the effect is engendered and developed through the process in which individuals receive emotional support and involve in helpful and meaningful interactions within the online community. Practically, our study provides guidance for health practitioners and policymakers. To promote COVID vaccine acceptance and spur the actual uptake, practitioners should take advantage of the online communities developed on various social media platforms to design and spread normative messages about COVID vaccination, as well as offer emotional social support for the community members to create an encouraging environment that helps the members acquire a sense of belonging to the community and builds members’ confidence in COVID vaccines. Further, public health campaigns about COVID and other vaccines should be designed and developed in a way that is consistent with the cultures of online communities, so that the members may find it easier to absorb the information conveyed by the campaigns. For public health policymakers, online community-based citizen science can be a good source of policy and strategy innovation regarding future crisis management. Policymakers should consider working with individuals who can provide social support and play an active role in maintaining an online community to facilitate social norm interventions as well as policy implementation.

Limitations and future directions

Our present study has limitations that set the stage for further investigation. First, this research used a cross-sectional survey instead of an experimental analysis to speculate the associations among the constructs. As the pandemic continues, future research should use experiments to manipulate one or more factors and test their impacts on individuals’ attitudes toward COVID vaccination, in order to validate the causal relationships. Moreover, our study focused on Reddit users to examine the effects of online community participation and did not conduct replication studies on the other social media platforms; future research should triangulate the results of our current study using other types of online communities. Further, even though our study has attempted to employ an integrative analytical approach to inferring the results, there are still limitations caused by the current quantitative design and dataset. Future research should address the limitations by adopting a mixed-method approach and utilizing advanced datasets or analytical tools to achieve a more in-depth understanding of COVID vaccination attitudes and actual uptake.

CONCLUSION

Our study demonstrated the influence of perceived emotional social support, sense of online community, health self-efficacy, perceived social norms, social media usage and political ideology on the attitude toward COVID vaccines. Residing in the research context of an ongoing world pandemic, our research specified a behavioral mechanism of how participating in online communities can affect individuals’ attitudes toward health-promotion activities such as COVID vaccination. We also offered insights into the critical role played by perceived social support in generating a sense of community and health-promotion social norms that ultimately give rise to positive attitudinal outcomes when facing a crisis like the COVID-19 pandemic.

Funding

This work was funded by University of South Florida through the Interdisciplinary Research Grant Program.

Conflict of Interest

The authors declare no potential conflict of interest with respect to the research, authorship, and/or publication of this article.

REFERENCES