Online vs. Face-to-Face Deliberation: Effects on Civic Engagement

Abstract

Deliberation is considered to produce positive effects on public opinion, in that it exposes participants to alternative perspectives and rational arguments. However, whereas benefits of face-to-face deliberation have been supported by many empirical studies, the effects of online deliberation remain unclear to date. This research compares the effects of online and face-to-face deliberation in experimental settings. A theoretical review of computer-mediated communication and deliberative democracy suggests that online deliberation is not necessarily inferior to face-to-face deliberation. An experiment was conducted to compare the relative outcomes of a deliberation performed in face-to-face and computer-mediated settings. The results suggest that both online and face-to-face deliberation can increase participants’ issue knowledge, political efficacy, and willingness to participate in politics.

Introduction

There is a growing concern that modern democracy is in danger because citizens lack interest in politics and are little informed of political affairs and important policy issues (Delli Carpini & Keeter, 1996; Kinder, 2002; Putnam, 2000). Scholars emphasize informed public discussion or “political deliberation” as one cure to address this malaise of modern democracy (e.g., Barber, 1984; Fishkin, 1991; Gutmann & Thompson, 1996). Deliberation or, more broadly, “deliberative democracy,” refers to the concept that democratic practice and rule making should depend on informed discussion of citizens. It is a normative political theory that assumes rational communicative behavior and voluntary participation in public affairs on the part of citizens. Deliberative democracy has received both popular and scholarly attention, because it is believed that deliberation produces norms beneficial to democracy, such as political efficacy and a willingness to participate in politics.

With the rise of new communication technologies, new deliberative potentials are being explored. Online interactions may enhance the scope of political deliberation while maintaining deliberation’s beneficial effects. This study reviews theories of deliberative democracy and computer-mediated communication (CMC) in an attempt to understand those new deliberative possibilities. An experiment was conducted to compare the relative outcomes of a deliberation performed in face-to-face and computer-mediated settings. The results suggest that both online and face-to-face deliberation can increase participants’ issue knowledge, political efficacy, and willingness to participate in politics.

Democracy and Deliberation

Political deliberation is considered a “discursive system where citizens share information about public affairs, talk politics, form opinions, and participate in political processes” (Kim, Wyatt, & Katz, 1999, p. 361). In a stricter definition, deliberation refers to the combination of careful problem analysis and an egalitarian process in which participants have adequate speaking opportunities and engage in attentive listening or dialogue concerning public issues (Burkhalter, Gastil, & Kelshaw, 2002). While there is some disagreement over the nature of talk that constitutes deliberation, scholars agree that rational human communication is the essence of deliberation, and deliberative democracy can be a good alternative or complement to the ailing liberal democracy of today.

Although its popularity has increased in recent years, deliberation is not a new idea. Since the writings of such theorists as Bryce (1888/1973), Tarde (1898/1969), and Dewey (1927), many democratic theorists have recognized the benefits of deliberation. Earlier scholars did not use the term “deliberation” as it is employed currently, but they emphasized the importance of everyday political conversation in forming healthy democracy. For many years, deliberation has been thought to be valuable because it helps the participants reach a more considered opinion by providing new information and by exposing the participants to alternate perspectives. As Curtin (1997) argues, “The deliberative view relies on a person’s capacity to be swayed by rational arguments and to lay aside particular interests and opinions in deference to overall fairness and the common interests of the collectivity” (p. 54).

Political deliberation exemplifies the ideal of deliberative democracy envisioned by Habermas (1984, 1987, 1989, 1996), in which rational arguments in the public sphere help form refined public opinion. American theorists such as Gutmann and Thompson (1996) also suggest that deliberation is beneficial for democracy because deliberation among citizens not only facilitates healthy public opinion but also forms attitudes and norms that are supportive of engagement, such as social trust and political efficacy.

Another important aspect of deliberation is that it can, at least theoretically, facilitate political participation. Political participation, in general, refers to action by ordinary citizens directed toward influencing some political outcomes (Brady, 1999) and is considered part of an informed citizenry. Katz (1992) writes how deliberation can increase citizens’ political participation: “By the very process of talking to one another, the vague dispositions which people have are crystallized, step by step, into specific attitudes, acts, or votes” (p. 80). Many other theorists (e.g., Calhoun, 1992; Chambers, 1996; Warren, 1992) also maintain that deliberation and participation are inextricably intertwined. Warren, for example, argues that deliberation exercises some transformative effects on participants by infusing them with a public spirit.

Political participation is closely connected to the concept of political efficacy. Political efficacy is an individual’s sense of personal competence in influencing the political system (Reef & Knoke, 1999). Many studies have shown reciprocal effects of political participation and efficacy (e.g., Finkel, 1985; Herbst, 1999). Through deliberation, it is expected that participants become confident in their views and willing to express them and hence increase their self-efficacy in political affairs. The logic here is simple: The more citizens learn, think, and talk about something, the more they tend to feel capable of dealing with it.

Empirical research on deliberation supports the claim that deliberation can create “better citizens” for democracy by improving citizens’ issue knowledge, political efficacy, and participation. Findings show a clear positive association between deliberation and citizens’ political knowledge (e.g., Gastil & Dilliard, 1999; Fishkin & Luskin, 1999; Luskin, Fishkin, & Jowell, 2002). Studies by Fishkin and his colleagues (Fishkin & Luskin, 1999; Luskin, Fishkin, & Jowell, 2002) additionally show that deliberation increases political efficacy. The study by Kim et al. (1999) demonstrates that citizens’ news media use, political conversation, and political participation are closely associated. Price, Nir, and Cappella (2002) show that deliberation contributes to people’s ability to generate reasons and thereby improves the quality of their opinions. Indeed, a wealth of empirical research supports the beneficial effects of deliberation.

Of course, deliberation is not a panacea. Research on deliberation and deliberative democracy tends to be guided by normative ideals and thus has been criticized as idealistic. The main criticism is that deliberative democracy is not practical in the current large-scale, complex democratic system (Przeworski, 1991). The second set of criticisms, which mostly come from the social psychology literature, argues that deliberation does not necessarily produce desirable outcomes, and even though the aim may be to reach more socially just decisions, deliberative processes are often biased against socially disadvantaged groups such as racial minorities and women. This criticism, usually accompanied by empirical evidence, is often given greater credence. Social psychological research on small group dynamics has produced findings that deliberation sometimes favors the preexisting views of the majority (Myers & Lamm, 1976; Schkade, Sunstein, & Kahneman, 2000), maintains racial conflicts (Mendelberg & Oleske, 2000), produces competition rather than cooperation (Young, 2000), causes opinion polarization (Mendelberg, 2002), and is biased against African Americans (Sanders, 1997).

The above criticism is not unwarranted. Deliberation produces beneficial outcomes in principle, but it can also generate anti-democratic processes and outcomes. As observed by Delli Carpini, Cook, and Jacobs (2004), deliberation is highly “context dependent.” That is, deliberation may produce good or negative effects depending on the situation. If so, a critical issue in deliberation research is to identify contexts that can promote procedures and outcomes conducive to democracy. This will be most readily achieved by satisfying the key assumptions of deliberation. Participants in deliberation should freely join and enjoy the discussion, carefully weigh both the consequences of various options for action and the views of others, and have sufficient and equal opportunities to speak (Mathews, 1994). Also, the participants should remain civil to each other and respect differing viewpoints. When these assumptions are met, deliberation is most likely to produce beneficial outcomes. The present research attempts to create idealized speech conditions in order to satisfy these assumptions of deliberation.

The second defense against the criticism of deliberation is that, even though deliberation sometimes creates conflicts, frustration, and opinion polarization among discussants, it still can have some positive effects. Even in situations of opinion polarization or majority domination, there is some evidence that participants still learn new perspectives from others and empathize with others’ views (Nemeth, 1986; Turner, 1991). In particular, the positive effects of deliberation on discussants’ issue knowledge, political efficacy, and intentions of political participation have been documented in numerous empirical studies (e.g., Eveland, 2004; Fishkin, 1999; Gastil & Dilliard, 1999; Gastil & Levine, 2005; Loycano, 1992).

Deliberation Online

The development of new communication technologies provides new ways of conducting deliberation, including deliberation in CMC settings. Whereas many people acknowledge the importance and benefits of deliberation in face-to-face settings, they are less certain about the effects of deliberation conducted in CMC settings. This is partly because there has been little empirical research investigating the effects of online deliberation on public opinion. An increasing number of studies deal with so-called e-democracy, but they do not directly concern online deliberation and mostly discuss citizens’ Internet usage and political participation patterns by analyzing secondary survey data. To the knowledge of this author, very few studies so far have compared the effect of online deliberation with that of face-to-face deliberation in experimental settings.

In one such study, Luskin, Fishkin, & Iyengar (2004) argue that online deliberation can generate a positive effect on public opinion that is comparable to face-to-face deliberation. In that study, however, the experimental settings for face-to-face and online deliberation groups were not identical. Because different expert panelists, moderators, and survey questionnaires were used to gauge the effect of deliberation in the two settings, it would have been difficult to compare the effects of the two types of deliberation directly. Furthermore, online deliberation in Luskin et al.’s study was conducted not by text but by voice. Given that most CMC is done by text, it can be said that the study’s design did not accurately address the reality of online deliberation. Thus, it is worth investigating the effects of online deliberation further. The current project probes the effects of textual online deliberation on opinion formation in an experimental setting and compares the effects of online deliberation with those of face-to-face deliberation.

Turning to a theoretical discussion of CMC, many theorists are skeptical of the power of online deliberation. With the exception of Walther’s (1992) social information processing theory, most other communication theories suggest that online discussion will be less effective than face-to-face discussion. Traditional social presence theory (Short, Williams, & Christie, 1976) asserts that communication is most effective when nonverbal cues are present. Thus, given the lack of social context cues in online settings, online deliberation among participants will not be effective, according to the theory. Similarly, media richness theory (Daft & Lengel, 1984) suggests that “lean” media such as text-based CMC would not be efficient for exchanging emotionally complex messages.

From a rather different perspective, social identity/deindividuation (SIDE) theory proposes that online opinion tends to polarize easily due to in-group out-group discrimination (Lea & Spears, 1991; Postmes, Spears, & Lea, 1998; Spears & Lea, 1992). According to the theory, online discussion participants remain anonymous and thus lose individual identity, instead developing a strong group identity. SIDE theory would probably be the most powerful critic of the value of online discussion, and a number of empirical research projects support the premise that in-group identity causes stereotyping of out-group members. Supporting SIDE theory, Sunstein (2001) also argues that there is an enhanced polarization of attitudes among online discussants who have been reminded of a group identity.

The above critical CMC theories, however, do not adequately address the issues associated with online deliberation. Social presence and media richness theories posit that it is difficult to communicate rich emotions in CMC due to the lack of non-verbal cues. However, the key to deliberation, whether it is online or offline, is that the discussion should be rational. “Rationality” is the core concept of Habermas’ (1984, 1987) communicative action. Rationality emerges from reasoning and argumentation and contributes to the quality of public opinion. A perusal of Habermas’ arguments suggests that rationality is achieved mostly by linguistic exchanges. If so, there is no reason to believe that online communication conducted through text should be inferior to face-to-face communication in terms of deliberativeness.

As Walther (1992) claims, verbal cues are sufficient for information processing. Online verbal communication may even be superior to face-to-face communication in terms of rationality, because online deliberation, which is solely dependent on text exchanges, is emotionally more detached and perhaps more cognitively-oriented. Online communication, in principle, is considered more democratic because in online communication one can observe reduced patterns of individual dominance and increased contributions by low-status participants (Rice, 1993).

If online deliberation has similar formal settings to face-to-face deliberation, as practiced currently in such organizations as the National Issue Forums and AmericaSpeaks, then there is no reason to believe that online deliberation is necessarily inferior to face-to-face deliberation. As Luskin et al.’s finding suggests, “Balanced deliberation with the sole purpose of helping the participants clarify their own thinking does not seem to produce any strong polarization” (2004, p. 20).

After all, what is important for deliberation—whether it is online or face-to-face—is to create an environment and processes that are conducive to effective deliberation. Setting up an ideal speech situation can be done by considering the norms of deliberation described in the first section. To reiterate, participation in deliberation needs to be open to all and governed by rules of equality, symmetry, and civility; participants need to be informed of deliberation issues; and deliberation needs to be reflexive and rational, meaning that everyone who deliberates agrees to advance positions either by appealing to the common interest or by making arguments of a sort that all participants could accept (Cohen, 1997; Shane, 2004). If we approximate such an ideal deliberation environment, both online and face-to-face deliberation should produce similar effects. An important part of this research is to create such ideal speech situations both for online and face-to-face deliberation groups.

Hypotheses

In sum, the suggestion is that deliberation, whether online or face-to-face, can create positive effects on public opinion formation and increase participants’ issue knowledge, political efficacy, and engagement. In particular, the theoretical review above suggests that online deliberation can be as effective as face-to-face deliberation. In light of the body of literature reviewed, the following research question and hypotheses are presented:

RQ: How does online deliberation compare with face-to-face deliberation? Do they produce similar opinion changes?

H1: Both online and face-to-face deliberation will increase participants’ issue knowledge to an extent that is significantly different from the control group.

H2: Both online and face-to-face deliberation will increase participants’ political efficacy to an extent that is significantly different from the control group.

H3: Both online and face-to-face deliberation will increase participants’ intention to participate in political affairs to an extent that is significantly different from the control group.

Method

Procedure

The research design involves a true experiment to gauge the effects of online and face-to-face deliberation on participants. A total of 81 students in a large mid-western university were recruited from communication courses. Students volunteering for the project received extra credit for the courses. After providing informed consent, the participants were randomly assigned to one of three conditions: face-to-face, online, and control. Participants in the two treatment groups took a pre-survey to measure their initial opinions on the issue to be deliberated. Their pre-deliberation opinions were compared with their post-deliberation opinions to identify changes. For the measurements of issue knowledge, political efficacy, and political participation, a posttest-only control group design was used. After deliberation, the participants in the two treatment groups took a post-survey concerning knowledge, efficacy, and participation, and these measurements were compared with those of the control group.

Out of the 81 students, 27 were randomly assigned to the face-to-face condition, 24 to the online condition, and 30 to the control condition. The 27 in the face-to-face condition were then divided into three small groups, as were the 24 in the online condition. Each small group contained seven to 10 members. This procedure was intended to facilitate deliberation, since it is generally assumed that deliberation can take place better among a small, manageable number of participants (Burkhalter et al., 2002). The deliberation procedures followed were identical across the small groups.

Between May 22 and May 25, 2006, the six small groups engaged in 90-minute long deliberations on a current school issue. The three face-to-face groups were invited to a school conference room. Each face-to-face discussant was given a name tag and information materials about the issue. After reading the materials, the participants started a deliberation moderated by a trained moderator. As they were recruited from many different classes, the face-to-face discussants were largely strangers to each other; thus there was little interaction among them before the start of actual deliberation. Online deliberation took place in a large computer lab. The online discussants could not see each other as they sat facing different directions, scattered throughout the large, round-shaped lab. After reading information materials online, the online discussants joined a specialized chat room that allowed moderation and transcription of the conversation. They used their real names in the chat room but were not able to identify each other due to the visually impeding seating. Those in the control group only answered the survey questions without reading any information materials or engaging in deliberation.

Operationalizing Deliberation

Deliberation is not an easy concept to investigate empirically, because it is difficult to operationalize. As described in the previous sections, deliberation requires rationality, reflexivity, equality, and civility in communication. However, these are very challenging to implement in practice. Indeed, some may well question whether the communication in many deliberation forums currently in existence can really be called deliberation. Perhaps a perfect deliberation session that will satisfy all the demanding criteria is unattainable. Thus, this project attempted to create the best possible idealized speech conditions that were conducive to deliberation.

For idealized speech conditions, some adjustments were necessary. First of all, participants in the two treatment conditions received information materials consisting of pro and con arguments on the issues to be deliberated. They then were advised to contemplate the arguments for a time. In addition, the participants were told that their voice would count a lot for school policy, and they were asked to weigh others’ arguments carefully during their discussion. Some may question whether positive effects of deliberation, if any, may come from reading the information materials, rather than from deliberation per se. However, reading background information is an integral element of deliberation, and thus it should be considered as part of deliberation. Practically speaking, it is difficult for discussants to deliberate on a topic without any background information. To be acquainted with the issue and to have time to think about it is vital in quality deliberation. If the use of information materials is inevitable, what is critical is how to present the information. That is, information should be objectively framed and presented to discussants to ensure balanced discussion of the issue.

Second, deliberation was moderated by a trained moderator who assured balanced discussion among participants. The role of the moderator is critical to obtain and maintain rationality, reflexivity, civility, and equality of discussion. Thus, someone who was familiar with the deliberation issues and who had some authority was chosen as moderator. The moderator controlled the discussion so that all discussants could have an equal chance to participate. The same moderator presided over all six deliberation sessions, both online and face-to-face, to make the deliberation setting as similar as possible across the sessions. The moderator led the deliberation according to set standards. First, the moderator shared discussion protocols with the participants so that everyone could understand the nature of the discussion. In the protocols, the moderator emphasized that the session was not a debate, but rather a reflexive discussion to help formulate school policies. The moderator also stressed that there was no “right” opinion or policy choice and that each participant should respect others’ opinions. The moderator then organized the discussion according to several topic areas and worked to keep it on topic. Other than controlling the flow of discussion and intervening to provide equal speaking opportunities, the moderator largely let the discussants talk freely over the main course of deliberation. Toward the end, the moderator suggested working toward a decision that was acceptable to all, but noted that a consensus was not necessary. This role of the moderator was similar across the six deliberation sessions.

Finally, as a measure to see how the discussants felt toward deliberation, they were asked to evaluate how much they enjoyed the discussion; whether they had equal opportunity to speak; and whether they respected differing viewpoints and exchanged rational opinions. As will be shown in the results and discussion sections, almost all participants answered the above questions positively, suggesting that the assumptions of deliberation were met.

Deliberation Topic

Since the participants were all students, the topic needed to address something that was relevant to their lives. Thus, a current, important school issue that had been under discussion in the undergraduate community was chosen as the deliberation topic. The issue was whether to allow students to carry concealed handguns on campus. The state laws allow licensed individuals to carry concealed handguns in many public settings, but they prohibit carrying handguns on campus. With a rising concern about crime inside and around the university, however, it has been suggested that students should be allowed to carry concealed handguns on campus as well. The participants thus were asked to deliberate on this issue. For about 90 minutes, they discussed whether they were concerned about crime on campus; whether they believe it is individuals’ free choice to carry concealed handguns; whether carrying concealed handguns could be an effective measure to prevent crimes on campus; and what would happen if students were allowed to carry handguns on campus.

Measurement

Public Opinion

Measurement involves determining participants’ opinions on the issue. The pre- and post-polls surveyed the students’ initial opinions and following changes using a Likert-type scale. The participants were asked during the pre- and post-polls, “On a scale of 1 to 7, how strongly do you agree or disagree with the following statement: “Students should be allowed to carry concealed handguns on campus for self defense.” Selecting 1 denoted “strongly disagree” and 7 denoted “strongly agree.” The pre- and post-polls measured the mean opinions of the two treatment groups before and after the deliberation.

Issue Knowledge

To test the hypothesis that deliberation increases issue knowledge, participants’ knowledge on the issue was measured after deliberation and compared with the measurement for the control group. The knowledge scale consisted of seven multiple choice-type, factual questions concerning the issue. Scores were calculated according to the number of questions the students answered correctly.

Political Efficacy

The political efficacy scale was adapted from the scale that has been used for the American National Election Studies in order to address the school issue (for the original ANES efficacy items, see Niemi, Craig, & Mattei, 1991). The efficacy scale asks three questions concerning the participants’ political potency and feeling of understanding concerning school issues. These three items are: “I think that I am better informed about school issues than other students,” “I feel I have a pretty good understanding of the important issues facing my school,” and “I consider myself well qualified to participate in school affairs.” The items were measured on a 7-point Likert scale. When administered to the 81 participants, the scale was found to be reliable, Cronbach’s α = .83. Theoretically, political efficacy is positively correlated with measures of political participation. That is, the higher one’s sense of efficacy, the higher one’s level of participation. The efficacy scale was found to be positively correlated with the political participation scale developed in this project, r = .62, satisfying convergent validity. The scale is attached in the appendix.

Participation

This scale measured participants’ intention to engage in political affairs after deliberation. This is not an actual political participation scale, but rather measures intention or willingness to participate in politics. Willingness to participate in politics, of course, does not tell us whether the participants will actually participate. Still, measuring intention/willingness is a useful indicator to gauge the participants’ motivations, which may then lead to political action. Indeed, social psychological studies show that attitudes and motivations lead to behavior (e.g., Fazio, Powell, & Williams, 1989). The scale was developed following the recommendations of Brady (1999). Brady analyzed six major political participation scales and made a typology of political activities common in the studies. His classification of acts includes “actions meant to influence politics indirectly,” “voting-electoral activity,” and “non electoral activity.” Based on Brady’s recommendations, the scale asked the participants how likely they were to engage in such actions as “I am willing to discuss important school issues with others and advance my position” (persuasion), “I am willing to vote for the next Undergraduate Student Government election” (voting), “If there is a meeting or conference on important school issues, I will attend it” (non-electoral activity, meeting attendance), and “I am willing to work with others in school to try to solve some school problems” (non-electoral activity, local problem solving). The four-item scale was found to be reliable, Cronbach’s α = .75. Again, this participation scale is positively correlated with the efficacy scale. The scale is attached in the appendix.

Results

To ensure that deliberation really took place, the participants’ evaluations of deliberation were measured. As can be seen in Table 1, the participants in both face-to-face and online groups all agreed that they enjoyed the discussion, had an equal opportunity to speak, respected differing viewpoints, and exchanged rational opinions. More than 90% of the participants said they enjoyed the discussion (M = 5.59 on a 7-point scale, SD = 1), 100% agreed that they had an equal opportunity to speak (M = 6.2, SD = .49), 78.5% said the discussants respected differing viewpoints (M = 5.63, SD = 1.47), and 90.2% believed they exchanged rational opinions (M = 5.87, SD = 1.25). This shows that the deliberation process was successful. However, the measurement scores were slightly higher for the face-to-face group.

Opinion Change

Although several participants changed their opinions after deliberation, no significant changes took place for the groups as a whole. When measured on a scale of 1 to 7 (with 1 being strongly disagree and 7 strongly agree), the average pre-deliberation opinion for the issue was 1.52 for the face-to-face group and 2.13 for the online group, which means that the students disagreed with the proposal that would allow them to carry concealed handguns on campus for self-defense. After deliberation, this average pre-deliberation opinion changed little (see Table 2). The average pre-deliberation opinion of the control group was 1.93.

Knowledge

As predicted in hypothesis 1, both face-to-face and online deliberation groups exhibited significant knowledge gains. A one-way ANOVA found deliberation to have a significant effect on the knowledge score of the participants, F(2, 78) = 32.95, p < .001, two-tailed. As shown in Table 3, the mean knowledge score for the face-to-face group was 5.70 with a SD of 1.10 and the mean score for the online group was 5.83 with a SD of 0.82. A Scheffe post hoc test showed that the knowledge scores of these two deliberation groups were significantly higher than the control group (M = 3.90, SD = 1.06). This means that on average, the deliberation groups scored about two more questions right than the control group.

Efficacy

The three efficacy items were added up to make one composite index. A one-way ANOVA found that the efficacy score was significantly higher for the two deliberation groups than the control group, F(2, 78) = 12.16, p < .001, two-tailed. A Scheffe post hoc test showed that both the face-to-face group (M = 15.96, SD = 3.11) and the online group (M = 15.30, SD = 1.90) had significantly higher scores than the control group (M = 12.40, SD = 3.34). Hypothesis 2 was thus supported. See Table 3.

Participation

Hypothesis 3 predicted that both online and face-to-face deliberation will increase the participants’ willingness to participate in politics. This hypothesis was partially supported. A one-way ANOVA found deliberation to have significant effect on increasing the participants’ intention to join political affairs, F(2, 78) = 4.25, p < .05, two-tailed. However, a Scheffe post hoc test showed that the significant effect mostly came from the difference between the face-to-face group and control group: Face-to-face’s groups participation score was 21.48 (SD = 4.37) and this was significantly higher from the score of the control group (M = 18.5, SD = 4.23), p < .05, two-tailed. The online group’s score (M = 20.5, SD = 2.93) was higher than the control group but not to a statistically significant degree at the α level of .05. See Table 3.

Discussion and Conclusion

Previous research shows that deliberation can change people’s opinions dramatically (e.g., Luskin et al., 2004). In the current research, however, little opinion change took place after deliberation for both groups. As shown in Table 2, the degree of opinion change was minimal and not statistically significant. Also, out of the 51 deliberation participants, only 13 changed their position after deliberation. This is probably because the gun issue had ideological and moral dimensions, which made a complete reversal of opinion—for example, from agree to disagree—less likely. Interestingly, when the 13 people who changed their positions were examined closely, it turned out that most of them intensified their opinions. That is, those who chose “agree” position in the pre-deliberation poll took “strongly agree” position in the post deliberation poll, and those who chose “disagree” position in the pre-deliberation poll leaned more toward “strongly disagree” position in the post deliberation poll. From this result, one may conclude that deliberation allowed the participants to adopt a stronger perspective, which is considered a criterion for a more sophisticated opinion (Gastil & Dilliard, 1999). However, this observation is open to interpretation because taking a stronger position may also be regarded as a marker of opinion polarization, as suggested by Sunstein (2001). Future research is needed on how to define and measure opinion sophistication.

More importantly, this research showed that deliberation can have positive impacts on important democratic values, whether the deliberation takes place online or face-to-face. The results support the hypotheses that both face-to-face and online deliberation will have positive impacts on the participants’ issue knowledge, political efficacy, and willingness to participate in politics, although the effect of online deliberation on the participation scale turned out to be somewhat smaller. This finding will serve as a good justification for doing online deliberation. Online deliberation can offer large benefits that cannot be realized in face-to-face deliberation. It is much more economical and can hold a larger number of participants regardless of geographic boundaries. Given that current face-to-face deliberation forums require citizens to commit a considerable amount of time and effort, this research suggests that online deliberation can be a good alternative to costly face-to-face deliberation.

The nature of online and face-to-face interaction requires more investigation. When the recorded transcripts of the online and face-to-face deliberation sessions were analyzed informally, it was observed that the online groups expressed more candid and direct opinions and engaged in more heated debates. The online participants remained visually anonymous, even though they used real names. The online participants may have exchanged more candid opinions in the absence of visual cues. Formal conversation analysis would be useful to understand the nature of online and face-to-face discourse.

This project, employing a true experimental approach, found it useful to study the causal relationship among deliberation, knowledge, political efficacy, and political participation. Previous experimental research on the effect of online deliberation employed voice communication and video images, not text (e.g., Luskin et al., 2004). Given that most CMC is carried out via text in actual practice, it can be said that the current research using textual online communication demonstrates higher ecological validity.

Moreover, the current project attempted to operationalize and evaluate deliberation, an endeavor that was not undertaken in previous research. Most deliberation research probing the effects of deliberation has not paid much attention to the operationalization of deliberation. But what exactly is deliberation, and how do we know whether deliberation really took place? Deliberation needs to be analytically separated from other forms of communication such as debate and discussion and appropriately operationalized for empirical investigation. This research discussed some assumptions and criteria of deliberation—equality, rationality, reflexivity, and civility of communication—and attempted to ensure that those criteria were met in the experiment. A post-deliberation survey of the discussants showed that these criteria were met, in that almost all of them agreed that they believed they enjoyed the discussion, had an equal opportunity to speak, exchanged rational opinions, and respected differing viewpoints.

A problem with the current project is its use of a convenience sample. College students used in this study might possess better computer skills and deliberative ability than the general public, and this might have influenced the outcome of the study. This study thus has limited generalizability. Even with this limitation, the findings of this study suggest that online deliberation can be as effective as face-to-face deliberation for those who have some computer and deliberative abilities, which are abilities that more and more of the general public are acquiring.

The relatively small number of participants (n = 81) can be an issue. There was also a problem with the randomization process. Some of the randomly assigned participants did not show up on the deliberation days, and some changed their schedule at the last minute; this might have interfered with the random assignment process. Post-experiment randomization tests, however, showed that the randomization process was successful. First of all, the demographic compositions of the experiment groups matched the approximate demographic profiles of the student population from which the sampled students were drawn. Second, a formal randomization test was conducted. Using an SPSS macro written by Hayes (1998), 100,000 possible random reassignments of the participants and their measurement scores on the dependent variables were produced. This test showed that the proportions of random assignments that produce an F ratio at least as large as the F ratio actually obtained in the study was less than .05 across the dependent variables, which suggests that the randomization was successful.

Future research comparing online and face-to-face deliberation should attempt to establish maximum resemblance for the experimental settings of the two groups. It would also be helpful to look at whether there is any difference between online voice deliberation and online text deliberation. Also, comparing the nature of online and face-to-face discourse, which was only briefly touched upon here, would be an interesting research agenda.

References

Appendix

Issue Knowledge Questions

• 1)  What is “Concealed Carry Law”?

• A)  A law that allows ordinary citizens to carry weapons in public settings in a concealed manner.

• B)  A law that allows public officials to carry weapons in public settings in a concealed manner.

• C)  A law that allows public officials to carry weapons in public settings in an open manner.

• D)  A law that allows ordinary citizens to carry weapons in public settings in an open manner.

• 2)  Which year did Ohio pass Concealed Carry Law?

• A)  2002   B) 2003   C) 2004   D) 2005

• 3)  In Ohio, people with a criminal record can apply for a license to carry weapons.

• A)  True   B) False

• 4)  According to Ohio’s Concealed Carry Law, which of the following places is banned from carrying weapons?

• A)  Parks

• B)  Liquor-selling restaurants

• C)  Train stations

• D)  Parking lots

• 5)  Weapon-carrying license applicants in Ohio are required to receive training. How long does this training last?

• A)  6 hours

• B)  12 hours

• C)  24 hours

• D)  36 hours

• 6)  Proponents of Concealed Carry Law say that it is citizens’ constitutional rights to carry weapons for self-defense. Which of the following Amendments is related to this argument?

• A)  First Amendment

• B)  Second Amendment

• C)  Third Amendment

• D)  Fourth Amendment

• 7)  An OSU body recently had a ballot asking the students whether it should ask for the Ohio General Assembly to change its current policies and allow students with concealed-carry licenses to carry weapons on campus. What is this body?

• A)  The Liberty Forum

• B)  Office of Safety

• C)  National Rifle Association (NRA) OSU chapter

• D)  Undergraduate Student Government

About the Author

Seong-Jae Min is a doctoral student in the School of Communication at The Ohio State University. His research focuses on political communication and new technologies, with a special interest in deliberative democracy. Other interests include international communication that concerns the global communication system, international news flow, and communicative action in world politics.

Address:3168 Derby Hall, The Ohio State University, Columbus, OH 43210, USA