People, places, and time: a large-scale, longitudinal study of transformed avatars and environmental context in group interaction in the metaverse

Abstract

The metaverse, persistent immersive virtual worlds often viewed through virtual reality (VR) headsets, is receiving increasing attention in industry, media, and academia. What makes these virtual worlds unique is that people can easily transform their avatar’s appearance or environmental context. With the touch of a button, a person can be anyone and anywhere. As the medium moves from gaming arcades and laboratories to consumers’ homes and universities, it is becoming increasingly important from both a theoretical and societal well-being standpoint to understand how these Transformed Social Interactions (TSI, Bailenson et al., 2004) influence people and their relationships with others, especially over prolonged periods of time.

One way of understanding the metaverse is through literature on collaborative virtual environments (CVEs). While researchers have been studying CVEs for decades (for a recent review, see Aseeri & Interrante, 2021), several important challenges have limited these investigations (for exceptions, see Khojasteh & Won, 2021, Moustafa & Steed, 2018, and Bailenson & Yee, 2006). First, due to the high expense and technical challenges involving VR implementation, researchers are often forced to rely on small sample sizes and either one-shot or a limited number of sessions (Lanier et al., 2019). This raises the question: As the novelty wears off over time, will people have a “better” or “worse” experience as they adapt to the medium? Second, while current metaverse platforms feature groups of various sizes, the majority of research on CVEs feature dyads or occasionally triads (for a review, see Han et al., 2022), which take on very different turn-taking, gaze behavior, and other group dynamics from larger groups.

The current research used two longitudinal field experiments to systematically examine multiple sets of larger groups and how social dynamics evolve over time in CVEs. From a statistical standpoint, we take a multivariate approach to observe how multiple constructs change over time and how they may be interrelated. From a theoretical standpoint, we extend the TSI paradigm to CVEs. Study 1 examines the visual appearance of avatars, a construct that has received much attention in the literature overall (for a review, see Ratan et al., 2020), by investigating the effects of assigning avatars customized to look like the self or uniform avatars to mask visual identity cues. Study 2 examines anenvironmental context by leveraging the ability to easily create VR scenes that differ in spaciousness and setting.

Background and previous work

Transformed social interaction

In CVEs, people are represented by avatars, or visual representations of the self. CVEs track various verbal and nonverbal signals and map them onto these digital beings. CVEs can also systematically filter avatars’ behavioral actions and physical appearance, amplifying or suppressing features and nonverbal signals in real-time. As predicted by TSI, the behaviors and appearance of avatars in CVEs can have a drastic impact on people’s perceptions, as well as their persuasive and instructional abilities (for a review, see Bailenson et al., 2008b). Of the three categories outlined by TSI: self-representations (i.e., avatars), contextual situations (i.e., virtual environments), and sensory capabilities, this current study focuses on the first two categories.

While research on TSI has occurred over two decades, there are still gaps in how these transformations affect people. For example, a recent article by Szolin et al. (2022) systematically reviewed the literature of avatar transformation on behavioral and attitudinal changes in the context of video games. The authors underscored how previous studies fail to separate different types of virtual environments, such as commercially available videogames and bespoke research-focused virtual environments. They concluded that this field of research is still in a relatively new stage of psychological investigation, and that further empirical investigation of avatar transformation and different contexts is needed to understand how they affect people both during and after the virtual experience. In the same vein, there has been growing literature on how other transformations lead to changed behavior, such as how body manipulations produce social, perceptual, and behavioral effects (for a review, see Gonzalez-Franco & Peck, 2018). However, most studies do not examine these transformations in the context of networked, social interactions or examine how the virtual environment itself may influence behaviors and attitudes.

Transforming self-representations

Previous research shows that even subtle transformations in avatars’ behavioral or visual (e.g., photographic or anthropomorphic) resemblance can impact the way people engage with and perceive others (Blascovich, 2002; Nowak & Biocca, 2003; Yee et al., 2011). For example, Roth et al. (2018) manipulated the type of gaze—natural, hybrid, synthesized, and random—exhibited by avatars in dyadic social interactions. They reported that, based on trends found in perceived virtual rapport, interpersonal attraction, and trust, natural gaze was superior, and synthetic and hybrid gaze were better than random gaze. Other research found that, overriding avatars’ actual head movements to mimic motions and increase synchrony led to greater liking between interaction partners (Bailenson & Yee, 2005). Finally, a longitudinal study by Bailenson and Yee (2006) implemented TSI conditions to examine the impact of visual and behavioral similarity on group cohesion and task performance. They found that in certain tasks, groups performed better when they saw their own face on their partners’ avatars (i.e., shared visual similarity). They also reported that, even with TSI manipulations, entitativity increased over time. However, the small sample size of this study precluded the generalizability of these data.

Transforming the situation

In VR, each person can have a unique viewpoint and sensory information, as it is possible to vary the number of visible avatars, spatial arrangement, or even colors in a scene. TSI has shown that these factors alter sensory perception, social interaction, and performance (Bailenson et al., 2008b). For example, Hasenbein et al. (2022) transformed the seating position of a student in a virtual classroom and manipulated how many rows of peer learners were between themselves, the teacher, and the screen. They used eye-tracking to show subsequent changes to attention and found that different seating arrangements led to different focus of gaze transitions on their virtual peer learners, teachers, and the screen, as well as different gaze distributions. Such spatial transformations of students’ seat positions have also been shown to influence memory (Bailenson et al., 2008a) and persuasion (McCall et al., 2009). Similarly, Miller et al. (2021) found that teams of designers had more positive interactions when working in a VR conference room compared to working in a VR garage. Other factors, such as the size of a virtual room and what kind of objects are placed within it, have also been shown to influence outcomes such as attention and navigation (Kim et al., 2022).

Visible space: panoramic and constrained environments

One environmental factor of interest pertains to the amount of space visible from a given viewpoint. Previous research has shown differences in outcomes resulting from being in a constrained or panoramic (i.e., environments in which people can see wide and far) environment. For example, taller (i.e., more spacious) ceilings have been shown to prime feelings of freedom and encourage a more global, abstract way of processing information (Meyers-Levy & Zhu, 2007). Similarly, compared to confining environments, spacious environments were found to foster more self-disclosure (Okken et al., 2012). Finally, compared to smaller rooms, larger rooms were found to promote more engagement in informal learning among students (Wu et al., 2017).

Setting: outdoor and indoor environments

Natural settings have been shown to have beneficial effects (Bratman et al., 2015; Lederbogen et al., 2011). Living in environments where people have access to green spaces have shown to lead to lower mental distress and greater well-being in the long term (White et al., 2013). Even shorter, simulated exposure to nature, such as viewing slides or pictures containing landscapes, have been shown to reduce stress (van den Berg et al., 2014), improve self-esteem (Barton & Pretty, 2010), and physiological restoration (Ulrich et al., 1991), and increase the ability to focus (Berto, 2005).

VR is a medium that is uniquely suited for simulating natural settings. For instance, Anderson et al. (2017) showed that 360° videos of natural environments, compared to those of indoor environments, led to greater relaxation and reduced negative affect. VR’s immersion enhances the restorative potential of mediated natural environments on physiological well-being (for a review, see Browning et al., 2020). To our knowledge, the effect of virtual nature has not been studied in the presence of other avatars within CVEs.

Group behavior

When individuals are linked by social relationships that make up a group, they become interdependent and influence each other’s behaviors, emotions, and perceptions (Janis, 1973; Milgram, 1963; Sherif, 1937). The moment a collection of individuals perceives itself as a group, a construct known as entitativity (Campbell, 1958), a series of psychological and interpersonal changes, occurs (Forsyth, 1990; Harasty, 1996). Given the social nature of CVEs, to meaningfully understand the effects of virtual experiences in such systems, it is important to investigate not only individuals, but also individuals as members of a group.

Nonverbal dynamics also play a critical role in groups. During the course of face-to-face communication, interactants tend to be in “synchrony,” or be in similar states or have similar behaviors at similar times (Condon & Ogston, 1966). VR’s ability to track motion allows for the examination of motion synchrony. Higher motion synchrony has been shown to be related to greater rapport between teachers and students (LaFrance, 1979) and more creativity (Won et al., 2014). Previously, researchers have manipulated synchrony with avatars displayed on screens (Oh Kruzic et al., 2020) and with agents (Tarr et al., 2018) and avatars (Sun et al., 2019; Miller et al., 2021) in VR.

Time

Human processes are complex and rarely, if ever, exist in isolation. It is critical to understand the processes by which human behavior and activity emerge as different components of a system, and influence and change one another over time (see Dynamic systems theory, Newman & Newman, 2020). Study of individuals repeatedly exposed to media stimuli and adapting to new technologies provide a unique opportunity to gain valuable insight about both the long-term and short-term processes invoked by those media (e.g., Bailenson & Yee, 2006; Brinberg et al., 2021). Inferences based on single-session exposures or obtained through analysis of just a few sessions when participants are adjusting to the novelty of a medium can be plagued with technical difficulties and be misleading. A number of researchers have theorized that, while first-time VR users may feel unfamiliar with the medium, with use, their experience in VR should improve (e.g., Loomis, 1992). Other perspectives suggest the opposite, arguing that the habituation effect can cause what was initially novel to diminish (e.g., Lombard & Ditton, 1997).

TSI is a paradigm that should be particularly sensitive to repeated exposures over time, as people learn to identify, adapt, and accommodate to the changes. For instance, while seeing a uniform avatar on everyone in a room may be jarring at first, perhaps with time people habituate. Alternatively, the effects of similarity may amplify. An early study by Bailenson and Yee (2006) followed three triads of participants for 15 sessions over a 10-week period as they collaborated for approximately 45 min per session. In addition to looking at time, the researchers manipulated two types of TSI—behavioral and visual similarity of group members. Results demonstrated changes in task performance, subjective ratings, nonverbal behavior, and simulator sickness over time as participants became familiar with the system. Furthermore, even in the presence of TSI, where there was a mismatch between types of behavior, entitativity was high over time, suggesting that people are able to retain symbolic meaning even with the starkest degree of social cues. However, the small sample reduced power, and further research is needed to examine how people evolve and respond to these transformations of people and place.

Current study

The current work aims to investigate how transformations of who you are and where you are evolve over time. Using TSI as its central framework, this work addresses two of the categories: self-representations and contextual situations, through two large-scale, longitudinal field experiments of how VR-based transformation of avatar appearance (Study 1) and environmental context (Study 2) influence interactants in group settings. The overarching research questions are as follows:

• (RQ1) How do people’s behaviors and attitudes change over time?

• (RQ2) How do people’s behaviors and attitudes change when they are embodying and surrounded by different avatars?

• (RQ3) How do people’s behaviors and attitudes change when they are interacting in different environmental contexts?

Given the critical role that self-presentations have on how individuals perceive their experience and their communication partners in a virtual environment, we manipulated the visual appearance of the avatars such that members of the same group were represented by either avatars that resembled their physical self or avatars that were uniform across all members. Similarly, given the critical role that where you are and what kind of environment you are surrounded by can lead to differing outcomes, we manipulated the type of virtual environment in which group members interacted. Across both studies, we measured both behavioral and self-report variables central to understanding people’s experiences in virtual environments, such as presence and realism. We additionally collected measures that aim to understand group outcomes, such as synchrony and entitativity. Using linear growth models with time-invariant and time-varying covariates, we built two models to understand how these outcomes change across time and vary based on individual differences.

Both field experiments were housed in a 10-week course about VR and its intersections with various disciplines. During each course, participants were provided with a VR headset, which they used to attend eight weekly instructor-led, medium-group size discussion sessions (n1 = 9–14, n2 = 5–8). The nature of each study being housed in a course allowed for naturalistic intervention of our variables of interest and unobtrusive measurement of behaviors. Nonverbal behavior was measured by recording 18 degrees of freedom of movement from each participant (e.g., pitch, yaw, and roll of head and both hands) to compute motion synchrony for each group. Attitudes of each participant’s experience were measured using weekly surveys. We additionally explored how much these outcomes may be mediated by individual differences.

Our key contributions to the field are as follows:

• Time plays a critical role in people’s experience in social VR. Across both studies, self-presence and realism increased over time and with VR use. In Study 1, entitativity, social and spatial presence, enjoyment, and realism increased over time and with VR use.

• Who you are and what you look like matter in social VR. When people are represented by avatars that resemble their physical selves, they are more nonverbally “in sync” with other people, and view the virtual environment and people as more realistic. On the other hand, having a uniform avatar leads to greater enjoyment.

• Where you are matters in social VR. When people are in more spacious virtual environments, they are more nonverbally “in sync” with other people, report feeling greater restoration, entitativity, pleasure, arousal, presence, enjoyment, and realism than when they are in constraining environments. This is an especially important finding as it is difficult to study very large indoor spaces in the real world, and the current findings are novel. Similarly, when there are elements of nature and people are in outdoor environments, they report feeling greater restoration and enjoyment than when they are in indoor environments.

Study 1

Study 1 focuses on the transformation of avatar appearance and investigates the following research questions: first, how will sharing visual similarity with group members influence nonverbal synchrony and entitativity over time? Second, how will perceived self, social, and spatial presence change over time and with different avatars? Third, how will perceived enjoyment of interacting in a virtual environment change over time and with different avatars? Finally, how will perceived realism change over time and with different avatars?

Method

Participants

Participants were 101 university students enrolled in a 10-week course about VR. At the beginning of the course, students were invited to participate in an Institutional Review Board-approved (IRB) study of how repeated exposure to VR influenced their individual and group behavior. While all students who were part of the course took part in all the VR activities, only those who consented to participate in the study had their data included in the study. Of the 101 students in the course, 93 consented to participate in the study. The 81 participants who participated in five or more of the eight weekly sessions (M = 47, F = 30, Other = 2, declined to or did not respond = 2) were between 18 and 58 years old (M = 22.26, SD = 5.19; n_18∼23 = 68, n_24∼29 = 7, n_30∼35 = 3, n_35∼40 = 1, n_55∼60 = 1, declined to or did not respond = 1) and identified as Asian or Asian-American (n = 30), White (n = 21), African, African-American, or Black (n = 11), Hispanic or Latinx (n = 9), multiracial (n = 5), Middle Eastern (n = 1), and declined to or did not respond (n = 4). Participants had varying levels of experience with VR, with 48 (59%) having never used VR before. Prior to the course, 38 participants were not familiar with anyone in their discussion group, and others reported knowing one (n₁ = 13) or more members (n₂ = 12, n₃ = 1, n₄ = 2, n₅ = 2). Safeguards implemented to ensure privacy and consent included review both by the IRB and a second university ethics organization, and third-party oversight of the consent process and data collection.

Hardware and VR equipment

Participants were provided with Oculus Quest 2 headsets (standalone head-mounted display with 1832 × 1920 resolution per eye, 104.00° horizontal FOV, 98.00° FOV, 90Hz refresh rate, and six-degree-of-freedom inside-out head and hand tracking, 503 g) and two hand controllers (126 g) for use in their personal environment. Of the 81 participants, 2 owned personal headsets (PC-based Valve Index) and participated using those devices.

Virtual environment: ENGAGE

Weekly sessions were hosted in ENGAGE, a collaborative social VR platform designed for education. Every week, the virtual environment consisted of a private (password restricted) “Engineering Workshop” room that was a large, open-space area that allowed participants to walk/teleport freely, create 3D drawings, write on personal whiteboards/stickies, add immersive effects/3D objects, and display media content. The large space accommodated the use of 3D audio, which allowed for splitting off into smaller groups without audio overlap.

Avatar: self vs. uniform

In ENGAGE, participants are represented by human avatars (Figure 1). Participants embodied one of two possible representations in the avatar conditions. In the self-avatar condition, participants were able to customize their avatars with various combinations of outfits, gender, age, skin complexion, weight, hairstyles, and facial features. In the uniform avatar condition, all participants used a pre-selected avatar within the customization options possible within ENGAGE. Through pre-testing and iteration, we chose an avatar that was gender and racially ambiguous. Prior to the study, we conducted a survey showing screenshots of five different avatars that varied in gender, skin tone, and facial features, and asked participants (n = 27) on their perceptions of each avatar’s gender presentation, racial category, and whether they felt comfortable being visually represented by the avatar in a virtual environment. We created the uniform avatar based on features that resulted in the highest perceived neutrality in gender, racial, and comfort in representation. A detailed description of the items, results, and sample avatars from the pre-test can be found in Appendix A. The final uniform avatar had no hair (to avoid racial marking tendencies, MacLin & Malpass, 2001) and had a neutral (given the available option) skin color (to be racially ambiguous).¹

Figure 1.

The uniform avatar, an example female customized self-avatar, and an example male customized self-avatar that participants embodied for some of their weekly discussion sessions.

Open in new tabDownload slide

Procedure

Participants selected a discussion group that fit their schedule and availability, resulting in eight consistent groups that met weekly for 8 weeks and varied in size from 9 to 14 members (M = 12.63, SD = 1.77). Two training sessions were held in the first 2 weeks of the course, during which participants were taught how to use the ENGAGE interface and navigate the virtual environment. During these training sessions, the teaching staff was available to assist in real-time both via video conferencing and within the virtual environment when participants faced technical mishaps. There was also a simultaneous Zoom call open during all discussion sessions, where participants could pull off their headsets and ask for technical support (Figure 2).

Figure 2.

A Zoom window with a subset of participants in their HMDs. A Zoom technical support call was open during all in-VR activities. Faces are blocked for the sake of privacy.

Open in new tabDownload slide

The structure of weekly activities varied with course content (Table 1). The sessions involved discussion with either the whole group or smaller groups of three to four, and typically followed a three-question format where participants were asked what they liked, what they were concerned about, and how what they learned might influence the future.² The discussions allowed for preservation of nonverbal spatial constraints like interpersonal distance, head orientation, and spatialized sound (Figure 3, bottom right). Some sessions leveraged physical activity affordances of ENGAGE, including working together on a shared object (Figure 3, top left), creating new computer graphic content together (Figure 3, top right), and design-thinking with a shared whiteboard and stickies (Figure 3, bottom left).

Figure 3.

Participants represented either by their customized self-avatar or a uniform avatar (top left) interacting with immersive effects/3D objects, (top right) drawing in 3D space, (bottom left) utilizing a whiteboard, and (bottom right) having a discussion during the weekly sessions. The bars floating above the avatar are blocking the participants’ names for the sake of privacy.

Open in new tabDownload slide

Each week, groups were assigned to one of the two avatar conditions (self vs. uniform) via a Latin square randomization scheme that ensured that each group spent 4 weeks in each condition, counterbalanced across weeks and meeting days (Table 2). In each session, all members of a group wore either their self or uniform avatar to attend the discussion.

Measures

Multiple aspects of individuals’ behaviors and attitudes were measured at the start of the study (pre-test), and during and after each of the eight weekly sessions (motion and weekly surveys; see Table 3).

Weekly repeated measures

Individual ratings were obtained after each weekly VR session through analysis of behavioral motion time series or surveys. To reduce fatigue, repetitiveness, and burden, item sets for each construct were purposely designed to be brief (Conner & Lehman, 2012).

Nonverbal behavior: motion synchrony

Following prior work on synchrony in VR (Miller et al., 2021; Sun et al., 2019), motion synchrony was first calculated for each pair of individuals in a session. Specifically, we calculated the Spearman correlation between all measurements of two individuals’ avatar head speeds obtained every one-thirtieth of a second during the 30-min session (30 Hz) for all offsets of ±2.5 s (not including offset = 0). These 150 correlations were then averaged to obtain a synchrony measure for each pair of participants for each week (pre-registration at https://osf.io/3c4aj/). Synchrony for a given individual on a given week was then calculated as the average of the synchrony scores for all pairs from a given session that included that individual. A detailed description of how motion synchrony was calculated can be found in Appendix B.

Entitativity

Entitativity was measured by seven items adapted from Rydell and McConnell (2005) using a 7-point Likert scale (1 = Strongly disagree, 7 = Strongly agree). Sample items include “My discussion group is important to its members” and “Members of my discussion group are affected by the behaviors of other members.” Weekly entitativity scores were calculated as the mean of the seven items (Cronbach’s α = 0.9), with higher scores indicating greater entitativity.

Self, social, and spatial presence

Self, social, and spatial presence were measured by items adapted from prior work (Herrera et al., 2020; Oh et al., 2019) using a 7-point Likert scale (1 = Strongly disagree to 7 = Strongly agree). Self-presence was measured as the level of agreement with two items: “I felt like my avatar’s body was my own body,” and “When something happened to my avatar, I felt like it was happening to me.” Social presence was measured as the level of agreement with two items, “I felt like I was in the same room as my classmates,” and “I felt like my classmates were aware of my presence.” Spatial presence was measured as the level of agreement with the two items: “I felt like I was really there inside the virtual environment” and “I felt as if I could reach out and touch the objects or people in the virtual environment.” Weekly scores for each of the three types of presence were calculated as the mean of the two items, with higher scores indicating greater perceived presence. Internal consistencies (calculated using Spearman–Brown formula, as recommended for two-item measures, Eisinga et al., 2013), across all participants and weeks were 0.86 for self-presence, 0.80 for social presence, and 0.85 for spatial presence.

Enjoyment

Enjoyment was measured as the level of agreement with two items: “How much did you like interacting in the virtual environment?” and “How much fun did you have in the virtual environment?” using a 5-point Likert scale (1 = Not at all, 5 = Extremely). Weekly scores for enjoyment were calculated as the mean of the two items (Spearman–Brown coefficient = 0.91), with higher scores indicating greater enjoyment in the virtual environment.

Realism

Perceived photorealism of the virtual environment and people, which refers to the rendering quality of the image, was measured weekly by a single item adapted from Nowak et al. (2009) using a slider scale (0 = Cartoon-like, 100 = Photorealistic). We used a one-item scale that focuses on one of the dimensions of realism, photorealism. There are multiple dimensions of realism that have distinct effects on people’s perceptions of the mediated environment and characters. In this study, the most critical dimension was the level of realism of the avatar that dealt not with whether it was a fantasy character or could occur offline, but instead the quality of the imagery. Because the original scale included other items that may relate to other dimensions, those items were excluded.

Individual differences measures

Individual differences measures were obtained during the pre-test and through analysis of motion data obtained throughout the entire study period.

Prior relationships

The number of discussion group members individuals were familiar with prior to the course was measured at the start of the study (e.g., 0, 1, 2, 3 people), to evaluate if there was an influence of having prior familiarity with any group members on how the dependent variables evolve over time (M = 0.98, SD = 1.24).

Prior VR use

Individuals’ prior experience with VR was measured at the start of the study. Individuals were asked if they had ever used a VR headset before (1 = Yes, 0 = No), and if they had, how many times they had experienced VR (n₀ = 41, n₁ = 6, n₂ = 6, n₃ = 7, n₃₊ = 20, declined to or did not respond = 1).

Group identification

Individual ratings for group identification, a person’s identification to a group they belong to, such as an organization, club, or sports team, were measured at the start of the study using eight items adapted from an in-group identification scale and an organizational identification scale (Leach et al., 2008; Mael & Ashforth, 1992). Sample items, each answered using a 7-point Likert scale (1 = Strongly disagree, 7 = Strongly agree), included: “The fact that I am part of my group is an important part of my identity” and “When I talk about my group, I usually say ‘we’ rather than ‘they.’” Individual group identification scores were calculated as the mean of the eight items (Cronbach’s α =  0.89), with higher scores indicating greater identification with the group (M = 5.35, SD = 0.97).

Other individual differences

Additional individual differences predictors were examined in our preliminary models, including gender, computer and online learning self-efficacy, loneliness, Zoom fatigue, and video game usage, but were eventually trimmed from the reporting because none of these variables were related to baseline levels, rates of change, or avatar effects for any of the seven outcomes.

Data analysis

Results

Results from growth models with time-varying predictors [week and avatar (uniform = 1 vs. self = 0)] and time-invariant predictors (prior relationships, prior VR experience, and group identification) are presented separately for all seven outcomes (synchrony, entitativity, self, social, and spatial presence, enjoyment, and realism). Plots of the raw data overlaid with relevant prototypical trajectories are given in Figure 5.

Synchrony

The prototypical participant’s synchrony decreased from an initial value of γ₀₀ = 0.0381, p = .006 (on the −1 to 1 correlation scale) at a rate of γ₁₀ = −0.0034, p < .001, per week. There was a significant effect of avatar manipulation on synchrony, such that participants synchronized less, γ₂₀ = −0.0122, p < .001, in sessions with uniform avatars than in sessions with self-avatars. There was no evidence that individual differences in prior relationships, prior VR experience, or group identification were uniquely related to baseline levels of synchrony (ps > 0.21), or rates of change in synchrony (ps > 0.14). Figure 4 indicates the relationship of synchrony to time offset.

Figure 4.

Effect of avatar on synchrony. This plot demonstrates that as the time offset of motion signals shifts away from zero (i.e., as one looks toward the right and left away from the center), synchrony (Y-axis) decreases. In this plot, synchrony for each group in each session is traced as a separate partially transparent line (60 total). The average of all sessions for a given avatar condition is the darker line, with the ribbon indicating 95% confidence intervals based on the underlying distribution. Each line is produced as the average of all unordered pairs in that session (from 6 to 78, M = 36.8, SD = 17.9), which is itself calculated from about 30 min of data per participant.

Open in new tabDownload slide

Entitativity

The prototypical participant’s entitativity increased from an initial value of γ₀₀ = 5.010, p < .001 (on a 7-point scale) at a rate of γ₁₀ = 0.059, p = .002 points per week. There was no evidence that the avatar manipulation influenced entitativity, γ₂₀ = −0.022, p = .62. A prototypical trajectory showing how entitativity changed over time is in Panel A of Figure 5. Individuals with more prior relationships had higher baseline levels of entitativity, γ₀₁= 0.22, p = .04, as evident in the contrast between the blue solid (+1SD on prior relationships) and dashed (−1 SD on prior relationships) lines in Panel A of Figure 5. There was no evidence that individual differences in group identification or prior VR experience were uniquely related to baseline levels of entitativity (ps > 0.07), or that individual differences in prior relationships or prior VR experience were uniquely related to the rate of increase in entitativity (ps > 0.59).

Figure 5.

Dependent variables over time. (A–F) Change over time and in relation to the avatar manipulation for each of the six survey outcome variables. Individual trajectories (raw data) are indicated by the light gray lines. Model-implied prototypical trajectories are indicated by the thick black lines and are shown for two hypothetical cases where the avatar conditions alternated weekly (and thus produce oscillations). When individual differences were related to baseline or rate of change, additional model implied trajectories for individuals 1 SD above (solid color) and 1 SD below (dashed color) the average score are indicated by thick colored lines.

Open in new tabDownload slide

Presence

Self-presence

The prototypical participant’s self-presence increased from an initial value of γ₀₀ = 3.75, p < .001 (on a 7-point scale) at a rate of γ₁₀ = 0.101, p = .004 points per week. There was a significant effect of the avatar manipulation, such that individuals reported lower self-presence when using uniform avatars than self-avatars, γ₂₀ = −0.21, p = .021.

Social presence

The prototypical participant’s social presence increased from an initial value of γ₀₀ = 5.23, p < .001 (on a 7-point scale) at a rate of γ₁₀ = 0.068, p = .014 points per week. There was no evidence that the avatar manipulation influenced social presence, γ₂₀ = 0.055, p = .40. Individuals with higher group identification had higher baseline levels of social presence, γ₀₃ = 0.25, p = .021, as evident in the contrast between the green solid (+1 SD on group identification) and dashed (−1 SD on group identification) lines in Panel C of Figure 5. There was no evidence that individual differences in prior relationships or prior VR experience were uniquely related to baseline levels of social presence (ps > 0.30).

Spatial presence

The prototypical participant’s spatial presence increased from an initial value of γ₀₀ = 4.33, p < .001 (on a 7-point scale) at a rate of γ₁₀ = 0.083, p = .014 points per week. There was no evidence that the avatar manipulation influenced spatial presence γ₂₀ = 0.069, p = .38.

There was no evidence that individual differences in group identification, prior relationships, or prior VR experience were uniquely related to baseline levels of self (ps > 0.35) or spatial (ps > 0.106) presence, or rates of increase in self (ps > 0.63), social (ps > 0.49), or spatial (ps > 0.43) presence.

Prototypical trajectories showing how self-presence changed over time for hypothetical individuals who alternated weekly between the two avatar conditions are shown as bold black lines in Panel B of Figure 5. Prototypical trajectories showing how social and spatial presence changed over time are in Panel C and D, respectively, of Figure 5.

Enjoyment

The prototypical participant’s enjoyment increased from an initial value of γ₀₀ = 3.057, p < .001 (on a 5-point scale) at a rate of γ₁₀ = 0.061, p = .002 points per week. There was a significant effect of the avatar manipulation, such that individuals reported greater enjoyment during weeks when using uniform avatars than self-avatars, γ₂₀ = 0.16, p = .011. Prototypical trajectories showing how enjoyment changed over time for individuals who alternated weekly between the two avatar conditions are shown as bold black lines in Panel E of Figure 5. Individuals with more prior relationships had higher baseline levels of enjoyment, γ₀₁= 0.22, p = .023, and more prior VR experience had higher baseline levels of enjoyment, γ₀₂= 0.24, p = .035, as evident in the contrast between the colored lines (blue prior relationships, yellow prior VR experience; +1 SD solid, −1 SD dashed) in Panel E of Figure 5. There was no evidence that individual differences in group identification were uniquely related to baseline levels of enjoyment (p = .37). Although there was no evidence that individual differences in prior relationships were uniquely related to rate of increase in enjoyment (p = .96), the enjoyment of individuals with more prior VR experience did not increase as much as those with no prior VR experience, γ₁₂ = −0.044, p = .0079, as seen in differential rates of increase of the yellow solid and dashed lines.

Realism

The prototypical participant’s perception of realism increased from an initial value of γ₀₀ = 35.62, p < .001 (on a 0–100, cartoon-like to photorealistic scale) at a rate of γ₁₀ = 0.88, p = .057 points per week. There was a significant effect of the avatar manipulation, such that individuals reported lower realism (i.e., more “cartoon-like”) when using uniform avatars than self-avatars, γ₂₀ = −2.028, p = .035. Prototypical trajectories showing how realism changed over time for individuals who alternated weekly between the two avatar conditions are shown as bold black lines in Panel F of Figure 5. Individuals with more prior relationships had higher baseline levels of realism, γ₀₁ = 5.89, p = .0106, as evident in the contrast between the blue solid (+1 SD on prior relationship) and dashed (−1 SD on prior relationship) lines in Panel F of Figure 5. There was no evidence that individual differences in group identification or prior VR experience were uniquely related to baseline levels of realism (ps > 0.41), or rate of change in realism (p = .108). The realism of individuals with more prior relationships increased less than that of individuals with fewer prior relationships, γ₁₁ = −0.704, p = .0405, as evident in the contrast between the slopes of the blue solid and dashed lines in Panel F of Figure 5.

Discussion

Study 1 examined the role of time and transformed visual appearance on participants’ experience and group dynamics. Every week for 8 weeks, 81 participants, separated into eight groups, met for approximately 30 min in a CVE to engage in a discussion on the course material. Overall, the results showed that almost all measures, including entitativity, presence (self, social, and spatial), enjoyment, and realism increased over time. The remaining measure, synchrony, decreased over time. These effects underscore the critical role that time plays in how people’s experience in VR evolves. Given this, it is possible that once participants adapt to the medium and are no longer uncomfortable with the novelty of the technology, they can reap the advantages that VR and CVEs provide and feel more presence and connectedness.

The investigation of synchrony demonstrated that motion synchrony occurs even when mediated in VR, consistent with previous research. It also indicated that synchrony both decreased over time and was lower in the uniform avatar condition (i.e., visually similar to one another) compared to the self-avatar condition. This may mean that synchrony serves a balancing function, where synchrony acts as a tool to increase entitativity when needed (Dale et al., 2020). Indeed, it is possible that transforming nonverbal behavior to induce synchrony can improve entitativity (Bailenson & Yee, 2005). However, future work should examine these possibilities.

Furthermore, when participants were in the uniform avatar, they had lower motion synchrony, reported lower self-presence, and perceived the virtual environment and others as more cartoon-like (less photorealistic), but reported greater enjoyment interacting in the virtual environment. Furthermore, while entitativity did increase over time, visual uniformity did not have an effect on entitativity. Similarly, while those who had prior relationships with group members did start with a higher level of entitativity, there was no evidence that this individual difference was uniquely related to the increase in entitativity.

Returning to the TSI paradigm, having limited cues about others’ offline bodies in a virtual environment may make differences among group members more salient and interfere with the group identification process, though this does not hold true over time. It is possible that sharing identical visual features with everyone in the group creates a more recreative environment (i.e., leading to lowered photorealism), which may place less stress on how an individual is presented in front of others and less emphasis on individual behavior, ultimately leading to a lowered sense of self-presence and greater enjoyment. If all members of the group look the same, the stress of individuality and being present in the environment may be distributed across group members. The visual cue that every member of the group shares identical features may lower an individual’s sense of ownership of their self and embodiment, affecting their sense of self as an individual more than how it affects their identification with the group.

Visual uniformity or similarity is often taken into consideration when wanting to create a stronger sense of group identity (Kim, 2009). However, how this transformation influences social interactions and behavior in virtual environments with time and use has remained open to question. It could be argued that visual appearance used in certain contexts can serve specific purposes in shaping social interactions. Given avatar appearance did not have an effect on entitativity and lowered motion synchrony, it may not make sense to use visual cues as a unifier for group identification. Conversely, avatar appearance did have an effect on variables such as self-presence, which, given its role in immersion, and in turn, attention in and connection to the environment, it may be unfavorable to have a uniform avatar in a group setting and suppress individuals’ visual cues. At the same time, if the goal of social interactions is for enjoyment purposes, having uniform avatars may allow people to focus less on their individual role in a group setting and more on enjoying the task at hand. However, we note that further research is needed to better understand how such transformations impact enjoyment. While we designed our own measurement of enjoyment, it may require a more nuanced understanding to draw definite conclusions about how it interacts with shared visual cues.

Lastly, we found that it is important to consider individual differences in how people’s experience in VR changes over time, as evidenced by our findings that individual differences accounted for different initial baselines and differences in how people’s experiences evolved to varying degrees.

Study 2

Complementary to Study 1’s focus on transformation of avatar appearance, Study 2 focuses on the transformation of environmental context. Based on the preliminary findings of Study 1, we generated and pre-registered hypotheses related to time and the virtual environment for Study 2 (pre-registration at https://osf.io/s37xc). As Table 4 lays out, given the beneficial effects that being in spacious, panoramic environments, and outdoor, natural environments provide, we hypothesized that participants will be able to interact with one another more freely in panoramic environments than in constrained environments. We anticipate that this increase in interaction and engagement will foster a greater sense of entitativity and enjoyment. Similarly, outdoor, natural environments have been shown to have restorative properties, which should improve perceived restorativeness for these environments.

In this study, participants at each weekly session were exposed to one of four possible types of virtual environments (2 spaciousness × 2 setting conditions): a panoramic outdoor environment, a panoramic indoor environment, a constrained outdoor environment, and a constrained indoor environment. Along with the dependent variables examined in Study 1, Study 2 examines the influence of time and virtual environment on additional variables such as perceived restorativeness and affect (pleasure and arousal).

Method

Participants

Participants were 171 university students enrolled in a 10-week course about VR. At the beginning of the course, students were invited to participate in an IRB-approved study of how repeated exposure to VR influenced their individual and group behavior. While all students who were part of the course took part in all the VR activities, only those who consented to participate in the study had their data included in the study. Of the 171 students in the course, 158 consented to participate in the study. The 137 participants who participated in five or more of the eight weekly sessions (M = 78, F = 59) were between 18 and 49 years old (M = 20.9, SD = 2.78; n_18∼20 = 62, n_21∼23 = 71, n_24∼49 = 5) and identified as Asian or Asian-American (n = 47), White (n = 41), multiracial (n = 19), African, African-American, or Black (n = 12), Hispanic or LatinX (n = 8), Native Hawaiian or other Pacific Island (n = 5), Indigenous/Native American, Alaska Native, First Nations (n = 2), declined to or did not respond (n = 2), Middle Eastern (n = 1), and a racial group not listed (n = 1). Participants had varying levels of experience with VR (n₀ = 50, n₁ = 29, n₂ = 23, n_3∼10 = 26, n_20∼50 = 4, n₉₀ = 2, n₁₀₀ = 4). Prior to the course, 86 participants were not familiar with anyone in their discussion group and others reported knowing one (n₁ = 40) or more members (n₁ = 13, n₃ = 5, n₄ = 6, n₅ = 1, n₇ = 1).

Virtual environments

As in Study 1, weekly discussion sessions were hosted in ENGAGE. There were four types of virtual environments (2 spaciousness × 2 setting): (a) panoramic outdoors, (b) panoramic indoors, (c) constrained outdoors, or (d) constrained indoors (Figure 6). Each environment was built by research personnel using 3D objects. In total, there were 192 uniquely-built environments that differed in size of moving area and height. As suggested by Reeves et al. (2015), as variance in media is growing, so should variance in media research. As the authors argue, any media chosen as a stimulus can have a list of features that may be psychologically relevant and interact with the primary factors in an experiment. Selecting one idealized representative stimuli from each end of the distribution can increase Type I, II, and III errors. Through stimulus sampling and statistical methods (e.g., using a mixed statistical model that factors in fixed and random effects), we are able to better understand media that may be found in real-world experiences (Judd et al., 2012; Westfall et al., 2014). To evaluate whether these manipulations work across a range of environments and examine generalization of results across stimuli, we created 192 unique environments which were rigorously controlled in terms of our theoretical variables related to context, but also contained diverse thematic features, as opposed to relying on a single stimuli manipulation.

Figure 6.

Environment types used every session. There were four possible types of virtual environments (2 spaciousness × 2 setting): (1) panoramic outdoors, (2) panoramic indoors, (3) constrained outdoors, or (4) constrained indoors.

Open in new tabDownload slide

The moving area of the environments was measured by adding markers to the corners and ceilings of the environments inside ENGAGE and then calculating the areas using the positional data of the markers. By design, the panoramic environments (n = 48, M = 39494.52, SD = 51231.27) were 1778.67% larger than constrained environments (n = 48, M = 2102.26, SD = 9367.35) [t(50.139) = 4.97, p < .0001] and were 84.17% greater in max heights (n = 48, M = 29.062, SD = 26.05) than those of constrained environments (n = 48, M = 15.78, SD = 15.78), [t(77.41) = 2.781, p = .0068]. As a design artifact (i.e., indoor environments could not be infinitely large), the moving areas of the outdoor environments (n = 48, M = 36745.89, SD = 53455.15) were 657.51% larger than the indoor environments (n = 48, M = 4850.89, SD = 7030.96), (t(48.63) = 4.099, p = .00016), but did not differ in max height [outdoor environments (n = 48, M = 22.33, SD = 23.45), indoor environments (n = 48, M = 23.56, SD = 21.29), [t(93.13) = 0.27, p = .79]].

Avatar

All participants were asked to use the customization tool to make an avatar that looked and felt like their offline selves.

Procedure³

Participants selected a discussion group that fit their schedule and availability, resulting in 24 groups that met weekly for 8 weeks and varied in size from five to eight members (M = 6.71, SD = 0.81). The sizes of actual attended groups ranged from 2 to 11 members (Week 1 M = 6.38, SD = 1.47; Week 2 M = 6.25, SD = 1.48; Week 3 M = 6.08, SD = 1.18; Week 4 M = 6.29, SD = 1.23; Week 5 M = 6.38, SD = 1.35; Week 6 M = 6.25, SD = 1.33, Week 7 M = 6.00, SD = 1.50; Week 8 M = 5.75, SD = 2.01). Each week, each group was assigned to a set of four between-subject conditions (2 × 2 design) via a Latin square randomization scheme that ensured each group experienced each condition once and that each condition appeared equally across the weekly schedule (Table 5). The sessions were led by one of three instructors. Each instructor led the same eight groups every week.

A training session was held in the first week of the course, during which participants were guided through how to use the ENGAGE interface and the controllers to navigate the virtual environment. As in Study 1, during these training sessions, the teaching staff was available to assist via Zoom when participants faced technical mishaps in hardware and software.

The first discussion session began in the second week, during which participants completed a series of small-group activities to further familiarize them with the ENGAGE environment and its tools. All discussions, except in the fifth session, had a creative activity, which involved creating, brainstorming, or prototyping an idea using the tools available on ENGAGE (e.g., drawing with the 3D pen, bringing in 3D models, writing on whiteboards). The 30-min sessions were divided into a 10-min full-group discussion and recap of the course material, a 15-min individual creative activity based on a prompt, and a 5-min sharing of the final product of the activity portion (Table 6).