Exploring Social Interaction With a Tangible Music Interface

RESEARCH HIGHLIGHTS

• Presentation of a video-based field study describing cross-group interaction.

• Description of exemplar vignettes of video for illustrating liminal situations in social interaction.

• Discussion of the nature of different levels of cross-group interaction and their relevance for social interaction and collaboration.

• Lessons learned in how best to enable liminal situations in the design of collaborative tangible user interfaces for promoting social interaction and collaboration.

1. INTRODUCTION

A new generation of multi-user technologies such as interactive tabletops and tangible user interfaces (TUIs) are increasingly being used in public settings to promote group activities. Interactions in these settings tend to be casual, unstructured, walk-up-and-use, dynamic and fast-paced, as described in an emerging corpus of studies done in real-world settings, often referred to in-the-wild studies (e.g. Bengler and Bryan-Kinns, 2015; D'Angelo et al., 2015; Hinrichs and Carpendale, 2011; Hornecker and Stifter, 2006; Marshall et al., 2011a). However, despite some discussion of tensions between strangers trying to use a tabletop application at the same time (Marshall et al., 2011a), there is a paucity of detailed analysis of how groups share access to these technologies and manage transitions of use. In addition, previous tabletop studies have investigated social interaction using relatively simple systems designed for casual users (Hinrichs and Carpendale, 2011; Hornecker, 2008; Marshall et al., 2011a). To our knowledge, little is known about the nature of social interaction with more complex tabletop systems in public settings.

In this article, we draw on the extended usage of the sociocultural notion of liminality and liminal experiences (Turner, 1964), proposed by Thomassen (2009), for understanding social interaction in public settings. Liminality has been used in anthropology to refer to any betwixt and between situation or object, with rites of passage as an early and influential example (Thomassen, 2009; Turner, 1964). Thomassen (2009) outlines that liminality, understood as situations of transition, is applicable to various dimensions, such as space (e.g. specific places, thresholds, areas, borderlands, larger regions), time (e.g. sudden moments, longer periods, epochs) or types of subjects (e.g. single individuals, social groups). We borrow the term to understand the situations of group changeovers in space and time that occur when people use a sophisticated musical interactive exhibit—the Reactable—in a public museum setting. Our approach aligns with the definition by MacDonald (2014) of liminal spaces within interactive exhibits as spaces in-between two areas or states, for example changes between two physical areas (e.g. exhibits spaces) or two group or visitors’ states (e.g. staring at an exhibit or interacting with an exhibit). The rationale of this research is that predicting the number of visitors in public settings can be a challenge, thus designing interactive exhibits that promote on-demand fluid transitions can facilitate a smooth flow between exhibits. Within an environment of visitors who come and go, exhibits that can only be used by one group at a time can disrupt the flow of traffic and create a bottleneck effect. Facilitating a fluid group change can also be helpful for supporting cross-group social interaction (e.g. shared experiences between strangers), in particular collaboration (e.g. parallel individual tasks, performed by members from different groups, towards a common goal offered by the exhibit) and learning (e.g. understanding how to use the exhibit by seeing and interacting with other groups).

Previous work has studied liminality in technology-mediated performance focusing on individual practice (Broadhurst, 2006). The concept of liminality can, Broadhurst (2006) argues, help us to understand digital performance: the space between the performer and the technology as meaningful for understanding the connections and limits between both entities. In this article, we are interested in exploring musical tabletops within multi-user situations in public settings, and the ‘spaces’ in-between groups of visitors and the technology as a potential situation for promoting social interaction and collaboration. Up to the present, collaborative music performance has been researched using highly constrained systems for visitors (cf. Blaine and Fels's ( 2003) survey on collaborative instruments for novices). Musical tabletops¹ that have a tangible interface appear to be a promising platform for social interaction and collaboration among visitors, with the potential to promote rich and complex interaction.

Through detailed observation of interactions with a musical tabletop, our findings point to rich social interaction at different levels of cross-group interaction. Cross-group interaction was either synchronous (simultaneous between groups) with fluid transitions and overlaps in use between groups or asynchronous (lapse of time between two groups’ interactions). We argue that fluid cross-group interaction is as important as inter-group interaction and is an overlooked aspect when designing tabletops and TUIs for public settings. Here, we prefer to use the term liminal situations over liminal spaces as it is more accurate with our observations of physical or experiential group change in the passage of space and time, in particular cross-group actions over time within a place. Liminal situations include physical change in group constellation (e.g. synchronous and asynchronous interaction) and experiential group change between and within exhibits (e.g. auditory transitions between exhibits or transitions between types of audio delivery within exhibits). For example, one group leaving and a new group taking over (transition) or the moment when a group member puts on headphones or takes them off (change of state). Designing liminal situations for fluid cross-group interactions in interactive exhibits can promote a rich palette of shared and collaborative experiences. Accordingly, we provide design recommendations on how to best support liminal situations of group change in public environments. This study focuses on the nuances of non-verbal interaction between groups that emerged from manipulating a musical tabletop. It is beyond the scope of this article to investigate communication aspects such as verbal vs non-verbal communication or the level of engagement, which has been studied extensively by other scholars (e.g. Block et al., 2015). It is also beyond the scope of this article to assess the resulting quality of the musical output, and thus we do not discuss the aesthetic value of the musical outcome.

2. BACKGROUND

2.1. Collaboration with interactive surfaces and tabletops

In computer-supported cooperative work (CSCW), work coupling refers to the level of intensity of communication required during the performance of a task, related to the level of interaction between group members (Neale et al., 2004). According to Neale et al. (2004) and in order from less to more demanding work coupling, there are five levels: light-weight interactions, information sharing, coordination, collaboration and cooperation. Collaboration entails a common goal, yet tasks are interdependent and work is done individually. By contrast, cooperation demands the highest level of work coupling and thus the highest quality of communication. Goals and tasks are shared, and consultation with others is needed before proceeding with the work. In this article, we focus on music performance with a tangible music interface as a collaborative task, in which visitors are interdependent with the common goal of creating music or following up the music created by others.

There is a large corpus of work on interactive surfaces and tabletops addressing a range of topics, such as size and orientation of display (Kruger et al., 2004), territoriality (Scott et al., 2004), interaction techniques (Nacenta et al., 2010) and awareness (Hornecker et al., 2008). Work on tabletop awareness is a topic of research in the literature that provides evidence of the connection between awareness and resolving interferences with others’ activities for successful collaboration (Hornecker et al., 2008). Tabletop studies tend to be group-centric, and thus the nature of awareness in cross-group interaction and beyond single groups is an open question, a topic that is investigated in this article.

In CSCW and HCI, awareness is a widely used term. An early definition of awareness was provided by Dourish and Belloti: “an understanding of the activities of others, which provides a context for your own activity” (Dourish and Belloti, 1992, p. 1). For successful collaborative applications, Dourish and Belloti (1992) suggested that mechanisms should be provided that could address (i) group coordination, (ii) data sharing and (iii) the provision of information about group and individual actions. In particular, shared feedback appears as a suitable mechanism for collaboration that provides awareness information, which can be meaningful in both synchronous and asynchronous interaction. More recent definitions of awareness include the notion of real-time awareness. For example, Gutwin and Greenberg provide a framework for the usability design of CSCW systems, focusing on workspace awareness, defined as “the up-to-the-moment understanding of another person's interaction with a shared workspace” (Gutwin and Greenberg, 2002, p. 412). Yuill and Rogers define the awareness of others as “the degree to which awareness of users' ongoing actions and intentions is present or made visible moment-to-moment” (Yuill and Rogers, 2012, p. 1:4), particularly emphasizing the intentionality of individuals. These three definitions of awareness foresee the necessity of perceiving and understanding the activities of others, in real time. Robertson (2002) discusses the relations between awareness, human perception and the public availability of actions and resources, lessons that can be utilized to facilitate the design of CSCW systems, which provide availability of resources supporting people's peripheral awareness.

A seminal ethnographic study about the importance of awareness during collaborative activity in the workplace was conducted by Heath and Luff (1992). Awareness has been traditionally researched in work settings (Heath et al., 1992; Schmidt, 2002). This research can be transferred to public settings, as some of the newer work by vom Lehn et al. (2001) in museums has shown. For example, there is peripheral awareness between visitors who mutually monitor each others’ behaviour and participation.

In-the-wild studies of large displays in public settings have looked into topics that relate to visitors’ social interactions (discussed more broadly in Section 2.2), such as awareness and collaboration (Huang and Mynatt, 2003), issues of attention and visibility (Dalton et al., 2015; Huang et al., 2008; Müller et al., 2010), also referred as display blindness (Dalton et al., 2015; Müller et al., 2009) and conflict management (Peltonen et al., 2008). Thresholds between phases of attention and issues in crossing these thresholds are discussed by Brignull and Rogers (2003) and Müller et al. (2010) as a design issue, which relates to liminality and the design of liminal spaces within public settings discussed here. Certain information needs to be provided, such as how to interact with the artefact, time needed, the type of experience and how to leave easily (Brignull and Rogers, 2003). Smooth transitions and overlaps between groups are reported by Peltonen et al. (2008). Accordingly, transitions emerge typically from floor and turn-taking, in which group change is because there are spaces available so that the existing group hands over the floor to the next group, while overlaps tend to emerge from two groups that work in parallel, and a conflict between them needs to be solved.

Visibility of interactions has been researched in both large public displays and tabletop studies. For example, Hinrichs and Carpendale (2011) distinguish between group actions and individual actions in multitouch gestures on an interactive tabletop exhibit, where collaborative gestures are characterized by groups avoiding the occlusion of an item's view from the group members. Hinrichs and Carpendale (2011) identified that generating distinguishable gestures for other people, including strangers, serves to mentor or promote imitation. Brignull and Rogers (2003) explored how to create smoother transitions between activities and spaces around a public display, from onlooking to participant roles. Three activities were identified: (i) peripheral awareness activities, in which visitors peripherally notice the display's presence from an another location, (ii) focal awareness activities, in which visitors pay more attention to the display's presence and learn from it and (iii) direct interaction activities, in which visitors interact with the display. Brignull and Rogers (2003) recommend a good visibility of the display for a fluid flow between the three spaces, for example by avoiding bodily occlusion or by seeing others interactions with it. Brignull and Rogers (2003) and Müller et al. (2012) identify as requirements for a successful display that not only should it be visible but it should also raise understanding and motivation. In addition, the honeypot effect raises awareness of the public display to passers-by from viewing a group of people interacting with or watching around the exhibit (Brignull and Rogers, 2003; Müller et al., 2012).

Tabletop collaboration has been investigated in terms of territoriality (Scott et al., 2004), orientation (Kruger et al., 2004) or coupling styles between individual and group work (Tang et al., 2006). Scott et al. (2004) presented three tabletop territories in group collaboration: personal, group and storage territories, Kruger et al. (2004) identified intentional communication with a lack of verbal communication when orienting an object to oneself, to another person or to the group and Tang et al. (2006) discussed implications for design in terms of supporting fluid transitions between coupling styles. There are a number of studies of tabletops in public settings focusing on social interaction in groups. These studies include how visitors manipulate media files on a large multitouch table, focusing on visitors’ gestures and their social context (Hinrichs and Carpendale, 2011); how visitors interact with a multitouch table for tourism planning, focusing on situated group interactions (Marshall et al., 2011a); how visitors interact around a largely information-centric table, with conversations focused on how to control the interface elements (Hornecker, 2008); or how visitor groups collaborate with a puzzle game on a multitouch tabletop, focusing on visitor engagement and learning (Horn et al., 2012). These studies on tabletop collaboration and social interaction tend to focus on inter-group interaction. Little is known of to what extent these concepts are suitable for cross-group social interaction and collaboration, which is explored in this article.

In-the-wild tabletop studies in public settings have considered conflict management and resolution between groups as a mechanism of group change, such as Marshall's et al. (2011a) work on a multitouch tabletop interface in a tourist centre and Block's et al. (2015) work on multitouch tabletop science exhibits. These studies focus on co-located synchronous multitouch interaction and generally from an inter-group perspective. Block et al. (2015) identify overlaps between groups and measure the percentage of the overlap as an indicator of collaboration, where collections of visitors that mutually overlap for over 50% of their time are considered a group. By contrast, our approach focuses on liminal situations and cross-group interaction in both synchronous and asynchronous interaction, and the benefits of using tangible interaction for this. We argue that the richness of social interaction in the real world can be modelled with tabletops and TUIs if transitory states and spaces are enabled with the interactive exhibit to support better liminal experiences, of which some of them can be collaborative. Block et al. (2015) instead focus on overlaps understood as a shared and stable collaborative experience between strangers, rather than a shared and collaborative experience about change. To our knowledge, investigating cross-group interaction with a tabletop system in a real environment is novel. A similar study on transitions between groups, but with large displays, was conducted by Peltonen et al. (2008); however, we contribute a detailed look at the different levels of space and time thresholds between groups.

2.2. Social interaction in museum settings

There is a shift of the role of museums from object-centred to visitor-centred (Bedford, 2014; Perry, 2012; vom Lehn, 2006): visitor experience is becoming a principal aspect of exhibit design. Social interaction plays a key role in how people make sense of exhibits, both between companions and strangers (Falk and Dierking, 2012; Perry, 2012; vom Lehn, 2006; vom Lehn et al., 2001). However, the behaviour of visitors interacting with computer-based exhibits is difficult to predict because of their novelty (Hindmarsh et al., 2005; vom Lehn et al., 2001): visitors may interact differently than the designer's expectations. In particular, interactive installations need to be immediately apprehendable, so that visitors can quickly determine whether it is worthwhile to interact and understand how to get started (Allen, 2004). Museum educators have also become sceptical of the traditional ‘button-pushing’ didactic model known from science museums, and aim for installation design that supports discovery, sense making, imagination and constructive learning (Bedford, 2014; Falk and Dierking, 2012; Perry, 2012).

The active relationship between audience and exhibits, creative audience engagement, has been investigated in interactive art (Costello and Edmonds, 2007; Edmonds et al., 2006). In interactive art, a range of levels of audience's control of the system's behaviour is possible, from controlled and predictable, to unpredictable behaviours, depending on the system's design (Edmonds et al., 2006). Stimulating playful audience behaviour has been researched as a mechanism for increasing audience engagement (Costello and Edmonds, 2007). The Reactable can be seen as an interactive artwork, with a high level of audience engagement because the audience can control the system's reaction. In addition, it has unpredictable behaviours (cf. Xambó et al., 2013), along with a potential playful activity (creating music) that subsequently promotes playful audience behaviour.

Video has been widely used in social sciences as a research tool for understanding interactions with technology, including seminal work on museum exhibits (Heath et al., 2010). Social interaction has been broadly studied using video-based studies drawing on semiotic, linguistic, sociological or anthropological perspectives. For example, the volume edited by Streeck et al. (2011) included articles by different authors on body movement and talk in different environments, including musical performance (Haviland, 2011). In particular, Haviland's (2011) work looks into the interaction between individual musicians, their bodies and their instruments, which includes the space occupied by those bodies, instruments and audience. In the context of public settings, vom Lehn et al. (2001) analyzed interpersonal communication between companions and strangers drawing on ethnomethodology and conversation analysis using video data from a range of museums and galleries. Design sensitivities are provided by Hindmarsh et al. (2005) for encouraging social interaction between visitors around exhibits. They also discussed the potential of collaborative work between companions and strangers in public settings whether co-located or remote, and the need of creating ‘opportunities of interaction’. This research builds on Hindmarsh et al. (2005) by discussing how to enable liminal situations as a mechanism for promoting a range of opportunities of interaction. This includes the support of smooth overlaps, a need highlighted by Block et al. (2015).

2.3. Social interaction and shareability in TUIs

Dourish (2001) coined the term embodied interaction to refer to TUIs and social computing, in particular how users create and communicate meaning through bodily interaction mediated by these systems. Hornecker and Buur's (2006) tangible interaction framework and Shaer and Hornecker's (2009) review on TUIs highlight that tangible interaction is suited for social and collaborative settings and the importance of supporting social interaction when designing TUIs. As pointed out by Fernaeus et al., the shareability of TUIs involves “designing for collaboration, sharing and social interaction” (Fernaeus et al., 2008, p. 226). With tangible manipulation in TUIs, the world is perceived through tools, which contrasts with pure touch interfaces, even though both may use tactile feedback. This resonates with Gibson's (1966) notion of haptic perception as an active exploration, and the different perceptual experiences between using the body to manipulate digital information and using a tool as an extension of the body to manipulate digital information. Since the popularization of touch-based devices such as smartphones and tablets, users are familiar with both types of interaction. Yet, this study focuses on the implications of tangible input because it resembles more interaction with the physical world.

In tabletop design, features such as enabling joint work and tailorability of parts of the system (Dalsgaard and Halskov, 2014), or group engagement (Block et al., 2015; D'Angelo et al., 2015), are common and salient design goals that relate to inter-group interaction. The design principle of shareability is also discussed (Hornecker et al., 2007). Shareability is linked to the notions of entry points and access points (Hornecker et al., 2007): entry points provide visibility and advanced information of the artefact, while access points provide active engagement with the artefact. Hornecker and Buur (2006) highlight multiple access points as a characteristic of TUIs that distributes control among individuals and invites everyone to join. In this article we use the term access points to indicate components that both invite visitors to interact with an artefact by providing visibility and advanced information, and, at the same time, allow visitors active engagement with the artefact.

2.4. Collaborative tangible music in public settings

There are a number of systems and studies based on TUIs for music performance, of which some are tabletop-based, and some are designed for public settings. With a few exceptions (Jordà, 2008), these systems are highly constrained to facilitate casual visitors’ use.

An early example of a highly constrained musical TUI for novices is Squeezables (Weinberg and Gan, 2001), a set of six squeezable balls, which can be controlled by continuous squeezing and pulling hand gestures. Some examples of highly constrained TUIs for music in public settings include the Music Bottles (Ishii et al., 2001), a set of bottles that contain sounds controlled by opening and closing the bottles; the Audiocubes (Schiettecatte and Vanderdonckt, 2008), a set of interconnected cubes with different functions that trigger sounds; or the Polymetros (Bengler and Bryan-Kinns, 2013), a collaborative musical instrument based on 3–4 individual controllers with illuminated grids that are interconnected, which allow for musical pattern manipulation. Audiocubes was informally tested in an art installation, positioned on a table, by video recording visitors’ interactions and then providing descriptions of different types of interactions. Polymetros was also evaluated in an art museum exhibition by collecting questionnaires, data logging, field notes and video observations in order to assess the collaborative experience.

We also find tabletops for music performance in public settings such as Iwai's Composition of the Table (Iwai, 1999), an audiovisual installation consisting of four tables that can be touched; the Jam'O Drum (Blaine and Perkis, 2000), a drum table for musical improvisation or the Reactable (Jordà, 2008), a tabletop system that uses tangible controls and multitouch input, which implements a virtual modular synthesizer. Previous work suggested that novice users prefer the social experience over virtuosity and expressiveness (Blaine and Fels, 2003). Yet, we argue that observing the use of a complex system among visitors can tell us about rich interactions as found in real environments.

3. THE REACTABLE EXPERIENCE

At WSC, the exhibits are organized spatially into sets of categories, one of which is Computers in Music, in which the Reactable is set up permanently in an open-plan space close to other interactive exhibits. The purpose of the Reactable in this context is to demonstrate concepts of computer music using tangible objects. Next to the Reactable, there is a TFT display at eye level, which shows a looping demo video of basic interactions with the system, with the aim at guiding casual users by watching possible configurations of objects. The exhibit has four pairs of headphones attached to the tabletop, one of which was appropriated for sound recording for the study and thus not available to users. The exhibit also has open audio via two loudspeakers embedded within the table. In addition, there is an artificial metal ceiling above the area for sound insulation purposes. In the exhibition space, there are several stools that can be moved, for visitors to sit or children to stand on.

4. THE STUDY

4.1. Research question

The study aimed to explore what a complex musical tabletop with a tangible interface can tell us about social interaction in public settings, in particular unstructured and casual interactions. Following a qualitative study design, more detailed questions for our data analysis emerged from this overarching research question when reviewing the video data using a thematic analysis (Braun and Clarke, 2006) approach. In particular, we noticed frequent cross-group interactions, and of various kinds, between visitor groups. Our analysis thus focused on this unexplored phenomenon from a perspective of liminal situations and the factors promoting it. We looked at:

• the nature of cross-group interaction, reflected in the coding scheme presented in Section 4.4 and described in the findings of Sections 5, 6, and 7;

• in particular, how groups managed transitions and interactions between their performances and those of others, described in the findings of Sections 6 and 7; and

• factors related to liminality that promoted cross-group interaction, discussed in Section 8; particularly, the role of tangible interaction with physical objects in cross-group interaction, discussed in Section 8.1; the role of system's autonomous audiovisual feedback and a rim territory, discussed in Section 8.2; and the role of different system's states, discussed in Section 8.3.

4.2. The setting

The Winchester Science Centre² is a hands-on science and technology centre containing a number of interactive exhibits (most of them developed in house) and a planetarium. The goal of the centre is to promote public knowledge and understanding in the science, technology, engineering and mathematics fields among younger generations. The centre is visited both by schools and the general public. It is generally attended by school visitors on weekdays and by the general public on weekends. It has an upper and a lower exhibition area which both feature an open floor plan.

4.3. Set-up, data collection and procedure

We collected 8 hours of video data over one well-attended weekend (reported by the centre as 666 visitors in total), consisting of four 2-hour slots allocated in two mornings and two afternoons. The study was given ethical approval (HREC/2011/#906/1).³

Throughout the data collection period, a researcher adopted a non-interventionist, observational approach, observing the Reactable from a distance, while walking around the exhibition space. A poster and leaflets informing visitors of the study were placed both at the entrance to the museum and next to the Reactable; these included research details and a timetable of the video recordings. Data collection was anonymous. If visitors had any concerns about being recorded, they could ask the researcher to switch off the camera or delete the footage at any time.

For data collection we used two cameras set up non-obtrusively: these were positioned to provide a general view of the Reactable and the bodily interactions of visitors, and a close-up view of the visitors’ hand gestures on the tabletop surface (Fig. 3). We also collected the Reactable's audio output with a handheld recorder replacing a headphone's output channel as additional data for potential future research. Thus, three out of four headphones were available to the public during the study data collection. The researcher only approached the tabletop if a visitor wanted to ask a question, which happened a few times. None of the visitors involved in the study requested their data be withdrawn.

4.4. Analysis approach

In order to produce an initial overview of the full data set, we coded all of the videos according to the number of groups, the group members and time of group interaction. We used the ELAN software, which aids video coding.⁴ Since the data were collected in the wild, it was rich but unstructured. We went on to use inductive and iterative thematic analysis (Braun and Clarke, 2006) to identify relevant themes within the data and thus derive an analytical structure. This enabled us to qualitatively identify relevant themes that progressively focused in on liminality and cross-group interactions, both synchronous and asynchronous, and finally to choose vignettes illustrating these themes.

As part of an initial exploration of the data, in addition to group discussions within the group of authors, group discussions were conducted with experts in the fields of museum installation design, multimodal interaction and video analysis in the course of two workshops.^5,6 Discussing our data as a case study with experts in the field enabled us to gain further insight into the data and to better understand the science centre environment. The discussions were based on short video excerpts of both cross- and inter-group interaction and included topics such as different phases of interaction, roles, engagement or learning.

We iteratively developed a set of behavioural codes through video analysis. Our analysis focused on the behaviour of groups around the Reactable, with a special interest in liminality and cross-group interaction.⁷ A group here refers to one or more people who approached and interacted with the Reactable. We recorded the time spent with the interactive tabletop starting from when the first object was positioned on the table and ending when the last object was used before the group moved to a new exhibit. In the case of groups that approached the table more than once, we added up the different intervals. A session refers to one or more participants interacting with the Reactable over a certain period of time. The coding scheme presented here includes: transitions and overlaps (Section 4.4.1) and continuists vs rupturists (Section 4.4.2). This nomenclature is based on our own terminology.

4.4.1. Co-located synchronous cross-group interaction: transitions and overlaps

Through the data analysis we identified a set of mutually exclusive styles of liminality and cross-group interaction, using social groups as the unit of analysis, ranging from:

• Transition: a process of change from a current group, whereby another group joined them at the table and started to interact with it, and after some period of time, the original group left. In these cases there could be a ‘handover’ of objects (e.g. headphones or tangibles).

• Overlap: a process whereby a new group joins or ‘merges’ with a current group for a certain amount of time, and both groups interact, mediated by the table, including observation of the practice of the current group. Then the second group leaves, letting the first group to continue interacting.

• Transition and overlap: a process whereby a new group overlaps with an existing group (a new group joins an existing group) and then leaves before or after there is a group transition from an existing group to the new group.

• No interaction: no co-located synchronous interaction with other groups, there is no overlap or transition in the time that sequential groups spend interacting with the table.

4.4.2. Co-located synchronous and asynchronous cross-group interaction: continuists vs. rupturists

Another level of liminality and cross-group interaction relates to how the tangible objects were left by a group and ‘inherited’ by the next group: a ‘heritage’ that could be either synchronous or asynchronous, that is, sequentially continuous (e.g. during a transition) or sequentially spaced in time (e.g. without human interaction). Each group adopted a mutually exclusive style of a continuist–continuist, a continuist–rupturist or a rupturist–continuist approach to their musical compositions (patches), which was either synchronous or asynchronous in relation to the next group. The ‘continuists’ started or ended their sessions in such a way as to continue with others’ patches or to allow others to take over, whereas the ‘rupturists’ started their sessions from scratch or ended with no objects on the table. The three approaches to liminality and cross-group interaction from a musical patch level (‘cross-group patch interaction’) are described in the following:

• Continuist–continuist: leaving a patch that produces sound on the table for the next group to continue and subsequently inheriting an existing patch from a previous group and modifying it.

• Continuist–rupturist: inheriting an existing patch from a previous group and removing all the tangibles to start afresh. A system's standby mode is automatically activated after several minutes of inactivity, which can be also considered continuist–rupturist.

• Rupturist–continuist: removing or tidying up all the tangible objects from the table and setting a ‘blank page’ before leaving the Reactable exhibit and subsequently not inheriting an existing patch from a previous group.

5. FINDINGS: GROUP SIZES AND INTERACTION TIMES

We collected no data about the identities of participants; rather we counted the groups as they arrived at the Reactable exhibit. We use the following nomenclature from here on: G# for groups and P# for participants, in which b# refers to a boy, g# refers to a girl, M# refers to a man and W# refers to a woman (e.g. W1-G1 refers to Woman 1 of Group 1). This nomenclature considers teenagers as adults because they can potentially attend the museum independently. From the video recordings, we observed 54 groups, 5 of which were clearly subgroups of five bigger groups, such as families (G4 and G47) or school field trips (G35), that arrived together, resulting in a total of 49 groups. The Letter A or B is appended (e.g. G4A and G4B) to subgroups.

Table 1 outlines the minimum, maximum and median duration of groups. These results support the median duration found in museum studies, in which it is common to find up to 4 minutes spent in different installations and a long tail that exponentially fades out (Bengler and Bryan-Kinns, 2013; Hornecker and Stifter, 2006), ranging from traditional object exhibits to computer-enhanced hands-on exhibits, and, in the ‘long tail’, interactions of up to 15 minutes (Hornecker and Stifter, 2006). The two peaks are unusual and indicate different levels of engagement with the Reactable.

A qualitative analysis with four vignettes to complement these descriptive overall results is provided in Sections 6 and 7. We observed both synchronous and asynchronous interactions between groups. As presented in Section 4.1, we were especially interested in these phenomena because it showed that tabletops promote a wide social interaction in public settings beyond preformed groups.

6. FINDINGS: FLUID TRANSITIONS AND OVERLAPS

We next present the nature of these cross-group interactions with two vignettes: fluid transitions (Section 6.1) and smooth overlaps (Section 6.2). We also describe the two combined (Section 6.3).

6.1. Fluid transitions

This vignette illustrates a fluid transition between two groups. This is possible because group members are aware of others’ actions making their intentions explicit and visible to all. The girl noticed the arrival of a new group with an interest in playing with the table, along with the lack of sufficient headphones for them. Leaving the headphones on the table, typically orienting the object to oneself, makes visible a change of turn to the next group and creates potential access points, with no need of verbal exchange. Group change is smoothly supported by the system because headphones indicate access points to the exhibit, and tangible objects play autonomously. The new group has access to the exhibit at points where there are free headphones and smoothly reconfigures the objects that the girl has left. An indirect handover happens when the girl leaves the headphones on the table orienting the object to herself (Frame 4), and the third boy of the group moves where they are and wears them (Frame 5). In other groups, there was a similar handover of headphones although made more explicit than in this vignette. Such a handover of headphones reveals the mutual group awareness of an impending group change, and it can be seen as an instance of coordination between groups. Headphones are used here to make a group change visible as well as to communicate that they are to be used to better experience the interactive exhibit.

6.2. Smooth overlaps

This vignette shows a smooth overlap between two groups, mutual group awareness, cross-group coordination and musical collaboration. Three features of the Reactable make it possible for a group to join an existing group seamlessly: (i) its obvious visibility and audibility when passing by that is associated with music, which is supported by the presence of headphones that signal intentional communication, both when in use or left on the table; (ii) the rounded shape of its surface allows visitors to reconfigure their positions dynamically and to have different points of access and (iii) its object-based interface consisting of an extensive collection of objects available. Although headphones do not seem to be necessary for engagement, they are a high-value resource that is in demand. The pairs of headphones left available on the tabletop surface (Frames 1–5) seem to work as entry points of access to the interactive experience for the newcomers.

6.3. Fluid transitions and overlaps

Other groups fluidly overlapped and transitioned. In these cases, a group or some of the members of a group overlapped with another group or groups, before or after formally interacting with the exhibit. Here formal interaction refers to a group clearly positioned around the table and playing. We found examples of overlaps prior to, or after, transitions. A prior overlap followed by a transition, which was different from the simple overlap in Vignette 2, tended to be observational or with little interaction. A prior overlap and transition was constituted as: (i) part of a new group or whole new group approaching the table as observers, or casual players, usually to get a sense of how the interactive exhibit works and (ii) a new whole group returning and transitioning with the current group. Overlaps after transitions happened less often (2 out of 8) and involved an earlier group, or part of an earlier group, which already overlapped and transitioned with other groups, returning to join the current group.

7. FINDINGS: CONTINUISTS VS. RUPTURISTS

We were particularly interested in understanding the continuist approach as an instance of co-located collaboration in collaborative music systems that includes both synchronous and asynchronous interaction (cf. Barbosa, 2003). Continuist interaction with a musical interactive tabletop in a public setting can be seen as a form of collaboration between strangers because there is a follow-up of a work developed by a previous group, which can be heard and seen by a new group, with a lapse of time in some cases.

In the following section we present two vignettes focusing on asynchronous interaction between groups: asynchronous continuist (Section 7.1) and asynchronous rupturist (Section 7.2). We focus on asynchronous interaction because it can tell us a different perspective about the nature of liminal situations than synchronous interaction, which was manifested during transitions and overlaps already discussed in Section 6.

7.1. Asynchronous continuist

Vignette 3: Asynchronous continuist–continuist (G33 and G34A). 6 minutes 32 seconds. The Group G33 (g1 and W1) starts with one sound produced by a cube object (Frame 1). Each of the two wears a pair of headphones. Both keep adding objects onto the table until there are eight active objects on the table (Frame 2), of which four objects produce sound and the other four modify the output of these sounds. Both of the two members of the group take off their headphones, put them on the table and leave (Frame 3). There are now 11 objects on the active area of the table, with audiovisual feedback, of which 6 objects produce sound and the other 5 modify the sound output. There is audiovisual feedback of the patch left by G33, but there is no human interaction (Frame 4). About 2 minutes later, the girl g1’ (g1-G34A) approaches the table (Frame 5). She puts her cardigan on the table border, takes the headphones next to her and puts them on. She calls one of her friends. She rotates two objects from the patch, one with each hand (Frame 6). The girl g2 of G34A approaches the table and takes another pair of headphones. The girl g1’ says to g2: ‘Twist it, twist it!’, and g2’ rotates an object (Frame 7). A third girl, g3, arrives and observes how g1’ and g2 create music by rotating objects (Frame 8).

This vignette illustrates how asynchronous interaction can promote a different type of cross-group collaboration compared to synchronous cross-group collaboration. The first group leaves a patch with a certain configuration of objects, and after a few minutes, another group approaches the table and modifies the patch by changing the configuration of the tangibles. The audiovisual feedback of objects seems to encourage visitors to approach the table and interact with the objects. There is a certain intention to leave an active patch by the first group, as well as to continue the existing configuration by the second group. The benefit of inheriting a patch is that visitors can continue a patch rather than start from scratch and can learn how sounds and effects work by modifying it. The potential downside of this situation is that the group loses the opportunity to experience the discovery process of starting from scratch, discussed in the next section. The possibility of continuing a patch asynchronously indicates an open-ended collaboration because there is an asynchronous continuation in a meaningful way. By contrast, other groups continued the work asynchronously by modifying the patch from no sound output instead, with only the automatic visual standby mode along with the visual feedback of objects. In this situation, there is a continuist–rupturist approach to asynchronous cross-group interaction, which is explained in the following.

7.2. Asynchronous rupturist

Vignette 4: Asynchronous continuist–rupturist (G52). Woman W1 approaches the table, on which there are already a number of tangible objects producing sound (Frame 1). W1 starts to remove each object from the active area to the rim area until the active surface is empty (Frames 2–4) and the screen saver appears (Frame 5). She then positions a cube in the middle (Frame 5) and a step sequencer (controller) that affects the cube (Frame 6).

This vignette indicates a strategy by some people of tidying up the existing tangible objects and starting from an empty canvas. Some visitors seemed to prefer this strategy. There could be a number of reasons for this, including a preference for having more personal control of the patch. It could also relate to a controlled, progressive, step-by-step discovery process and learning. A risk of this strategy is that the visitor may by coincidence fail to create a sound-producing patch, which then hampers learning, as there is no audio feedback. As the name indicates, a continuist–rupturist approach can be interpreted as a non-collaborative instance compared to the continuist–continuist approach, because the previous group's patch is removed and there is no continuity. By contrast, a rupturist–continuist approach can be considered a neutral approach to collaboration, because the second group has to start from scratch due to the previous group's choice, as opposed to by own choice.

8. DISCUSSION

Our findings indicate the importance of enabling on-demand fluid transitions and overlaps for smooth flow, and asynchronous interaction for a wider spectrum of social interaction between groups, in collaborative interactive systems for public settings. Encouraging social interaction, and in particular collaborative group experiences with exhibits that orchestrate group interactions between strangers, aligns with the museums’ agenda of moving from object-centred to visitor-centred experiences (Bedford, 2014; Falk and Dierking, 2012; Perry, 2012; vom Lehn, 2006). These findings show that fluid transitions and smooth overlaps between groups can be orchestrated by an exhibit and reflect the nature of social interaction in a public setting. Furthermore, a system that facilitates asynchronous group interaction promotes more social interaction, which can enrich visitors’ experience because there are more possibilities and levels of group interaction. In particular, a system that facilitates a continuist approach to asynchronous interaction can promote a wider range of types of collaboration beyond continuist synchronous interaction.

From our observations, we give design recommendations about how to support social interaction between groups through liminal situations, considering: (i) liminal components that allow for access, change and levels of intimacy; (ii) liminal spaces that allow for getting to grips with the system and (iii) liminal states that allow for synchronous and asynchronous interaction, including cross-group collaborations. Promoting liminal components, liminal spaces and liminal states are expected to suit both synchronous and asynchronous group interaction.

8.1. Liminal components for access, change and levels of intimacy: the role of speakers, headphones and tangible objects

Liminal components refer to interface elements of a TUI system, which support liminal situations of physical or experiential group change between and within exhibits, which allow for access, change and levels of intimacy when interacting between and within exhibits, as discussed in the following.

The purpose of the exhibit is clear to visitors at first sight, as seen in Vignette 2 when G11 passes by, hearing loud music coming from the speakers of the table and recognizing headphones as a familiar object. Moreover, as seen in Vignette 1, when g1-G21 leaves the headphones on the table, the headphones facilitate a change of turn between groups as they have to be exchanged. The use of liminal components that indicate the type of activity and the available access points from a walking distance appear useful for both activity access and group change: hearing the sound from the table speakers combined with seeing headphones allow for smooth overlaps and fluid transitions between groups because they indicate the type of activity and available spaces in which to join the activity. Orienting headphones to oneself before leaving the exhibit signals intentional communication using non-verbal conversational acts (Kruger et al., 2004); the person is leaving and ensures there is a free available access point. This evidences that orienting an object to oneself in cross-group interaction is different from orienting an object to oneself in inter-group interaction, in which group members understand that the person is doing her or his own personal work instead (Kruger et al., 2004).

The fact that visibility of interactions supports smooth overlaps and fluid transitions is found in the literature of tabletop and public display systems (Brignull and Rogers, 2003; Hinrichs and Carpendale, 2011; Müller et al., 2012). This connects with the importance of providing awareness information for successful collaboration discussed by Dourish and Belloti (1992), which enables peripheral monitoring of others’ activities (Brignull and Rogers, 2003; vom Lehn et al., 2001). Providing awareness information (in this case visual and auditory) allows for getting a sense of the available access points, and smooth group change between loose vs close collaboration, where transitions can be seen as an instance of loose collaboration (Vignette 1), while overlaps as an instance of close collaboration (Vignette 2). Liminal components, such as headphones, function as activity indicators (awareness indicators) for observers, providing information about both the activity and dedicated access points (but are not the only access points due to the loudspeaker-based open audio).

It has been argued that tangibility supports manipulative access and fluidity of sharing (Hornecker et al., 2007; Speelpenning et al., 2011). We observed that the availability of an extensive collection of objects and a large surface allows for a collective haptic exploration and experience, in which visitors can smoothly join or leave at any time. Previous research on large multitouch wall displays has reported similar fluid transitions and overlaps in cross-group interaction; however the interaction between groups tends to be less ‘conversational’ (e.g. handovers) (Peltonen et al., 2008). Overlaps consist in groups’ work that is generally in parallel but independent, and cross-group interactions that happen due to conflict management between groups (Peltonen et al., 2008). Transitions tend to be based on turn-taking by queuing, and so visitors in the queue pay attention for free available slots, a situation also referred to as floor and turn-taking (Peltonen et al., 2008). By contrast, our observations reveal overlaps and transitions between groups that smoothly coexist together, in which groups work in a shared space on a single activity using a conversational style. A horizontal surface that allows visitors to see each other seems to facilitate this. No queuing is observed: the radial shape and medium size of the Reactable's tabletop interface, as opposed to the left-right orientation and large size of a multitouch wall display, allows for a wide angle of visibility and makes it easier to join an existing group configuration around the table.

The physicality of liminal components is important in cross-group interaction for their role of awareness indicators and for supporting non-verbal communication. In multitouch interfaces, only gestures are possible: fluid transitions between gestures have been reported during cross-group interaction (Hinrichs and Carpendale, 2011). However, sharing media content between groups with no physical elements beyond gestures makes group change focalized around the active display and less visible from a passer-by's view. The physical objects in the rim area provide awareness information about access points, a feature that is missing in multitouch-only tabletops. In the particular case of tangible objects, they are also modular, a relevant design characteristic which can also benefit cross-group interaction for supporting both individual and group manipulation, as well as multiplicity of combinations. This characteristic can support learning by doing as shown in Vignettes 3 and 4.

Enabling activity access and group change with liminal components, in particular physical components that can be manipulated, such as headphones and tangible objects, recalls the notion of multiple access points, a characteristic of TUIs highlighted by Hornecker and Buur's (2006) tangible interaction framework and Hornecker's et al. (2007) entry and access point model. Not only relevant places, such as multiple access points, but also suitable moments are important for group change (e.g. transitions, overlaps). Peltonen et al. (2008) refer to this notion as transition relevant places (TRPs). Physical liminal components function as multiple access points and TRPs, by: (i) distributing control among users by means of physical components, thus promoting a more democratic and decentralized setting and (ii) providing visibility of the activity and available free spaces, thus potentially promoting participation and learning from seeing others’ interactions (entry and access points). For example, during overlaps as in Vignette 2, seeing others manipulate the objects was helpful for newcomers to understand how to use tangibles. This aligns with Peltonen's et al. (2008) observations of newcomers learning from seeing groups interacting with a large multitouch display, in which the presence of people next to the exhibit helps newcomers to both notice the display and see how it works. Similarly, the visibility of others’ gestures allows for learning from demonstration and by imitation, reported by Hinrichs and Carpendale (2011) as a key element for social interaction in casual situations. Accordingly, the design of liminal components needs to consider the space required for these gestures between groups to happen.

Using tangible artefacts also appears useful for enabling different levels of intimacy with the activity: as soon as a visitor starts to wear headphones, there is a subtle auditory change from a less to a more immersive experience, from an average volume from the speakers to a louder volume of sound from the headphones. Thus, this transition is less abrupt than if it were from silence. However, there is a limit to the number of headphones that can be made available, and thus to the number of visitors who can join to the activity as fully immersive, as shown in Vignette 1, where there is transition of headphones’ holders. It is a challenge to improve the auditory experience by providing a slightly more immersive experience for those who are not wearing headphones: there seem to be several trade-offs between the isolation of groups engaged in an immersive experience and the need to attract and interest other visitors, between interest in the Reactable and interest in other exhibits and between concentration and distraction in an open environment, such as a museum.

Our findings are relevant to other tabletop and TUI installations that could take on similar functions. For example, the tabletop application TellTable (Helmes et al., 2009) allows children to create storytelling by using a portable camera module that can be snapped on to a range of viewfinders such as a magnifying glass or a telescope. These physical devices allow for different levels of intimacy with the content that is photographed depending on the viewfinder frame used. Moreover, seeing the use of these viewfinders from a distance indicates the type of activity and facilitates handovers between visitors. Similarly, Futura (Speelpenning et al., 2011) is a collaborative tabletop game about sustainability that includes two tangible magnifying glasses that provide players with additional information using real-time visualizations. Another example is the use of passive polarized glasses for playing with a tabletop children's game that shows a stereoscopic display (Hoberman et al., 2012). The transition for accessing the content is more disruptive here, as it is not possible to see the image correctly until wearing the glasses. The alternative of using an autostereoscopic display (stereoscopic images that can be seen with no glasses) would exclude a transition to a greater level of intimacy, which can be detrimental for inter-group interaction in a public setting because a closer group experience becomes less distinct.

We argue that the design of these liminal components for access, change and levels of intimacy is aligned with the approach to designing CSCW systems that take account of awareness, perception and the notion of public availability of resources, as proposed by Robertson (2002). In order for visitors to be aware of the liminal components, they need to be perceived, which is achieved by making explicit their availability, for example, with visual and auditory information. This connects with the notion of peripheral awareness among visitors (Brignull and Rogers, 2003; vom Lehn et al., 2001), a human perception mechanism that helps visitors to coordinate between themselves when interacting with exhibits, in particular, for moving across the available liminal spaces. The use of liminal components can facilitate coordination between groups of strangers in a self-organized way to access and leave the exhibit.

In particular, liminal components promote smoothness (as opposed to abruptness) and self-regulation (as opposed to external regulation) during activity access and group change. Self-regulation of turn-taking is interesting in a public venue context, in which social interaction is recognized as an important factor in visitors’ experience of museums and galleries (Falk and Dierking, 2012; Heath and vom Lehn, 2009; Perry, 2012). For example, our observations have shown that asynchronous interaction can be useful when the public venue is less crowded (Vignette 3), allowing visitors to notice the exhibit with no presence of others, a desired feature in large display design (Peltonen et al., 2008). With a few exceptions in public large displays’ research (Brignull and Rogers, 2003), previous work has been less focused on the temporal space between groups as a potential mechanism for cross-group interaction, whose benefits are fully discussed in Section 8.3.

Liminal components allow for group coordination of turn-taking, by means of both explicit and indirect handovers, as shown in the group transition of Vignette 1. Similarly, liminal components also allow for cross-group collaboration of groups working in parallel on the same activity, as shown in the group overlap of Vignette 2. In Vignette 2, the overlap of two groups would have been considered, potentially, an exclusive group from the perspective of fluid grouping by Block et al. (2015), in which group members spend most of their time interacting together and so share most of their experience. The advantage of our approach is that instead of analysing overlaps as the quality of group collaboration measured by the time spent together (centripetal approach), we analyse transitions as well as overlaps to gauge the quality of group interaction and collaboration measured by patterns of cross-group social interaction (centrifugal approach).

8.2. Liminal spaces for getting to grips with: the role of an autonomous system and a rim territory

Liminal spaces refer to areas of a TUI system that support liminal situations of physical or experiential group change between and within exhibits, which allow for access and change when interacting between and within exhibits. They are understood as spaces of transition.

The Reactable operates in real time using audiovisual feedback, which prompts newcomers to join the activity smoothly and either: (i) explore the objects together with others with no need for the verbal communication (Vignettes 1 and 2) or (ii) explore the objects with a lack of human interaction with other groups (Vignettes 3 and 4).

When objects are placed on the Reactable surface, they have an autonomous behaviour with no need of human manipulation. However, objects can be manipulated as well. This feature connects to interactive composing systems, a term coined by Chadabe, in which “the computer responds to the performer and the performer reacts to the computer, and the music takes its form through the mutually influential, interactive relationship” (Chadabe, 1984, p. 23). This twofold feature allows groups to approach the table, observe the system's behaviour and intervene smoothly at any moment. This allows groups to notice the table even though there is nobody interacting. This approach overcomes the issue of visibility reported by Brignull and Rogers (2003) in public displays: when there are no people interacting with the device, it is harder for other people to notice the device's presence and to discern how it works. As shown in Vignette 3, when the second group (G34A) approached the table, our observations indicated that visual autonomous behaviours alone can attract the attention of visitors. However, it is unclear whether an autonomous visual standby mode with no sound is useful in musical tabletops and audiovisual exhibits, as this can mislead visitors about how the system works. As discussed in Section 8.3, allowing visitors a range of states beyond standby mode as starting points can be useful. Furthermore, if the purpose of the interactive tabletop is to demonstrate concepts during social interaction, usually in a short period of time, that could start to be clear when approaching the table before beginning the interaction, during the liminal space from walking to starting the session. This aligns with Brignull and Rogers's (2003) suggestion of providing meaningful awareness information for the passer-by, so that she or he can smoothly become a participant by means of supporting fluid transitions from the ‘onlooker’ threshold to the ‘participant’ threshold, and back.

The notion of liminal space relates to the concept of territoriality, introduced by Scott et al. (2004), and to previous guidelines on designing tabletop interfaces, which suggest the support of fluid transitions between different areas as they may occur during co-located activities (Scott et al., 2003). Similar to tabletop territories, tabletop liminal spaces serve to coordinate tabletop interactions (cf. Scott et al., 2004). The three types of tabletop territories (personal, group and storage) are regions centred in the tabletop workspace, while liminal spaces also include the outer space between the exhibit and other exhibits, which is fundamental for group change. Thus, a broader scope of the exhibit space is relevant to liminal situations in cross-group interaction, compared to the territoriality approach. From a liminal situation perspective, the storage territory (rim area) is the most relevant from the territoriality literature, because it can be seen as a liminal space between the outer space of the exhibit and the tabletop workspace. A threshold space between the outer and inner spaces of the exhibit combined with liminal components can serve as an awareness indicator of available access points (as discussed in Section 8.1), facilitating fluid transitions between start and ending group activities.

Along with the design recommendations of supporting a number of transitions, such as fluid transitions between activities, transitions between personal and group work and transitions between tabletop collaboration and external work (Scott et al., 2003), our findings point out that transitions between liminal spaces should be supported. The characteristics for supporting transitions between tabletop collaboration and external work, in which collaborative tabletop activities are considered within an ecosystem of activities in the workplace, are relevant here because it considers tabletop practices within a broader context. However, the nature of a casual setting differs from the workplace (and from formal learning environments) in that, in the former, activities tend to be open-ended, self-directed and sometimes goalless (cf. Falk and Dierking's (2012) notion of ‘free-form learning’), performed by groups of strangers, rather than a large group of people who knows each other. As long as tabletop systems are designed for public settings, supporting external playful activities between strangers, taking into account the characteristics of a museum's environment needs to be addressed. An understanding of visitors’ practices is thus relevant for getting a sense of the type of liminal changes that need to be supported, as discussed here.

8.3. Liminal states for synchronous and asynchronous interaction: the role of a tabula rasa state

Liminal states refer to moments during the interaction with a TUI system, which support liminal situations of physical or experiential group change between and within exhibits, which allow for access and change when interacting between and within exhibits. They are understood as moments of transition.

As seen in Vignette 4, for some of the visitors it was useful to remove all objects from the table and start from the state of a tabula rasa (manual tabula rasa). Similar behaviours have been observed during interactions with a large multitouch wall display (Peltonen et al., 2008). However, our observations suggest that an automatic tabula rasa state, initiated by the standby mode state, has rupturist consequences for collaboration within an asynchronous interaction. This is because a configuration of objects (a patch) left by a previous group can stop playing for a later group. This situation can disrupt fluid transitions and collaboration between groups because there is no sound, as we observed with half of the groups that dealt with the standby mode. It is unclear how an automatic system's standby mode, after a few minutes of inactivity (a lack of interaction with the tangibles), can support social interaction between groups. By contrast, a manual tabula rasa state seems useful.

A speculative design question emerges about whether other liminal states apart from tabula rasa could have a similar function of allowing visitors to understand how the tabletop system works. For example, a state with suggested start points (i.e. predefined by the designer) could show a range of configurations from basic to complex, including concepts of computer music, such as how an effect works (e.g. a delay effect) or the difference between two effects (e.g. a low and high pass filter). This would support tailorability, an important feature in tabletop design (cf. Dalsgaard and Halskov, 2014). Another possibility would be to allow visitors to load previous groups’ configurations. Conversely, visitors could store their configurations for future groups, sharing their patches with others over time. This history feature could provide an approach for asynchronous interaction, suitable for collaboration, and allow new visitors to learn from and build upon past groups’ contributions. Previous research (e.g. Block et al., 2015) has identified the need of supporting smooth overlaps between groups with system's states beyond the initial state of the system but has not detailed states for supporting overlaps and group change (liminal states) beyond resetting the system.

A continuist–continuist approach to co-located asynchronous interaction can be seen as an instance of collaboration. As shown in Vignette 3, an asynchronous continuist approach recalls the game cadavre exquis, also known as exquisite corpse, in which participants create a written or graphical collaborative piece by contributing in sequence, sometimes only seeing part of the piece. An asynchronous continuist group only sees and hears the last patch left by the previous group. This also recalls post-it based interfaces, used synchronously (Klemmer et al., 2001) or asynchronously (Guerreiro et al., 2014), which support a collaborative activity where participants can leave messages as well as read others’ messages. Promoting asynchronous interaction in public settings is a promising approach to supporting collaboration based on visitors’ traces. This aligns with Hindmarsh's et al. (2005) suggestion of leaving an ‘activity trace’ for forthcoming visitors, in this case a collaborative musical piece. Furthermore, the need for generating ‘opportunities of interaction’ is claimed. Our findings draw on this work by exploring the opportunities of not only synchronous but also asynchronous collaboration provided by a musical tabletop.

An asynchronous continuist approach across groups seems to support social interaction beyond immediate simultaneous physical interaction. There appears a particularity of musical tabletops, which promote the aforementioned multiple access points, so that visitors can access it from any direction and at any time, especially tabletops with a circular surface. Audiovisual feedback, and in particular, the use of the auditory channel sound in TUIs, seems to strengthen not only co-located synchronous but also co-located asynchronous musical collaboration, because visitors can hear others’ work and modify it later in time. This fills a gap in Barbosa's (2003) classification of computer-supported collaborative music, based on user's location and performance synchronicity. We here expand the term local interconnected musical networks, classified by Barbosa (2003) from just synchronous co-located collaboration in networked systems, to include also asynchronous co-located collaboration, within the context of CSCW for music: a collaborative musical installation, in this case a particular configuration of tangible objects on the Reactable, can be modified over time, not only synchronously (generally within groups) but also asynchronously (generally between groups).

8.4. Study limitations and future work

As shown in this study, musical tabletops such as the Reactable can be representative of the complex nature of social interaction with interactive exhibits. We have provided evidence that a complex tabletop TUI for music performance can tell us about social interaction in public settings and how an interactive exhibit can best support change and liminal situations between groups in public environments. We discuss study limitations and potential future work from this research in the following.

8.4.1. Limitations

Our study only investigated social interaction with a focus on cross-group interactions with a single exhibit rather than conducting a comparative study with other exhibits. We argue that the Reactable can generate a rich and natural stream of data from visitors’ interactions, and studying these phenomena should prelude a comparative study. The Reactable is useful for the type of exploratory research we are conducting here.

Another possible problem is that our study was conducted using three out of the four available headphones. However, the majority of groups were pairs, and so the limitation in the number of headphones was rarely a constraint. Moreover, it was noticeable that the built-in loudspeakers allowed groups to share sound and experience the interactive exhibit without wearing headphones, as well as had a role as mediators to a more immersive experience.

Concerns can be raised about whether 2 days of data collection within 4 time slots is sufficient to characterize social interactions. The museum gatekeeper suggested that collecting data during one weekend would be representative for the site, although perhaps busier than on weekdays. We analyzed an 8 hour sample of video data taken at 2 different times of the day as: (i) a compromise between sample size and the time/labour consuming task of video analysis (cf. Heath et al., 2010) and (ii) with a reasonable variation of crowdedness over the course of the day. This approach has proved useful for exploring the nature of cross-group interaction. Another potential issue is whether consented video studies can reflect visitors’ behaviours in the field: the study by Block et al. (2015) on fluid grouping used an intrusive approach of stopping visitors first to collect personal data; visitors’ flow thus intervened. By contrast, we adopted a non-intrusive approach of interacting with the visitors as minimum as possible, an approach that, as evidenced here, is relevant for understanding in-the-wild interaction.

8.4.2. Future work

As follow-on work to extend the findings of this study we suggest the design and assessment of tabletop and TUI features for enabling fluid transitions and smooth overlaps between groups based on the provided design recommendations. This could include the exploration of other approaches to tangible music in public settings (e.g. interactive exhibits based on portable or wearable devices or whole immersive tangible installations) as a platform for democratic settings of social interaction and collaboration beyond within groups. In line with Block et al. (2015), it would be interesting to develop an algorithm that could model cross-group interaction around an interactive exhibit, which would require a large-scale study and quantitative methods. An interesting avenue of research would be to analyse spatial patterns of interaction, and the connection with Kendon's F-formation theory of spatial organization, as discussed in Marshall et al. (2011b). An open question is whether there might be a potential effect of gender roles and power balance among participants during transitions and overlaps, as well as how age and the balance of gender at any given time effects use, which points to future research. Furthermore, the nature of asynchronous collaboration could be explored more fully by analysing patterns from audio recordings, for example. Future research can look at social interaction from a more institution-led approach, considering whether and how visitors are involved in processes of enquiry related to either a set of exhibits, or the whole centre, as other scholars have proposed (Hornecker and Stifter, 2006).

9. CONCLUSION

Supporting social interaction is part of improving visitors’ experience in public settings. This study researched social interaction of visitors interacting with the Reactable in the Winchester Science Centre, with the aim of understanding better the nature of social interaction in a public setting by observing in-the-wild interaction around a complex system. We found that the musical tabletop promoted different approaches to cross-group interaction beyond common inter-group interaction. There were instances of synchronous interaction between groups, of which some were collaborative (e.g. overlaps). We also observed instances of asynchronous interaction between groups, of which some of them were collaborative, which is a form of social interaction that this particular musical tabletop afforded to participants in the context of the science museum. We argued that designing tabletops and TUIs for public spaces, and more generally interactive exhibits, includes thinking on supporting fluid cross-group interaction (both synchronous and asynchronous) in terms of enabling liminal situations. Liminal situations can be enabled by means of liminal components, spaces and states, which promote and self-regulate smooth group change, both physical and experiential, including different forms of collaboration. We provided design recommendations accordingly.

ACKNOWLEDGEMENTS

We thank the help and support of Stuart Creal and Tracey Creal from the INTECH Science Centre. Thanks to Gerard Roma for all his constant support, help and insightful conversations. We are grateful to Anna Weisling for thoughtful comments of a final version of this manuscript. We are thankful to Leslie Bennett for a final proofreading. Thank you to Joe Mills, Richard Hearne and the E&AVP department (The Open University) and to the Statistics Advisory Service (The Open University), for all their technical help and support. Thanks to Yvonne Rogers for initial advice on data collection. We are thankful to the organizers and participants of the workshops SISCA Museum Workshop at the Strathclyde University and Data Analysis Workshop at the London Knowledge Lab: special thanks to Carey Jewitt, Sara Price and Loraine Clarke.

REFERENCES

Author notes