Contour Integration over Time: Psychophysical and fMRI Evidence

Abstract

The brain integrates discrete but collinear stimuli to perceive global contours. Previous contour integration (CI) studies mainly focus on integration over space, and CI is attributed to either V1 long-range connections or contour processing in high-visual areas that top-down modulate V1 responses. Here, we show that CI also occurs over time in a design that minimizes the roles of V1 long-range interactions. We use tilted contours embedded in random orientation noise and moving horizontally behind a fixed vertical slit. Individual contour elements traveling up/down within the slit would be encoded over time by parallel, rather than aligned, V1 neurons. However, we find robust contour detection even when the slit permits only one viewable contour element. Similar to CI over space, CI over time also obeys the rule of collinearity. fMRI evidence shows that while CI over space engages visual areas as early as V1, CI over time mainly engages higher dorsal and ventral visual areas involved in shape processing, as well as posterior parietal regions involved in visual memory that can represent the orientation of temporally integrated contours. These results suggest at least partially dissociable mechanisms for implementing the Gestalt rule of continuity in CI over space and time.

Introduction

Integrating local image fragments into global shapes is critical for object recognition in complex environments. This contour integration (CI) process has extensively been investigated in psychophysical, neurophysiological, neuroimaging, and computational modeling studies (e.g., Field et al. 1993; Li 1998, 2006; Kourtzi et al. 2003). However, the neural mechanisms under CI remain controversial. One distinct characteristic is that it follows the Gestalt rule of good continuation. That is, adjacent contour segments that are similarly oriented and aligned are more likely to be integrated. Because long-range horizontal connections in V1 are known to connect neurons with similar orientation preferences (Gilbert and Wiesel 1989), many CI theories assume that such connections would mediate CI through contextual modulation (e.g., Field et al. 1993; Li 1998; Kapadia et al. 2000).

However, neuroimaging evidence indicates that CI involves multiple areas from V1 to lateral occipital complex (LOC; Altmann et al. 2003; Kourtzi et al. 2003). There are reports that high-level visual regions such as LOC are activated earlier than V1 by contour stimuli (Mijovic et al. 2013; Shpaner et al. 2013). These results are consistent with monkey data that contour-related responses in V1 are delayed with respect to initial neural responses to visual stimuli (Li et al. 2006; Gilad et al. 2013; Chen et al. 2014), and that V1 responses to contour stimuli are strongly modulated by top-down feedback (Li et al. 2006, 2008; McManus et al. 2011; Chen et al. 2014). However, it is unclear whether V1 horizontal connections are an indispensable machinery for CI under all viewing conditions.

In this study, we examine whether CI could still take place in a viewing condition where V1 horizontal connections are likely rendered ineffective. We ask the observers to detect a collinear contour embedded in random orientation noise, which is similar to stimuli used in many CI studies (e.g., Field et al. 1993), while the whole stimulus image is moving behind a fixed vertical slit (Fig. 1a). The contour is tilted while the whole stimulus image moves horizontally. Therefore, the viewable elements of the contour move either up or down within the vertical slit, one at a time. These contour elements would be encoded over time by V1 neurons that are not arranged along the contour path, but are parallel to each other. If the contour is still detectable when the slit is narrow enough to allow only up to one contour element to be viewed at any moment, we would argue that V1 horizontal connections may not play a significant role in this particular temporal integration process. Rather, we reason that higher-level mechanisms responsible for visual working memory and shape perception may play more prominent roles.

Materials and Methods

Observers

Twenty-two observers (12 males and 10 females, mean age = 24 years) with normal or corrected-to-normal vision participated in psychophysical and fMRI experiments. Some observers participated in more than one psychophysical and/or fMRI experiments (see Results). All, except S.-G.K., were new to psychophysical and fMRI experiments and were unaware of the purposes of the study. Informed written consent was obtained from each observer prior to data collection.

Stimuli

The stimuli comprised 256 Gabors (Gaussian windowed sinusoidal gratings), each occupying one of 16 × 16 invisible square grids (0.825° × 0.825° each). Some Gabors formed collinear contour paths, and the remaining ones were randomly oriented. The Gabors had a spatial frequency at 3 cpd, a standard deviation at 0.15°, and a contrast at 70%. The phases of the Gabors varied randomly from 0° to 315° in 45° steps. When slit-viewed, the stimulus image moved behind a vertical slit at a speed of 6.4°/s in psychophysical experiments or 6.8°/s in fMRI experiments.

In psychophysical experiments, a straight contour path formed by 9 collinear Gabor elements was embedded in a field of randomly oriented Gabors (Fig. 1a). The center of the path was randomly positioned within a range of ±2.5° from the stimulus center. To control the density cues, the center-to-center horizontal distance of adjacent contour elements varied from 0.9 to 1.1 times the average interelement distance (AIED) that was equal to the grid width. The global orientation of the contour path was randomized either between 15° and 60° or between 120° and 165°. A random stimulus image (without any contour path) was also generated within the same trial in a different stimulus interval by randomly shuffling the positions of all Gabors in the contour stimulus image.

In fMRI experiments, to maintain a sufficiently strong signal-to-noise ratio of blood oxygen level-dependent (BOLD) responses, each contour stimulus image consisted of 5 nearly parallel contours, each containing 4–10 Gabor elements. The 5 contours of different lengths were randomly placed, and the distance between 2 neighboring contours was randomized from 1.5° to 2.25°. The global orientation of each contour was randomized from 30° to 45° or from 135° to 150°.

All stimuli were generated with Matlab-based Psychtoolbox3 (Pelli 1997). In psychophysical experiments, the stimulus images were presented on a 21 in. CRT monitor (1280 × 1024 pixels, 0.3 mm × 0.3 mm pixel size, 85 Hz frame rate, and 47 cd/m² mean luminance). The luminance of the monitor was linearized by an 8-bit look-up table. Viewing was binocular at a distance of 46 cm. A chin-and-head rest was used to stabilize the head of the observer. Experiments were run in a dimly lit room. In fMRI experiments, the stimuli were presented through a projector onto a mirror in front of the observer (1280 × 1024 pixels, 0.44 mm × 0.44 mm pixel size, 60 Hz frame rate). The luminance of the projector was linearized by an 8-bit look-up table. Viewing was binocular at a distance of 67 cm.

Psychophysical Procedure

Contour detection performance was measured with a two-interval forced-choice method of constant stimuli. In each trial, the stimulus image moved behind a vertical slit of various widths from left to right or reversely for 2 s. Two stimulus intervals, one containing the contour image and the other containing the random stimulus image, were separated by a 500-ms interstimulus interval. Observers were instructed to report which interval contained a contour. Each slit width was repeated in 60 trials.

fMRI Experimental Procedure

Each observer was scanned for 7–8 runs using a block design. Each run consisted of 16 stimulus blocks that were interleaved with 3 fixation blocks (10 s each) at the beginning, in the middle, and at the end of the run, respectively. Each stimulus block consisted of 8 trials and lasted for 20 s. Each trial consisted of a 2-s stimulus image and a 0.5-s blank with slit-viewing, or four 0.5 s regenerated similar stimulus images (to minimize adaptation) and a 0.5-s blank with full-field viewing. The color of the fixation dot changed in 25% of the trials randomly. Color changes never occurred in 2 consecutive trials. Observers were asked to report the stimulus type (e.g., contour or random stimulus) when color changed. Since the participants could not predict the trials to which they needed to be responded, they had to keep their attention to the stimulus images throughout the experimental runs.

fMRI Data Acquisition

fMRI data were acquired in a 3-T Achieva Philips scanner at the University of Birmingham Imaging Centre using an eight-channel head coil. Anatomical images were obtained using a three-dimensional T₁-weighted sequence (voxel size = 1 × 1 × 1 mm, slices = 175) for localization and visualization of the functional data. Functional images were acquired by a high-resolution gradient echo-pulse sequence covering the occipital and posterior temporal cortex (28 slices; repetition time = 2000 ms; time to echo = 34 ms; resolution = 1.5 × 1.5 × 2 mm).

fMRI Data Analysis

Anatomical data were transformed into Talairach space and then inflated using BrainVoyager QX (Brain Innovations, Maastricht, the Netherlands). Pre-processing of functional data included slice scan time correction, three-dimensional motion correction, linear trend removal, and temporal high-pass filtering (3 cycles per run), but not spatial smoothing. Trials with head motion larger than 1 mm of translation or 1 degree of rotation were excluded from analysis (<5% of total trials). The functional images were aligned to anatomical data and the complete data were transformed into Talairach space. For each observer, the functional imaging data between the 2 sessions were co-aligned, registering all volumes for each observer to the first functional volume of the first run and session. This procedure ensured a cautious registration across sessions.

Regions of Interest

For each individual observer, we identified retinotopic motion-related [V3B/kinetic occipital (KO)] and shape-related [lateral occipital (LO)] areas using standard procedures. Specifically, visual areas (V1, V2, V3, V3a, hV4, and V7) were defined by standard retinotopic mapping procedures using rotating wedge stimuli (Engel et al. 1994; Sereno et al. 1995; DeYoe et al. 1996). hV4 was identified as the region comprising the ventral but not the dorsal subregion of V4 (Wandell et al. 2007). V3B/KO was defined as the set of contiguous voxels anterior to V3A and inferior to V7 showing significantly higher response to kinetic boundaries than transparent motion (Dupont et al. 1997; Zeki et al. 2003). One observer in the full-field viewing condition did not complete the localizer scan of V3B/KO. For this observer, we defined the V3B/KO based on known Talairach coordinates for this region (left hemisphere: −29.5, −83.9, 1.8; right hemisphere: 31.5, −81.4, 3.6). Area middle temporal (MT) was defined as the set of voxels in the lateral temporal cortex demonstrating significantly higher activation to an array of moving dots than to a static array dot (Zeki et al. 1991). LO was defined as a set of contiguous voxels in the posterior ventral occipitotemporal cortex showing significantly stronger activation for intact than for scrambled images (Malach et al. 1995; Kourtzi and Kanwisher 2001). Note that the anterior LOC subregions and ventral regions around the fusiform and the parahippocampal gyrus were not included, since they were covered by high-resolution slices scanned in this study. Finally, intraparietal areas [ventral intraparietal sulcus (VIPS), left hemisphere: −24.2, −73.2, 25; right hemisphere: 27.4, −72, 25.8 and parieto occipital intraparietal sulcus (POIPS), left hemisphere: −20.3, −66.1, 42.6; right hemisphere: −20.8, −65.2, 42.7] were defined on the basis of known Talairach coordinates for these regions (Orban et al. 1999), since we did not have a functional localizer for these areas. Data from different hemispheres were merged for further analysis. The final regions of interest (ROIs) spanned both hemispheres, as both hemispheres showed the same pattern of results.

Multivoxel Pattern Analysis

To identify brain patterns that discriminate between stimuli (e.g., contour vs. random stimuli, or left- vs. right-tilted contour paths), we performed multivoxel pattern classification analysis (Haynes and Rees 2005; Kamitani and Tong 2005). Voxels that showed stronger responses for all stimulus conditions compared with fixation were selected and ranked in a descending order based on their t-values. The first 500 voxels for each ROI per observer were then selected for analysis, as prediction accuracy had saturated at this pattern size across areas. The time course of each voxel was extracted and normalized (z-score) in each run to minimize baseline differences across runs. The fMRI data were shifted by 4 s due to the hemodynamic response delay and were then averaged within each experimental block.

We trained binary linear support vector machine (SVM) classifiers to discriminate fMRI responses evoked by contour versus random stimuli or by different contour orientations using a leave-one-run-out cross-validation procedure. There were 112–128 training patterns and 16 test patterns in the slit-viewing fMRI experiment, and 64 training patterns and 8 test patterns in the full-field viewing fMRI experiment. It is important to note that the classification comparisons were independent of the voxel selection procedure (stimulus vs. fixation). Specifically, we ranked voxels by their activations to all stimulus conditions in contrast to the fixation condition and selected the top 500 voxels in each ROI. That is, voxel selection was independent of pattern classification contrasts (e.g., contour vs. random), avoiding circularity in the multivoxel pattern analysis (MVPA) procedures. Furthermore, we subtracted the univariate signal for each condition to prevent MVPA classification relying simply on differences in fMRI responses between stimulus conditions. That is, for each run, the mean fMRI signal across volumes per condition was subtracted from the fMRI signal per volume. For each observer, we calculated the mean accuracy of the classifier's predictions over cross-validations. To calculate the classifier's baseline performance, stimulus labels were randomly assigned to experimental blocks and the same MVPA as described above was conducted for 1000 times. In addition, the distribution of classification accuracies was estimated from actual data by 1000 times of bootstrap resampling. The probability of overlap between distributions of classifier predictions using ordered versus shuffled stimulus labels was estimated to assess significance.

Results

Psychophysical Experiments

We found that when the contour stimulus moved behind a narrow fixed (nonmoving) slit that allowed as few as one contour element to be viewed at any given moment (Fig. 1a), the global contour was still detectable. Six observers' data showed that contour detection was near chance when measured at a slit width of 0.2 times the AIED (Fig. 1b). However, the performance was improved to approximately 70% correct when the slit width was increased to 0.4 AIED, where only one contour element or part of it was visible at any given moment. The slope of a line connecting the data points at 0.2 and 0.4 AIED was 0.86. A further increase in the slit width to 0.8 AIED, where parts of 2 neighboring contour elements were sometimes visible at the same time, elevated contour detection rate to approximately 80%, but at a much slower speed (slope = 0.19). There was additional slight improvement of performance by approximately 6.4% as the slit width increased from 1 to 2 AIED when 2 neighboring contour elements were sometimes fully visible at the same time (slope = 0.06). Further increasing the slit width, which allowed spatial interaction of neighboring contour elements, had a smaller incremental impact on contour detection. These results show that global contour detection under the slit-viewing condition is possible with temporal integration of neighboring contour elements. As we pointed out earlier, such contour processing with our particular stimulus configurations may not be mediated by V1 horizontal connections.

CI over space is known to obey the Gestalt rule of good continuity (Field et al. 1993). To examine whether collinearity is also crucial for CI over time, 4 observers from the previous experiment (Fig. 1b) perform the contour detection task again. However, the orientations of individual contour elements were now jittered within a range of 0°, ±10°, ±20°, ±30°, or±45° from the contour path, while the slit was 1 AIED wide. Contour detection deteriorated with increasing orientation jitter and reached chance level at ±45° orientation jitter (Fig. 1c). These results indicate that collinearity is as important in CI over time as over space.

As a control, we examined whether contour detection could be alternatively due to the perception of similarly oriented contour elements along the slit over time through a simple probability summation mechanism across time. Contour detection performance was compared with iso-oriented contour elements all deviating from the contour path by 0°–90° at steps of 15°. Such orientation deviations reduced collinearity while keeping the local orientation similarity unchanged. The same 4 observers from Figure 1c performed this experiment. Their detection performance decreased as the orientation deviation increased, reaching the chance level when the orientation deviation was larger than 30° (Fig. 1d). The performance slightly recovered when the contour elements were all orthogonal to the path (90° deviation). Therefore, the potential detection of similarly oriented contour elements along the vertical slit could not account for contour detection under the slit-viewing condition.

For the collinear contour stimuli shown in Figure 1a, the observers might have responded on the basis of a barberpole illusion kind of percept: The up or down movements of the contour elements along the vertical slit. This possibility has not been completely ruled out by Figure 1d because in the latter experiment, the orientation of contour elements deviated from the contour path. To further examine this possibility, we separately jittered the positions of individual contour elements along the direction perpendicular to the contour path. This lateral position jitter destroyed the collinearity without changing the orientations of the contour elements. As a result, the up or down motions of individual contour elements were largely unchanged when viewed through the narrow slit, and so was the potential barberpole illusion. Data from 4 observers showed that the contour detection performance started to decrease after the average position jitter was over ±0.5 AIED, and rapidly approached the chance level when the jitter was larger than ±1.5 AIED (Fig. 1e). These results suggest that contour detection under slit-viewing is unlikely a result of the barberpole illusion.

fMRI Experiments

The above psychophysical data suggest that CI over time may not necessarily involve V1 horizontal connections. We conducted the following fMRI experiments to examine the cortical mechanism underlying this new CI format.

In the first fMRI experiment, 9 observers were presented with 4 types of stimuli with slit-viewing in a blocked design: Right-tilted contour stimuli (collinear contours oriented at 30°–45°, Fig. 2b) and their counterpart random images (all the positions of individual Gabors were randomly reshuffled), and left-tilted contour stimuli (collinear contours oriented at 150°–165°) and their counterpart random images. The ROIs shown in Figure 2a include early ventral and dorsal visual areas and intraparietal sulcus (IPS). These areas are highly relevant to visual information processing and were covered by our high-resolution fMRI sequence (1.5 mm × 1.5 mm × 2 mm). However, a comparison of fMRI responses (i.e., percent signal change from the fixation baseline) between contour versus random stimuli showed no significant differences in these ROIs (F_1,8< 1, P = 0.60, η² = 0.036). We thus used MVPA, a more sensitive measure in discriminating activation patterns distributed across voxels.

We tested the accuracy of a linear SVM in classifying fMRI signals associated with contour versus random stimuli in each ROI. A repeated-measures ANOVA showed a significant effect of ROI (F_3.7,30 = 7.18, P < 0.001 with the Greenhouse–Geisser correction, η²= 0.473). In particular, accuracies in higher dorsal visual areas (e.g., V3B/KO, P = 0.003), IPS (VIPS, P = 0.016; POIPS, P < 0.001), and LO (P = 0.001) were significantly higher than baseline as calculated by bootstrapping (Fig. 2c; see Materials and Methods for baseline calculation). In contrast, no significant differences were observed in the early visual areas (V1, P = 0.33; V2, P = 0.17) and ventral visual areas (V3v, P = 0.71; hV4, P = 0.34).

We reasoned that contours could be perceived only after a number of local elements have been integrated over time. As a result, brain areas involved in visual memory should contain information that supports the discrimination of contours of different global orientations. To test this hypothesis, we examined fMRI selectivity for contour orientation by training an SVM classifier to discriminate activations for the right- versus left-tilted contours (Fig. 2e). Only in POIPS was the classification accuracy significantly higher (P = 0.001) than baseline (Fig. 2f), suggesting that POIPS, which is involved in visual memory (Linden et al. 2003; Todd and Marois 2004; Marois and Ivanoff 2005), may play a critical role in storing and integrating contour elements for CI over time (also see Discussion).

Next, we measured fMRI responses to full-field stationary stimuli (the same right- and left-tilted contour stimuli and their counterpart random images used in the previous experiment, which were no longer viewed through a slit) in 7 observers to compare brain areas involved in CI over space versus over time. Detection performance (hit rate) was over 90% for contours and random stimuli (Fig. 3a). Again, MVPA was used to discriminate between activations for contours versus random stimuli. In contrast to CI over time, the classification accuracies were now significantly higher than baseline (P < 0.001) across all ROIs, including visual areas as early as V1 (Fig. 3b), consistent with previous fMRI results (Kourtzi et al. 2003). In addition, MVPA was also used to discriminate activations for different contour orientations (Fig. 3c,d), which also showed different classification accuracies across ROIs (F_3.4,20.4 = 13.03, P < 0.001, η² = 0.685). The accuracies were significantly higher than the baseline in early visual areas V1 (P = 0.001) and V2 (P < 0.001), early ventral and dorsal visual areas V3d (P < 0.001) and V3a (P < 0.001), but not in higher dorsal visual areas V3b/KO (P = 0.12) and V7 (P = 0.14), posterior parietal regions VIPS (P = 0.21) and POIPS (P = 0.36), and LO (P = 0.27). The lack of significant accuracies in LO may be due to neural populations that are size and orientation invariant in this region (Grill-Spector et al. 1999).

In Figure 1a, when the stimulus contained a diagonal contour path, a single contour element moved upward or downward along the slit. Such vertical motion was not present when the stimulus contained no contour. Figure 1d shows that when the contour elements were all equally rotated from the contour path by 30–75°, contour detection deteriorated to chance levels. In these conditions, a single contour element still moved upward or downward along the slit. Therefore, the observers did not knowingly use the illusory motion as a cue for contour detection. However, even under these null-detection conditions, the motion cues could be extracted by neurons in V3A/B and other visual areas, and the classifier applied to fMRI data could discriminate between contour and random stimulus patterns based on signals related to these motion cues. Such a confound is more likely to be evident in our fMRI contour stimuli that contained 5 contours in a single stimulus, in contrast to one in psychophysical experiments. We ran an fMRI control experiment to test for this confound.

The stimuli patterns contained collinear contours, no contours, or contour stimuli with all elements oriented 45° from the contour paths (“ladders,” Fig. 4a). Collinear contours and ladders resulted in similar vertical motion of local elements, but only collinear contours produced coherent contour perception (Fig. 4b). We compared activation patterns in visual and posterior parietal areas when 7 observers responded to collinear contours, ladders, and random stimuli. Performance detection was 83.4% for collinear contours and 45.7% for ladder contours, consistent with previous results that detection of ladder contours is impoverished (Schwarzkopf and Kourtzi 2008; Zhang and Kourtzi 2010). MVPA was used to classify fMRI responses related to collinear versus random stimuli and ladders versus random stimuli (Fig. 4c). The classifier's performance for discriminating ladders versus random contours was not significantly different from the baseline across ROIs. However, classification accuracy for collinear versus random contours differed from the accuracy for ladders versus random stimuli. There was a significant main effect of classifier (collinear vs. random and ladders vs. random, F_1,6= 16.8, P = 0.006, η² = 0.737) and a significant interaction between classifier and ROI (F_3.7,22.1 = 3.9, P = 0.017, η² = 0. 393). Classification accuracy was significantly higher for collinear versus random stimuli than for ladders versus random stimuli in the dorsal visual areas (F_1,6 = 25.6, P = 0.002, η² = 0.81), LO [t₍₆₎ = 2.88, P = 0.028, Cohen's d = 1.09], and IPS (F_1,6 = 21.51, P = 0.004, η² = 0.78), but not in the early visual areas (F_1,6 = 1.35, P = 0.29, η² = 0.18). Furthermore, the classification accuracy showed a similar pattern between collinear versus random stimuli and collinear versus ladder stimuli. There were a nonsignificant main effect of classifier (F_1,6= 1.5, P = 0.267, η²= 0.20) and a nonsignificant interaction between classifier and ROI (F_{3.5, 21.2}= 1.07, P = 0.39, η²= 0.15), confirming that the activities in these areas reflect integration signals under slit-viewing. These results suggest that higher visual (ventral and dorsal) and posterior parietal regions contain information about the perceived contours under the slit-viewing condition, rather than the perceived vertical motion of similarly oriented contour elements.

We performed additional control analyses to examine whether the observed fMRI activation patterns could be due to differences in general arousal levels or eye movements. We did not observe any significant univariate BOLD signal differences between contour stimuli and random stimuli in the first fMRI experiment (F_1,8< 1, P = 0.60, η²= 0.036), suggesting that the fMRI results with the slit-viewing condition could not be explained by higher general arousal levels for salient contours. Comparing slit-viewing (Fig. 2) and full-field viewing (Fig. 3) conditions did not reveal a significant main effect of viewing condition (F_1,14< 1, P = 0.51, η²= 0.031). Behavioral performance in contour detection did not differ between slit-viewing and full-field viewing either [90.9 ± 1.2% vs. 93.3 ± 4.2%, t₍₁₄₎ < 1, P = 0.55, Cohen's d = 0.29]. Thus, the different activation patterns could not be explained by the differences in task difficulty that may alter observers' general arousal levels.

We recorded eye movements from 4 observers with an ASL 6000 Eye-tracker (Applied Science Laboratories, Bedford, MA, USA). Eye-tracking data were preprocessed with the Eyenal software from the same company and analyzed with custom Matlab code. For each stimulus condition, we calculated the horizontal and vertical eye positions and the amplitude and number of saccades. For each stimulus type, histograms of the horizontal and vertical eye positions were centered on the fixation at 0°. Paired t-tests indicated no significant differences in horizontal eye position [t₍₃₎ < 1, P = 0.47, Cohen's d = 0.41], vertical eye position [t₍₃₎ < 1, P = 0.57, Cohen's d = 0.32], number of saccades [t₍₃₎ < 1, P = 0.61, Cohen's d = 0.29], horizontal saccade amplitude [t₍₃₎= 1.3, P = 0.27, Cohen's d = 0.68], and vertical saccade amplitude [t₍₃₎ < 1, P = 0.95, Cohen's d = 0.04], regardless of whether the slit-viewed stimulus image contained contours or not. In addition, we did not find significant difference of eye positions in the first half and second half of a trial in the horizontal eye position (F_1,3= 3.5, P = 0.16, η²= 0.54), vertical eye position (F_1,3 < 1, P = 0.41, η²= 0.23), number of saccades (F_1,3= 1.7, P = 0.29, η²= 0.36), horizontal saccade amplitude (F_1,3= 3.8, P = 0.15, η²= 0.56), and vertical saccade amplitude (F_1,3= 3.5, P = 0.16, η²= 0.54), suggesting that eye positions did not change systematically over time.

Discussion

Statistical analysis of natural scene images demonstrates that collinearity and co-circularity occur at a higher probability than other geometrical relationships (Sigman and Gilber 2000; Geisler et al. 2001). These regularities well fit the Gestalt grouping rule of good continuity (Feldman 2001; Geisler et al. 2001). Our slit-viewed contour stimuli by design minimize the roles of V1 long-range horizontal connections, so that the Gestalt rule of good continuation needs to be implemented by higher-level mechanisms. This is feasible since the visual system may acquire contour statistics from mechanisms that may not entirely rely on low-level V1 horizontal connections (Geisler et al. 2001). Indeed, our fMRI evidence indicates the engagement of posterior parietal regions known to be involved in visual memory, and dorsal and ventral visual areas known to be involved in motion and shape processing, in CI over time.

Our fMRI results suggest that the posterior parietal cortex (POIPS) contains information related to the orientation of contours that are integrated over time (Fig. 2f). In contrast, ventral areas like LO that are representation invariant are ill-suited for the precise spatial and temporal encoding of contour elements that are integrated over time (Fig. 2f). POIPS is known to be involved not only in visual working memory (Todd and Marois 2004; Marois and Ivanoff 2005; Xu and Chun 2006), but also in attentional processing (Corbetta et al. 1998). However, differential POIPS activation patterns for different contour orientations suggest that here POIPS is mainly responsible for CI over time in the visual memory, rather than for general attentional processing, consistent with other studies on slit-viewing (Mateeff et al. 1993; Nishida 2004; Silvanto and Cattaneo 2010). The roles of the dorsal areas in global percepts defined by motion signals have also been reported elsewhere (Caclin et al. 2012; Zaretskaya et al. 2013).

Our results also show that POIPS activation patterns cannot discriminate the orientations of full-field contours (Fig. 3d), consistent with Konen and Kastner (2008) in that regions of IPS are orientation invariant for spatial stimuli. As the key difference between CI over space versus time is the requirement of visual memory, the conflicting roles of POIPS in CI over space versus time suggest that POIPS may be involved in short-term visual memory processes necessary for CI under slit-viewing. We speculate that POIPS memorizes orientations of contour elements passed through a slit and reconstruct the image of contour paths. The findings in agreement with previous studies show that superior IPS maintains details of object features in visual memory (Xu and Chun 2006; Bettencourt and Xu 2016).

However, CI over time and space may share some component processes. Our fMRI evidence indicates that ventral areas like LO also participate in CI over time (Fig. 2d), and that both dorsal and ventral areas are involved in CI over space (Fig. 3b) as in other similar imaging studies (Murray et al. 2002, 2004; Altmann et al. 2003; Kourtzi et al. 2003; Shpaner et al. 2013). Moreover, 3D spatial contour interpolation, which involves integration of information from multiple depth cues, requires parietal areas as well (Sakata et al. 1997; Kellman et al. 2005). It is likely that, regardless of where and how the CI process is initially implemented in the brain, the higher-tier ventral areas responsible for global form processing, such as the LO, are necessary for global contour perception.

As a final note, in this study, the exclusion of the roles of V1 horizontal connections in CI over time is based on the psychophysical stimulus design, not the null difference of V1 BOLD patterns between the contour and random stimuli (Fig. 2d). Because of the limited spatial resolution of fMRI, in principle, a voxel is unable to separate the activations by the contour and random stimuli (Guo et al. 2007), regardless of whether or not V1 neurons are able to detect the slit-viewed contours.

Funding

This research was supported by the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 255577, the Biotechnology and Biological Sciences Research Council to Z.K. (D52199X and E027436), and National Natural Science Foundation of China Grants to C.Y. (31230030), S.K. (31571160), and W.L. (91432102).

Notes

Conflict of Interest: None declared.

References