Functional Neuroanatomy of Time-To-Passage Perception

Functional Neuroanatomy of Time-To-Passage Perception Abstract The time until an approaching object passes the observer is referred to as time-to-passage (TTP). Accurate judgment of TTP is critical for visually guided navigation, such as when walking, riding a bicycle, or driving a car. Previous research has shown that observers are able to make TTP judgments in the absence of information about local retinal object expansion. In this pa-per we combine psychophysics and functional MRI (fMRI) to investigate the neural substrate of TTP processing. In a previous psychophysical study, we demonstrated that when local retinal expansion cues are not available, ob-servers take advantage of multiple sources of information to judge TTP, such as optic flow and object retinal velocities, and integrate these cues through a flexible and economic strategy. To induce strategy changes, we introduced trials with motion but without coherent optic flow (0% coherence of the background), and trials with coherent, but noisy, optic flow (75% coherence of the background). In a functional magnetic resonance imaging (fMRI) study we found that coherent optic flow cues resulted in better behavioral performance as well as higher and broader cortical activations across the vis-ual motion processing pathway. Blood oxygen-level-dependent (BOLD) sig-nal changes showed significant involvement of optic flow processing in the precentral sulcus (PreCS), postcentral sulcus (PostCS) and middle temporal gyrus (MTG) across all conditions. Not only highly activated during motion processing, bilateral hMT areas also showed a complex pattern in TTP judg-ment processing, which reflected a flexible TTP response strategy. 1. Introduction As we drive along a road, a continuous optic flow pattern projected onto our retina is available to our visual system, and at any given point in time it can be used to compute the time remaining until we pass an object of interest. This time to passage (TTP), together with cues about motion trajectory, allows us to anticipate and judge oncoming objects regarding their path of movement and to prepare time-critical motor actions. Despite ample research on the topic, it still remains unresolved how TTP is computed and which other optical (such as ob-ject velocity and expansion cues) are being exploited when observers are asked to provide judgments regarding the time to passage of an oncoming object. One reason why these cues have been eluding identification may lie in the adaptive nature of the visual system. In a recent psychophysical study, we have shown that the visual system appears to employ an adaptive strategy that changes with the task at hand [1]. We presented a moving cloud of randomly placed dots viewed through a square aperture, consistent with forward observer motion but devoid of local expansion cues. Therefore, the dots remained of constant retinal size throughout the motion display. In each trial, two dots were colored red, and upon occluding the display, observers had to indicate which of the two dots would pass first the observer’s eye plane. Combined with two coherence levels (0% and 75%), that is, the proportion of background dots that could not deviate from the motion they should perform when thought of as stationary points in the 3D flow-field approaching the observer. Our results showed that, when no coherent optic flow was available (coherence 0%), observers resorted to the use of a relative velocity strategy and picked the dot with the faster screen velocity. However, when optic flow was highly coherent (75%), observers used a more complex strategy involving the global flow-field information. Global tau is a property of coherent optic flow that relies on the systematic change of distances on the retina and between-objects. Global tau makes the as-sumption that 3D-distances between the objects in the world remain constant thus producing coherent optic flow, and it can be computed from the relative rate of change of the angular displacement of the target from the observer’s line of sight [2]. This flow-field analysis is exploiting properties of coherent optic flow which cannot be reduced to, but sometimes are correlated with local expan-sion cues (expanding retinal object size). For instance, in simulated forward mo-tion through a cloud of fixed spherical objects, the systematic change of retinal distances among these objects specifies the direction of the observer’s motion through the cloud, provided that the objects remain static. The angular subtense between the observer’s path (track vector) and a given object, or more precisely the relative rate of change of this angular subtense, gives away the object’s TTP. Far-away objects typically produce less centrifugal retinal motion than close-by objects [2]. Interestingly, TTP is specified for expansionless objects as long as they do not coincide with the tracking direction. Thus, the optic flowfield pro-vides TTP information even when it is devoid of local expansion cues [3] [4]. In naturalistic scenarios, in which the retinal size of the targets does expand, both the local expansion cues (local tau) and the global tau cues can be exploited by the visual system to predict TTP of the target. Studies on the utilization of local tau information report that the neural substrate involved in the extraction of local tau expansion cues is the locus rotundus in pigeons [5], and more re-cently fMRI studies in humans point to the superior colliculus, the pulvinar nucleus of the thalamus, and cortical regions associated with motor preparation [6] [7]. fMRI studies of timeto-contact estimation tasks (TTC) have demon-strated significant cortical activation in left inferior parietal regions [8] [9], su-perior parietal, motor and cortical regions around the central sulcus [10], the insula, and inferior and middle frontal areas [6]. Bilateral hMT areas play im-portant roles in optic flow processing [2] [11] [12]. All of these regions show more or less specific activation in response to the local retinal expansion of looming objects that move toward the observer. However, the specific neural substrate associated with global tau has not yet been identified. In the present study we used fMRI to identify such regions in human observ-ers. Given that at the behavioral level, observers differ in the strategies they use between the case of local expansion scenarios and expansionless global optic flow, one would expect some shared but also some specific cortical areas to be involved in TTP judgment. Imagine a cloud of fixed expansionless objects (dots) through which an ob-server is moving. Two dots are marked red while all the others are white. If these dots are at equal lateral distance on opposite sides of the track vector, then the dot that is sagittally farther away from the observer will project closer to the fo-cus of expansion in the retinal flow pattern. If the observer is asked to judge which of the two marked dots is closer, she/he could base the decision on this fact. In other words, in the case of such symmetrical lateral spacing, observers might use an imagebased strategy once they have discerned the track vector from the optic flow. Reducing the coherence of the optic flow makes it harder to determine the track vector, and performance should break down or resort to some other strategy. For instance, subjects may merely base their judgments on how far a target is from the center of the screen. We have previously found that observers employ flexible strategies that can use a combination of global flow analysis and image-based cues [1]. Thus, we created stimuli that provide infor-mation about the direction of selfmotion (track vector and track velocity) and others that do not. The former provides global information containing a certain amount of noise (75% coherence), the latter preserves the local motion magni-tude but removes all global information (0% coherence). Note that local tau in-formation was absent at all times. We have collected task-based fMRI data while observers were making TTP judgments in the absence of local tau information. By manipulating the initial positions of the target objects relative to the observers’ track vector and by changing the coherence of optic flow (75% or 0%), we used a limited number of specific information sources that observers could exploit when making TTP judgments. Based on our previous psychophysical study [1], we hypothesized that observers use global flow information in the case of 75% coherent flow, but resort to guesswork when global flow is incoherent. Our behavioral results are consistent with previous psychophysics findings: TTP judgments reflected the differential use and integration of multiple sources of information, including global optic flow, object retinal velocities, and other depth cues [13] [14] [15]. In the experiment detailed below, we will focus on and interpret the cortical and subcortical activations in light of the likely response strategies. 2. Methods 2.1. Subjects Seven subjects (5 females, 2 males, mean age = 24.42 years, SD = 4.82 years) par-ticipated in the study. They were graduate students at Boston University, re-cruited from our pool of subjects. All of them had normal or corrected-to-normal vision. All underwent a psychophysical testing session prior to the scan, to make sure that their performance was at least 70% correct for ∆τ = 0.5 sec for the symmetric configuration regardless of the initial x-offset and background mo-tion coherence (0% or 75%) [16]. All participants signed an informed consent form before the start of the experimental sessions in accordance with the re-quirements on research involving human subjects, as approved by the Massa-chusetts General and Boston University Institutional Research Boards. All sub-jects fully satisfied the inclusion criteria for participating in MRI/fMRI studies and none of the exclusion criteria were met. They participated previously in other psychophysical and functional imaging tasks conducted by our research team. Those studies had no similarity with the task reported here. All subjects reassured us they could pay attention throughout an experimental task, maintain fixation, and stay still during the imaging experiments. 2.2. Apparatus and Data Acquisition The stimuli were generated on an Intel-based Macintosh laptop and displayed at a resolution of 1024 × 768 pixels and a refresh rate of 75 Hz. Two of the dots, re-ferred to as target dots, were red (51.20 cd/m2) and the rest of them were white (79.55 cd/m2), all displayed against a gray background (10.22 cd/m2). They were backprojected onto a translucent screen (27.3 cm × 36.5 cm) using a LCD pro-jector. Subjects viewed the translucent screen through a mirror mounted on the head coil of a whole-body scanner. The distance between the eyes of the subject and the mirror was approximately 4 cm and the distance between the mirror and the screen was approximately 81 cm, therefore, the total viewing distance was about 85 cm. This setup provided a square viewing aperture subtending 17˚ × 17˚. fMRI data were acquired at Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital, using a 3T Siemens whole body scan-ner and a standard 8-channel head coil. Structural images were obtained as T1 weighted magnetization prepared rapid acquisition gradient echo images (MPRAGE) (128 slices with slice thickness of 1.33 mm, voxel size: 1.00 × 1.00 × 1.33, FOV = 256, TR = 2.53 sec, TE = 3.39 msec, flip angle = 90˚). Two T1-weighted images were collected for each subject. Functional images were obtained with gradient echo, echo planar (EPI) interleaved sequence (33 slices with slices oriented along the AC-PC line, slice thickness of 3 mm with 20% dis-tance factor, FOV = 200, TR = 2.00 sec, TE = 30 msec, flip angle = 90˚) for mea-surement of BOLD contrast. 2.3. Stimuli and Experimental Procedure A field of moving white dots simulated the observer’s forward self-motion in 3D and was presented through a square viewing aperture. The dots remained sta-tionary with respect to one-another in the simulated space. Subjects were asked to indicate which one of two red dots would pass their eye plane first, mimicking they were moving forward through the field. All of the dots subtended 2 pixels × 2 pixels (4 arcmin × 4 arcmin) throughout the simulated approach and were placed such that they maintained a density of 2 dots/deg2. The screen size of all dots, including the targets, remained constant, thus eliminating all local tau cues. The motion of the dots simulated the subjects’ forward self-motion along a straight-line trajectory at a speed of 150 cm/s. In each trial, the direction of si-mulated self-motion was toward the center of the aperture. Dots that moved out of the volume behind the observer’s eye plane were randomly assigned to new locations such that the density of the dots remained constant (Figure 1) [1]. Figure 1. Schematic of the virtual trapezoidal volume. White dots simulating forward self-motion were randomly distributed between 260 cm and 2060 cm from the observer. The two red target dots were embedded in the flow field and moved with the same speed as the flow field according to their instantaneous position within the volume. The direction of self-motion matched the center of the aperture [1]. The dots maintained constant screen size. The target dots were placed at different depths such that the difference be-tween their passage times (tau difference Δtau) was set to be 0.5 sec for all trials. This value was chosen because it should be just above detection threshold, rendering the task of identifying the leading target meaningful but still allowing for errors. The initial depth of the reference target was 1200 cm. Thus, the possible TTP values from stimulus onset until both targets would have passed the ob-server, ranged from 7.5 s to 8.5 s. This left ample time for the 2AFC responses, which had to be made before the leading target passed the vertical eye-plane of the observer. Observers were not required to respond as quickly as possible. The two target dots were placed such that they were always on opposite sides of the track vector. Their lateral distances from the track vector (x-offsets) were either 10 cm or 50 cm in the simulated space. This resulted in four different lat-eral target offset combinations: Two equidistant symmetric placements (leading target 10 cm to one side-trailing target 10 cm to the other side, or 50 cm - 50 cm). In the other two combinations, the targets were placed with asymmetric x-offsets (10 cm - 50 cm, and 50 cm - 10 cm). These 4 stimuli were paired with two coherence levels of the background dots. Remember that an entirely inco-herent flow-field no longer specifies observer motion. Our previous psycho-physical data [1] showed that 50% coherence of the dots begins to provide prominent global flow information relative to the condition of 0% coherence dots. This design led to eight unique stimuli, each of which was repeated 16 times within a run, using a randomized event-related design paradigm. The visual stimulus was occluded after 3 seconds. The next trial would not be presented until a decision had been made. The timing and order were rando-mized using optseq2. Inter-stimulus Inter-vals (ISIs) between trials varied from 1 - 7 s. Frames with static dots were pre-sented within the ISIs, serving as a baseline condition. During the whole scan-ning period, subjects were required to fixate a small central cross (40 × 40 arc-min). Stimuli were presented binocularly in a two-alternative forced choice (2AFC) paradigm without feedback. The subjects’ task was to determine which of the two targets would arrive at their eye plane first. Subjects entered their responses by pressing a designated key on a magnet-compatible button box. A separate block design employing a MT localizer task was performed by all subjects in two runs. The human middle temporal complex (hMT) has been shown to be highly involved in motion processing, including optic flow. Accor-dingly, area hMT was functionally localized by utilizing moving and static dot patterns [17], so that we could locate the exact position of hMT for further anal-ysis [18]. Other anatomical regions were defined with normalized functional images, using the Automated Anatomical Labeling (AAL) atlas. 2.4. Data Analysis Imaging data analysis was performed using the Statistical Parametric Mapping software package (SPM12, Wellcome Department of Cognitive Neurology, Lon-don, UK) and utilizing MATLAB (The MathWorks, Natick, Massachusetts, USA). The preprocessing steps were as follows: 1) format conversion by con-verting data from original DICOM files to Nifti files, 2) slice timing eliminated the time shift for all voxels of functional images, 3) realignment by motion correction, 4) co-registration of the anatomical image with the mean functional image, 5) spatial normalization normalized all images to a standard space (Mon-treal Neurological Institute, MNI) with a voxel size of 3 × 3 × 3 mm, and 6) smoothing: all functional data were smoothed with a FWHM (Full Width at Half Maximum) kernel of 4 mm. For each subject, the onset and duration of each condition was modeled by a general linear model (GLM). The motion parameters from realignment were al-so used as multiple regressors in generating the design matrix. Trials with cor-rect responses and incorrect responses were separated as different conditions. With the contrast images of each subject, a group level randomeffect analysis was performed for each condition. The resulting t-value maps were set as un-corrected for multiple comparisons, p < 0.05. Clusters with less than 10 conti-guous voxels were excluded. Based on group-level activation maps in normalized space, we defined several functional regions of interest (ROI) for each single subject. Every functional ROI was defined as a sphere, with its center at the respective local maximum of the activation cluster and with a 5mm radius. Subsequently, we calculated the per-cent BOLD signal change for each functional ROI using Marsbar [19]. Bilateral hMT areas were defined using localizer tasks [18]. We set the mini-mum overlapped proportion on individual activation maps as 0.5, based on the group-constrained subject-specific (GSS) method [20]. 3. Results 3.1. Behavioral Performance Response accuracy for each condition was first calculated per subject and then averaged across subjects (Figure 2). As expected, subjects could do the task and performed well above chance when the two target dots were symmetric around the track vector of the simulated motion. This was the case for both 0% cohe-rence and 75% coherence conditions. Thus, subjects exploited the simple im-age-base cue of eccentricity to guide their answer choices. In contrast, when the two target dots were placed asymmetrically, such that the correlation of target eccentricity and proximity to the observer was severely reduced, the response accuracy was around chance level. Thus, the behavioral performance showed that in the absence of local expansion cues, TTP judgments were based on retinal eccentricities. Interestingly, when the flow field contained additional information about the track vector, performance improved. When the target dots were symmetric, subjects performed better under the 75% coherence condition compared to 0% coherence. Thus, global motion information provided by background dots Figure 2. Behavioral performance accuracy in % as a function of lateral target-offset pairing and coherence of optic flow. Offset pairings indicate the lateral distance of the leading and trailing targets from the track vector. They were always placed on opposite sides. Thus, a pairing of 10 vs 50 signifies that the leading target was placed at a lateral offset of 10 cm in virtual space, whereas the trailing target was offset by 50 cm to the op-posite side. Error bars indicate standard derivations. enhanced subjects’ performance only if the targets were spaced symmetrically around the track vector. However, when the target dots were asymmetric, there were no significant differences between 0% coherence and 75% coherence (p > 0.05 in paired t-test). These results replicate previous behavioral results we col-lected with a similar experimental design [1]. 3.2. Functional Imaging We contrasted activation during the trials against the activation within the static dots presentation (baseline) to obtain significance activation maps. The analysis was performed separately for 0% and 75% coherence levels in the optic flow field Table 1 & Table 2. The runs with the same coherence value were grouped to-gether and the contrasts were done separately for each experimental condition. Based on behavioral performance, we separated correct responses and incorrect responses for each condition. Therefore, the experimental conditions were the cases where initial target x-offsets were symmetric and 10 cm from the center of the aperture (10 vs 10), symmetric and 50 cm from the center of the aperture (50 vs 50), and the leading target’s initial x-offset was 10 cm from the center of the aperture (10 vs 50) or the leading target’s initial x-offset was 50 cm from the center of the aperture (50 vs 10), either with correct responses or incorrect res-ponses (Figure 3). Table 1. Activation position of the local maxima in MNI space for 0% coherence trials by lateral target-offset pairing. Experimental Conditions Offset 10 cmvs 10 cm Offset 10 cm vs 50 cm Center coordinates (MNI) t score Region Experimental Conditions Center coordinates (MNI) t score Region −10 5.3335 IFG_L 26 −14 5.2095 IFG_R −22 29 50 4.3096 MFG_L PreCS_L −43 −43 50 4.0464 IPG_L 4.8364 PreCS_L −19 44 44 3.8016 SFG_L 45 4.6973 PreCS_L −51 3 37 3.4871 PreCS_L −76 22 4.3901 MTG_R −24 30 47 3.4871 MFG_L −37 −65 44 4.295 AG_L −52 −70 28 3.4667 AG_L −26 31 43 4.1308 MFG_L 52 −62 19 2.9655 MTG_R −43 15 46 3.7597 MFG_L −42 −43 51 2.7081 IPG_L 24 −48 73 3.6131 SPG_R 39 −57 52 2.686 AG_R −64 −19 −23 3.5173 MTG_L 32 −70 57 2.3657 SPG_R −50 13 25 3.2137 IFG_L −64 −24 −23 2.3448 ITG_L −56 −1 −25 3.164 MTG_L −51 −72 33 2.0063 AG_L −25 −6 65 3.103 SFG_L 32 2 59 10.3475 MFG_R −49 11 27 3.0358 IFG_L 50 42 6 10.0239 IFG_R −27 32 42 2.9486 MFG_L 29 5 59 8.7421 MFG_R 45 12 31 2.6691 IFG_R 45 42 22 8.4385 MFG_R 11 39 42 2.4373 SFG_R 47 43 16 6.8521 MFG_R 62 −40 25 2.1131 SMG_R −49 −33 58 6.1049 PostCS_L 15 −72 28 4.9103 Cuneus_R 57 −40 50 5.9746 IPG_R −41 −41 57 4.4563 PostCS_L 44 −46 52 5.9718 IPG_R 61 −33 45 3.6884 SMG_R −39 15 1 4.6148 Insula_L 22 11 65 3.3684 SFG_R 47 45 13 4.3168 MFG_R 42 19 37 4.1277 MFG_R −50 11 25 3.9546 IFG_L x y z x y z −62 −45 1 5.8782 MTG_L −48 38 −45 30 31 5.6209 MFG_L 39 38 −9 61 5.4201 PreCS_R −42 −10 58 5.0111 −30 −23 68 −45 9 52 Offset 50 cm vs 10 cm Offset 50 cm vs 50 cm 60 −37 47 3.3336 IPG_R 32 4 57 3.2927 MFG_R −39 28 −14 3.2274 IFG_L −39 −25 63 3.7812 PreCS_L 48 42 14 2.8553 MFG_R 27 −86 42 3.6851 SOG_R 55 −67 10 2.6978 MTG_R 55 −61 31 3.4739 AG_R −23 61 15 2.6792 SFG_L −44 48 18 3.1624 MFG_L 39 −51 57 2.6526 SPG_R −20 25 56 3.0011 SFG_L 59 −64 −9 2.4649 ITG_R −50 34 11 2.6471 IFG_L −46 36 19 2.4134 MFG_L −45 10 42 2.5428 PreCS_L −45 21 41 2.2472 MFG_L −61 −62 −8 2.2485 ITG_L −9 −65 49 2.1421 Precuneus_L −62 −57 −2 2.1452 MTG_L 40 22 −10 2.1283 Insula_R −44 20 43 2.0606 MFG_L Table 2. Activation position of the local maxima in MNI space for 75% coherence trials by lateral target-offset pairing. Experimental Conditions Offset 10 cm vs 10 cm Offset 50cm vs 50 cm Center coordinates (MNI) Experimental Conditions Center coordinates (MNI) t score Region −19 6.4753 MTG_L −44 63 5.1668 PostCS_R 28 −10 68 5.1602 SFG_R SPG_L 51 −67 24 4.9411 MOG_R 5.5004 PostCS_L 59 2 33 4.7903 PreCS_R 6 5.317 Insula_L −58 −59 31 4.7567 AG_L t score Region x y z x y z −67 −25 −21 6.8192 ITG_L −56 −2 −43 12 49 6.4968 MFG_L 34 −47 33 −6 6.0027 IFG_L −34 −57 64 5.5322 −49 −10 54 −42 14 −35 −28 65 5.0485 PreCS_L 51 8 −29 4.3979 MTG_R 64 −40 23 4.822 SMG_R −44 10 48 4.3957 PreCS_L −68 −43 −5 4.5121 MTG_L 52 −74 22 4.3723 MTG_R −40 18 49 4.2904 MFG_L 34 −41 67 4.0675 PostCS_R −62 −36 −1 4.1254 MTG_L −19 −25 74 3.856 ParaCL_L 42 −12 61 4.0469 PreCS_R −43 20 45 3.7685 MFG_L −20 33 51 3.9973 SFG_L −21 36 48 3.3604 SFG_L −45 5 0 3.8583 Insula_L 57 1 −21 3.301 MTG_R 64 −58 2 3.8517 MTG_R −53 −68 22 2.6059 MTG_L 59 −4 −14 3.6064 MTG_R −57 −40 50 2.2667 IPG_L 68 −32 38 3.4659 SMG_R 43 −60 48 8.139 AG_R Offset 10 cm vs 50 cm −49 32 18 3.4331 IFG_L 65 −8 30 6.9923 PostCS_R −66 −24 −19 2.7305 ITG_L −53 −68 38 6.8438 AG_L −64 −57 −9 2.5988 MTG_L −19 43 45 6.1479 SFG_L −61 −53 2 8.8346 MTG_L −54 −67 29 5.4362 AG_L −49 5 35 7.6157 PreCS_L −59 −11 −31 5.381 ITG_L −65 −21 −16 7.4735 MTG_L −44 13 45 5.3593 MFG_L −59 −59 27 5.4359 AG_L −29 −72 56 5.2191 SPG_L −18 29 55 5.2575 SFG_L 45 −50 52 5.1744 IPG_R −58 −65 −9 5.056 ITG_L 31 61 −3 4.7883 SFG_R −32 −68 58 4.4519 SPG_L −22 24 58 4.7555 SFG_L 47 47 0 4.2252 IFG_R −42 6 52 4.7128 MFG_L 65 −36 −11 4.0454 MTG_R −44 −50 52 4.6378 IPG_L 64 −52 −12 3.94 ITG_R 60 −8 −22 4.582 MTG_R 15 36 54 3.852 SFG_R −31 −69 58 4.5712 SPG_L −43 43 19 3.8006 MFG_L −55 23 13 4.5267 IFG_L −51 −72 31 3.0919 AG_L −62 −19 −12 4.4022 MTG_L 54 32 14 2.9253 IFG_R −10 32 53 4.2148 SFG_L −40 4 2 2.7089 Insula_L 56 20 15 4.0634 IFG_R Offset 50 cm vs 10 cm 49 −73 38 2.6896 AG_R 42 23 37 4.02 MFG_R 34 −1 −32 2.5066 Fusiform_R −50 20 24 3.67 IFG_L −60 −14 −30 2.2694 ITG_L −38 22 49 3.6474 MFG_L −34 17 −18 2.2464 IFG_L −63 −41 −1 3.556 MTG_L Continued 51 −26 55 2.1535 PostCS_R −64 −45 −3 3.4299 MTG_L −58 −12 40 3.2885 PostCS_L −46 46 −5 3.2316 IFG_L 47 −69 47 3.0358 AG_R 66 −30 −20 2.7127 ITG_R 52 −26 53 2.5203 PostCS_R −59 −11 −11 2.1069 MTG_L AG: angular gyrus, IFG: inferior frontal gyrus, IPG: inferior parietal gyrus, ITG: inferior temporal gyrus, MFG: middle frontal gyrus, MOG: middle occipital gyrus, MTG: middle temporal gyrus, ParaCL: paracentral lobule, PostCS: postcentral cortex, PreCS: precentral cortex, SFG: superior frontal gyrus, SMG: supramarginal gyrus, SOG: superior occipital gyrus, SPG: superior parietal gyrus. (a) (b) (c) (d) Figure 3. Cortical activation maps as a function of lateral target-offset pairing and cohe-rence of optic flow. The threshold tvalue was at 1.943, uncorrected p < 0.05, extent 10 voxels. Panel (a) condition 10 vs 10; (b) 10 vs 50; (c) 50 vs 10; (d) 50 vs 50. Area hMT showed activation in all conditions based on the findings from the localizer task. Figure 3 ((a) condition 10 cm vs 10 cm; (b) 10 cm vs 50 cm; (c) 50 cm vs 10 cm; (d) 50 cm vs 50 cm) shows the activation maps of trials with correct res-ponses with a threshold t-value 1.943, extent of 10 voxels (p < 0.05, uncorrected), across all the subjects. Table 1 and Table 2 show the corresponding coordinates of the local maxima of these clusters, in MNI space. In general, across all the subjects, the activation areas were distributed along the motion processing pathway [21]. Consistent with previous research [22], the activation was not only showing along the ventral pathway (also known as “what” pathway), it also showed along the dorsal pathway (also known as “where” path-way). Activation during the stimulus motion was significantly elevated com-pared to baseline in the occipital lobe, including the superior occipital gyrus (SOG), temporal lobe including middle temporal gyrus (MTG), inferior tempor-al gyrus (ITG) and superior temporal sulcus (STS), parietal lobe including supe-rior parietal gyrus (SPG) and intraparietal sulcus (IPS), frontal lobe including middle frontal gyrus (MFG) and inferior frontal gyrus (IFG). In addition, the precentral sulcus and post-central sulcus were activated bilaterally across all subjects. For all x-offset target conditions, when we compared the activation for 0% coherence and 75% coherence conditions, the latter resulted in more distributed activation aroundthe bilateral pre-central and post-central temporal areas, MTG, IFG and IPS. Based on previous functional imaging data related to TTP and TTC processing (e.g. Field & Wann, 2005), we defined several functional regions of interest based on the Automated Anatomical Labeling (AAL) atlas. Figure 4 shows the BOLD percentsignal changes in these functional ROIs. By and large the BOLD percentsignal changes increased from 0% to 75% coherence levels in bilateral precentral, postcentral, and middle temporal areas, whereas they de-creased in superior and middle frontal areas. In the inferior frontal and parietal cortical regions, including intra-parietal sulcus, there was not much difference between the two coherence levels. Specifically, in the left precentralsulcus (PreCS: p = 0.045), left postcentralsulcus (PostCS: p = 0.027) and bilaterally in the middle temporal (MTG: p = 0.025) cortical regions, percentsignal changes at 75% coherence were significantly larger than those at the 0% coherence level, for the symmetric condition (10 vs 10). In the right hemisphere MTG: p = 0.024), significant activations were also found in the asymmetric condition (10 vs 50) at both coherence levels. Bilaterally in the middle frontalgyrus (MFG: left: p = 0.012, right: p = 0.005) and right PreCS (p = 0.046) areas, there were significant differences between the 0% and the 75% coherence level in the asymmetric con-dition (50 vs 10). In the left inferior frontal gyrus (IFG: 0.034), percentsignal changes were also found significantly higher under 75% coherence than 0% co-herence, in the symmetric condition (50 vs 50). In MFG in the left hemisphere (p = 0.030), percent signal changes were found to be significantly higher at 0% coherence than at 75% coherence, in the symmetric condition (10 vs 10). Inte-restingly, bilateral significant differences in percent signal change were found in hMT areas for asymmetric conditions, but not for symmetric conditions (left hMT: p = 0.013 in 10 vs 50, right hMT: p = 0.032 in 10 vs 50 and p = 0.036 in 50 vs 10). The percent signal changes suggest that bilateral precentral and postcentral sulci as well as a MTG are highly involved in the processing of global optic flow. The activation in hMT bilaterally suggests that more complicated visual processing is performed when there is more than one cue that subjects might use (e.g. global Figure 4. BOLD percent signal changes plotted by functional region of interest (ROI) and lateral target-offset condition (*indicates p < 0.05; **indicates p < 0.025). optic flow, object velocities, and symmetry heuristics) for their TTP judgments. The activation of IFG and MFG may be underlying the process of decision mak-ing for solving the task. 3.3. Correlation of Behavioral Responses with BOLD Percent Signal Change For each condition described above, we also compared the trials in which sub-jects gave the correct responses to those with incorrect responses. For the sym-metric conditions (offsets of 10 cm vs 10 cm and 50 cm vs 50 cm), the activa-tions were more extended for trials with correct responses than for those with incorrect responses. The activations in precentral postcentral, IPS, MTG and MFG regions, in both correct and incorrect trials, suggest the involvement of these areas in solving the underlying task. For asymmetric conditions with the leading target at 10 cm x-offset (10 vs 50), invalid image velocity information was provided to the subjects, resulting in more distributed activation in incorrect trials than in correct trials, at 0% coherence. When coherent background dots were presented (75% coherence), thereby providing global motion information, subjects exploited multiple cues, resulting in more distributed activation in cor-rect trials than in incorrect trials. Conversely, for the asymmetric conditions with the leading target at 50 cm x-offset (50 vs 10), valid image velocity informa-tion was provided, and activations were more distributed in correct trials than incorrect ones in both 0% and 75% coherence level. 4. Discussion In this study, we have used fMRI to record observers’ TTP judgments in the ab-sence of local expansion information. During simulated forward motion, the observer had to judge, which of two red dots would pass him/ her first. We have presented the information indicative of forward motion of the observer (global flow information) by manipulating the coherence of the flow field (no coherence vs. 75% coherence). We also manipulated the lateral offsets of the targets from the track vector and the initial target depths from the observer. Since local ex-pansion information was not present in the optic flow, only global flow informa-tion could be used for the task. From this global flow information observers could, in principle, utilize simple image cues and/or the complex pattern provided by the entire dot field, but they could do so only when the dot field (RDK) moved coherently. In the case of incoherent motion of the dots, observers could have used other cues, such as relative screen velocity of the targets, which re-mained available in the display. However, the latter image cues were only valid when the targets were spaced symmetrically around the track vector. In cases of asymmetric spacing and incoherent optic flow, no useful information about rela-tive TTP remained and, as expected, subjects’ behavioral performance was at chance. When the two targets were spaced asymmetrically, coherent optic-flow could in principle produce good performance if the relative rates of change of the re-spective angle between the target object and the track vector are considered. However, our results showed that this was not the case suggesting that in the presence of asymmetric targets, observers failed to exploit the global flow infor-mation for judging TTP. Simpler image cues, such as the targets’ positions and relative velocities would only provide valid information if the targets are spaced symmetrically to either side of the track vector (or the center of the aperture). Thus, only in the presence of symmetric targets, could above-chance perfor-mance be reached with incoherent flow. This did in fact improve performance but failed to approach perfection. Thus, with multiple sources of information, when judging TTP, subjects appear to integrate several cues through an eco-nomic strategy that mostly rely on image cues. This strategy becomes clearly no-ticed when local tau information is missing and the symmetry assumption holds. The cortical activities during TTP judgments reflect this economic strategy. In general, subjects showed higher and broader activations on trials with 75% co-herence than on those with 0% coherence. This suggests that they did processop-tic flow information when making TTP judgments, which is consistent with pre-vious studies [3]. When the two targets were located symmetrically around the observer’s track vector, the percent-signal changes on bilateral PreCS, PostCS and MTG proved significantly higher when global optic flow was available, as compared to when it was absent. This is where coherent global flow provided a behavioral advantage. Previous retinotopic mapping and fMRI studies in humans have established a continuum of several motionselective regions, including cortical areas hMT and superior parietal gyrus [23] [24]. In our study, bilateral hMT cortical regions were activated across all subjects, regardless of their performance on judging TTP. Significant differences in percent signal change were found bilaterally in hMT when the two targets were asymmetric. Activity during stimuli with 0% cohe-rence was higher than during stimuli with 75% coherence in the 10 vs 50 condi-tion, whereas activity during stimuli with 75% coherence was higher than during stimuli with 0% coherence in the 50 vs 10 condition. This points to lateralized differences that reflect the complex reaction of hMT to changes in global and local information. Remember that only when global cues were unavailable, sub-jects based their TTP judgments on the velocity discrepancy between the targets [2] [25]. Consistent with previous research, we also found activation in bilateral supe-rior colliculus (SC), which is an area involved in motor preparation and atten-tion [6]. The percent signal change was not significantly different in the condi-tions of 0% coherence and 75% coherence, which is expected as there should not be a difference in motor (response) preparation between different coherence le-vels. 5. Conclusion In summary, in this study we investigated the neural substrate of the mechan-isms involved in TTP judgments in the absence of local expansion cues. Previous behavioral results suggested that the subjects base their TTP judgments on the integration of multiple sources of information, with emphasis on image cues, such as target velocity, which are supplemented by global optic flow informa-tion, if the latter is coherent. Accordingly, and consistent with previous studies [6] [8] [9] [10], out fMRI results show a broad range of activation along the vis-ual motion processing pathway, which reflects the complex information processing strategy. Unlike in pigeons, there does not seem to be one area dedi-cated to TTP processing [5] [10] [26]. Instead, the BOLD percent-signal changes show that PreCS, PostCS and MTG are involved in global information processing. Strong activation has also been found in bilateral hMT areas. Further investigation of the cortical involvement in TTP judgments will contribute to-wards a better understanding of how temporal and spatial perceptual mechan-isms are integrated.