Cortical Dynamics of Perception and Decision in Sensory Tasks

Cortical dynamics of perception and decision in sensory tasks: an MEG study Introduction Perception and perceptual decisions arise from the spatiotemporal orchestration of activity distributed across brain networks. Functional MRI (fMRI) studies have shown that discrete networks mediate the sensory processing and the representation of visual search task1 . However, fMRI does not have the temporal precision required for revealing the dynamic cortical networks that integrate sensory information and coordinate the decision making process during perceptual tasks. Here we used anatomically constrained (by MRI) and magnetoencephalography (MEG) to compute the temporal signature and the oscillatory synchrony that modulates cortical interactions in two search tasks. In one, VS, visual, moving observers search for a moving object (target), and in the other, VAS, an auditory cue congruent with the target facilitates performance on the task. To understand where, when, and how cortical areas connect to each other during these tasks, in both time- and frequency- domains (α, β, and γ bands), we compared the direction and dynamics of the cortical networks using Granger causality in the time domain (DGC) and PLV in the frequency domain in 8 healthy observers. MEG Acquisition Time-domain Cortical Connectivity MEG datasets were acquired in a magnetically shielded room using a whole head 306-channel MEG (VectorView, Elekta-Neuromag, Helsinki, Finland) The MEG signals were band-pass filtered to the frequency range 0.5 – 200 Hz and digitized at 600 samples/s. There are 102 measurement locations each equipped with three sensors, one magnetometer and two planar gradiometers. Vertical and horizontal electrooculogram (EOG) measurements were also recorded to monitor eye-movement artifacts. For stimulus presentation we employed a DLP projector and a back-projection screen, plastic tubes connected to frequency-compensated loudspeakers outside the magnetically shielded room (ADU1c, Unides Design, Helsinki). Responses were recorded with a fiberoptic response pad. We compute temporal cortical connectivity through Granger Causality methods, which allow for the study of causal connections between cortical areas. We employ both time and frequency domain dynamic Granger-Geweke causality to investigate the transient connections between our cortical areas. Granger causality is calculated over a sliding window allowing for the study of transiently-active Granger causal connections over time. We take 30 random sets of 50% of the total trials and we compute averaged ROI time courses for each set. Time series are temporally demeaned and detrended. We compute the optimal sliding window size and AR order through the SURE criterion. For time-domain Granger causality, each time point’s statistic is calculated using the residuals of prediction with the optimal AR model over the window given only its own time series, y, over the residuals when also provided the additional time series x: Time Windows The stimulus is split into four time windows: T1: sensory (150 - 450 ms), T2: representation (450 - 1150 ms), T3: decision 1 (1150 1500 ms), T4: decision 2(1500 ms - 2000 ms). Behavioral Results Figure 2 Gx "y ( $# (t)* ) t=1 n 2 x,y For frequency domain Granger causality, each time point’s statistic is calculated using: ! % ($ # $2 / $ ) | H ( f ) | 2 ( xy yy yx Gx "y ( f ) = # ln'1# xx * Syy ( f ) & ) Stimuli and Task The stimuli (adapted from 1,2) consisted of: one second fade-in of nine textured spheres (1.5 degrees in diameter); one second static frame displaying the 9 static spheres; one second where 8 of the spheres, randomly selected, portray simulated forward motion of the observer, and the other sphere (target) moves independently with its own speed and looming motion (forward or backward). In the following 3 seconds the spheres are again shown static but numeric labels (1-4) are shown on four spheres, one of which is the target. In a 4AFC subjects indicated via a button press which was the target sphere. Percent correct and reaction times were collected. Two conditions of the experiment were run: Visual-only (V-O), and VisualAuditory (V-A). In the latter, a suprathreshold auditory pure tone is colocalized and in % of Figure trials1 moves in depth congruent with the target, while and in the colocalized in other % of % of trials is static. trials moves in depth congruent with the target, while in the other % of trials is static. ROI Selection ROIs (shown below) were chosen based on fMRI activation of the same task for the visual-only condition, and on MEG activation in both conditions, guided by the Freesurfer anatomical parcellation. Figure 2 Aud - Primary Auditory Csinf - Central Sulcus, Inferior Cssup - Central Sulcus, Superior DISPM = Dorsal Intraparietal Sulcus ITS - Inferior Temporal Sulcus MPFC - Middle Prefrontal Cortex %n = ln'$#y2 (t) & t=1 hMT =- human MT PreCS - Precentral Sulcus PoCS - Postcentral Sulcus STP - Superior Temporal Polysensory Area STS - Superior Temporal Sulcus VIP - Visual Intraparietal Figure 3 VS • Performance of all observers subjects was significantly better when auditory cues were available (proportions test7 VS vs VAS: p= 2.561x10-8) . • Given the small number of trials obtained during the imaging sessions, RT analysis should be considered more descriptive than quantitative. However, they are important here to show that in all subjects most of the responses occurred within the first 2 seconds of the stimulus, in particular starting at around 1.5 seconds from stimulus onset. Thus subjects made their decision in both VS and VAS tasks within the T4 time window. Onset Timing We consider an onset to occur in an ROI if its average time course within a time window exceeds 3 SD of the noise level computed from the 500 ms prestim baseline and lasts least 20 ms above this level (Raij et. al., 2010). We utilize the bootstrap method (250 bootstrap samples across epochs) and apply an approximate permutation test (using 1000 permutations) to compare timings across test conditions (VS/VAS). Group statistics are computed through Fisher’s method. False Discovery Rate (FDR) control is applied to the resulting p-values with an FDR of 0.001. The table below shows which condition has faster onsets in each time window. overall, the areas during the Visual Search (VS) condition are activated faster than in Visual Auditory Search (VAS). ! We compute the statistic for every time point in the time domain and, due to computation time constraints, every 40 ms. In time-domain dynamic Granger causality, a t-statistic is obtained for each datapoint by testing the set of granger scores in a part of the prestimulus (-500ms to -200ms) to the set of granger scores at a certain time point. The same is done in the frequency domain, but this is computed separately for each sampled frequency (1 Hz sampling, 5-120 Hz). Due to the large density of connections produced by the time domain, we use the data from the frequency domain representation to compute the temporal snapshots below. For each time point, the statistic is averaged over the highest 15 scores across all frequencies for each directed connection between two ROIs. The resulting value is thresholded at a -log10 P-value of 3.5, so that any edge that is less significant is thrown out. Dynamic Granger Causality Network Snapspots: Results In Fig 3, we show snapshots of dynamic Granger to illustrate the dynamics of the time-varying network. · In the early time windows (T1,T2) the main cortical interactions occur between areas involved in the sensory processing and computation of the representation, both modulated by attention and working memory. In T4, decision making stage, there is significant interaction between areas encoding the representation and the frontal areas involved in working memory (for maintenance of the representation) and decision making. VAS Figure 5 · In the VAS task, there is an early and transient connection of the Aud area with visually responsive areas. These connections no longer exist later on, when the mutimodal area (STP) have integrating visual-auditory information. The Aud area is again connected late (in T3&T4) in a network possibly involved in checking the detection of the target object. · In T3 the unimodal VS network is significantly more connected than the VAS network, suggesting that the computation of the representation might be more difficult (without the benefit of additional cues). Time-Domain and Frequency-Domain Dynamic Granger Causality: Results Figure 4 VS 5 60 time 0 1 0 2 0 1 1 0 1 60 0 20 0 0 1 2 1 2 5 0 10 0 1 2 0 0 time 1 2 0 5 2 VIP 50 0 2 2 7 0 2 1 Figure 9 α β γ 10 0 0 time 1 5 2 1 0 2 aud 50 0 5 2 Dynamic frequency-band correlations are discovered between each pair of ROIs through phase locking. The trial-by-trial ROI time courses are decomposed into complex time-frequency coefficients through the Morlet wavelet transform. Phase differences are computed between two ROIs by finding the difference between the phase angles of the Morlet wavelet coefficients at corresponding trials, times, and frequencies. For each pair of ROIs at each frequency, the phase difference across trials in the prestimulus from 500 to -200 ms is compared against the phase difference at each time point across trials during the two-second stimulus interval using the Uniform-Scores Test. The test statistic is fit to a χ 2 distribution (df=2) to obtain p-values, using Fisher’s method8 to combine controls. MPFC T1 50 1 0 Frequency Granger causality is computed over the entire windows for T1-T4. The statistic is computed using the same frequency granger method discussed in the previous section across each frequency band (sampled at every 1 Hz in each band). The resulting statistics are summed over the frequency band of interest. Through the permutation method, multiple samples of the distribution of the null granger score are found by permuting the time courses. We test significance by comparing the true statistic to the null distribution, with a significance threshold of 0.05. To combine across subjects, we use Fisher’s method. To correct for multiple comparisons, we apply the False Discovery Rate (FDR) method. Figure 7 0 5 Phase Locking VS 0 5 2 20 t-statistic of G X→Y t-statistic of G X→Y 0 60 0 20 10 5 2 STP 3 0 2 Frequency-domain Cortical Connectivity Time Frequency Representation Due to the large-banded nature of Frequency-Domain GrangerGeweke causality, we investigate the frequency domain in more detail using other methods (TFR and PLV). We are interested in the α band (5-15 Hz), the β band (15-30 Hz), and the γ band (3050 Hz). We study the frequency domain by observing the amplitude of activity at different frequencies through the time-frequency representation (TFR) (Fig 5) and the phase coherence between ROIs through phase locking (Fig 6-8). In order to assess the levels of activity at various frequencies, the complex Morlet wavelet filter is applied to the data. Each sensor measurement is passed through the complex Morlet wavelet transform from 5 Hz to 120 Hz at 1/3 Hz intervals. Then, the wavelet transformed coefficients are mapped back onto the cortex by applying the MNE inverse solution the surface normal signal at each vertex. The magnitudes of the complex coefficients are used to infer activity at each frequency and time point. The magnitudes between 0 and 2 seconds are normalized by the average magnitude in the baseline (-0.5 to 0 seconds), producing scores equivalent to those in the dSPM β · In the VS task the frequency directionally connected networks show a progressive increase in connectivity from T1 to T3, and across frequency bands from low (alpha) to high (gamma) · In the networks representing phase and time related connectivity during the VAS task, MT represents a source of outgoing and incoming connections in T1 and in T2-T4 the networks remain sparse. T2 T3 STP T4 T1 T2 T3 T1 T2 T1 T2 T3 T4 6 35 20 1.5 5 0 STS T4 T4 T3 Time (sec) 2 VIP T4 T1 T2 T3 aud T4 T1 T2 T3 T4 6 20 5 1.5 35 20 5 0 Figure 8 2 Time (sec) DIPSM T1 T2 T3 hMT T4 T1 T2 T3 STP T4 T1 T2 T3 STS T4 T1 T2 T3 VIP T4 T1 T2 T3 aud T4 T1 T2 T3 T4 6 35 20 5 1.5 50 35 20 5 γ T1 RH T3 -log(p ) · In VS task, the network seeded on STS, this area is significantly sustained and reciprocally connected with MT, STP, and DIPSM suggesting their continuous interaction during stimulus representation, decision and selection of response. The network extends to both hemispheres involving the same cortical areas (L-hem not shown). · In VAS there is an early but transient connection of the Aud to STP, which returns as reciprocal connection in T3. This pattern would suggest that once Aud has provided information to STP this area will be involved in building the multimodal representaton of the target. The Aud interaction returns at the level of consolidation the representation and decision making (T3&T4). · Aud is also strongly connected with MPFC, in T3, using working memory in the target selection in the 4AFC choice task. In both, the VS and VAS tasks, (results not shown) there is strong connectivity in T3 with DIPSM, STS and STP, all significantly involved in the tasks representation network. hMT T4 T2 35 50 α T3 STP, aud LH T1 50 50 2 time T2 Figure 6 -log(p ) 0 2 5 aud-rh →VIP-rh VIP-rh →aud-rh 120 7 frequency frequency 60 0 120 2 time aud-rh →STP-rh STP-rh →aud-rh 7 0 -10logP ITS 50 2 1 10 time MPFC-rh →aud-rh 5 50 Frequency Granger in Windows VAS aud-rh →MPFC-rh 2 STS DIPSM VAS 0 VAS 120 0 50 Frequency (Hz) 0 0 VAS 1 5 50 Frequency (Hz) 0 2 VS 0 0 hMT 50 7 20 10 2 5 Frequency (Hz) 1 2 VIP Seeded on DIPSM-lh 2 1 t-statistic of G X→Y t-statistic of G X→Y 0 0 MPFC 50 VAS 0 1 5 50 VAS-lh -log(p ) t-statistic of G X→Y 0 2 STP ITS Frequency (Hz) 0 20 10 0 60 2 1 0 50 VS 0 1 5 50 DIPSM Frequency (Hz) 0 20 2 STS 50 STS-rh →DIPSM-rh DIPSM-rh →STS-rh 120 7 frequency frequency frequency 60 0 hMT-lh →STS-rh STS-rh →hMT-lh 120 7 hMT 50 Seeded on MPFC-lh STP-rh →STS-rh STS-rh →STP-rh MPFC 0 50 120 frequency 50 5 Below we show sample frequency dynamic granger (top plots) and time-domain dynamic granger (bottom) for VS and VAS. t-statistic of G X→Y VS-lh 0 Time (sec) 2 · Fig 6 shows significant correlation in T1&T2 between STP and AUD in the VAS task; there is more significant and sustained correlation in the gamma-band in the RH, suggesting an involvement in the representation of the multimodal target object in STP. Phase Locking in the L hemisphere, is earlier in the beta band. In T3&T4 periods there is more significant gamma band correlation in the L hemisphere, where presumably the task is represented. · Fig 8 (MPFC) shows a higher coherence at all frequency bands between MPFC and MT in the VS task than in VAS. Aud and DIPSM regions are coherent with MPFC, first in the beta band and later in the gamma band. · Strong and sustained gamma and beta band coherence are seen between DIPSM and other cortical areas involved in both VS and VAS tasks, suggesting the importance of his area in the representational networks.