Publications

Whether short-term learning of new words can induce rapid changes in cortical areas involved in distributed neural representation of the lexicon is a hotly debated topic. To answer this question, we examined magnetoencephalographic phase-locked responses elicited in the cerebral cortex by passive presentation of eight novel pseudowords before and immediately after an operant conditioning task. This procedure forced participants to perform an active search for unique meaning of four word-forms that referred to movements of their own body parts. While familiarization with novel word-forms led to bilateral repetition suppression of cortical responses to all eight pseudowords, these reduced responses became more selectively tuned towards newly learned action words in the left hemisphere. Our results suggest that stimulus repetition and active learning of semantic association have separable effects on cortical activity. They also evidence rapid plastic changes in cortical representations of meaningful auditory word-forms after active learning.

The current study aimed to find out whether attention is required for visual feature binding. Participants performed a visual discrimination task with Gabor grates, which differed in two features: spatial frequency and angle. Detection of deviant stimuli against standards was possible exclusively by feature conjunctions rather than by separate features. We analyzed event-related potentials under four experimental conditions: selective attention to the target stimulus; selective ignoring of the non-target stimulus; nonselective distributed attention to all stimuli within visual modality; cross-modal shift of attention from the visual to the audial modality. Mismatch negativity was present only in conditions of attention to visual stimuli – both selective and distributed. The finding evidences that feature binding occurred only under conditions of attention towards visual stimuli.

Recent theories of cognitive control put large emphasis on theta oscillations in relation to action monitoring. Multiple EEG studies of cognitive control revealed increased power of theta oscillations restricted to midfrontal areas, while there is a substantial body of functional connectivity data demonstrating that theta oscillations may be a carrier of informational exchange over multiple cortical regions. fMRI studies revealed immense distributed networks involved in cognitive control. Paradoxically, MEG has been considered almost insensitive to theta oscillations in such an experimental context. It also remains debatable what is the functional role of such theta oscillations. An influential line of evidence links feedback-related theta oscillations to two types of prediction errors (unsigned and signed), but this distinction has not been tested during trial-end-error learning with theta activity measured beyond the midfrontal cortex.

We recorded MEG while participants were involved in trial-and-error learning within a novel multiple-choice behavioral task with complex stimulus-to-response mapping. Three conditions were analyzed: correct and erroneous trials during the initial stage of learning acquisition, as well as correct trials during stable performance. Sources of MEG activity were analyzed using minimum-norm estimation method within 4-6 Hz frequency range.

We revealed a number of bilateral cortical areas that displayed theta oscillations to the feedback signal: in addition to the "classical" medial frontal areas (the anterior part of the medial cingulate cortex and the pre-supplementary motor area), this network included the insula and the auditory cortex, the frontal operculum and posterior inferior frontal gyrus, the premotor cortex, the paracentral lobule, and the posterior part of the medial cingulate cortex. Granger causality analysis revealed overall communication directed from lateral to medial sites. During the initial stage of trial-and-error learning, we observed a strong non-differential response to feedback signal that reflected an unsigned component of the prediction error. The signed component of the prediction error was observed later – with greater theta activations after errors compared with correct responses.

Thus, using MEG, we were able to reveal a distributed network of brain areas in relation to feedback-related processing that included not only medial frontal, but also auditory areas, insula, lateral frontal, and medial parietal areas. The data obtained confirm the existence of two components of the prediction error, and this distinction was evident all over the network revealed.

The study was implemented in the framework of the Basic Research Program at the National Research University Higher School of Economics (HSE) in 2018.

Embodied cognition theory implies that speech is largely based on the body motor and sensory experience. The question, which is crucial for understanding the origin of language, is how our brain transforms sensory-motor experience and gets access to word semantic representation. We developed an auditory-motor experimental procedure that allowed investigating neural underpinning of word meaning acquisition by way of associative "trial-and-error" learning paradigm that mimics basic aspects of natural language learning. Participants were presented with eight pseudowords; four of them were assigned to specific body part movements during learning blocks – through commencing actions by one of participant’s left or right extremities and receiving a feedback. The other pseudowords did not require actions and were used as controls. Magnetoencephalogram was recorded during passive listening of the pseudowords before and after learning blocks. The cortical sources of the magnetic evoked responses were reconstructed using distributed source modeling. Learning of novel word meaning through word-action association selectively increased neural specificity for these words in the auditory parabelt areas responsible for spectrotemporal analysis, as well as in articulatory areas, both located in the left hemisphere. The extent of neural changes was linked to the degree of language learning, specifically implicating the physiological contribution of the left perisylvian cortex in the speech learning success.

Currently there are two opposing views on feature binding in the auditory modality: according to behavioral studies, this process requires focused attention, while electrophysiological studies suggest that feature binding may be fully automatic and independent from attention. Here, we examined whether feature binding depends on higher-level attentional processes, by manipulating the attentional focus. We used four auditory stimuli that differed in two features: pitch and location. Two rare deviants could be detected within a sequence of two frequent standards exclusively by feature conjunctions rather than by any single feature alone. Event-related potentials to auditory stimuli were analyzed for four conditions: selective attention to target auditory deviants, selective ignoring of non-target auditory deviants, nonselective distributed attention to all stimuli within auditory modality, and selective attention diverted from auditory to visual modality. The negative difference (Nd) between ERPs to deviants and standards was measured within two time intervals, corresponding to mismatch negativity (MMN, 100-200 ms) and N2b (200-300 ms). Only under the condition of selective attention to specific feature conjunctions, prominent Nd was observed in MMN as well in N2b time ranges, while no significant Nd was observed in other conditions. Since Nd is considered a marker of deviance processing, our results support the view that deviance was not detected unless attention was focused on the stimuli, thus supporting the view that feature binding requires attention.

Response commission in cognitive tasks comprises a complex set of mechanisms including decision making and action execution. Decision-making processes in multi-choice tasks involve sensory evidence integration and action selection, that are strongly related to the activity of the intraparietal cortical areas. Action execution is regulated within motor areas that may implement response initiation and inhibition. Failures within one or the other of the two systems may lead to qualitatively different kinds of errors that differ in the response time. Presumably, fast errors, with response latency shorter than average response latency on correct trials, are caused by inability to inhibit irrelevant prepotent responses, while slow errors, with response latency longer than average response latency on correct trials, result from disruptions forming the decision variable.

Errors can be detected and appropriate corrective adaptations can be initiated in top-down fashion by performance monitoring systems, with errors detected either internally (following error commission), or externally (following negative feedback presentation).

We recorded EEG while participants performed the auditory version of the condensation task with high cognitive demand. In order to detect modulations of oscillatory activity and functional connectivity patterns between areas, we used spectral power measures and weighted phase-locking index, which is a measure of stability of phase difference between signals recorded from two electrodes.

We observed alpha power suppression in the pre-response time window with the minimum at the left parietal electrodes, which presumably reflects generation of a decision variable after sensory evidence integration. Midfrontal theta power and phase coupling between midfrontal and left parietal regions increased after negative feedback presentation, associated with performance monitoring and top-down adjustments of the decision variable in the intraparietal cortical areas. Moreover, functional coupling was more pronounced for subjects who tended to commit slower errors, compared to correct responses.

This finding suggests that functional connectivity patterns differ depending on the type of error committed. That is, in the case of slow errors, which resulted from failures in decision-making processes, negative feedback elicited robust functional connectivity, that might be used in top-down influence on decision-making systems.

The study was implemented in the framework of the Basic Research Program at the National Research University Higher School of Economics (HSE) in 2018.

A response action consists of at least two stages: initiation and execution. Recording keystrokes and button presses is the method most commonly used in the field of cognitive psychophysiology; this method provides data on response accuracy and response time, which seem to be mostly related to the initiation stage. On the contrary, mouse tracking provides continuous data on response dynamics. Particularly, we assume that mouse movement duration is an important response parameter that is related to the execution stage of the response. Here, we applied this method to probe the functional significance of the response-related event-related potential (ERP) components such as correct-related negativity (CRN) and a Pe-like positivity.

We used the condensation task, which involves complex stimulus-to-response mapping: participants had to make responses to four auditory stimuli relying on the combination of two independent stimulus features. During each trial, participants had to respond to auditory stimuli by moving a computer mouse either to the top-left or to the top-right mousepad corner. EEG was recorded during the experiment. The following parameters of mouse movement were assessed: movement initialization time and movement duration. Within each subject, we divided the trials with correct responses into four quartiles for each of the mouse movement parameters separately. We compared ERP waveforms for trials within each pair of marginal quartiles.

Both movement initialization time and movement duration were higher for errors compared to correct responses. These mouse movement parameters were uncorrelated. We found that CRN amplitude within 10-110 ms time window was higher for early correct responses compared to late ones (p=0.004). In addition, we found a significant effect of mouse movement duration on ERP in early Pe time window (120-265 ms): amplitude of the Pe-like positivity was significantly higher for long correct responses compared with short correct responses (p<0.001).

We suggest that the early Pe-like component is not specifically related to errors; rather, both CRN and Pe-like component seem to be related to response uncertainty. Particularly, uncertainty during response execution stage seems to result in increased Pe-like component and prolonged mouse movement. We also assume that early correct responses are mostly premature responses, and increased CRN may indicate stronger performance monitoring arising after response initiation.

The study was implemented in the framework of the Basic Research Program at the National Research University Higher School of Economics (HSE) in 2018.

Despite the impressive progress achieved recently in the study of brain mechanisms of speech and language, the current understanding of the processes that implement assigning meaning to novel words is limited. We developed an experimental procedure that allowed investigating acquisition of word meaning by way of rapid associative trial-and-error learning. Eight pseudowords were presented to the participants; four of them were assigned to left and right hand and foot movements, while the other pseudowords did not require actions and were used as controls. Participants were instructed to learn the relations between the pseudowords and actions through the trial-and-error motor learning procedure. Auditory feedback was delivered on each trial informing whether response was correct or erroneous. Magnetoencephalogram was recorded during passive listening of the pseudowords before and after learning. The cortical sources of the magnetic evoked responses were reconstructed using distributed source modeling (MNE software). Neural responses to newly learnt words compared to control pseudowords were significantly enhanced in temporal and frontal cortical regions surrounding the Sylvan fissure of the left hemisphere. This activation was inversely related to the number of trials needed for participants to reach the learning threshold. Thus, our findings revealed a neural signature of rapid associative learning of word meaning and highlighted the role of sensory-motor transformation for association-grounded word semantics.

According to embodied cognition theory, speech is largely based on the body motor and sensory experience. The question that is crucial for our understanding of the origin of language is how our brain transforms sensory-motor experience into word meaning. We have developed an auditory-motor experimental procedure that allowed investigating neural underpinning of word meaning acquisition by way of associative "trial-and-error" learning that mimics important aspects of natural word learning. Participants were presented with eight pseudowords; four of them were assigned to specific body part movements during the course of learning – through commencing actions by one of a participant’s left or right extremities and receiving a feedback. The other pseudowords did not require actions and thus were used as controls. A magnetoencephalogram was recorded during passive listening to the pseudowords before and after the learning. The cortical sources of the magnetic evoked responses were reconstructed using distributed source modeling. The learning of novel word meanings through word-action associations selectively increased neural specificity for these words in the auditory parabelt areas responsible for spectrotemporal analysis, as well as in articulatory areas, both located in the left hemisphere. The extent of neural changes was linked to the degree of language learning, specifically implicating the physiological contribution of the left perisylvian cortex in the speech learning success.

Mechanisms of cognitive control include monitoring and regulation of both task-specific attentional processes and non-specific motor threshold. Failures in one or the other of these two mechanisms may lead to different kinds of responses, post-response adaptations and, importantly, distinctive behavioral correlates. Slow responses can be interpreted as responses committed after attentional lapses and, therefore, during the state of uncertainty, while fast responses can be interpreted as responses committed in conditions of lowered motor threshold. Thus, slow and fast errors have different nature and require different brain adaptations.

The aim of the current study was to confirm the idea that modulations in oscillatory brain activity can distinguish between these two types of responses.

Methods

EEG was recorded during performance of the auditory two-choice condensation task, which requires sustained attention and does not require inhibition of prepotent responses.

Results

Increased frontal midline theta (FMT) power was observed during pre-response time interval for both correct responses and errors. Enhanced error-related FMT power was found in post-response and post-feedback time intervals. Increased frontal beta power was observed in post-feedback time interval. We also observed significant positive trial-to-trial correlation between pre-response FMT power and response time (RT) for both correct responses and errors, negative trial-to-trial correlation between post-response FMT power and RT for errors, and positive trial-to-trial correlation between post-feedback frontal beta power and RT.

Discussion

Thus, slow erroneous responses characterized by high uncertainty were accompanied by increased FMT power before the response and by increased frontal beta power following the feedback; these effects, presumably, reflect enhanced cognitive effort and feedback processing, respectively. On the contrary, fast erroneous responses characterized by low uncertainty led to increased post-response FMT power, which, presumably, reflects internal error detection. Thus, this study confirmed the idea that RT can be a valid index of uncertainty level, with high uncertainty occurring due to attentional lapses and low uncertainty occurring due to failures to keep a sufficiently high motor threshold.

The study was implemented in the framework of the Basic Research Program at the National Research University Higher School of Economics (HSE) in 2018.

Speech is largely based on the body motor and sensory experience. The question, which is crucial for understanding the brain mechanisms of human language, is how our brain transforms sensory-motor experience into word meaning. Multiple evidence hints that natural language acquisition involves biological mechanisms of associative learning. The ability to quickly acquire word-picture associations was shown to depend on reorganization in neocortical networks including the left temporal area, especially the left temporal pole, as well as temporoparietal, premotor, and prefrontal regions.

We developed an auditory-motor experimental procedure that allowed investigating neural underpinning of word meaning acquisition by way of associative "trial-and-error" learning that mimics important aspects of natural word learning. Participants were presented with eight pseudowords; four of them were assigned to specific body part movements during the course of learning – through commencing actions by one of participant’s left or right extremities and receiving a feedback. The other four pseudowords did not require actions and were used as controls. Magnetoencephalogram was recorded during passive listening of the pseudowords before and after learning. The cortical sources of the magnetic evoked responses were reconstructed using distributed source modeling.

We found a significant effect in the middle part of the STS/STG that mostly includes the auditory parabelt areas responsible for spectrotemporal analysis and initial steps of word recognition. Processing of new words also activated the posterior opercular part of the inferior frontal gyrus that is involved in subvocal rehearsal and articulatory coding of the perceived speech sounds, this fact emphasizing the role of articulatory sensory-motor experience in acquisition of word meaning. Our analysis did not reveal significant effects in the temporal pole or in the temporoparietal regions.

Juxtaposition of our findings with the current body of literature may imply that rooting the word meaning into one's sensory-motor experience is an initial stage, which is prerequisite but not sufficient for its embedding into the full associative structure of semantic memory.

Taken together, our findings show that learning of novel word meaning through word-action association selectively increased neural specificity for these words in the auditory areas responsible for spectrotemporal analysis, as well as in articulatory areas, both located in the left hemisphere. The extent of neural changes was linked to the degree of language learning, specifically implicating the physiological contribution of the left perisylvian cortex in the learning success.

According to the embodied cognition theory, speech is largely based on the body motor and sensory experience. The question, which is crucial for our understanding of the origin of language, is how our brain transforms sensory-motor experience into word meaning. We have developed an auditory-motor experimental procedure that allowed investigating neural underpinning of word meaning acquisition by way of associative "trial-and-error" learning that mimics important aspects of natural word learning. Participants were presented with eight pseudowords; four of them were assigned to specific body part movements during the course of learning – through commencing actions by one of participant’s left or right extremities and receiving a feedback. The other pseudowords did not require actions, and were used as controls. Magnetoencephalogram was recorded during passive listening of the pseudowords before and after learning. The cortical sources of the magnetic evoked responses were reconstructed using distributed source modeling (MNE software). Neural responses to newly learnt words were significantly enhanced as compared to control pseudowords in a number of temporal and frontal cortical regions surrounding the Sylvan fissure of the left hemisphere. Learning-related cortical activation was inversely related to the number of trials needed to acquire the word meaning (this value varied between participants from 74 to 480 trials to the learning criterion). Our findings revealed a neural signature of associative learning of meaning of nonsense words and highlighted the role of sensory-motor transformation for association-grounded word semantics.

Feature binding is an essential aspect of sensory perception, since most realistic objects can be identified only by grasping conjunctions of multiple features and their patterns. Psychophysiological mechanisms of this phenomenon are still under debate; importantly, mutually exclusive points of view exist concerning the role of attention in feature binding. The current study aimed at testing the hypothesis that mismatch negativity (MMN) to specific feature conjunctions may depend upon attention. Two experiments were conducted in the auditory and visual modalities respectively. Within each experiment, we used four stimuli that differed in two distinctive features, with two feature conjunctions designated as standards, and two feature conjunctions designated as deviants. Features used in the auditory modality were tone pitch and location; Gabor grating orientation and spatial frequency were used in the visual modality. Attentional modulation involved four conditions: selective attention to targets, selective ignoring of nontargets, nonselective attention within a given modality, and deviation of attention to a task in a different modality. The basic finding was that MMN was evident only in conditions of within-modality attention. MMN was reduced or abolished in response to ignored feature conjunctions, as well as in conditions of the cross-modal distraction of attention. Thus, contrary to previous studies of MMN under feature conjunctions, our data show that the preattentive stage of feature conjunction processing requires a proper top-down attentional influence.

 

Supported by Russian Foundation for Humanities, project No 15-06-10742. 

Cognitive control includes maintenance of task-specific processes related to attention, and non-specific regulation of motor threshold. Generally, two different kinds of errors may occur, with some errors related to attentional lapses and decision uncertainty, and some errors – to failures of sustaining motor threshold. Error commission leads to adaptive adjustments in brain networks that subserve goal-directed behavior, resulting in either enhanced stimulus processing or increased motor threshold depending on the nature of errors committed. We report here two studies using the auditory version of the two-choice condensation task, which is highly demanding for sustained attention while involves no inhibition of prepotent responses. We analyzed power and topography of EEG oscillations in theta, alpha, and beta frequency bands.

Experiment 1. We studied post-error adaptive adjustments resulting in optimized brain processing and behaviour on subsequent trials. Errors were followed by increased frontal midline theta (FMT) activity, as well as by enhanced alpha band suppression in the parietal and the left central regions; parietal alpha suppression correlated with the task performance, left central alpha suppression correlated with the post-error slowing, and FMT increase correlated with both behavioral measures. On post-error correct trials, left-central alpha band suppression started earlier before the response, and the response was followed by weaker FMT activity, as well as by enhanced alpha band suppression distributed over the entire scalp. These findings show the existence of three separate neuronal networks involved in post-error adjustments: the midfrontal performance monitoring network, the parietal attentional network, and the sensorimotor network.

Experiment 2. We studied if response time may be a valid approximation distinguishing trials with high and low levels of sustained attention and decision uncertainty. We found that error-related FMT activity was present only on fast erroneous trials. The feedback-related FMT activity was equally strong on slow erroneous and fast erroneous trials. Late post-response posterior alpha suppression was stronger on erroneous slow trials. Feedbackrelated frontal beta oscillations were present only on slow correct trials. The data obtained cumulatively suggests that response time allows distinguishing the two types of trials, with fast trials related to higher levels of attention and low uncertainty, and slow trials related to lower levels of attention and higher uncertainty.

Flexible goal-directed behavior in cognitive tasks relies on multiple task-specific processes, as well as on functioning of the monitoring system. In multiple-choice tasks, the task-specific processes include sensory evidence integration and action selection that partially occur in the lateral intraparietal area (LIP). The performance monitoring system is located in the medial frontal regions of the cortex. Activation of this system is associated with increased frontal midline theta (FMT) power, and increased theta coherence between midfrontal areas and the task-specific areas. One of the situations that require the increase of cognitive control is receiving a negative feedback after an erroneous response. There are two possible types of errors. One of them originates from failures in task-specific processes and is associated with increased response times with high outcome uncertainty; the other is related to failures of non-specific motor inhibition and is characterized by decreased response times with low levels of outcome uncertainty. In the present study, we aimed to investigate whether post-feedback activity of the performance monitoring system depends on the type of committed errors. We recorded EEG while subjects performed an auditory version of the two-choice condensation task, in which both types of errors described above could occur. In the time window between the stimulus and the response, we observed significant decrease of alpha power in the left central-parietal sites (compared to the baseline), which presumably reflects the task-specific activation of the LIP area. Higher frontal midline theta (FMT) power and theta-band coherence with left parietal electrodes were observed after negative feedback, compared to positive one, reflecting error detection by medial frontal structures and their interaction with the LIP aimed to prevent future errors, respectively. Furthermore, the difference in theta coherence and the mean response time ratio between erroneous and correct responses were positively correlated, i.e. subjects that tended to commit slow errors demonstrated stronger increase of theta coherence after negative feedback. These findings support the idea that slow errors are associated with high outcome uncertainty, and the feedback information in this case is used to a greater degree in the processes aimed at performing post-error adaptations.

Feature binding is believed to be critical for perception, while brain mechanisms of this process are still under debate. Studies measuring mismatch negativity (MMN) demonstrate that binding occurs at a low level of the cortical hierarchy, while behavioral experiments suggest a much higher integrative level. The current study aimed at testing the hypothesis that processing of feature conjunctions is distributed and may involve two different levels. Experiment 1 combined recording both MMN and behavioral measures. Two types of deviant target stimuli were used – complex stimuli, which required feature conjunctions to be identified, and simple stimuli, which could be identified by a single feature. Responses to complex stimuli were slower and less accurate than responses to simple stimuli. For simple stimuli, errors were associated with increased response time, while there was no such effect for complex stimuli. Errors in response only to complex stimuli – but not to simple stimuli – were associated with decreased MMN amplitude. P300 amplitude was greater for complex stimuli than for simple stimuli. For simple stimuli, P300 amplitude was reduced on erroneous trials. Thus, we replicated within a single experiment the major effects reported in two opposing lines of binding research. Our MMN data hint that the neuronal population encoding feature conjunction is closely associated with (or coincides with) the neuronal population that generates MMN. Our P300 data and behavioral data are compatible with the explanation that higher processing levels receive sensory representations of conjoined features as well as of separate features. Our findings provide resolution to conflicting views concerning the nature of feature binding and support the notion that feature binding is a distributed multi-level process involving bottom-up and top-down interactions. Experiments 2 and 3 involved attentional manipulation – with target stimuli in the auditory and visual modality respectively. Feature conjunction involved tone pitch and location for auditory modality, and Gabor grating orientation and spatial frequency for visual modality. The basic finding from these two experiment was that that MMN was evident only in conditions of voluntary attention directed to a particular feature conjunction. MMN was reduced or abolished in response to ignored nontarget feature conjunctions, as well as in conditions of non-focused attention and cross-modal distraction of attention. Thus, contrary to previous studies of MMN under feature conjunctions, we found that early preattentive feature conjunctions happen on condition of a corresponding topdown attentional control.