What is a Sequence? The Neural Mechanisms of Perceptual, Motor, and Task Sequences Across Species and Their Interaction with Addiction
Summary and Keywords
Sequences permeate daily life. They can be defined as a discrete series of items or states that occur in a specific order with a beginning and end. The brain supports the perception and execution of sequences. Perceptual sequences involve tracking regularities in incoming stimuli, such as the series of sounds that make up a word in language. Executed sequences range from the series of muscle activations used by a frog to catch a fly to a chess master mapping her next moves. How the brain controls sequences must therefore scale to multiple levels of control. Investigating how the brain functions to accomplish this task spans from the study of individual cells in the brain to human cognition. Understanding the neural systems that underlie sequential control is necessary to approach the mechanistic underpinnings of complex conditions such as addiction, which may be rooted in difficult-to-extinguish sequential behaviors. Current research focuses on studies in both animal and human models and spans the levels of complexity of sequential control and the brain systems that support it.
When cooking a meal, it is intuitive that this process includes a series of steps. Most recipes include items that cannot be interchanged and must be performed in a particular order, such as chopping vegetables, searing vegetables, pouring in sauce ingredients. Many of these tasks involve executing practiced motor actions, such as chopping. However, some steps in cooking require monitoring a series of events before and after performing an action. For example, bringing a simmering liquid to a boil briefly and then returning it to a simmer requires sequential monitoring. Simply looking at the simmering pot would not be informative as to whether or not the boiling step had been performed. Together, these subtasks and goals make up the overarching goal of preparing a meal. This goal-subgoal structure is a common element of task sequences (Lashley, 1951).
The management of tasks on multiple levels requires cognitive control, and specifically, hierarchical cognitive control. Cognitive control is our general ability to flexibly select appropriate actions (Badre & Nee, 2018; Miller & Cohen, 2001). Hierarchical cognitive control is a more complex form of control necessary for selecting actions based on a higher-level or more abstract context (Badre, 2008; Badre & Nee, 2018; Botvinick, 2008). Sequential control is then further defined by hierarchical decisions that evolve through time, as is the case when deciding how to use the knife (subgoal) based on the specific step within the recipe (overarching goal). These control processes are subserved by common brain areas and key nodes within larger networks, such as the prefrontal cortex and striatum. Control networks, in turn, encompass a wide variety of areas spanning from those brain areas canonically known for perceptual processing to those brain areas performing more integrated and hierarchical processing of abstract content.
Because of the large and overlapping nature of these control networks engaged in sequential control, and the involvement of many brain areas spanning many levels of processing, investigating sequential control rapidly becomes intractable without a suitable, encompassing definition. What, then, is a sequence? Here a sequence is defined as a series of steps or states that are in a particular order and have a beginning and an end. State is used as a general term to signify the conjunction between the current context and stimuli in the system. This definition is not new. There are entire lines of inquiry, notably in computer science, that investigate how control systems define sequences de novo (e.g., Sun & Giles, 2001). Although how a system identifies a sequence as a sequence is a compelling question in its own right, it is not the focus of this review. Here we focus on bringing together multiple literatures across levels of inquiry and analysis that have self-defined as the study of “sequence.”
What are the different levels of sequence? One classically defined level is that of action or motor sequences. Motor sequences involve a precise order of activated muscles. Playing the piano is a canonical example (Figure 1).
Playing a song has a clear beginning and end, and a wrong move or muscle activation will mean that the sequence has been executed incorrectly. However, the purpose of defining a sequence more generally as a series of states is to encompass sequences that do not necessarily involve actions. A perceptual sequence, such as the visual cues of water changing from simmer to a boil or the series of stops on a subway train, could be defined as a series of stimuli that are in a specific order. Perceptual sequences do not necessarily require a specific action but could engage many of the same mechanisms responsible for monitoring the order of motor sequences. Task sequences, as in the cooking example, are the series of subgoals that are performed in a particular order in the service of an overarching goal. Together, perceptual and task sequences can be thought of as discrete endpoints of a continuum on which sequence can be defined.
Sequences intersect with everyday behaviors that range from benign (e.g., cooking) to harmful and even deadly as in the drug seeking and taking actions that occur in addiction. To examine this intersection and integrate across the levels of sequences, the following review presents work that is organized by sequence modality: perceptual, motor, perceptual and motor, and task. Within each modality, literature is examined across animals and humans; behavior and brain function; and health and addiction, ordered by approximately increasing sequence complexity. Sequence modalities are typically examined in isolation. These individual modalities may form a scaffold and interact to enable complex behavioral repertoires, such as naturalistic sequential actions. The literature reviewed here elucidates parallels across modalities and species, and suggests that common activity dynamics may represent sequences across different brain areas. Further, the same neural circuits necessary for sequential control, and processing across levels of sequence behavior more generally, have been shown to be dysfunctional within addiction. Though there is evidence to suggest that addiction is closely linked to habitual motor sequences, the aim of this review is to broaden the scope for how sequences are associated with addiction more generally. Sequences that exist simultaneously across multiple levels (perceptual, motor, and task) may create structures that are particularly embedded in the neural circuitry and, in disorders such as addiction, may be particularly difficult to extinguish.
Addiction is a disorder that affects more than 19.7 million people in the United States (SAMSHA, 2018). The current U.S. opioid abuse crisis illustrates our lack of knowledge in understanding the complex structure of drug seeking and taking behaviors (Burke et al., 2018). Integrating ideas about sequential processing across levels may reorient approaches to treating addiction, which is itself a notoriously heterogeneous disorder that is difficult to develop therapies for and has a high relapse rate. While this review highlights the overlap between sequential control and addiction, existing findings within the addiction field demonstrate an imbalanced and, in some instances, absent, examination of the sequential processing components. The goal of this review is to connect fields of research typically studied in isolation across behavioral and neural levels to scaffold future research. In sum, approaching sequences across levels will further our understanding of the brain mechanisms underlying sequential control in health and disorder.
Repeated Stimuli and Statistical Learning
Perhaps the most basic sequence is a perceptual sequence, which is a series of stimuli in the environment that occur in a particular order through time. Perceptual sequences themselves can vary in complexity. This subsection discusses the perceptual sequences that are built up through varying levels of experience with order of the transitions that occur between the stimuli themselves.
Simple perceptual sequence processing paradigms often use tasks in which a series of stimuli are repeated. The evaluation of whether these perceptual sequences have been learned consists of examining rare violations, or deviants, from the established series. Inferences about the capability of the underlying circuitry to encode the repeated patterns of stimuli are then made from the responses to the deviants. The first such studies were published in the late 1970s describing the mismatch negativity (MMN) component of event-related potentials (ERPs) in response to an infrequent deviant auditory stimulus (for review, see Näätänen, Astikainen, Ruusuvirta, & Huotilainen, 2010). Evidence exists for the presence of the MMN across a broad array of species and conditions including human adult and infancy sleep, sedated humans, and anesthetized animals, suggesting that the pattern detection machinery is ubiquitous and foundational. The MMN also exists in the visual domain (for review, see Pazo-Alvarez, Cadaveira, & Amenedo, 2003). This section introduces some of the neural mechanisms putatively responsible for coding these visual (ir)regularities and to suggest that these mechanisms may be the building blocks on which more complex sequential representations are constructed.
Neural signals detecting visual regularities have been described in very early visual areas. The responses of salamander and mouse retinal ganglion cells to omitted flash stimuli are robust to minor changes in light level and duration (Schwartz, Harris, Shrom, & Berry, 2007). Note that though the authors use the word “sequence” to describe the series of stimuli in this study, the stimuli are uniform and the observed neural responses reflect a very early prediction of the timing of subsequent stimuli. Therefore, the detection of regularities is not necessarily equal to the detection of sequence, and differs from the definition of sequence presented here as an ordered series of states.
Further studies have examined the mechanisms underlying neural responses to similar repeated stimuli throughout the visual stream. Neurons in the superior colliculus of monkeys showed an adaptation of responses to repeated stimuli along with much more rare novelty signals to oddball stimuli (Boehnke et al., 2011). Functional magnetic resonance imaging (fMRI) in humans has shown that similar adaptation occurs in early visual cortex (V1, V2, and V3, Figure 2), with novelty being signaled in V4, unlike earlier visual areas (Gardner et al., 2005).
Therefore, though early visual areas may be capable of detecting differences in repeated stimuli, it is only when proceeding into later stages of processing that changes in the pattern of the stimuli, rather than just the properties of the stimuli themselves, begin to be detetcted consistently. The idea of hierarchical representation of stimulus (ir)regularities has garnered some support from a human magnetoencephalography (MEG) study. In a test of 21 causal models of responses to sequence deviants, they found an expectancy signal was delivered to the prefrontal cortex (PFC, Figure 2; Phillips, Blenkmann, Hughes, Bekinschtein, & Rowe, 2015). Further investigation will be necessary to examine the network properties of these sequence deviant responses.
Abnormal responses to stimuli within the environment suggest that the basic underlying mechanisms of perception and processing of (ir)regularities of sequence structure are altered by drugs of abuse. Studies in human addiction show that disruptions in pattern detection can result from drugs of abuse in both auditory and visual MMN components. While these studies are reviewed extensively elsewhere (see Kremláček et al., 2016; Näätänen et al., 2012), findings across both the visual and auditory domain demonstrate a reduction in the MMN to unpredicted stimuli as the result of alcohol use and dependence, particularly the frontal subcomponent. In contrast, nicotine and methamphetamine addiction have been shown to enhance MMN in the visual domain, which was related to severity of addiction for methamphetamine use. Changes in the neural responses during perceptual processing may therefore underlie maladaptive addictive behaviors that reflect heterogeneity-based substance type; alcohol is generally a depressant, whereas nicotine and methamphetamine are stimulants. These results suggest a potential mechanism by which relatively simple perceptual cues and context could influence complex drug seeking and taking behavior within the domain of perceptual sequence processing.
More complex sequences that do not just involve repeated items can also be constructed through experience (Figure 1A). Learning about these sequences can be classified as statistical in nature, in that the sequence is built up by consistent relationships among the elements over time. Perceptual sequences in the context of such statistical learning have been studied perhaps most commonly in the auditory domain, where language is an intuitive example of learning to associate sequences of sounds and group them together (Saffran, Aslin, & Newport, 1996). While an extensive review of the statistical learning literature is outside the scope of this article (for review, see Aslin, 2017), the article attempts to illustrate that statistical learning takes place in the visual domain and that the neural underpinnings of constructing these regularities may share common neural substrates with those associated with other levels of sequential performance and control.
Temporal correlations are theoretically as important in the visual domain as in the auditory domain. For example, when viewing an object that is rotating, the image on the retina will be correlated from time point to time point, and that fact is arguably what helps construct the object as an object. This observation can be extrapolated to situations where the image on the retina changes from one object or scene to another, such as when making saccades (Fiser & Aslin, 2002). In a series of studies, Fiser and Aslin (2001, 2002) adapted a paradigm used in the auditory domain to study the effects of temporal correlations among visual stimuli and how humans perceive them. They found that participants not only learned the transitions from one shape to the next, but that higher-order “conditional probabilities” (the probability of one object appearing given another) were also learned. Subsequent research showed that stimulus features are learned in a manner where the features (such as shape and color) are bound such that learning sequential regularities of consistent shape-color objects does not transfer when shape or color are tested alone (Turk-Browne, Isola, Scholl, & Treat, 2008). Further, repeating stimuli and features preferentially bias attention in an automatic manner (Zhao, Al-Aidroos, & Turk-Browne, 2013). These results suggest that people automatically extract correlated temporal sequences of visual information.
Existing behavioral evidence suggests that monkeys are also capable of learning perceptual sequences in a statistical manner. Statistical sequence learning in monkeys has been found to be similar to sequence learning in humans (for review, see Conway & Christiansen, 2001) and similar across auditory and visual sequences (Milne, Petkov, & Wilson, 2017). Further, monkeys may be capable of more complex statistical learning than originally thought possible, in that they were capable of generalizing a learned sequence “grammar” to new sequences (Heimbauer, Conway, Christiansen, Beran, & Owren, 2018). Thus, it is possible that perceptual sequence learning has been conserved across species.
Perceptual statistical sequence learning activates many brain regions observed in other kinds of sequences, as discussed further in subsequent sections. When comparing structured versus random sequential stimuli, greater activation was observed in a wide network of areas (Turk-Browne, Scholl, Chun, & Johnson, 2009). These areas included the medial and middle frontal gyrus, precentral gyrus, and insula in the frontal cortex; superior and middle temporal gyrus; hippocampus; caudate nucleus; and parahippocampal gyrus (Figure 2). Further research proposed that the representation of these related stimuli may arise from clusters of mutually predicting stimuli, and not just transient events at the borders of sequences (Schapiro, Rogers, Cordova, Turk-Browne, & Botvinick, 2013). Specifically, such representations were found in the inferior frontal gyrus, insula, anterior temporal lobe, and superior temporal gyrus. Many of these areas overlap with those previously observed in the broader study of areas responding to statistical learning and therefore suggest a common network.
These fMRI investigations have been extended to monkeys by a small number of studies that examine the neural bases of statistical learning in both humans and monkeys. One study found that ERPs to deviant auditory stimuli were similar between monkeys and human infants, though these results were not specific to particular brain regions (Milne et al., 2016). Another study examined fMRI in both human and monkey participants while they were exposed to auditory sequences constructed according to a set of transition “rules” and violations, or deviants from those rules (Wilson et al., 2015). The authors found strikingly similar activation produced by deviants in humans and monkeys in the frontal cortex (including rostral frontal cortex) and middle temporal gyrus (Figure 2). Caudate nucleus activation was also present in the response to deviants in monkeys, but was not discussed or explicitly tested in humans. Therefore, although there are similarities across species, there may also be differences that underscore the necessity of examining sequences across species.
The neural mechanisms underlying statistical learning have also been investigated in electrophysiological studies in animals. Studies of primary visual cortex in mice found evidence of the predictive coding of the timing and identity of sequences of oriented gratings such that responses in the cortex were present even when sequence items were omitted (Gavornik & Bear, 2014). Preliminary evidence exists that the predictive code in primary visual cortex may consist of a transient coding of novel stimuli and a more sustained encoding of familiar stimuli (Homann, Koay, Glidden, Tank, & Berry, 2017). Notably, the maintained response was seen in different cells for different sequences, indicating that a code may exist for the identity of learned stimulus sequences as early as in the primary visual cortex. A series of studies investigated the responses of neurons in the inferotemporal cortex (ITC) when monkeys were repeatedly exposed to pairs or triplets of images (Meyer & Olson, 2011; Meyer, Ramachandran, & Olson, 2014; Ramachandran, Meyer, & Olson, 2017). They found evidence for enhanced responses in the ITC to items substituted from other sequences to be due to the suppression of expected items, rather than the surprise of unexpected items. Thus, there is evidence that sensory cortex represents statistical sequences, possibly through mechanisms that both maintain the current context and suppress responses to expected items.
Classic statistical learning paradigms have not been explicitly examined in addiction or in studies of drugs of abuse. However, learning about statistical transitions within the environment are essential aspects of determining drug availability and consumption. In addiction, individuals may prioritize cues and actions regularly associated with the drug seeking and taking ritual at the expense of other behaviors that could help the individual overcome continuous drug use despite negative consequences. Alcohol dependent individuals performing a decision-making task demonstrated disruption in the ability to use inference to guide alternative selection of choices, suggesting deficits in statistical learning of the structure of the decision environment (Reiter et al., 2016). FMRI data and computational modeling supported this conclusion by showing decreased efficacy of the medial prefrontal cortex (mPFC) and posterior cingulate cortex signal in coding these inferences (Figure 3A).
Similar brain areas are found to be active across species for encoding relationships among elements within a sequence. Difficulty in learning from consistencies or changes within the environment may render behavioral repertoires in addiction inflexible and thus hard to change. Understanding pattern detection and statistical learning as fundamental building blocks of sequential processing and how these components interact across both perceptual and motor levels will enable the development of new experimental paradigms for studying addiction.
Perceptual Sequence Pattern Detection
In contrast to repeated stimuli, studies of perceptual sequences involve a repeating pattern of stimuli where deviations cannot be detected by differences from previous stimuli alone. The progress through these perceptual sequences can be thought of as progress through states, as the stimuli are only cohesive as a group in the broader context of the sequence. Any individual transition would not be sufficient to define the state.
One of the first paradigms developed to study the detection of sequences was the local-global paradigm (Bekinschtein et al., 2009). In this paradigm people were exposed, multiple times, to a standard pattern of tones (e.g., XXXXY) and then rare deviants from that pattern (e.g., XXXXX). In this way responses to “local” deviants such as the X-Y in the standard pattern could be compared to “global” deviants such as the change from XXXXY to XXXXX that are only relevant because of the larger context. The authors found in electroencephalography (EEG) recordings that responses to the local deviants were earlier in time after the stimulus than the global deviants and more similar to those reported for the MMN. Responses to global deviants were present in frontal areas commonly associated with motor sequences (see section on “Motor Sequences”), such as the dorsolateral prefrontal cortex (DLPFC) and anterior cingulate cortex (ACC) along with the superior temporal gyri, parietal lobe, and occipital pole (Figure 2). These global responses were not present if the participant was not conscious or paying attention, suggesting that a higher level of processing was necessary than for the more automatic responses to the local deviants, which were present under all consciousness conditions. These results suggest that perceptual stimuli are processed as sequences, even without the need for action.
Subsequent work further investigated the role of attention in sequentially grouping perceptual stimuli. One study used MEG to investigate the perception of bi-stable sequences of sounds (Billig, Davis, & Carlyon, 2018). These repeated sequences could be perceived as integrated XYX-XYX or as XX, Y. They found that attention to the sequence as a whole biased the percept of the series of tones to the integrated sequence. Further, the ability to decode sequential information in the auditory cortex was also modulated by attention, suggesting that the neural representation itself changed according to the attended features of the sequence. Though these perceptions are modulated by attention, there has been some evidence to suggest that attention may not be necessary to detect all sequential features. Deviants from the standard XXXY tones, such as XXXXXY, evoked significant responses at frontal EEG electrodes during the fourth position (where the “Y” tone would have been in the standard) whether participants were passively or actively listening to the tones (Symonds et al., 2017). However, these responses were significant earlier in time and distributed differently across other cortical areas in the passive condition, suggesting that different networks and processes may be engaged with the frontal cortex during conditions with and without attention to perceptual sequences.
Given the ubiquity of the MMN across species, the question then arises if these perceptual sequence detections also generalize across species. An elegant study by Wang, Uhrig, Jarraya, and Dehaene (2015) examined fMRI activation to a passive sequence listening task, similar to the local-global paradigm, in both humans and monkeys. They found responses to global pattern deviants in the prefrontal cortex and basal ganglia across species (Figure 2). These results suggest that perceptual sequence detection is not limited to humans and that humans and monkeys may share similar underlying neural responses to perceptual sequences.
All of the studies thus far have used auditory stimuli. Though less frequently studied, evidence has shown that perceptual sequence detection is similar in the visual domain. One study compared ERPs in auditory and visual local-global paradigms and found their timing and distribution across the cortex to be similar across the domains (Blundon, Rumak, & Ward, 2017). Therefore, it is possible that perceptual sequence detection recruits similar neural resources, regardless of stimulus modality.
Studies of addiction measuring perceptual sequences have not been directly performed. However, limited behavioral evidence from humans viewing groups of neutral and alcohol-related stimuli suggests that the motivation to consume alcohol biased attentional focus (Hicks, Friedman, Gable, & Davis, 2012). In the local-global paradigm, the processing of local deviants may occur automatically while attending to and recognizing the global context may require more effortful executive control. Enhanced automatic behaviors with the use of drugs of abuse (see subsection on “Habitual Sequences”) suggests that perceptual processing may be biased to the relatively automatic or local aspects of patterned stimuli within the local-global task. Addicted individuals also exhibit dysfunction within the attention and executive control networks, potentially further biasing processing away from the global context (Goldstein & Volkow, 2011; Hicks et al., 2012). Testing these hypotheses directly by applying the local-global paradigm to studies of addiction will result in a better understanding of the behavioral and neural correlates of perceptual sequence pattern detection. Such experiments could also determine whether perceptual processing is misprioritized for actions that are typically associated with the drug use ritual, and suggest avenues of subsequent treatment.
Instructed Perceptual Sequences
In the previous section on “Perceptual Sequence Pattern Detection,” the relationships among stimuli were not explicitly instructed (though evidence suggests that they had been learned through experience). This section discusses sequences that depend on explicit association and memory, yet are still perceptual in nature. These tasks also do not require classic sequences of movements, and so can address questions on how sequences are remembered and encoded, without the confound of simultaneous action planning.
Human neuroimaging studies have focused on the features and location of remembered sequential information. In one series of studies, participants were asked to memorize sequences of visual objects; however, during testing while undergoing scanning, participants did not have to use the sequential information and instead made unrelated (e.g., size) judgments of the objects (Hsieh, Gruber, Jenkins, & Ranganath, 2014; Hsieh & Ranganath, 2015). The authors found sequence representations in areas traditionally associated with memory functions such as the hippocampus, medial PFC, retrosplenial cortex, and angular gyrus (Figure 2). They also found sequence representations in areas traditionally associated with motor sequences such as the caudate nucleus and putamen in the striatum. However, the nature of the sequential representation differed across these areas. The hippocampus coded information both about the object identity and the position in learned sequences. Cortical areas represented the sequence position, without coding for the object identity. Striatal areas carried information about objects either specific to the learned sequences, as in the putamen, or to objects themselves regardless of sequence, as in the caudate. These studies suggest that even when sequential information is not actively needed for the task, it is still represented in areas classically associated with motor sequences.
The previous study utilized memory that was presumably more long-term. However, people and animals perform sequences with ease even when they have not had long-term encoding practice with them. What, then, is the representation of sequences in short-term memory? An early study found that the mid-DLPFC and ACC participated in encoding the order of novel, abstract stimuli (Amiez & Petrides, 2007). A subsequent study examined both encoding and retrieval by asking participants to remember a sequence of up to three items over a brief delay and then determine if the order presented matched the order of the test stimuli (Kalm & Norris, 2017). The authors found a large network of brain areas that represented sequence position information either at sequence encoding or test, including large regions in the prefrontal cortex, temporal lobe, intraparietal sulcus, and occipital cortex. However, only two areas were specifically active during both phases of the task, the rostrolateral prefrontal cortex (RLPFC) and the anterior temporal lobe. A similarly broad network of areas was observed for target detection during a sequential target detection task (Farooqui, Mitchell, Thompson, & Duncan, 2012). These areas, specifically including the RLPFC, will be discussed further in the section on “Task Sequences” and provide an example of sequential information being potentially coded by common areas of the brain across levels of sequences.
The PFC has also been implicated in monkey studies of short-term memory. These studies required the monkey to remember the order of presented stimuli, and then either recognize whether or not the test order was correct or recall the order by reproducing a portion of or the entire sequence. Earlier studies used shorter sequences of two items and found that neurons in the PFC encoded the order of the objects, with the second object being represented more strongly in both neural spiking (Warden & Miller, 2010) and gamma-band local field potentials (LFPs; Lundqvist, Herman, Warden, Brincat, & Miller, 2018). Neurons in the PFC also encoded whether the animal was performing the recognition or recall task, indicating that neurons represented the higher-order context along with the sequence position. Further, a study of perceptual sequences found that in the PFC and other frontal cortical areas a position code was consistent across sequence lengths (Carpenter, Baud-Bovy, Georgopoulos, & Pellizzer, 2018). Together these studies illustrate that neurons in the PFC code sequence position during perceptual, nonmotor sequences.
Across monkeys and humans, it is therefore possible that similar brain areas, e.g., the PFC, are performing similar functions in coding for sequence position and broader context (Figure 2). However, this possibility has not been directly tested for instructed perceptual sequences. There is behavioral evidence that animals and humans may perform these tasks in the same or similar manner. Rodents and humans were trained to monitor a sequence of odors or abstract images, respectively, for one that was out of the pre-instructed order (Allen, Morris, Mattfeld, Stark, & Fortin, 2014). The authors found a similar pattern of performance, particularly on out of order “probe” trials, across humans and rodents. These results indicate that not only do humans and animals remember perceptual sequences, but that they may use similar cognitive processes to do so. Further, there is preliminary evidence that neurons recorded in the PFC of rodents respond to sequential items while performing this task (Quirk, Allen, & Fortin, 2014). Whether these similarities will extend to monkeys and humans is an assumption that remains to be tested directly.
Laboratory studies of drugs of abuse and addiction in both humans and animals focus on the neural responses to discrete drug-associated cues, rather than accounting for the sequential nature of the image presentations and underlying memory processing. However, brain regions that are activated to cue presentations in substance-dependent or heavy-use individuals overlap with those that process instructed perceptual sequences. For example, a meta-analysis of human addiction studies showing drug cues during fMRI demonstrated that areas such as the ventral striatum, ACC, and ventromedial PFC (vmPFC) show increases in activation in addicts or heavy users to drug cues, and areas significantly more active in drug users relative to controls included posterior cingulate, precuneus, and superior temporal gyrus (Figure 3A; Schacht, Anton, & Myrick, 2013). Further, drug-related stimuli activate common areas of visual cortex, including both primary and secondary, across studies (Hanlon, Dowdle, Naselaris, Canterberry, & Cortese, 2014). These results again highlight commonalities in the behavioral and neural substrates across the levels of perceptual sequences and addiction, suggesting that addiction-related cues may produce neural activations related to addictive sequential behaviors. It is likely that changes in processing of perceptual modalities interact, such that attentional bias to cues and statistical regularities in the environment become linked during the sequential actions of drug seeking and consumption. How the brain encodes such relationships is an outstanding question, warranting future studies that capitalize on the utility of studying addiction within the realm of perceptual sequences. A better understanding of disruptions in perceptual processing will provide a novel foundation for studying the heterogeneity of cues and responses present within addiction.
Motor sequences encompass the acquisition and production of skilled actions, themselves a series of muscle movements. Motor sequences can either be voluntarily planned or potentially evolve to be performed automatically, with limited cognitive resources devoted to monitoring processes. The behavioral and neural correlates of motor sequence execution and learning have been extensively studied in both animals and humans. However, studies have only recently begun to expand the study of motor sequences and habitual behavior to the study of addiction. This section will introduce common paradigms utilized across species to discuss the application and relevance to addiction, with the goal of drawing parallels between the neural underpinnings of motor sequences across studies of health and addiction.
Serial Reaction Time Tasks
Aside from muscle activation sequences that are not explicitly controlled, such as breathing and walking (for brief discussion, see Desrochers, Burk, Badre, & Sheinberg, 2016), one of the simplest forms of motor sequence involves responding to a stimulus as quickly as possible in a manner that follows a sequence. Over time, participants will learn the sequence of the stimuli so that they are faster to respond. Specifically, two common paradigms used to measure motor sequence learning are the serial reaction time task (SRTT) and the discrete sequence production (DSP) task. In the SRTT, participants execute a repeated series of button presses that correspond to the spatial location of stimuli as they are highlighted on a computer screen (Nissen & Bullemer, 1987; Figure 1C). Reaction time decreases steadily with learning of practiced sequences relative to elevated reaction time for newly introduced sequences, often without the explicit awareness of the participant (Abrahamse, Jiménez, Verwey, & Clegg, 2010). In contrast, the DSP task is typically used to measure explicit learning as participants are instructed to execute separate trials of sequences (Verwey, 2001). For example, tasks may incorporate one or both hands, use longer sequences to examine how they are executed in segments or “chunks,” and require 500 to 1,000 repetitions to measure changes in reaction time as these skilled motor sequences are learned (Abrahamse, Ruitenberg, de Kleine, & Verwey, 2013). Although the behavioral parameters of these task paradigms vary, the underlying neural correlates are proposed to follow a dual process theory in which both frontal cognitive control regions and basal ganglia areas interact during motor task learning and execution.
At the neural level, overlapping brain areas and networks underlie the learning and execution of motor sequences in the SRTT and DSP task. This neural data has been summarized extensively elsewhere (Abrahamse et al., 2010, 2013; Gheysen & Fias, 2012; Keele, Mayr, Ivry, Hazeltine, & Heuer, 2003), but key regions include the hippocampus, lateral prefrontal cortex, parietal and motor cortices such as the supplementary motor area (SMA), premotor cortex (PMC), and the striatum (Figure 2). The activity of these regions has been shown to interact during early stages of learning, with shifts in engagement from frontal cortex to subcortical areas, such as the striatum during training to test phases of learning dependent upon task requirements (training level, dual-task processing, novelty; Abrahamse et al., 2010). Further, this research also demonstrates a rostral to caudal gradient in the predominance of brain areas engaged as learning progresses, both within prefrontal cortical areas and within the striatum itself (Gheysen & Fias, 2012; Sakai et al., 1998).
Despite the simplistic nature and relative ease of using the SRTT task paradigm in studying motor sequences in humans, the translation to studies in animals has required species-dependent adaptations. Studies in rodents have used a combination of light cues, levers, and nose-poke apertures in varying spatial locations to enable serial responding that mimics the task in humans, with an emphasis on examining subcortical contributions to sequential learning in the striatum and hippocampus (Schwarting, 2009). Further studies in monkeys have expanded the study of the SRTT to examine frontal neural contributions and behavioral correlates during motor sequence learning and execution (Heimbauer et al., 2012; Hikosaka, Rand, Miyachi, & Miyashita, 1995; Minier, Fagot, & Rey, 2016; Ohshiba, 1997). In monkeys, practice of cued response sequences shapes the activity of M1 neurons during repeated presentations of the sequences (Matsuzaka, Picard, & Strick, 2006), and the pre-SMA relative to SMA functions to encourage learning of novel sequences (Nakamura, Sakai, & Hikosaka, 1998, 1999). Together, these studies show the importance of integrated neural processing during motor sequences that is dynamically adjusted over time throughout learning.
Examining the behavioral and neural findings across species demonstrates that similar cortical and subcortical structures are active during the learning and execution of motor sequences (Figure 2). Cortical areas including motor cortex and SMA as well as subcortical structures such as the hippocampus and striatum are key components of distributed networks across the brain that play a major role in simple motor sequence execution. These brain structures and networks show aberrant processing in nonsequential studies of addiction in humans, demonstrating that there is overlap in the brain areas that govern motor reaction sequences and addictive behaviors (Figure 3A). Motor sequence studies in addicted individuals are limited, and the results are sometimes conflicting. For example, alcohol-dependent individuals showed similar SRTT execution as controls, despite diminished executive functioning (Virag et al., 2015). In contrast, cocaine users exhibited impaired performance on finger tapping sequences and deficits in frontostriatal connectivity (Lench, DeVries, & Hanlon, 2017). Given these mixed findings, studies applying these paradigms to investigate the neural substrates of motor sequence learning in addiction represent an essential avenue for future research.
The SRTT involves making a series of responses to a stimulus. It has been hypothesized that making repeated stimulus-response (S-R) associations can drive actions that were initially reward- or goal-driven to become habitual. The scope of literature that has studied habits and habit formation in both health and addiction is vast (Balleine & O’Doherty, 2010; Belin, Belin-Rauscent, Murray, & Everitt, 2013; Dolan & Dayan, 2013; Graybiel, 2008). The aim of this section is to first briefly describe paradigms that have focused on potential components of entire habitual sequences, such as habitual S-R associations, because of their potential connections to processes underlying addiction. Further discussion will then focus on the body of work that studies multistep habitual sequences.
Though the habitual S-R component of habitual sequences was studied initially and most extensively in rodents, it has since been studied across species. Habitual S-R associations are defined by the execution of actions no longer being affected by the reward or goal, such that the reward can be “devalued” and the action will persist (Dickinson, 1985). Human studies have employed paradigms stemming from animal models to measure habits as opposed to goal-directed behavior (Balleine & O’Doherty, 2010; de Wit, Corlett, Aitken, Dickinson, & Fletcher, 2009; de Wit, Niry, Wariyar, Aitken, & Dickinson, 2007; O’Doherty, 2004; Tricomi, Balleine, O’Doherty, 2009; Valentin, Dickinson, & O’Doherty, 2007). The direct translation from rodent studies to humans provided a basis for developing additional paradigms to study these behaviors in both health and neuropsychiatric disorders (Banca, Harrison, & Voon, 2016; Daw, Niv, & Dayan, 2005; Dolan & Dayan, 2013; Gillan et al., 2011). However, recent studies have failed to replicate previous findings (de Wit, Kindt, Knot, & Verhoeen, 2018). Further, conflicting results have been reported on validating the constructs of goal-directed and habitual behavior across tasks (Friedel et al., 2014; Miller, Shenhav, & Ludvig, 2019; Sjoerds et al., 2016), suggesting that these behaviors may be more complex to parse across species than originally demonstrated.
The shift from goal-oriented to habitual behavior has been associated with changes in processing within the brain from prefrontal control and value associated regions to strengthened engagement of the striatum (Balleine & O’Doherty, 2010; Dolan & Dayan, 2013). Specific brain areas in humans that have been implicated in these behaviors include the OFC, vmPFC, PMC, caudate, and putamen (de Wit et al., 2009; de Wit et al., 2012; Tricomi, Balleine, O’Doherty, 2009; Valentin et al., 2007). A study in monkeys used lesions to value associated regions and a devaluation manipulation to demonstrate the necessity of the orbital frontal cortex (OFC) in overriding habitual responding and ventrolateral PFC (vlPFC) in creating habitual actions (Rudebeck, Saunders, Lundgren, & Murray, 2017). Findings across species converge on the importance of the prefrontal cortex in flexibly switching between goal-directed and habitual action selection strategies, but outstanding questions remain on the role of more rostral PFC in such behaviors and more complex action sequences that incorporate these behavioral components.
Human addiction studies have demonstrated deficits in goal-directed actions and the enhancement of habitual responding (Belin-Rauscent, Everitt, & Belin, 2012; Belin & Everitt, 2008; Ersche et al., 2016; McKim, Bauer, & Boettiger, 2016; McKim, Shnitko, Robinson, & Boettiger, 2016; Ostlund & Balleine, 2008; Vandaele, Pribut, & Janak, 2017). Further, the same brain regions and associated networks as in goal-directed and habitual behaviors have been implicated as dysfunctional within addiction. For example, alcohol-addicted individuals had decreased activation in the vmPFC and anterior striatum associated with goal-directed behavior, and increased activation in the posterior striatum associated with habitual behaviors (Sjoerds et al., 2013). The decrease in vmPFC activity was negatively correlated with duration and severity of alcohol dependence. Together, these studies of addiction are similar to animal studies demonstrating that drugs of abuse shift the balance between goal-directed and habit-based responding. However, further research is necessary to better understand changes in the neural circuits that underlie such complex habitual action sequences that are resistant to change in humans.
Expanding habitual paradigms to include a multistep decision structure within the task have shown the involvement of several prefrontal regions such as lateral PFC, vmPFC, and rostral PFC (Colas, Pauli, Larsen, Tyszka, & O’Doherty, 2017; Glascher, Daw, Dayan, & O’Doherty, 2010; Lee, Shimojo, & O’Doherty, 2014; Wunderlich, Dayan, & Dolan, 2012). While some of these areas (e.g., vmPFC) overlap with those involved in simpler habitual associations, others (e.g., rostral PFC) remain open for investigation as to their precise role in habitual multistep decisions and addiction. This complexity is illustrated by a study that required a sequence of actions in order for rodents to self-administer cocaine (Singer, Fadanelli, Kawa, & Robinson, 2017). Although the animals displayed behaviors that are defined as “addiction like” in humans, the animals did not develop habitual behaviors based on a devaluation test. That is, the sequence of behaviors did not persist after decreasing the value of the reward. Other studies support these findings (Olmstead, Lafond, Everitt, & Dickinson, 2001); however, there is evidence to suggest that longer training may render these sequential behaviors insensitive to devaluation (Zapata, Minney, & Shippenberg, 2010) and negative consequences (Chen et al., 2013; Pelloux, Everitt, & Dickinson, 2007; Vanderschuren & Everitt, 2004). Variability in results across species highlights the need to further refine existing task paradigms. These results suggest that while habitual action sequences may underlie the consumption of drugs, more complex sequences may constitute the behavioral sequence of drug use as a whole and may be a resulting combination of habitual and goal-directed behaviors.
More complex habitual motor sequences have been studied outside the context of the reward devaluation paradigm and have begun to elucidate the neural bases of these more habitual motor sequences. Studies in animals have shown that frontostriatal activity is necessary during the learning and execution of entire action sequences that may become habitual. The striatum signals the initiation and execution of an action sequence whereas the prefrontal cortex represents more abstract boundary signals across many neurons (Jin & Costa, 2015). Rodent work has consistently found that striatal neurons mark the beginning and end of sequences of lever presses (Jin & Costa, 2010; Jin, Tecuapetla, & Costa, 2014; Tecuapetla, Jin, Lima, & Costa, 2016) or maze running (Barnes, Kubota, Hu, Jin, & Graybiel, 2005; Jog, Kubota, Connolly, Hillegaart, & Graybiel, 1999; Smith & Graybiel, 2013) as the behaviors evolve through learning. In monkeys, electrophysiological recordings have demonstrated that frontostriatal circuits mark sequences boundaries. In studies of sequential saccades, frontal cortex signals the completion of learned eye movement sequences (Fujii & Graybiel, 2003) as well as striatal activity at movement boundaries (Desrochers, Amemori, & Graybiel, 2015; Fujii & Graybiel, 2003; 2005). Further, these striatal signals again develop through learning and underlie a cost-benefit signal that can then drive acquisition of future sequence movements (Desrochers, Amemori, & Graybiel, 2015). Together, these studies suggest the importance of connections between frontal cortex and striatum that demarcate learned action sequences, enabling the execution of such habitual sequences.
Computational models have been employed to demonstrate how routine sequential behavior may be represented within and across brain areas. Two influential models that have been applied are briefly discussed in this section. Cooper and Shallice (2000; 2006a, b) proposed that a hierarchically structured interactive activation network model best accounted for a complex routine sequential task such as making coffee. In contrast, Botvinick and Plaut (2004; 2006a, b) put forth an alternative framework using a recurrent neural network model that does not impose a hierarchical structure but can result in a hierarchical architecture through learning. Together these models raised key questions as to how sequential information is learned and parsed. Though these questions remain unresolved, neural evidence has provided support for both models (see section on “Task Sequences”).
These models provided a foundation for models employing reinforcement learning algorithms to explain routine hierarchical behavior (Botvinick, 2008; 2012). Reinforcement learning models have further expanded from the study of habitual choices (model-free) to goal-directed action selection (model-based; for review, see Dolan and Dayan, 2013). Model-free and model-based computational algorithms have been used to explain behavior in both healthy controls and addicted populations. In addicted populations, results have been mixed in modeling behaviors across samples and within drug use categories (Banca et al., 2016; Sebold et al., 2014; Voon et al., 2015). These results suggest that the model-free and model-based distinctions may not be sufficient to capture the complexity of sequential behaviors in human addiction.
Despite mixed evidence for separable behaviors, particularly in addicted individuals, different brain regions have been associated with behaviors that are categorized as model-free or model-based. Prefrontal brain regions underlying cognitive control have been associated with goal-directed behavior, and subcortical areas such as the striatum (caudate/putamen) show greater activation during habit-based responding (Colas et al., 2017; Glascher et al., 2010; Lee et al., 2014; Wunderlich et al., 2012). However, the direction of the deficits were not consistent in addicted individuals, as in some cases behavior was shifted in one domain (enhanced habit) while the other remained intact (normal goal-directed; Banca et al., 2016; Sebold et al., 2014; Voon et al., 2015). Thus, while computational models have been key in defining features of the acquisition, execution, and representation of routine sequential behaviors in brain and behavior, the inconsistencies in explaining the complex and perhaps overly routine sequential behaviors in human addiction highlight the need for further research.
Within these circuits, corticostriatal pathways and dopamine transmission are critical components to the execution of seeking and taking behavioral sequences for both natural reward and drug studies. Key brain regions necessary for drug seeking and taking for cocaine include the dorsolateral striatum (Zapata et al., 2010), the prelimbic PFC (mPFC) (Chen et al., 2013), and connections between the nucleus accumbens and dorsolateral striatum (Belin & Everitt, 2008). Studies using natural rewards instead of drugs extend these findings to also include the dorsomedial PFC in regulating the execution of action sequences (Ostlund, Winterbauer, & Balleine, 2009) and a role for phasic mesolimbic dopamine within the nucleus accumbens to motivate the execution of action sequences over time (Wassum, Ostlund, & Maidment, 2012). Increasing levels of dopamine have been demonstrated to underlie progress toward a goal in sequential habitual actions in rodents (Howe, Tierney, Sandberg, Phillips, & Graybiel, 2013). These findings demonstrate the overlap between the neural circuits underlying habits and the execution of habitual responses in more complex motor sequences.
Despite the relative ease with which motor sequences are executed in daily life, the literature discussed within this section demonstrates that such sequential processing is complex and, when combined or interacting with drugs of abuse, can have lasting effects on both behavior and neural circuits that regulate sequences of actions. Evidence across species suggests that similar brain regions are engaged during both initial learning and the evolution of motor and habitual sequences over time. The basic underlying circuitry includes connections between frontal cortex and striatum across species. These are the same circuits that have been shown to function abnormally in addiction in studies examining executive, motor, and value-based processing (Ersche et al., 2005; Goldstein & Volkow, 2011; Konova et al., 2012; Olausson et al., 2007; Park et al., 2010; Woicik et al., 2011). Studies from animal models of drug seeking and taking demonstrate that changes within these circuits causally relate to shifts in sequential actions (Belin & Everitt, 2008; Ostlund, 2010; Zapata et al., 2010). In contrast, results from human studies cannot currently differentiate whether shifts in behavior are a result of drug use experience or may be a predisposing factor that ultimately leads to drug seeking and taking responses. Further research is warranted to better understand the neural correlates of motor and habitual sequential processing in addiction. These studies will not only be essential to our understanding of the basic components that build complex sequential processing, but will provide novel paradigms for the development of treatment options for addiction.
Volitional Motor Sequences
As noted previously, motor sequences can be goal-directed, habitual, or a combination of the two. This section briefly focuses on motor sequences that are goal-directed and under direct volitional control and their neural underpinnings (for longer discussion, see Desrochers et al., 2016). Tasks used to study these motor sequences explicitly instruct participants to perform actions in a particular order. Early studies in humans implicated the frontal cortex in sequence performance. For example, the SMA showed increased activity during pre-learned saccade sequences using positron emission tomography (PET; Petit et al., 1996). FMRI studies in humans have also shown that the supplementary eye field (SEF) is active during multistep saccade sequences (Heide et al., 2001) and a stop-signal saccade task (Curtis, Cole, Rao, & D’Esposito, 2005), suggesting this area contributes to planning and monitoring of executed sequences (Figure 2).
Studies in monkeys have implicated some of the same frontal cortical brain regions as in humans. Activity in the DLPFC of monkeys has been shown to carry information about executed sequences of decisions during intervals between trials (Averbeck & Lee, 2007) and the primary motor cortex demonstrates anticipatory activity when executing learned motor actions from memory toward spatial locations (Lu & Ashe, 2005). Similar neural activity has been observed in the pre-SMA and SMA of monkeys performing remembered sequences made up of push, pull, and turn motor movements (Shima & Tanji, 2000). Inactivation of SMA disrupted execution of the sequence as whole, but not the individual motor components (Shima & Tanji, 1998). The learning of serial order of sequences has also been found to be coded in the ACC (Procyk, Tanaka, & Joseph, 2000) as well as in SEF neurons (Berdyyeva & Olson, 2010), which can selectively encode order that is sequence specific (Lu, Matsuzawa, & Hikosaka, 2002). Even in this limited sample of studies, there is a broad array of frontal cortical areas involved in volitional motor sequences across species (Figure 2). Precisely how these sequences interact with executive control, or whether executive control deficits in addiction extend to volitional motor sequences, is an open avenue for future investigation.
Perceptual and Motor Sequences
Motor sequences, despite their name, are rarely motor sequences in isolation in the laboratory. For example, in the SRTT, even though what is being produced is a series of movements, there is also a series of perceptual cues that indicate which movement to perform next (Figure 1C). Humans and animals implicitly and automatically learn perceptual sequences (see section on “Repeated Stimuli and Statistical Learning”; Fiser & Aslin, 2002). Therefore, it is possible that participants are not only learning a series of muscle activations or motor actions, but also learning a series of stimuli to expect. Not long after the SRTT was introduced (Nissen & Bullemer, 1987), studies began to investigate if and how perceptual and other kinds of learning may be incorporated into motor sequence tasks like the SRTT.
Some of the first research to investigate whether perceptual learning accompanied motor sequence learning found that neither happened in isolation, and that it was most likely that perceptual and motor sequence learning happened together (Willingham, Nissen, & Bullemer, 1989). A rich literature subsequently investigates the details of this assertion. Subsequent work found that perceptual and motor sequence learning had long-term effects (Fendrich, Healy, & Bourne, 1991), spatial sequences were learned (Mayr, 1996), and that attention sequences were also integrated and learned (Willingham, 1999). Together these studies indicate that, on the behavioral level, motor sequences are integrated with perceptual sequences.
Few studies have directly investigated the neural bases of the integration of motor and perceptual sequences. One study observed activation in the caudate nucleus of the striatum and hippocampus for both perceptual and motor sequences, and activation was evident earlier in learning for motor sequences in the hippocampus (Gheysen, Van Opstal, Roggeman, Van Waelvelde, & Fias, 2011). The authors mention greater activation for sequences than random in the occipital cortex as well, but do not detail whole brain analyses. Therefore, it is difficult to draw further conclusions from this work other than to say that at least one area classically associated with motor sequences, the striatum, also shows activation in the same experiment, to perceptual sequences.
Open questions remain as to whether and how the integration of perceptual and motor sequences occur in the context of addiction, as these hypotheses have not been directly tested. A paradigm frequently studied in animals that combines the complexity of perceptual and motor learning in a nonsequential manner is known as Pavlovian instrumental transfer (PIT). Sessions are used to associate a Pavlovian cue (light) and reward (food pellet), and separately, an instrumental lever press with reward presentation. At test, an increase in responding on the lever in the presence of the light stimulus (cue) demonstrates that behavior transferred between the two Pavlovian and instrumental sessions, although they had never been trained or paired together prior to the testing session (for reviews, see Cartoni, Moretta, Puglisi-Allegra, Cabib, & Baldassarre, 2015; Holmes, Marchand, & Coutureau, 2010). In animals, the infralimbic mPFC and its connections to the nucleus accumbens shell of the striatum and the basolateral amygdala mediate PIT (Corbit, 2005; Keistler, Barker, & Taylor, 2015). Further, signaling in the nucleus accumbens shell has been implicated in enhancing the behavioral responding in PIT after cocaine exposure in rats (Saddoris, Stamatakis, & Carelli, 2011). Together these results suggest common neural substrates for integrated learning of perceptual and motor sequences within the brain, and these areas within frontostriatal circuitry are associated with addictive behaviors across animal and human studies (Balleine & O’Doherty, 2010; Goldstein & Volkow, 2011; Hu, Salmeron, Gu, Stein, & Yang, 2015; Kim, Lee, Yun, & Kim, 2017).
It is commonly assumed that S-R and hierarchical contextual associations are components of addiction that become linked with the reinforcement of behavior. Evidence from studies of extinction training suggest that modifying these associations by pairing drug seeking and taking behaviors with the absence of the actual outcome of consuming the drug may have a more lasting impact on reducing addictive behaviors. In healthy humans working for points, PIT responding could be extinguished if both the cue and the opportunity to respond were available in the absence of reward (Gámez & Rosas, 2005). The PIT extinction paradigm has further been used in nontreatment seeking smokers and current alcohol drinkers to demonstrate that they also show decreased responding after experiencing the cue in a context where the absence of reward is associated with responding (Hogarth et al., 2014), and that drug cues provide context for expectations of trained alcohol response associations (Hardy, Mitchell, Seabrooke, & Hogarth, 2017). While these extinction procedures reduce cue-evoked craving in the laboratory setting, they may have limited long-term efficacy when applied to real-world contexts of drug-use behavior (Conklin & Tiffany, 2002; Xue et al., 2012). Repetition strengthens the relationship between associations within the drug seeking and taking ritual, ultimately resulting in a sequence of chained behaviors.
Studies of chained behaviors in rodents typically include a combination of seeking and taking responses that provide access to a reward, as well as cues that act as context for each behavioral response (Corbit & Balleine, 2003; Dickinson & Balleine, 1995). Animals trained on a seeking/taking action chain followed by extinction of one response within the chain also showed weakening of other behaviors, but cue presentation alone did not weaken the seeking/taking chain (Thrailkill & Bouton, 2015, 2016). Further, extinction of chained behaviors was facilitated by the presence of cues and the simultaneous availability of responding, similar to findings in the human PIT studies. Findings from this paradigm also demonstrate that perceptual and motor learning are integrated, further suggesting that their combined influence on behavior may result in persistent drug-related behaviors
Understanding the making, and breaking, of entire sequences of associations and how they interact with the presence of cues and context will be necessary to make significant advances in our understanding of natural sequential behaviors in health and addiction. Studies in animals and humans suggest that single events or cues can be triggers for entire sequences of addictive behaviors. Nicotine administration in rats acts as an internal cue for the behavioral sequence driving drug-seeking (Troisi, 2013). After one month of abstinence, rats that had been trained to perform a chained behavioral sequence reinstated heroin seeking behavior after an injection in the training context (Lu et al., 2010). Similar results have been observed with nondrug rewards. Chained seeking-taking behaviors to receive reward were classified as habitual because they persisted after the reward had been devalued; that is, the reward was no longer desirable or consumed (see section on “Habitual Sequences”; Thrailkill & Bouton, 2017). This situation could be analogous to persisting in drug seeking and taking despite increased tolerance to the drug and, consequently, decreased value. In the real-world setting of drug-taking behavior, motor and perceptual sequences are integrated; for example, cues and context are constantly present while preparing a drug for consumption (paraphernalia such as a pipe and lighter) and are intertwined with the motor actions (packing the pipe, lighting, and smoking). These findings suggest that to address the complex actions and associations in drug seeking and taking, they cannot be considered as unitary events but as entire sequences of behaviors.
Sequences were defined as a series of states, where states are the conjunction of context and stimuli that do not necessarily have to be concrete. However, the discussion to this point has been about sequences that are either in perceptual stimulus, motor action, or both. There are sequences that cannot be defined by motor actions or perceptual stimuli alone. These sequences are defined as task sequences where a series of abstract operations are performed in a sequence (Figure 1D). A simple example is completing a mental math problem such as 3 + 2 x 5. The mathematical operations have to be completed in the correct order to obtain the correct answer. This example illustrates some common features of task sequences. First, this sequence, multiplying and then adding, does not have a series of motor actions associated with it, nor does it have a series of perceptual stimuli. The numbers in the problem could change and the order of operations would still be the same. Second, the order that the mathematical symbols are listed in does not necessarily correspond to the order of the operations. This lack of correspondence illustrates another feature of task sequences: that often the components are performed in the absence of external cues. Last, this task involves selecting and flexibly switching between different subtasks (mathematical operations) at each serial position in the sequence. Rarely in life is it possible to maintain a single task multiple times to complete a sequence (multiplying all the numbers would not be correct).
More complex task sequences are pervasive in daily life. They can range from cooking, as discussed in the introduction (Figure 1E), to landing a fighter plane on an aircraft carrier. In cooking, though exact motor sequences are required (e.g., chopping, slicing, scooping) the goal requires a whole sequence of tasks for completion (e.g., chop the carrot, slice the onion). These sequences, by their nature, cannot be represented by a series of motor actions, but by a more abstract notion of the component tasks necessary to accomplish the goal (e.g., the same knife will be used for chopping at one point and slicing at another). These task sequences are distinct from motor sequences, as they have unique demands. For example, a common complaint of patients with frontal lobe dysfunction is that they are unable to complete task sequences in the course of daily living, such as making their own breakfast, despite the ability to perform other tests of executive function normally (Eslinger & Damasio, 1985; Shallice & Burgess, 1991). These deficits then render these patients incapable of independent living.
Task sequences have been studied behaviorally to examine the goal, subgoal structure. One such study investigated task switching in sequences (Schneider & Logan, 2006). Participants were instructed to remember and repeat a sequence of simple categorization tasks, (e.g., color and shape) throughout a block of trials. The response on each trial depended both on the identity of the stimulus, which was unpredictable, and the position in the task sequence. Thus, a hierarchical structure was created with the task sequence governing the individual trial decisions. Importantly, the motor responses and the stimuli could not be predicted or prepared for on a trial-by-trial basis, so there was no motor or perceptual sequence learned or performed as part of these task sequences. They found increased reaction times (RTs) at the first position in the sequence, over and above the increased RT found at task switches (e.g., color to shape). These results provided evidence for task sequences as a construct and the hierarchical control of task sequences. Without the task sequences, the RT at the first position would not be elevated beyond that of task switching alone.
Briefly, hierarchical control has been studied extensively in the nonsequential context and provides a starting point for identifying the brain areas potentially involved in task sequence control. Numerous studies have illustrated a caudal-to-rostral gradient of control in the frontal cortex where progressively more rostral areas represent progressively more abstract levels of hierarchical control (Badre & D’Esposito, 2007; Badre & D’Esposito, 2009; Koechlin, Ody, & Kouneiher, 2003; for review, see Badre & Nee, 2018). These brain areas range from premotor cortex to DLPFC and RLPFC. Many of these same brain areas are implicated in sequential control and more research is needed to fully enumerate their role in executive function and sequential control.
Whether control over sequential processes that appear hierarchical in nature (i.e., with a goal and subgoal structure) are represented hierarchically in the brain has been a topic of extensive debate. Though these models were introduced in the context of routine/habitual sequential behavior (see section on “Habitual Sequences”), they are applicable to task sequences as defined in this context because they address how actions may be grouped into abstract routines. Models that explicitly impose hierarchical structure (Cooper & Shallice, 2000; 2006a, b), and those that do not and instead rely on recurrent connections (Botvinick & Plaut, 2004; 2006b), are both capable of explaining complex task sequences. Neuroimaging studies have provided varied evidence for these models. In support of the explicitly hierarchical account, activation with the frontal cortex and other brain areas may rely on anatomical connections associated with the hierarchical structure reviewed in the section on perceptual sequences (Hsieh et al., 2014; Hsieh & Ranganath, 2015). Also, boundaries between and associations among groups of sequential items may facilitate remembering their order (DuBrow & Davachi, 2013). Coding the position within a sequence during short-term memory in the PFC and anterior temporal lobe (ATL) may be indicative of nonhierarchial neural network representations (Kalm & Norris, 2017). Neural responses to image sequences that are part of a group and are shaped through learning may rely on recurrent connections (Schapiro et al., 2013). Together, these studies and the authors of the debate suggest that the representation of complex sequences may be a combination of the explicitly hierarchical and recurrent networks (Botvinick & Plaut, 2006a, b; Cooper & Shallice, 2006a, b). Representation may also differ across brain area for sequential information. Though several brain regions were found to respond to the order of stimuli, only two regions (PFC and ATL) were shown to encode order in a manner that could guide behavior (Kalm and Norris, 2017). Further experiments are necessary to explicitly test these hypotheses and determine if the models forwarded for complex routine sequential actions also apply to routine and nonroutine sequences of tasks as defined in this review.
A small number of studies have directly investigated the representation of task sequences in the human brain. One of the first studies used cued sequences of three tasks where the task (e.g., consonant/vowel judgement) was indicated by the color of the letter presented on each trial (Dreher & Berman, 2002). They observed activation in the ACC when the sequence of three tasks was initiated. This study differs from other task sequence studies in that the relevant task was cued on each trial and that participants were not aware of the sequences. Studies where the task sequences were explicitly instructed and performed from memory (Desrochers, Chatham, & Badre, 2015; Desrochers, Collins, & Badre, 2019; Koechlin, Corrado, Pietrini, & Grafman, 2000; Koechlin & Jubault, 2006) found many of the same areas engaged as those found in nonsequential hierarchical control, including RLPFC, PFC, and pre-premotor cortex (pre-PM, an area of cortex rostral to premotor cortex, Brodmann area 6, that includes Broca’s area, Figure 2). Specifically, activation in pre-PM was observed when hierarchical processing was necessary, irrespective of timing (Koechlin & Jubault, 2006), and RLPFC was involved preferentially in the execution of sequences of tasks where the series of stimuli was less predictable (Koechlin et al., 2000).
The relative contributions of these frontal brain regions that have common involvement between task sequences and nonsequential hierarchical control remains to be fully established. One task sequences study, based on behavioral work by Schneider and Logan (Schneider & Logan, 2006), dissociated some of these frontal regions and illustrated their necessity in task sequences using fMRI and transcranial magnetic stimulation (TMS) in humans (Desrochers, Chatham, & Badre, 2015). Specifically, the authors observed novel dynamics in the RLPFC across the items in the task sequence; activation in the RLPFC increased progressively (“ramped up”) through the four positions in the sequence each time it was repeated (Figure 3B, C). Further, TMS to the RLPFC disrupted sequential task performance in a manner that increased through the positions in the sequence, mirroring the ramping activation observed with fMRI. These effects in the RLPFC dissociated from other frontal control regions, such as predorsal premotor cortex (pre-PMd) where no ramping activation was observed and task performance was disrupted more in the beginnings of sequences with TMS. Sequential control is consistently found to underlie ramping activation in the RLPFC (Desrochers, et al., 2019). A similar role for the RLPFC was found in a separate task sequences study (Holroyd, Ribas-Fernandes, Shahnazian, Silvetti, & Verguts, 2018). The authors found sequence state representation in the midcingulate cortex and RLPFC and proposed task sequence selection in the rostral frontal cortex with monitoring and execution by the midcingulate cortex. Strikingly, these areas overlap with ramping activation observed in Desrochers et al. (2015) and Desrochers, Collins, and Badre (2019). Collectively, these studies indicate that task sequences may engage some of the same areas that are active in perceptual and motor sequences and nonsequential cognitive control, and that task sequences may utilize novel dynamics and provide dissociation among a network of areas that is commonly activated together.
No studies in animals or related to addiction or drugs of abuse have, to our knowledge, been performed using task sequences. However, studies examining the drug seeking and taking ritual in humans provide preliminary behavioral and neural evidence of sequential processing. Procedures utilizing task sequences in the lab to extinguish drug rituals include drug videos showing the seeking and taking process in combination with mock drug-taking rituals (Childress, McLellan, & O’Brien, 1986). An fMRI study in smokers showed ramping dynamics to drug-related images based on their position within the sequence of events associated with drug consumption (Stippekohl et al., 2010). Participants were shown four images of the smoking ritual (e.g., taking a cigarette out and lighting a cigarette). Responses to images at the beginning of the smoking sequence were located in areas typically associated with sequence and reward such as the OFC, ACC, DLPFC, and the insula. Responses to the end images showed deactivations in the DLPFC and activations in the OFC. Further, sequence position activations decreased over the time course of the smoking ritual images within the ACC, OFC, and insula, thus showing a negative “ramping” signal that has previously been associated with sequential control (Desrochers, Chatham, & Badre, 2015; Desrochers et al., 2019). These results begin to draw an explicit connection between sequential control and drug seeking and taking behaviors, and highlight the importance of understanding the neural impact on the entire sequence of behaviors in the development of potential treatments and therapies.
Conclusion: Integration of Perceptual, Motor, and Task Sequences
Returning to the cooking example, this review has illustrated the many levels that are layered within such sequences (Figure 1). Though each sequence can be summarized as a series of states, these states and sequences can be defined at many levels themselves: perceptual, motor, combined perceptual and motor, task, and the combination of all of them. In cooking, the perceptual sequence may be the series of colors observed when cutting up vegetables or the series of scenes and locations when moving around the kitchen. The motor sequences are the specific muscle movements involved in actions such as chopping, slicing, scooping, and stirring. In the case of cooking, the perceptual and motor sequences are intertwined. Further, a task sequence governs the entire process of cooking a particular recipe in which a series of steps need to be performed in a particular order to accomplish preparing the meal (e.g., cooking the vegetables without cutting them first would most likely result in poorly cooked vegetables).
In light of these multilevel real-world sequences, we put forward the hypothesis that addictions and addictive behaviors may be difficult to extinguish precisely because of the multilayered scaffolding on which they are constructed. First, we will discuss some common brain dynamics among perceptual, motor, and task sequences that suggest that there may be a common neural substrate, and then we will discuss how these ideas about multilevel sequences may more broadly be applied to existing research in addiction (Figure 3A). Several lines of research in animal studies of motor sequences and addiction demonstrate that similar neural circuits underlie sequential actions, and studies in humans have begun to rapidly expand our understanding of the connection between addiction and habitual motor sequences. We posit that the study of the integration across levels of sequence processing and developing an understanding of the underlying neural mechanisms in humans are promising areas for novel research in the field of addiction.
Across the levels of sequences, ramping dynamics have been observed in conjunction with their performance in both humans and animals. We propose that sequential control and processing may underlie these dynamics across many brain areas, and may participate in binding different levels of sequence into a single construct. Preliminary data from the primary visual cortex during statistically learned perceptual sequences show that a small fraction of neurons there show a progressively increasing firing rate as sequences of images proceed (Homann et al., 2017). Though such a small sample of preliminary data is not sufficient to draw conclusions from, the existence of the cells signifies that it is not implausible that ramping signals may exist to signal sequential information in the early visual system.
Similar ramping signals have been observed in instructed perceptual sequences. Examples of recordings from PFC, PM, and M1 of monkeys during a sequential working memory task can display increased firing as a position code (Carpenter et al., 2018; Lundqvist et al., 2018). A study of short-term memory of sequences in humans did not present data on ramping per say, but did indicate that the ability to classify position in the sequence changed with sequence position, particularly in the RLPFC (Kalm & Norris, 2017). It is purely speculative, but difficulty in decoding the first position in the sequence may be due to decreased signal at that position, which might be the case if there were ramping activation present.
Studies of motor sequences have also observed ramping activity. In the activity of neurons in the ACC of rodents performing a sequential lever press task, a smooth ramp in firing rate was observed for correct choices (Ma, Hyman, Phillips, & Seamans, 2014). As discussed previously, activity in ACC is frequently observed in sequential tasks (Figure 2). Asymmetric, ramp-like receptive fields develop in the hippocampus of rats performing repetitive motor sequences (Mehta, 2015; Mehta, Quirk & Wilson, 2000). In an fMRI study performed in humans, frontal cortical regions of interest appear to show ramping dynamics during a modified SRTT task that is potentially related to state uncertainty during the sequence (Konovalov & Krajbich, 2018). Though the ramping dynamic is not discussed for rostral PFC, activation was observed in the RLPFC in relation to state uncertainty. Therefore, the possibility exists that ramping could be a component of activity there.
Together with the task sequences study that observed ramping activation in a network of areas including the RLPFC during task sequences (Desrochers, Chatham, & Badre, 2015; Desrochers et al., 2019), these studies suggest that sequential processing may underlie ramping dynamics in an array of sequence-related areas. Particularly striking is the finding that ramping dynamics in the RLPFC may underlie motor sequence (SRTT) performance as well as task sequence performance and may be related to resolving state uncertainty shown by Konovalov and Krajbich (2018) and hypothesized by Desrochers, Chatham, and Badre (2015). In monkeys, neural recordings in the basal forebrain show ramping related to reward uncertainty (Zhang, Chen, & Monosov, 2018). Further investigation is necessary to determine if sequential processing underlies ramping signals across brain areas, conditions, and species.
Dopamine ramping in animal studies has been found during goal-directed behavior to obtain a reward. Dopamine concentrations in the striatum show ramping that is modulated by reward and goal variables (Howard, Li, Geddes, & Jin, 2017; Howe et al., 2013). Levels of dopamine release also encode reward rate and vigor during learning (Hamid et al., 2015). Additionally, computational modeling shows that dopamine enables frontostriatal circuitry to encode the temporal structure of reward that underlies interval timing across task paradigms in rodents (Mikhael & Gershman, 2019). Alterations of dopamine signaling linked to dysfunction of reward processing in addiction have been found across both animal and human studies. Outstanding questions include whether changes in ramping dynamics are an underlying mechanism for deficits in reward processing within the brain and if and how they are related to neural ramping signals elsewhere in the brain.
In substance addiction, a sequential process typically unfolds in the pursuit to obtain and consume drugs: procuring money, seeking out access to and arranging to obtain the drug, preparing the drug, and finally self-administration. In addition to these actions that unfold over time, contextual cues within the environment and conditioned responses during this sequential process likely contribute to the strengthening of associations between the order of drug pursuit, acquisition, and consumption. Ramping dynamics in the ACC, OFC, and insula have been demonstrated in smokers when viewing images of the drug taking ritual (Stippekohl et al., 2010). Due to this multifaceted and complex repertoire of behaviors that integrates perceptual, motor, and task sequences, addiction can be construed as a disorder of sequential processing. This hierarchical integration of processing may contribute to the chronic and relapsing aspects of addiction in which the intertwined nature of sequential processing levels becomes especially difficult to change with continued drug use.
Data examining the underlying neural basis of multistep actions that comprise behavioral sequences have demonstrated that connections between frontal and striatal brain circuits are necessary for efficient sequential processing (Balleine & O’Doherty, 2010; Belin & Everitt, 2008; Daw, Gershman, Seymour, Dayan, & Dolan, 2011). Data from rodent studies of action sequences have shown that both exposure to drugs of abuse, as well as chronic drug use, in combination with cues and context of the environment, markedly influence the strength of associations formed between action components of drug seeking and taking within sequences (Belin & Everitt, 2008; Ostlund, 2010; Zapata et al., 2010). Preliminary neural data from these studies also support the necessity of intact connections between mPFC areas and striatal regions including the dorsolateral striatum (homolog of human putamen) and ventral striatum.
Human studies of drug use demonstrate similarities to animal studies on the interaction between Pavlovian and instrumental learning, resulting in a combination of cues linked to action sequences that may drive the transition from initial drug use to more habitual sequence execution of actions (Stock, 2017). A striking parallel finding across animal and human drug use studies shows that perceptual (cues) as well as motor (lever presses) components need to be experienced together during the absence of reward to successfully extinguish complex sequences (Hardy et al., 2017; Hogarth et al., 2014; Thrailkill & Bouton, 2016); without learning that these integrated components of sequential behavior no longer result in reward, behavior persists in the absence of the reward outcome, a hallmark of addiction. These significant contributions to our understanding of why behaviors may be hard to change represent a foundation on which to determine whether deficits in frontostriatal circuitry, which have been found among tasks of executive function in addiction (Goldstein & Volkow, 2011), result in a resistance to extinction. Further, research in both humans and animals of simplistic habitual responding suggests that a shift in the neural circuits away from prefrontal control toward a reliance on striatal circuits may results in inflexible control of drug-seeking behavioral sequences.
Others have suggested that addiction and addictive behaviors are disorders of sequences of action (Dezfouli & Balleine, 2012, 2013; Dezfouli, Lingawi, & Balleine, 2014; Graybiel, 2008; Thrailkill & Bouton, 2017; Troisi, 2013). Here we further these ideas by presenting evidence to support the hypothesis that integration across all domains of sequential processing may underlie the persistence of addiction as a disorder. Taken together, findings across domains of sequential processing demonstrate that intact network connectivity between various prefrontal areas and the striatum promote integration across levels of perceptual, motor, and task sequences to enable efficient interaction with our environment on a daily basis (Figure 3A). However, in the context of addiction, deficits of prefrontal control in combination with enhanced habitual behaviors may result in aberrant sequential behavior.
The addiction literature discussed throughout this review suggests that the interaction of sequential modalities (e.g., perceptual and motor sequences) forms the foundation for more complex and overarching task sequences and underlies drug seeking and taking rituals. Abnormalities in processing of stimuli within the brain and behavioral interactions with the environment may contribute to the heterogenous behaviors exhibited within drug seeking and taking rituals. These ideas encourage the development of novel therapeutic options with testable behavioral and neural correlates across species. Ongoing studies and clinical trials in cocaine addiction have demonstrated reduced cue-reactivity to drug-related stimuli after theta burst TMS to the vmPFC (frontal pole; Hanlon et al., 2015; Hanlon, Dowdle, Moss, Canterberry, & George, 2016). Considered in the context of sequences presented in this review, these findings implicate the RLPFC as a potential therapeutic target for sequential processing in addiction. Our ideas thus present the opportunity to explore and develop new paradigms to ultimately better understand sequences of behavior as ubiquitous as cooking or making coffee and as detrimental in the context of drug seeking and taking found in addiction.
Botvinick, M. M. (2008). Hierarchical models of behavior and prefrontal function. Trends in Cognitive Sciences, 12(5), 201–208.Find this resource:
Dehaene, S., Meyniel, F., Wacongne, C., Wang, L., & Pallier, C. (2015). The neural representation of sequences: From transition probabilities to algebraic patterns and linguistic trees. Neuron, 88, 2–19.Find this resource:
Kalm, K., & Norris, D. (2017). Reading positional codes with fMRI: Problems and solutions. PLoS One, 12(5), 8–10.Find this resource:
Kremláček, J., Kreegipuu, K., Tales, A., Astikainen, P., Põldver, N., Näätänen, R., & Stefanics G. (2016). Visual mismatch negativity (vMMN): A review and meta-analysis of studies in psychiatric and neurological disorders. Cortex, 80, 76–112.Find this resource:
Wilson, B., Marslen-Wilson, W. D., & Petkov, C. I. (2017). Conserved sequence processing in primate frontal cortex. Trends in Neurosciences, 40(2), 72–82.Find this resource:
Abrahamse, E. L., Jiménez, L., Verwey, W. B., & Clegg, B. A. (2010). Representing serial action and perception. Psychonomic Bulletin and Review, 17(5), 603–623.Find this resource:
Abrahamse, E. L., Ruitenberg, M. F. L., de Kleine, E., & Verwey, W. B. (2013). Control of automated behavior: Insights from the discrete sequence production task. Frontiers in Human Neuroscience, 7(March), 1–16.Find this resource:
Allen, T. A., Morris, A. M., Mattfeld, A. T., Stark, C. E. L., & Fortin, N. J. (2014). A sequence of events model of episodic memory shows parallels in rats and humans. Hippocampus, 24(10), 1178–1188.Find this resource:
Amiez, C., & Petrides, M. (2007). Selective involvement of the mid-dorsolateral prefrontal cortex in the coding of the serial order of visual stimuli in working memory. Proceedings of the National Academy of Sciences of the United States of America, 104(34), 13786–13791.Find this resource:
Aslin, R. N. (2017). Statistical learning: A powerful mechanism that operates by mere exposure. Wiley Interdisciplinary Reviews. Cognitive Science, 8(1–2), 1–7.Find this resource:
Averbeck, B. B., & Lee, D. (2007). Prefrontal neural correlates of memory for sequences. Journal of Neuroscience, 27(9), 2204–2211.Find this resource:
Badre, D. (2008). Cognitive control, hierarchy, and the rostro-caudal organization of the frontal lobes. Trends in Cognitive Sciences, 12(5), 193–200.Find this resource:
Badre, D., & D’Esposito, M. (2007). Functional magnetic resonance imaging evidence for a hierarchical organization of the prefrontal cortex. Journal of Cognitive Neuroscience, 19(12), 2082–2099.Find this resource:
Badre, D., & D’Esposito, M. (2009). Is the rostro-caudal axis of the frontal lobe hierarchical? Nature Reviews. Neuroscience, 10(9), 659–669.Find this resource:
Badre, D., & Nee, D. E. (2018). Frontal cortex and the hierarchical control of behavior. Trends in Cognitive Sciences, 22(2), 170–188.Find this resource:
Balleine, B. W., & O’Doherty, J. P. (2010). Human and rodent homologies in action control: Corticostriatal determinants of goal-directed and habitual action. Neuropsychopharmacology, 35(1), 48–69.Find this resource:
Banca, P., Harrison, N. A., & Voon, V. (2016). Compulsivity across the pathological misuse of drug and non-drug rewards. Frontiers in Behavioral Neuroscience, 10, 154.Find this resource:
Barnes, T. D., Kubota, Y., Hu, D., Jin, D. Z., & Graybiel, A. M. (2005). Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature, 437(7062), 1158–1161.Find this resource:
Bekinschtein, T. A., Dehaene, S., Rohaut, B., Tadel, F., Cohen, L., & Naccache, L. (2009). Neural signature of the conscious processing of auditory regularities. Proceedings of the National Academy of Sciences of the United States of America, 106(5), 1672–1677.Find this resource:
Belin, D., Belin-Rauscent, A., Murray, J. E., & Everitt, B. J. (2013). Addiction: Failure of control over maladaptive incentive habits. Current Opinion in Neurobiology, 23(4), 564–572.Find this resource:
Belin, D., & Everitt, B. J. (2008). Cocaine-seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron, 57(3), 432–441.Find this resource:
Belin-Rauscent, A., Everitt, B. J., & Belin, D. (2012). Intrastriatal shifts mediate the transition from drug-seeking actions to habits. Biological Psychiatry, 72(5), 343–345.Find this resource:
Berdyyeva, T. K., & Olson, C. R. (2010). Rank signals in four areas of macaque frontal cortex during selection of actions and objects in serial order. Journal of Neurophysiology, 104(1), 141–159.Find this resource:
Billig, A. J., Davis, M. H., & Carlyon, R. P. (2018). Neural decoding of bistable sounds reveals an effect of intention on perceptual organization. Journal of Neuroscience, 38(11), 2844–2853.Find this resource:
Blundon, E. G., Rumak, S. P., & Ward, L. M. (2017). Sequential search asymmetry: Behavioral and psychophysiological evidence from a dual oddball task. PloS One, 12(3), e0173237.Find this resource:
Boehnke, S. E., Berg, D. J., Marino, R. A., Baldi, P. F., Itti, L., & Munoz, D. P. (2011). Visual adaptation and novelty responses in the superior colliculus. European Journal of Neuroscience, 34(5), 766–779.Find this resource:
Botvinick, M. M. (2007). Multilevel structure in behaviour and in the brain: A model of Fuster’s hierarchy. Philosophical Transactions of the Royal Society B: Biological Sciences, 362(1485), 1615–1626.Find this resource:
Botvinick, M. M. (2008). Hierarchical models of behavior and prefrontal function. Trends in Cognitive Sciences, 12(5), 201–208.Find this resource:
Botvinick, M. M. (2012). Hierarchical reinforcement learning and decision making. Current Opinion in Neurobiology, 22(6), 956–962.Find this resource:
Botvinick, M., & Plaut, D. C. (2004). Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired routine sequential action. Psychological Review, 111(2), 395–429.Find this resource:
Botvinick, M. M., & Plaut, D. C. (2006a). Short-term memory for serial order: A recurrent neural network model. Psychological Review, 113(2), 201–233.Find this resource:
Botvinick, M. M., & Plaut, D. C. (2006b). Such stuff as habits are made on: A reply to Cooper and Shallice (2006). Psychological Review, 113(4), 917–927.Find this resource:
Burke, D. S., Jalal, H., Balmert, L. C., Roberts, M. S., Buchanich, J. M., & Zhang, K. (2018). Changing dynamics of the drug overdose epidemic in the United States from 1979 through 2016. Science, 361(6408), eaau1184.Find this resource:
Carpenter, A. F., Baud-Bovy, G., Georgopoulos, A. P., & Pellizzer, G. (2018). Encoding of serial order in working memory: Neuronal activity in motor, premotor and prefrontal cortex during a memory scanning task. Journal of Neuroscience, 38(21), 4912–4933.Find this resource:
Cartoni, E., Moretta, T., Puglisi-Allegra, S., Cabib, S., & Baldassarre, G. (2015). The relationship between specific Pavlovian instrumental transfer and instrumental reward probability. Frontiers in Psychology, 6(Nov.), 1–7.Find this resource:
Chen, B. T., Yau, H. J., Hatch, C., Kusumoto-Yoshida, I., Cho, S. L., Hopf, F. W., & Bonci, A. (2013). Rescuing cocaine-induced prefrontal cortex hypoactivity prevents compulsive cocaine seeking. Nature, 496(7445), 359–362.Find this resource:
Childress, A. R., McLellan, A. T., & O’Brien, C. P. (1986). Abstinent opiate abusers exhibit conditioned craving, conditioned withdrawal and reductions in both through extinction. British Journal of Addiction, 81(5), 655–660.Find this resource:
Colas, J. T., Pauli, W. M., Larsen, T., Tyszka, J. M., & O’Doherty, J. P. (2017). Distinct prediction errors in mesostriatal circuits of the human brain mediate learning about the values of both states and actions: evidence from high-resolution fMRI. PLoS Computational Biology, 13.Find this resource:
Conklin, C. A., & Tiffany, S. T. (2002). Applying extinction research and theory to cue-exposure addiction treatments. Addiction, 97(2), 155–167.Find this resource:
Conway, C. M., & Christiansen, M. H. (2001). Review: Sequential learning in non-human primates. Trends in Cognitive Sciences, 5(12), 539–546.Find this resource:
Cooper, R., & Shallice, T. (2000). Contention scheduling and the control of routine activities. Cognitive Neuropsychology, 17(4), 297–338.Find this resource:
Cooper, R. P., & Shallice, T. (2006a). Hierarchical schemas and goals in the control of sequential behavior. Psychological Review, 113(4), 887–916.Find this resource:
Cooper, R. P., & Shallice, T. (2006b). Structured representations in the control of behavior cannot be so easily dismissed: A reply to Botvinick and Plaut (2006). Psychological Review, 113(4), 929–931.Find this resource:
Corbit, L. H. (2005). Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian-instrumental transfer. Journal of Neuroscience, 25(4), 962–970.Find this resource:
Corbit, L. H., & Balleine, B. W. (2003). Instrumental and Pavlovian incentive processes have dissociable effects on components of a heterogeneous instrumental chain. Journal of Experimental Psychology: Animal Behavior Processes, 29(2), 99–106.Find this resource:
Curtis, C. E., Cole, M. W., Rao, V. Y., & D’Esposito, M. (2005). Canceling planned action: An fMRI study of countermanding saccades. Cerebral Cortex, 15(9), 1281–1289.Find this resource:
Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P., & Dolan, R. J. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69(6), 1204–1215.Find this resource:
Daw, N. D., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711.Find this resource:
de Wit, S., Corlett, P. R., Aitken, M. R., Dickinson, A., & Fletcher, P. C. (2009). Differential engagement of the ventromedial prefrontal cortex by goal-directed and habitual behavior toward food pictures in humans. Journal of Neuroscience, 29(36), 11330–11338.Find this resource:
de Wit, S., Kindt, M., Knot, S., & Verhoeen, A. (2018). Shifting the balance between goals and habits: Five failures in experimental habit induction. Journal of Experimental Psychology, 147(7), 1043–1065.Find this resource:
de Wit, S., Niry, D., Wariyar, R., Aitken, M. R., & Dickinson, A. (2007). Stimulus-outcome interactions during instrumental discrimination learning by rats and humans. Journal of Experimental Psychology: Animal Behavior Processes, 33(1), 1–11.Find this resource:
de Wit, S., Watson, P., Harsay, H. A., Cohen, M. X., van de Vijver, I., & Ridderinkhof, K. R. (2012). Corticostriatal connectivity underlies individual differences in the balance between habitual and goal-directed action control. Journal of Neuroscience, 32(35), 12066–12075.Find this resource:
Desrochers, T. M., Amemori, K., & Graybiel, A. M. (2015). Habit learning by naive macaques is marked by response sharpening of striatal neurons representing the cost and outcome of acquired action sequences. Neuron, 87(4), 853–868.Find this resource:
Desrochers, T. M., Burk, D. C., Badre, D., & Sheinberg, D. L. (2016). The monitoring and control of task sequences in human and non-human primates. Frontiers in Systems Neuroscience, 9, 185.Find this resource:
Desrochers, T. M., Chatham, C. H., & Badre, D. (2015). The necessity of rostrolateral prefrontal cortex for higher-level sequential behavior. Neuron, 87(6), 1357–1368.Find this resource:
Desrochers, T. M., Collins, A. G., & Badre, D. (2019). Sequential control underlies robust ramping dynamics in the rostrolateral prefrontal cortex. Journal of Neuroscience, 39(8), 1471–1483.Find this resource:
Dezfouli, A., & Balleine, B. W. (2012). Habits, action sequences and reinforcement learning. European Journal of Neuroscience, 35(7), 1036–1051.Find this resource:
Dezfouli, A., & Balleine, B. W. (2013). Actions, action sequences and habits: Evidence that goal-directed and habitual action control are hierarchically organized. PLoS Computational Biology, 9(12), e1003364.Find this resource:
Dezfouli, A., Lingawi, N. W., & Balleine, B. W. (2014). Habits as action sequences: Hierarchical action control and changes in outcome value. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 369(1655).Find this resource:
Dickinson, A. (1985). Actions and habits: The development of behavioral autonomy. Philosophical Transactions of the Royal Society of London Series B-Biological Sciences, 308(1135), 67–78.Find this resource:
Dickinson, A., & Balleine, B. (1995). Motivational control of heterogeneous instrumental chains. Current Directions in Psychological Science, 4(5), 162–167.Find this resource:
Dolan, R. J., & Dayan, P. (2013). Goals and habits in the brain. Neuron, 80(2), 312–325.Find this resource:
Dreher, J.-C., & Berman, K. F. (2002). Fractionating the neural substrate of cognitive control processes. Proceedings of the National Academy of Sciences of the United States of America, 99(22), 14595–14600.Find this resource:
DuBrow, S., & Davachi, L. (2013). The influence of context boundaries on memory for the sequential order of events. Journal of Experimental Psychology: General, 142(4), 1277–1286.Find this resource:
Ersche, K. D., Fletcher, P. C., Lewis, S. J. G., Clark, L., Stocks-Gee, G., London, M., . . . Sahakian, B. J. (2005). Abnormal frontal activations related to decision-making in current and former amphetamine and opiate dependent individuals. Psychopharmacology, 180(4), 612–623.Find this resource:
Ersche, K. D., Gillan, C. M., Jones, P. S., Williams, G. B., Ward, L. H. E., Luijten, M., . . . Robbins, T. W. (2016). Carrots and sticks fail to change behavior in cocaine addiction. Science, 352(6292), 1468–1471.Find this resource:
Eslinger, P. J., & Damasio, A. R. (1985). Severe disturbance of higher cognition after bilateral frontal lobe ablation: Patient EVR. Neurology, 35(12), 1731–1741.Find this resource:
Farooqui, A., Mitchell, D., Thompson, R., & Duncan, J. (2012). Hierarchical organization of cognition reflected in distributed frontoparietal activity. Journal of Neuroscience, 32(48), 17373–17381.Find this resource:
Fendrich, D. W., Healy, A. F., & Bourne, L. E. (1991). Long-term repetition effects for motoric and perceptual procedures. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17(1), 137–151.Find this resource:
Fiser, J., & Aslin, R. N. (2001). Unsupervised statistical learning of higher order spatial structures from visual scenes. Psychological Science, 12(6), 499–504.Find this resource:
Fiser, J., & Aslin, R. N. (2002). Statistical learning of higher-order temporal structure from visual shape sequences. Journal of Experimental Psychology: Learning Memory and Cognition, 28(3), 458–467.Find this resource:
Friedel, E., Koch, S. P., Wendt, J., Heinz, A., Deserno, L., & Schlagenhauf, F. (2014). Devaluation and sequential decisions: Linking goal-directed and model-based behavior. Frontiers in Human Neuroscience, 8, 587.Find this resource:
Fujii, N., & Graybiel, A. M. (2003). Representation of action sequence boundaries by macaque prefrontal cortical neurons. Science, 301(5637), 1246 LP–1249.Find this resource:
Fujii, N., & Graybiel, A. M. (2005). Time-varying covariance of neural activities recorded in striatum and frontal cortex as monkeys perform sequential-saccade tasks. Proceedings of the National Academy of Sciences of the United States of America, 102(25), 9032–9037.Find this resource:
Gámez, M. A., & Rosas, J. M. (2005). Transfer of stimulus control across instrumental responses is attenuated by extinction in human instrumental conditioning. International Journal of Psychology and Psychological Therapy, 5(3), 207–222.Find this resource:
Gardner, J. L., Sun, P., Waggoner, R. A., Ueno, K., Tanaka, K., & Cheng, K. (2005). Contrast adaptation and representation in human early visual cortex. Neuron, 47(4), 607–620.Find this resource:
Gavornik, J. P., & Bear, M. F. (2014). Learned spatiotemporal sequence recognition and prediction in primary visual cortex. Nature Neuroscience, 17(5), 732–737.Find this resource:
Gheysen, F., & Fias, W. (2012). Dissociable neural systems of sequence learning. Advances in Cognitive Psychology, 8(2), 73–82.Find this resource:
Gheysen, F., Van Opstal, F., Roggeman, C., Van Waelvelde, H., & Fias, W. (2011). The neural basis of implicit perceptual sequence learning. Frontiers in Human Neuroscience, 5(November), 1–12.Find this resource:
Gillan, C. M., Papmeyer, M., Morein-Zamir, S., Sahakian, B. J., Fineberg, N. A., Robbins, T. W., & de Wit, S. (2011). Disruption in the balance between goal-directed behavior and habit learning in obsessive-compulsive disorder. American Journal of Psychiatry, 168(7), 718–726.Find this resource:
Glascher, J., Daw, N., Dayan, P., & O’Doherty, J. P. (2010). States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron, 66(4), 585–595.Find this resource:
Goldstein, R. Z., & Volkow, N. D. (2011). Dysfunction of the prefrontal cortex in addiction: Neuroimaging findings and clinical implications. Nature Reviews Neuroscience, 12(11), 652–669.Find this resource:
Graybiel, A. M. (2008). Habits, rituals, and the evaluative brain. Annual Review of Neuroscience, 31, 359–387.Find this resource:
Hamid, A. A., Pettibone, J. R., Mabrouk, O. S., Hetrick, V. L., Schmidt, R., Vander Weele, C. M., . . . Berke, J. D. (2015). Mesolimbic dopamine signals the value of work. Nature Neuroscience, 19(1), 117–126.Find this resource:
Hanlon, C. A., Dowdle, L. T., Austelle, C. W., Devries, W., Mithoefer, O., Badran, B. W., & George, M. S. (2015). What goes up, can come down: Novel brain stimulation paradigms may attenuate craving and craving-related neural circuitry in substance dependent individuals. Brain Research, 1628, 199–209.Find this resource:
Hanlon, C. A., Dowdle, L. T., Moss, H., Canterberry, M., & George, M. S. (2016). Mobilization of medial and lateral frontal-striatal circuits in cocaine users and controls: An Interleaved TMS/BOLD functional connectivity study. Neuropsychopharmacology, 41(13), 3032–3041.Find this resource:
Hanlon, C. A., Dowdle, L. T., Naselaris, T., Canterberry, M., & Cortese, B. M. (2014). Visual cortex activation to drug cues: A meta-analysis of functional neuroimaging papers in addiction and substance abuse literature. Drug and Alcohol Dependence, 143(1), 206–212.Find this resource:
Hardy, L., Mitchell, C., Seabrooke, T., & Hogarth, L. (2017). Drug cue reactivity involves hierarchical instrumental learning: evidence from a biconditional Pavlovian to instrumental transfer task. Psychopharmacology, 234(13), 1977–1984.Find this resource:
Heide, W., Binkofski, F., Seitz, R. J., Posse, S., Nitschke, M. F., Freund, H.-J. J., & Kömpf, D. (2001). Activation of frontoparietal cortices during memorized triple-step sequences of saccadic eye movements: An fMRI study. European Journal of Neuroscience, 13(6), 1177–1189.Find this resource:
Heimbauer, L. A., Conway, C. M., Christiansen, M. H., Beran, M. J., & Owren, M. J. (2012). A serial reaction time (SRT) task with symmetrical joystick responding for nonhuman primates. Behavior Research Methods, 44(3), 733–741.Find this resource:
Heimbauer, L. A., Conway, C. M., Christiansen, M. H., Beran, M. J., & Owren, M. J. (2018). Visual artificial grammar learning by rhesus macaques (Macaca mulatta): Exploring the role of grammar complexity and sequence length. Animal Cognition, 21(2), 267–284.Find this resource:
Hicks, J. A., Friedman, R. S., Gable, P. A., & Davis, W. E. (2012). Interactive effects of approach motivational intensity and alcohol cues on the scope of perceptual attention. Addiction, 107(6), 1074–1080.Find this resource:
Hikosaka, O., Rand, M. K., Miyachi, S., & Miyashita, K. (1995). Learning of sequential movements in the monkey: process of learning and retention of memory. Journal of Neurophysiology, 74(4), 1652–1661.Find this resource:
Hogarth, L., Retzler, C., Munafò, M. R., Tran, D. M. D., Troisi, J. R., Rose, A. K., . . . Field, M. (2014). Extinction of cue-evoked drug-seeking relies on degrading hierarchical instrumental expectancies. Behaviour Research and Therapy, 59, 61–70.Find this resource:
Holmes, N. M., Marchand, A. R., & Coutureau, E. (2010). Pavlovian to instrumental transfer: A neurobehavioural perspective. Neuroscience and Biobehavioral Reviews, 34(8), 1277–1295.Find this resource:
Holroyd, C. B., Ribas-Fernandes, J. J. F., Shahnazian, D., Silvetti, M., & Verguts, T. (2018). Human midcingulate cortex encodes distributed representations of task progress. Proceedings of the National Academy of Sciences of the United States of America, (31), 1–6.Find this resource:
Homann, J., Koay, S. A., Glidden, A. M., Tank, D. W., & Berry, M. J. (2017). Predictive coding of novel versus familiar stimuli in the primary visual cortex. Unpublished manuscript.Find this resource:
Howard, C. D., Li, H., Geddes, C. E., & Jin, X. (2017). Dynamic nigrostriatal dopamine biases action selection. Neuron, 93(6), 1436–1450.e8.Find this resource:
Howe, M. W., Tierney, P. L., Sandberg, S. G., Phillips, P. E., & Graybiel, A. M. (2013). Prolonged dopamine signalling in striatum signals proximity and value of distant rewards. Nature, 500(7464), 575–579.Find this resource:
Hsieh, L. T., Gruber, M. J., Jenkins, L. J., & Ranganath, C. (2014). Hippocampal activity patterns carry information about objects in temporal context. Neuron, 81(5), 1165–1178.Find this resource:
Hsieh, L.-T., & Ranganath, C. (2015). Cortical and subcortical contributions to sequence retrieval: Schematic coding of temporal context in the neocortical recollection network. NeuroImage, 121, 78–90.Find this resource:
Hu, Y., Salmeron, B. J., Gu, H., Stein, E. A., & Yang, Y. (2015). Impaired functional connectivity within and between frontostriatal circuits and its association with compulsive drug use and trait impulsivity in cocaine addiction. JAMA Psychiatry, 72(6), 584–592.Find this resource:
Jin, X., & Costa, R. M. (2010). Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature, 466(7305), 457–462.Find this resource:
Jin, X., & Costa, R. M. (2015). Shaping action sequences in basal ganglia circuits. Current Opinion in Neurobiology, 33, 188–196.Find this resource:
Jin, X., Tecuapetla, F., & Costa, R. M. (2014). Basal ganglia subcircuits distinctively encode the parsing and concatenation of action sequences. Nature Neuroscience, 17(3), 423–430.Find this resource:
Jog, M. S., Kubota, Y., Connolly, C. I., Hillegaart, V., & Graybiel, A. M. (1999). Building neural representations of habits. Science, 286(5445), 1745 LP–1749.Find this resource:
Kalm, K., & Norris, D. (2017). A shared representation of order between encoding and recognition in visual short-term memory. NeuroImage, 155(April), 138–146.Find this resource:
Keele, S. W., Mayr, U., Ivry, R., Hazeltine, E., & Heuer, H. (2003). The cognitive and neural architecture of sequence representation. Psychological Review, 110(2), 316–339.Find this resource:
Keistler, C., Barker, J. M., & Taylor, J. R. (2015). Infralimbic prefrontal cortex interacts with nucleus accumbens shell to unmask expression of outcomeselective Pavlovian-to-instrumental transfer. Learning and Memory, 22(10), 509–513.Find this resource:
Kim, H. J., Lee, H., Yun, K., & Kim, J.-H. (2017). Alterations in striatal circuits underlying addiction-like behaviors. Molecules and Cells, 40(6), 379–385.Find this resource:
Koechlin, E., Corrado, G., Pietrini, P., & Grafman, J. (2000). Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. Proceedings of the National Academy of Sciences of the United States of America, 97(13), 7651–7656.Find this resource:
Koechlin, E., & Jubault, T. (2006). Broca’s area and the hierarchical organization of human behavior. Neuron, 50(6), 963–974.Find this resource:
Koechlin, E., Ody, C., & Kouneiher, F. (2003). The architecture of cognitive control in the human prefrontal cortex. Science, 302(5648), 1181–1185.Find this resource:
Konova, A. B., Moeller, S. J., Tomasi, D., Parvaz, M. A., Alia-Klein, N., Volkow, N. D., & Goldstein, R. Z. (2012). Structural and behavioral correlates of abnormal encoding of money value in the sensorimotor striatum in cocaine addiction. European Journal of Neuroscience, 36(7), 2979–2988.Find this resource:
Konovalov, A., & Krajbich, I. (2018). Neuron neurocomputational dynamics of sequence learning. Neuron, 98(6), 1282–1293.e4.Find this resource:
Kremláček, J., Kreegipuu, K., Tales, A., Astikainen, P., Põldver, N., Näätänen, R., & Stefanics, G. (2016). Visual mismatch negativity (vMMN): A review and meta-analysis of studies in psychiatric and neurological disorders. Cortex, 80, 76–112.Find this resource:
Lashley, K. S. (1951). The problem of serial order in behavior. In L. A. Jeffress (Ed.), Cerebral mechanisms in behavior (pp. 112–146). New York, NY: Wiley.Find this resource:
Lee, S. W., Shimojo, S., & O’Doherty, J. P. (2014). Neural computations underlying arbitration between model-based and model-free learning. Neuron, 81(3), 687–699.Find this resource:
Lench, D. H., DeVries, W., & Hanlon, C. A. (2017). The effect of task difficulty on motor performance and frontal-striatal connectivity in cocaine users. Drug and Alcohol Dependence, 173, 178–184.Find this resource:
Lu, B., Li, M., Hou, Y. Y., Chen, J., Chi, Z. Q., & Liu, J. G. (2010). Initial experience of heroin use under a two-chained operant schedule influences drug-seeking behavior after one month of abstinence. Acta Pharmacologica Sinica, 31(4), 387–392.Find this resource:
Lu, X., & Ashe, J. (2005). Anticipatory activity in primary motor cortex codes memorized movement sequences. Neuron, 45(6), 967–973.Find this resource:
Lu, X., Matsuzawa, M., & Hikosaka, O. (2002). A neural correlate of oculomotor sequences in supplementary eye field. Neuron, 34(2), 317–325.Find this resource:
Lundqvist, M., Herman, P., Warden, M. R., Brincat, S. L., & Miller, E. K. (2018). Gamma and beta bursts during working memory readout suggest roles in its volitional control. Nature Communications, 9(1), 1–12.Find this resource:
Ma, L., Hyman, J. M., Phillips, A. G., & Seamans, J. K. (2014). Tracking progress toward a goal in corticostriatal ensembles. Journal of Neuroscience, 34(6), 2244–2253.Find this resource:
Matías Gámez, A., & Rosas, J. M. (2005). Transfer of stimulus control across instrumental responses is attenuated by extinction in human instrumental conditioning. International Journal of Psychology and Psychological Therapy, 5(3), 207–222.Find this resource:
Matsuzaka, Y., Picard, N., & Strick, P. L. (2006). Skill representation in the primary motor cortex after long-term practice. Journal of Neurophysiology, 97(2), 1819–1832.Find this resource:
Mayr, U. (1996). Spatial attention and implicit sequence learning: Evidence for independent learning of spatial and nonspatial sequences. Journal of Experimental Psychology, 22(2), 350–364.Find this resource:
McKim, T. H., Bauer, D. J., & Boettiger, C. A. (2016). Addiction history associates with the propensity to form habits. Journal of Cognitive Neuroscience, 28(7), 1024–1038.Find this resource:
McKim, T. H., Shnitko, T.A., Robinson, D.L., & Boettiger, C.A. (2016). Translational research on habit and alcohol. Current Addiction Reports, 3(1), 37–49.Find this resource:
Mehta, M. R. (2015). From synaptic plasticity to spatial maps and sequence learning. Hippocampus, 25(6), 756–762.Find this resource:
Mehta, M. R., Quirk, M. C., & Wilson, M. A. (2000). Experience-dependent asymmetric shape of hippocampal receptive fields. Neuron, 25(3), 707–715.Find this resource:
Meyer, T., & Olson, C. R. (2011). Statistical learning of visual transitions in monkey inferotemporal cortex. Proceedings of the National Academy of Sciences, 108(48), 19401–19406.Find this resource:
Meyer, T., Ramachandran, S., & Olson, C. R. (2014). Statistical learning of serial visual transitions by neurons in monkey inferotemporal cortex. Journal of Neuroscience, 34(28), 9332–9337.Find this resource:
Mikhael, J. G., & Gershman, S. J. (2019). Adapting the flow of time with dopamine. Journal of Neurophysiology, 121(5), 1748–1760.Find this resource:
Miller, E. K., & Cohen, J. D. (2001). An integrative theory of prefrontal cortex function. Annual Review of Neuroscience, 24, 167–202.Find this resource:
Miller, K., Shenhav, A., & Ludvig, E. (2019). Habits without values. Psychological Reviews, 126(2), 292–311.Find this resource:
Milne, A. E., Mueller, J. L., Männel, C., Attaheri, A., Friederici, A. D., & Petkov, C. I. (2016). Evolutionary origins of non-adjacent sequence processing in primate brain potentials. Scientific Reports, 6, 1–10.Find this resource:
Milne, A. E., Petkov, C. I., & Wilson, B. (2017). Auditory and visual sequence learning in humans and monkeys using an artificial grammar learning paradigm. Neuroscience, 389(October), 104–117.Find this resource:
Minier, L., Fagot, J., & Rey, A. (2016). The temporal dynamics of regularity extraction in non-human primates. Cognitive Science, 40(4), 1019–1030.Find this resource:
Näätänen, R., Astikainen, P., Ruusuvirta, T., & Huotilainen, M. (2010). Automatic auditory intelligence: An expression of the sensory-cognitive core of cognitive processes. Brain Research Reviews, 64(1), 123–136.Find this resource:
Näätänen, R., Kujala, T., Escera, C., Baldeweg, T., Kreegipuu, K., Carlson, S., & Ponton, C. (2012). The mismatch negativity (MMN): A unique window to disturbed central auditory processing in ageing and different clinical conditions. Clinical Neurophysiology, 123(3), 424–458.Find this resource:
Nakamura, K., Sakai, K., & Hikosaka, O. (1998). Neuronal activity in medial frontal cortex during learning of sequential procedures. Journal of Neurophysiology, 80(5), 2671–2687.Find this resource:
Nakamura, K., Sakai, K., & Hikosaka, O. (1999). Effects of local inactivation of monkey medial frontal cortex in learning of sequential procedures. Journal of Neurophysiology, 82(2), 1063–1068.Find this resource:
Nissen, M. J., & Bullemer, P. (1987). Attentional requirements of learning: Evidence from performance measures. Cognitive Psychology, 19(1), 1–32.Find this resource:
O’Doherty, J. P. (2004). Reward representations and reward-related learning in the human brain: Insights from neuroimaging. Current Opinion in Neurobiology, 14(6), 769–776.Find this resource:
Ohshiba, N. (1997). Memorization of serial items by Japanese monkeys, a chimpanzee, and humans. Japanese Psychological Research, 39(3), 236–252.Find this resource:
Olausson, P., Jentsch, J. D., Krueger, D. D., Tronson, N. C., Nairn, A. C., & Taylor, J. R. (2007). Orbitofrontal cortex and cognitive-motivational impairments in psychostimulant addiction. In G. Schoenbaum, J. A. Gottfried, & E. A. Murray (Eds.), Linking affect to action: Critical contributions of the orbitofrontal cortex (pp. 610–638). Oxford: Wiley Blackwell.Find this resource:
Olmstead, M. C., Lafond, M. V., Everitt, B. J., & Dickinson, A. (2001). Cocaine seeking by rats is a goal-directed action. Behavioral Neuroscience, 115(2), 394–402.Find this resource:
Ostlund, S. B., Maidment, N. T., Balleine, B. W. (2010). Alcohol-paired contextual cues produce an immediate and selective loss of goal-directed action in rats. Frontiers in Integrative Neuroscience, 4(July), 1–8.Find this resource:
Ostlund, S. B., & Balleine, B. W. (2008). On habits and addiction: An associative analysis of compulsive drug seeking. Drug Discovery Today: Disease Models, 5(4), 235–245.Find this resource:
Ostlund, S. B., Winterbauer, N. E., & Balleine, B. W. (2009). Evidence of action sequence chunking in goal-directed instrumental conditioning and its dependence on the dorsomedial prefrontal cortex. Journal of Neuroscience, 29(25), 8280–8287.Find this resource:
Park, S. Q., Kahnt, T., Beck, A., Cohen, M. X., Dolan, R. J., Wrase, J., & Heinz, A. (2010). Prefrontal cortex fails to learn from reward prediction errors in alcohol dependence. Journal of Neuroscience, 30(22), 7749–7753.Find this resource:
Pazo-Alvarez, P., Cadaveira, F., & Amenedo, E. (2003). MMN in the visual modality: A review. Biological Psychology, 63(3), 199–236.Find this resource:
Pelloux, Y., Everitt, B. J., & Dickinson, A. (2007). Compulsive drug seeking by rats under punishment: Effects of drug taking history. Psychopharmacology, 194(1), 127–137.Find this resource:
Petit, L., Orssaud, C., Tzourio, N., Crivello, F., Berthoz, A., & Mazoyer, B. (1996). Functional anatomy of a prelearned sequence of horizontal saccades in humans. Journal of Neuroscience, 16(11), 3714–3726.Find this resource:
Phillips, H. N., Blenkmann, A., Hughes, L. E., Bekinschtein, T. A., & Rowe, J. B. (2015). Hierarchical organization of frontotemporal networks for the prediction of stimuli across multiple dimensions. Journal of Neuroscience, 35(25), 9255–9264.Find this resource:
Procyk, E., Tanaka, Y. L., & Joseph, J. P. (2000). Anterior cingulate activity during routine and non-routine sequential behaviors in macaques. Nature Neuroscience, 3(5), 502–508.Find this resource:
Quirk, C. R., Allen, T. A., & Fortin, N. J. (2014). The nucleus reuniens and perirhinal cortex are critical to memory for sequences of events. In Society for Neuroscience. Washington, DC.Find this resource:
Ramachandran, S., Meyer, T., & Olson, C. R. (2017). Prediction suppression and surprise enhancement in monkey inferotemporal cortex. Journal of Neurophysiology, 118(1), 374–382.Find this resource:
Reiter, A. M. F., Deserno, L., Kallert, T., Heinze, H.-J., Heinz, A., & Schlagenhauf, F. (2016). Behavioral and neural signatures of reduced updating of alternative options in alcohol-dependent patients during flexible decision-making. Journal of Neuroscience, 36(43), 10935–10948.Find this resource:
Rudebeck, P. H., Saunders, R. C., Lundgren, D. A., & Murray, E. A. (2017). Specialized representations of value in the orbital and ventrolateral prefrontal cortex: Desirability versus availability of outcomes. Neuron, 95(5), 1208–1220.Find this resource:
Saddoris, M. P., Stamatakis, A., & Carelli, R. M. (2011). Neural correlates of Pavlovian-to-instrumental transfer in the nucleus accumbens shell are selectively potentiated following cocaine self-administration. European Journal of Neuroscience, 33(12), 2274–2287.Find this resource:
Saffran, J. R., Aslin, R. N., & Newport, E. L. (1996). Statistical learning by 8-month-old infants. Science, 274(5294), 1926–1928.Find this resource:
Sakai, K., Hikosaka, O., Miyauchi, S., Takino, R., Sasaki, Y., & Pütz, B. (1998). Transition of brain activation from frontal to parietal areas in visuomotor sequence learning. Journal of Neuroscience, 18(5), 1827–1840.Find this resource:
SAMSHA, Center for Behavioral Health Statistics and Quality. (2016). Key substance use and mental health indicators in the United States: Results from the 2015 National Survey on Drug Use and Health (HHS Publication No. SMA 16-4984, NSDUH Series H-51). Rockville, MD: SAMSHA.Find this resource:
Substance Abuse and Mental Health Services Administration (SAMSHA). (2018). Key substance use and mental health indicators in the United States: Results from the 2017 National Survey on Drug Use and Health (HHS Publication No. SMA 18-5068, NSDUH Series H-53). Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration.Find this resource:
Schacht, J. P., Anton, R. F., & Myrick, H. (2013). Functional neuroimaging studies of alcohol cue reactivity: A quantitative meta-analysis and systematic review. Addiction Biology, 18(1), 121–133.Find this resource:
Schapiro, A. C., Rogers, T. T., Cordova, N. I., Turk-Browne, N. B., & Botvinick, M. M. (2013). Neural representations of events arise from temporal community structure. Nature Neuroscience, 16(4), 486–92.Find this resource:
Schneider, D. W., & Logan, G. D. (2006). Hierarchical control of cognitive processes: Switching tasks in sequences. Journal of Experimental Psychology: General, 135(4), 623–640.Find this resource:
Schwarting, R. K. W. (2009). Rodent models of serial reaction time tasks and their implementation in neurobiological research. Behavioural Brain Research, 199(1), 76–88.Find this resource:
Schwartz, G., Harris, R., Shrom, D., & Berry, M. J. (2007). Detection and prediction of periodic patterns by the retina. Nature Neuroscience, 10(5), 552–554.Find this resource:
Sebold, M., Deserno, L., Nebe, S., Schad, D. J., Garbusow, M., Hagele, C., . . . Huys, Q. J. (2014). Model-based and model-free decisions in alcohol dependence. Neuropsychobiology, 70(2), 122–131.Find this resource:
Seidlitz, J., Sponheim, C., Glen, D., Ye, F. Q., Saleem, K. S., Leopold, D. A., . . . Messinger, A. (2018). A population MRI brain template and analysis tools for the macaque. NeuroImage, 170(April), 121–131.Find this resource:
Shallice, T., & Burgess, P. W. (1991). Deficits in strategy application following frontal lobe damage in man. Brain, 114(Pt 2), 727–741.Find this resource:
Shima, K., & Tanji, J. (1998). Both supplementary and presupplementary motor areas are crucial for the temporal organization of multiple movements. Journal of Neurophysiology, 80(6), 3247–3260.Find this resource:
Shima, K., & Tanji, J. (2000). Neuronal activity in the supplementary and presupplementary motor areas for temporal organization of multiple movements. Journal of Neurophysiology, 84(4), 2148–2160.Find this resource:
Singer, B. F., Fadanelli, M., Kawa, A. B., & Robinson, T. E. (2017). Are cocaine-seeking “habits” necessary for the development of addiction-like behavior in rats? Journal of Neuroscience, 38(1), 60–73.Find this resource:
Sjoerds, Z., de Wit, S., van den Brink, W., Robbins, T. W., Beekman, A. T., Penninx, B. W., & Veltman, D. J. (2013). Behavioral and neuroimaging evidence for overreliance on habit learning in alcohol-dependent patients. Translational Psychiatry, 3, e337.Find this resource:
Sjoerds, Z., Dietrich, A., Deserno, L., de Wit, S., Villringer, A., Heinze, H.-J., . . . Horstmann, A. (2016). Slips of action and sequential decisions: A cross-validation study of tasks assessing habitual and goal-directed action control. Frontiers in Behavioral Neuroscience, 10(December), 1–15.Find this resource:
Smith, K. S., & Graybiel, A. M. (2013). A dual operator view of habitual behavior reflecting cortical and striatal dynamics. Neuron, 79(2), 361–374.Find this resource:
Stippekohl, B., Winkler, M., Mucha, R. F., Pauli, P., Walter, B., Vaitl, D., & Stark, R. (2010). Neural responses to BEGIN-and END-stimuli of the smoking ritual in nonsmokers, nondeprived smokers, and deprived smokers. Neuropsychopharmacology, 35(5), 1209–1225.Find this resource:
Stock, A. K. (2017). Barking up the wrong tree: Why and how we may need to revise alcohol addiction therapy. Frontiers in Psychology, 8(May), 1–6.Find this resource:
Sun, R., & Giles, C. L. (2001). Sequence learning: Paradigms, algorithms, and applications. Berlin, Germany: Springer.Find this resource:
Symonds, R. M., Lee, W. W., Kohn, A., Schwartz, O., Witkowski, S., & Sussman, E. S. (2017). Distinguishing neural adaptation and predictive coding hypotheses in auditory change detection. Brain Topography, 30(1), 136–148.Find this resource:
Tecuapetla, F., Jin, X., Lima, S. Q., & Costa, R. M. (2016). Complementary contributions of striatal projection pathways to action initiation and execution. Cell, 166(3), 703–715.Find this resource:
Thrailkill, E. A., & Bouton, M. E. (2015). Extinction of chained instrumental behaviors: Effects of procurement extinction on consumption responding. Journal of Experimental Psychology: Animal Learning and Cognition, 41(3), 232–246.Find this resource:
Thrailkill, E. A., & Bouton, M. E. (2016). Extinction and the associative structure of heterogeneous instrumental chains. Neurobiology of Learning and Memory, 133, 61–68.Find this resource:
Thrailkill, E. A., & Bouton, M. E. (2017). Effects of outcome devaluation on instrumental behaviors in a discriminated heterogeneous chain. Journal of Experimental Psychology: Animal Learning and Cognition, 43(1), 88–95.Find this resource:
Tricomi, E., Balleine, B. W., & O’Doherty, J. P. (2009). A specific role for posterior dorsolateral striatum in human habit learning. European Journal of Neuroscience, 29(11), 2225–2232.Find this resource:
Troisi, J. R. (2013). Acquisition, extinction, recovery, and reversal of different response sequences under conditional control by nicotine in rats. Journal of General Psychology, 140(3), 187–203.Find this resource:
Turk-Browne, N. B., Isola, P. J., Scholl, B. J., & Treat, T. A. (2008). Multidimensional Visual Statistical Learning. Journal of Experimental Psychology: Learning Memory and Cognition, 34(2), 399–407.Find this resource:
Turk-Browne, N. B., Scholl, B. J., Chun, M. M., & Johnson, M. K. (2009). Neural evidence of statistical learning: Efficient detection of visual regularities without awareness. Journal of Cognitive Neuroscience, 21(10), 1934–1945.Find this resource:
Valentin, V. V., Dickinson, A., & O’Doherty, J. P. (2007). Determining the neural substrates of goal-directed learning in the human brain. Journal of Neuroscience, 27(15), 4019–4026.Find this resource:
Van Essen, D. C., Drury, H. A., Dickson, J., Harwell, J., Hanlon, D., & Anderson, C. H. (2001). An integrated software suite for surface-based analyses of cerebral cortex. Journal of the American Medical Informatics Association, 8(5), 443–459.Find this resource:
Vandaele, Y., Pribut, H. J., & Janak, P. H. (2017). Lever insertion as a salient stimulus promoting insensitivity to outcome devaluation. Frontiers in Integrative Neuroscience, 11(September), 1–13.Find this resource:
Vanderschuren, L. J. M. J., & Everitt, B. J. (2004). Drug seeking becomes compulsive after prolonged cocaine self-administration. Science, 305(5686), 1017–1019.Find this resource:
Verwey, W. B. (2001). Concatenating familiar movement sequences: The versatile cognitive processor. Acta Psychologica, 106(1–2), 69–95.Find this resource:
Virag, M., Janacsek, K., Horvath, A., Bujdoso, Z., Fabo, D., & Nemeth, D. (2015). Competition between frontal lobe functions and implicit sequence learning: Evidence from the long-term effects of alcohol. Experimental Brain Research, 233(7), 2081–2089.Find this resource:
Voon, V., Derbyshire, K., Ruck, C., Irvine, M. A., Worbe, Y., Enander, J., . . . Bullmore, E. T. (2015). Disorders of compulsivity: A common bias towards learning habits. Molecular Psychiatry, 20, 345–352.Find this resource:
Wang, L., Uhrig, L., Jarraya, B., & Dehaene, S. (2015). Representation of numerical and sequential patterns in macaque and human brains. Current Biology, 25(15), 1966–1974.Find this resource:
Warden, M. R., & Miller, E. K. (2010). Task-dependent changes in short-term memory in the prefrontal cortex. Journal of Neuroscience, 30(47), 15801–15810.Find this resource:
Wassum, K. M., Ostlund, S. B., & Maidment, N. T. (2012). Phasic mesolimbic dopamine signaling precedes and predicts performance of a self-initiated action sequence task. Biological Psychiatry, 71(10), 846–854.Find this resource:
Willingham, D. B. (1999). Implicit motor sequence learning is not purely perceptual. Memory and Cognition, 27(3), 561–572.Find this resource:
Willingham, D. B., Nissen, M. J., & Bullemer, P. (1989). On the development of procedural knowledge. Journal of Experimental Psychology: Learning, Memory, and Cognition, 15(6), 1047–1060.Find this resource:
Wilson, B., Kikuchi, Y., Sun, L., Hunter, D., Dick, F., Smith, K., . . . Petkov, C. I. (2015). Auditory sequence processing reveals evolutionarily conserved regions of frontal cortex in macaques and humans. Nature Communications, 6, 8901.Find this resource:
Woicik, P. A., Urban, C., Alia-Klein, N., Henry, A., Maloney, T., Telang, F., . . . Goldstein, R. Z. (2011). A pattern of perseveration in cocaine addiction may reveal neurocognitive processes implicit in the Wisconsin Card Sorting Test. Neuropsychologia, 49(7), 1660–1669.Find this resource:
Wunderlich, K., Dayan, P., & Dolan, R. J. (2012). Mapping value based planning and extensively trained choice in the human brain. Nature Neuroscience, 15(5), 786–791.Find this resource:
Xue, Y. X., Luo, Y. X., Wu, P., Shi, H. S., Xue, L. F., Chen, C., . . . Lu, L. (2012). A memory retrieval-extinction procedure to prevent drug craving and relapse. Science (New York, NY), 336(6078), 241–245.Find this resource:
Zapata, A., Minney, V. L., & Shippenberg, T. S. (2010). Shift from goal-directed to habitual cocaine seeking after prolonged experience in rats. Journal of Neuroscience, 30(46), 15457–15463.Find this resource:
Zhang, K., Chen, C. D., & Monosov, I. E. (2018). Novelty, salience, and surprise timing are signaled by neurons in the basal forebrain. Current Biology, 29(1), 134–142.Find this resource:
Zhao, J., Al-Aidroos, N., & Turk-Browne, N. B. (2013). Attention is spontaneously biased toward regularities. Psychological Science, 24(5), 667–77.Find this resource: