Classical Models of Decisions

Our daily activities involve many kinds of decision-making tasks. In uncertain situations, we are usually slow and hesitant to make up our mind, as opposed to straightforward situations in which we are fast and decide with confidence. A simple model introduced decades ago in the psychology/behavior literature has been very successful in explaining this basic observation in many various contexts. The central idea of the “accumulation-to-bound” model is that during deliberation, the brain accumulates evidence for or against different possible choices until the total accumulated evidence reaches some threshold value, at which point the decision is made. This mechanism is attractive in at least three ways: It is simple as it includes a limited number of parameters, it relates decision speed and accuracy for both correct and error trials, and it efficiently deals with noise in sensory information. However, despite accurately accounting for human and nonhuman decision-making behavior in a variety of tasks, classic accumulation models also have limits in their ability to capture important aspects of natural behavior.

Here, a few of these limitations are discussed, and the proposals that have been made to modify current models, as well as the empirical data providing evidence for or against these proposals, are reviewed. The discussion begins with a brief review of some of the mathematical foundations for expressing the questions at hand and the answers that have been proposed.

Mathematical Foundations of Accumulation-to-Bound Models

Suppose you are faced with a decision between two actions $A$ and $B$, exactly one of which is the correct choice, and you want your error rate to be no higher than some fraction $\alpha$. For example, if you want to be 95% accurate, then $\alpha =0.05$. If $p\left(X\right)$ denotes the probability that action $X$ is the correct choice, then you want to choose action $A$ if $p\left(A\right)>1-\alpha$, and action $B$ if $p\left(B\right)>1-\alpha$, and otherwise you want to wait to see if you can gather information to improve your chances. Another way of expressing this is to say that you choose $A$ if

and choose $B$ if

and otherwise you wait and try to gather more information.

Gathering information can be thought of as updating your estimate of the probability that an action is correct given some samples of gathered information. For instance, after observing a sample $s_1$, you can replace $p\left(A\right)$ with $p\left(A|s_1\right)$ (the probability of $A$ given $s_1$) and $p\left(B\right)$ with $p\left(B|s_1\right)$. This means that the policy for choosing $A$ is now written as

To calculate the left-hand side, we can use Bayes’ rule, which is defined as follows:

This gives us

and with some cancellation, we can rewrite our policy as

Next, if you observe another sample $s_2$, which is independent of the first sample, then you can apply Bayes’ rule again and this becomes

This complex product can be awkward to work with, but it can be turned into a summation by taking the logarithm, yielding

Thus, in general we can express the policy for choosing $A$ as calculating the sum of the logarithm of the ratio of prior probabilities plus the sum of the log-likelihood ratio (logLR) of each of the independent samples of evidence observed so far and comparing that to a criterion that depends on our desired level of accuracy. Therefore, we can define a “decision variable” $x_A$ for choosing $A$ as

We do the same for choice $B$ and then update these variables using samples of evidence until one of them crosses the criterion $C=log\frac{1-\alpha }{\alpha }$. This is known as the “sequential probability ratio test” (Wald, 1945), and it minimizes the number of samples needed to reach the desired level of accuracy (Wald & Wolfowitz, 1948). In continuous form, it is equivalent to the integration of sensory evidence over time up to a constant threshold related to desired accuracy, starting from a point that is related to prior probabilities. This is called the drift–diffusion model (DDM) (Bogacz et al., 2006; Gold & Shadlen, 2007; Ratcliff, 1978; Ratcliff & McKoon, 2008; Ratcliff et al., 2016; Schall, 2019).

The DDM usually assumes that the accumulation process is noisy, not just because of variations in the sensory information but also because of internal noise such as neural activity fluctuation. It can be written as the following differential equation:

where $x_A$ is the decision variable for choice $A$. The first term is the evidence signal $E_A\left(t\right)$, which can be defined as the logLR provided by the stimulus at time $t$, multiplied by a gain $a$. The second term implements a white noise process with mean $0$ and variance $c^2\mathrm{dt}$. The decision variable starts at an initial level related to prior information in favor of $A$ and grows to a constant threshold at a rate determined by the gain and the evidence in favor of choice $A$ over choice $B$. Equation 2 describes the decision variable $x_A$ for choice $A$, but a similar variable $x_B$ can be defined for choice $B$. Because the evidence term is always expressed as the evidence for a given choice and against the other (e.g., the logLR for $B$ will be the negative of that for $A$), these two variables behave like mirror images of each other, and the choice is determined by which reaches its threshold first. However, because of the noise, the decision variable will perform a “random walk” that does not always reach the threshold at the same time for the same strength of evidence, but instead produces a distribution of reaction times (RTs). Furthermore, if the evidence is weak, randomness can sometimes cause the system to make the wrong decision. In addition to the white noise modifying the intratrial rate of build-up, the model can also simulate intertrial variability in the gain parameter $a$. This latter type of variability could be due to changes in arousal or attention, and several analyses have suggested that it is the major source of RT variability in decision behavior (Carpenter & Reddi, 2001).

Figure 1 shows some of the implications of Equation 2 for the kinds of RT distributions that are observed with different levels of evidence and different levels of prior information. Note that the model predicts not only that RTs will be longer in trials with weaker evidence but also that their distribution will be broader. It also predicts that anticipation will reduce RTs and narrow the distribution. All of these effects have been shown in a very wide range of studies, with different kinds of stimuli, different types of responses, and different species of animals (Gold & Shadlen, 2007; Ratcliff & McKoon, 2008; Ratcliff et al., 2016; Schall, 2019).

Open in new tab

Figure 1. Schematic illustration of the reaction time distributions produced by the drift–diffusion model. (A) Comparison of reaction time distributions produced by the build-up of a decision variable to a threshold (red dashed horizontal line) during easy trials (green) and hard trials (red). In each case, three example trials are shown along with the average (black arrows). The variability of the build-up is subject to the same level of noise (purple arrows), which causes the time of threshold crossing to broaden over time. Because the threshold is reached later in hard trials, the distribution is both later and broader. (B) Comparison of reaction time distributions in hard trials that are anticipated (green) or not (red). Although the build-up slope is the same, the anticipated response is faster because the build-up starts at a higher level.

One widely used task summarizes many of these phenomena. It is called the random-dot motion discrimination task (Britten et al., 1992), and it involves subjects looking at a field of randomly moving dots, some small percentage of which are moving coherently to the left or right. The subject’s task is to detect that coherent motion and indicate its direction (with a button press, saccadic eye movement, or other type of response). Several variations of the task exist (Roitman & Shadlen, 2002). In the “reaction time” version, subjects can respond as soon as they think they know the correct answer—this makes it possible to study how RT distributions change as a function of coherence, prior information, rewards and penalties, etc. In the “fixed duration” version, subjects must wait for an external “GO” signal that tells them when to respond—this makes it possible to study how accuracy changes as a function of observation time.

Most importantly, many studies have recorded neural activity in the brain of monkeys performing the random-dot motion discrimination task and reporting their decisions with a saccadic eye movement. These have yielded a large body of results supporting many of the predictions of the DDM. In particular, early studies showed that neural activity in the medial temporal area (MT) reflects the strength of the motion signal presented to the monkey and predicts the monkey’s choices and accuracy (Britten et al., 1992). In contrast, activity in the lateral intraparietal area (LIP) grows at a rate related to that motion signal, as if it implements the integration of sensory evidence (Roitman & Shadlen, 2002; Shadlen & Newsome, 2001). Similar build-up of neural activity during motion viewing has been observed in the prefrontal cortex (Kim & Shadlen, 1999), the basal ganglia (Ding & Gold, 2010, 2013), and the superior colliculus (Horwitz & Newsome, 1999), implicating all of these regions in aspects of the evidence integration process. Furthermore, microstimulation in MT acts like an increase in sensory evidence (Salzman et al., 1992), whereas microstimulation in LIP acts like an increase in the integral (Hanks et al., 2006), consistent with a model in which MT signals are integrated by LIP (Mazurek et al., 2003).

Neuroimaging studies in humans have shown similar results in many of the same regions (Donner et al., 2009; Heekeren et al., 2008; Tosoni et al., 2008), leading to the widespread conclusion that the DDM accurately reflects the process by which the brain makes decisions over time. In particular, similar phenomena are seen in different kinds of tasks with different kinds of stimuli, such as brightness (Ratcliff, 2002; Ratcliff et al., 2007, 2011), letter (Ratcliff & Rouder, 2000), contrast and orientation discrimination tasks (Smith et al., 2004), decision tasks involving faces and objects (Ratcliff et al., 2009), decisions about vibrotactile frequencies (Romo et al., 2002, 2004), olfactory (Bowman et al., 2012) and auditory stimuli (Brunton et al., 2013; Piet et al., 2018), spatial distance discriminations (Ratcliff et al., 2003), and economic choices (Krajbich & Rangel, 2011). Indeed, the support for the DDM has been so common that currently when neural activity is seen to rise over time, it is often interpreted as the accumulation of evidence to a decision threshold.

Beyond low-level perceptual tasks, DDMs have been applied to explain data in item recognition (Ratcliff, Thapar, et al., 2004), lexical decisions (Ratcliff, Gomez, et al., 2004), numeracy judgments (Ratcliff et al., 2015), and text processing and priming tasks (McKoon & Ratcliff, 2012). DDM-based analyses are used to study working memory (Schmiedek et al., 2007), sleep deprivation (Ratcliff & Van Dongen, 2009), and intelligence (Ratcliff et al., 2010) and to assess deficits such as aphasia (Ratcliff, Perea, et al., 2004), dyslexia (Zeguers et al., 2011) and attention-deficit/hyperactivity disorder (Weigard & Huang-Pollock, 2014).

In summary, the DDM and other accumulation-to-bound models have been broadly applied to explain a wide variety of phenomena and have stimulated a great deal of informative research, both theoretical and empirical. However, like any model, they have limitations. The following sections discuss some of these limitations and the kinds of modifications they motivate.

What if the World Changes?

In natural behavior, we rarely confront an entirely static world, and the decisions we make are not merely about classifying objects around us but also about deciding how to interact with them. That decision must be both fast and accurate, and it must also be flexible if the situation changes. However, accumulators are always sluggish to respond to changes in the world. For example, if 500 ms was spent integrating toward the threshold for choice $A$, then if the world suddenly changes to favor choice $B$, it will take another 500 ms just to return to the initial state before starting to integrate toward the threshold for $B$. Such a mechanism seems poorly adapted to real-world situations, such as crossing a road that appeared safe and suddenly seeing a scooter coming at you at high speed.

One approach for dealing with changing information might be to reset the decision variable to its initial value or to increase the gain of integration whenever a change occurs, but this raises the question of what mechanism detects that change. A simpler approach is to have a leaky integrator (Usher & McClelland, 2001) with a time constant that is adapted to the volatility of the environment (Ghose, 2006; Ossmy et al., 2013; Veliz-Cuba et al., 2016). This means we can replace Equation 2 with

where $\tau$ is the time constant (the duration of time it takes for the integrator to reach approximately 63% of its asymptotic value). With a short time constant, this system will react quickly to changes in the evidence signal and reach a stable level, on average equal to

However, this implies that evidence accumulation stops quickly and is actually much shorter than the build-up of neural activity recorded during decisions and inferred from RT distributions (Ghose, 2006; Ludwig et al., 2005; Uchida et al., 2006). So then how could one set a single threshold high enough to prevent excessive errors and yet low enough to be reached even in trials with weak evidence? This brings us to the second question.

What Constitutes Evidence?

In the original work on the DDM, Ratcliff (1978) defined evidence as “information representing the goodness of match, over time” (p. 63) in the context of comparing a probe to an item in memory. In a more recent and highly influential review, Gold and Shadlen (2007) defined it as “information that bears on whether (and possibly when) to commit to a particular hypothesis.” (p. 536) Although it is not often explicitly stated, the implementation of the mathematical models stemming from these statements implies that during perceptual discrimination, the “evidence” is the information present at any moment within the stimulus—the term $E_A\left(t\right)$ in the previous equations. For example, during random-dot motion discrimination, it would be the difference in the motion strength to the left versus the right, which is present between every two frames of the video display.

However, is that really what one should accumulate? Let’s return to the derivation of the diffusion model and recall our assumption that the individual samples $s_k$ are statistically independent. In most perceptual discrimination tasks, that assumption does not hold. In fact, later samples are conditionally dependent on previous samples. For example, suppose you are sneaking glances at a person and trying to decide if the person is someone you know. The first glance gives you a lot of information. The second gives some more. But after the seventh or eighth glance, you are not actually getting any more relevant evidence because the information obtained during that glance is already predicted by previous glances you have taken. You are just looking at the same face over and over again. In mathematical terms, what this means is that Equation 1 must take the redundancy between the samples into account, using an expansion of Bayes’ rule to multiple variables.

Let’s consider the steps again. We start with the log ratio of the priors and add the first sample of evidence, which is obviously not redundant because there were no previous samples, so following Bayes’ rule we simply add its logLR:

Now let’s consider what happens when a second sample is presented. Adding the evidence from the second sample is more complicated than the logLR. Let’s go back to expressing $x$ as the logarithm of the ratio of the probabilities of $A$ and $B$ given the evidence presented:

To properly compute this, we need to use Bayes’ rule for three variables, or

which yields the following:

Consequently, our expression for $x$ after two samples is the following:

Note that the third term, which introduces the evidence from the second sample, is not just the logLR of $s_2$ given $A$ versus $B$ but also given $s_1$. This is more complicated than the simple logLR. From the definition of conditional probabilities, it can be written as

where the first factor is the logLR and the second is inversely related to the mutual information between the samples.

Now, let’s consider two extreme cases. First, suppose that the samples are in fact statistically independent. From the definition of independence, we know that $p\left(s_1,s_2|A\right)=p\left(s_1|A\right)p\left(s_2|A\right)$, so the second factor in the numerator and denominator of Equation 5 is equal to 1, and so we just add the logLR factor, as before.

In contrast, suppose the second sample is completely redundant and fully predicted by the first sample. This means that $p\left(s_1,s_2|A\right)=p\left(s_1|A\right)$, so now the entire Equation 5 collapses to $log\frac{1}{1}=0$. In other words, the redundant sample should be completely ignored. In the intermediate cases in which the first sample provides partial but incomplete information about the second, the amount that should be accumulated depends on the mutual information between the samples—as that mutual information increases, the difference between the numerator and the denominator in the logarithm on the right-hand side of Equation 5 goes to zero, and so does the entire logarithm.

The previous procedure generalizes to additional samples by extending Bayes’ rule to additional variables, yielding the following expression for the decision variable:

Equation 6 has important implications for the kinds of experiments discussed previously and the kinds of models used to explain the resulting data. In any task with constant evidence and without stimulus noise, the only informative sample is the first one, so the temporal profile of the decision variable should be a simple step function. In tasks with a constant evidence signal buried in noise, such as random-dot motion discrimination, each successive sample will be progressively less informative so the profile will be a decelerating rise to a stable asymptote (Figure 2). In fact, it will reach a level related to the strength of evidence present in the stimulus, rising with a decelerating time course that looks very much like a leaky integrator with a relatively short time constant, estimated in empirical studies to be on the order of 100 ms (Ghose, 2006; Ludwig et al., 2005; Uchida et al., 2006). But, again, if the decision variable reaches a stable asymptote in such tasks, then how can one set a threshold that will prevent excessive errors and yet still be reached even in trials with low evidence? Furthermore, if the decision variable quickly asymptotes, then how can it be reconciled with the numerous observations that neural activity continues to rise during decision-making tasks for a significant period of time, up to 800 or 1,000 ms (Roitman & Shadlen, 2002)? Before addressing that dilemma, let us consider the third question.

Open in new tab

Figure 2. Decreasing rate of build-up in tasks with partially redundant information. Each green step illustrates the novel information contributed by an individual sample. Because this decreases over time, the total accumulated information (black lines) slows down, resembling a low-pass filter or “leaky” integrator. Note that although this may not be a problem for very easy trials, in hard trials the accumulated information may never reach the threshold.

What Should One Seek to Optimize?

The sequential probability ratio test and the DDM are meant to minimize the time necessary to reach a given level of desired accuracy. In statistics, desired accuracy is set by convention (e.g., $p<0.05$ or $p<0.01$) and is not open to negotiation. In the natural world, however, animals are not obliged to live up to such high standards and have the freedom to occasionally take a guess. Consider an animal deciding about whether or not to approach a watering hole after having reached 93% certainty that no predator is hiding in the bushes. Should it wait until reaching 95% certainty, even if it would take an hour? What if it would take a year? Obviously, what animals must optimize during natural behavior is not simply time or expected rewards but, rather, a balanced compromise between the two: reward rate. Indeed, many studies have shown that both animals and humans sacrifice accuracy in favor of maximizing reward rates (Balci et al., 2011; Uchida et al., 2006).

A simple expression for reward rate is

where $U$ is the subjective utility of a favorable outcome, $p\left(t\right)$ is the probability of obtaining that outcome after deliberating for time $t,C$ is the cost of trying (including metabolic costs as well as the opportunity cost of not doing something else), $m$ is the time spent moving, and $d$ is the delay before one can try again. This expression is related to a time-discounted expected utility of a single decision and is similar to harvest intake in foraging theory (Charnov, 1976). Importantly, it can be analytically shown that in a wide range of situations, the best way to maximize Equation 7 is to have a decision criterion that decreases over time (Thura et al., 2012).

Briefly, for almost any reasonable decision-making process, the probability of making the correct choice that yields the favorable outcome will increase over time but with decreasing slope, eventually reaching an asymptote (if only because one can never be more than 100% accurate). This means that the first derivative of $p\left(t\right)$ is positive and the second derivative is negative. These two assumptions hold regardless of the mechanism for estimating the evidence (DDM, leaky integrator, etc.) and make it possible to find the peak of the reward rate function (Equation 7) by solving for when the first derivative of the reward rate is zero and the second is negative. With a bit of algebra, this yields the following policy:

An analytical solution to a simplified version of Equation 8 is provided by Thura et al. (2012). Here, we demonstrate a geometrical solution, shown in Figure 3. First, we find the time $t$ that corresponds to the peak reward rate, for a given trial, by drawing a tangent line from the point $\left(-\left(m+d\right),\frac{C}{U}\right)$ to the function $p\left(t\right)$, as shown in Figure 3A. This can be done for a set of trials, each with a different function $p\left(t\right)$, to delimit a “deliberation region” in the $(t,p)$ plane where one should continue to deliberate (Figure 3B). The edge of that deliberation region is an “accuracy criterion” that, when crossed, should lead one to commit to a choice in favor of their current best guess. Importantly, Figures 3C and 3D demonstrate that in almost all cases the accuracy criterion will be dropping over time, in a manner related to both the cost:benefit ratio $\frac{C}{U}$ and the time taken to move $m$ and the intertrial interval $d$. This expresses the following intuitive policy: If you are confident, make your choice; if not, think some more, but if time is running out, go with your current best guess. Importantly, this addresses the dilemma, posed previously, regarding how the decision variable can reach the accuracy criterion if it stops growing: The answer is that it is the accuracy criterion that drops to meet the decision variable.

Open in new tab

Figure 3. A geometric interpretation of equation 8 and its implications. (A) You can find the time $t$ that satisfies the equation for a single trial by drawing a tangent line from the point $(-(m+d),C/U)$ to the function $p\left(t\right)$. Note that the value of $p\left(t\right)$ at that time (blue dashed line) is equal to the sum of the two terms of equation 8. (B) The solutions for a family of trials, each with a different function $p\left(t\right)$, define a deliberation region (yellow shaded area) and a dropping accuracy criterion (dashed red line). (C) The accuracy criterion is higher (dashed purple line) for a condition with higher cost $C$. (D) The accuracy criterion is lower (dashed blue line) for a condition with shorter intertrial interval $d$.

The idea of the dropping accuracy criterion, or “collapsing bound,” has been discussed in several recent modeling and empirical studies (Chandrasekaran et al., 2017; Cisek et al., 2009; Drugowitsch et al., 2012; Thura et al., 2012, 2014). In particular, several studies have shown that the level of evidence at the time a decision is made is inversely related to the time taken to decide (Cisek et al., 2009; Gluth et al., 2012; Murphy et al., 2016; Thura et al., 2012) and that one’s confidence in one’s choice is also lower as more time is taken to decide (Kiani & Shadlen, 2009; Van den Berg et al., 2016). At the neural level, however, there is little evidence of a decreasing threshold for commitment. Instead, the level of activity prior to a choice appears to be remarkably stable across trials and task conditions (Gold & Shadlen, 2007; Hanes & Schall, 1996; Schall, 2019; but see Heitz & Schall, 2012). For this reason, most models of neural activity implement decreasing accuracy criteria by combining the decision variable with a rising, nonspecific “urgency signal,” and it is the two together that bring neural activity to a stable threshold for commitment (Cisek et al., 2009; O’Connell et al., 2018; Standage et al., 2011). Some of these models retain the assumption of a long time constant of integration that accumulates momentary evidence into a growing decision variable (Churchland et al., 2008; Drugowitsch et al., 2012), whereas others assume a much shorter time constant that produces a saturating decision variable (Cisek et al., 2009; Thura et al., 2012) so that it is urgency, not evidence accumulation, that is primarily responsible for neural activity build-up.

Taken together, the previous issues lead one to consider two potential modifications to the classic DDM. First, instead of accumulating all the evidence present at each moment in time, it is possible that the brain only accumulates novel evidence, which in constant signal tasks will resemble a leaky integrator with a short time constant. Second, instead of a constant criterion of accuracy, the brain might use a dropping criterion that is implemented by combining the decision variable with a rising urgency signal. Importantly, if both of these modifications are made, then the resulting model addresses the first question posed previously: What happens if the world changes? Because a system that emphasizes novel evidence effectively acts like a leaky integrator with a short time constant, it will respond to changes quickly. However, it will still always make a decision because, eventually, the dropping accuracy criterion will force it to take a best guess. We have called this the urgency-gating model (UGM) (Cisek et al., 2009; Thura et al., 2012), and it can be described with the following set of equations:

The first equation defines a leaky integrator, as in Equation 3. The second combines this with an urgency signal $U\left(t\right)$, which is defined in the third equation as a simple linear function with slope $m$ and intercept $b$. The variable $x_A$ is the “decision variable” that keeps track of the evidence in favor of choice $A$, the variable $y_A$ combines this with the urgency signal, and it is the latter that is compared to a constant threshold for committing to a choice. In principle, adjustments of the speed–accuracy trade-off could equivalently be accomplished by changing the threshold or by changing $m$ and $b$, but as discussed previously, neural data strongly favor the latter.

Note that the UGM is effectively equivalent to what Ditterich (2006a) proposed several years earlier as an integrator with a time-varying gain and a limited time constant. Although his analyses did not determine the setting of the time constant, the data we review later suggest that it is short, on the order of 100–200 ms. Such a short time constant has important implications for interpreting data. First, it implies that accumulation is in fact quite leaky and that decisions are most strongly based on the sensory evidence closest to but just before the moment of commitment. Second, with such a short time constant, neural activity will resemble a quickly saturating exponential and not the kind of linear ramping that has so often been observed in single-unit spiking activity or functional imaging experiments. Therefore, we propose that such ramping activity is not related to the integration of evidence, as so often assumed, but is primarily caused by the growing urgency signal. Importantly, that ramping activity could be related to and combine a number of potentially distinct components, such as the rising urge to commit, the growing preparation to act, and the anticipation of approaching events (Costello et al., 2013; Huk et al., 2017).

Attractor Models

An additional issue, orthogonal to those raised previously, concerns the actual neural mechanisms and computations involved in processing sensory information, combining it with a putative urgency signal, and determining the moment of commitment. Accumulation-to-bound models, whether leaky or not, whether with or without urgency, are abstract mathematical concepts, and few theorists would suggest that the brain implements them by literally having one neuron that accumulates evidence for one choice, another that accumulates for another, and a third that compares their output to some threshold. Instead, it is more likely that all of these processes emerge through interactions among large populations of neurons, each of which contributes to the overall behavior of a dynamical system that progresses from a state of deliberation to a state of commitment (Thura et al., 2020). Such models have been proposed for many decades, often based on the concept of dynamical “attractors” (Amari, 1977; Grossberg, 1973; Wang, 2002). These models have the advantage that they can more readily incorporate features of the real biology, such as different types of neurotransmitter receptors and distinct classes of excitatory and inhibitory neurons.

A useful analogy to understand this type of model is to think of a ball moving across a landscape consisting of valleys separated by flat plains. If the ball is on a flat part of the landscape, then it can be easily swayed to move in one direction or another by gusts of wind—this is analogous to deliberation, where the wind is the sensory evidence. At some point, the ball will fall into a specific valley into which it becomes trapped—this is analogous to commitment to a specific action. Although this is simply an analogy, the dynamics are mathematically similar to what happens in a network of mutually inhibiting cells. For example, Grossberg (1973) showed that if the inhibition between cells is shallow, the system behaves like a ball on a flat landscape that remains sensitive to the input pattern and tracks its changing state. However, if the inhibition is steep, the system behaves like a ball falling into a deep well from which it cannot escape—like a winner-take-all process of committing to one choice above others. Importantly, gradual changes of nonspecific input to all cells in the system can modulate the effective steepness of the interactions, shifting the system from tracking input patters to a system that behaves in a winner-take-all manner. Similarly, an influential model proposed by Wang (2002) consisted of pools of spiking neurons in which the network connectivity is tuned to achieve a flat landscape around the initial state, such that premature convergence to committing point attractors is avoided and the ball’s motion is initially mostly determined by the inputs. This initial phase corresponds to the accumulation of evidence. Eventually, the ball drops into either of two attractor states—that is, valleys that are difficult to escape from and correspond to the commitment to a choice. Similar models can be generalized beyond two-alternative tasks by using entire distributed populations of interacting neurons (Cisek, 2006; Erlhagen & Schöner, 2002; Furman & Wang, 2008).

Attractor network models thus simulate the two stages of the decision-making process but with less clear-cut boundaries between these stages. There is no explicit threshold to which activity must be compared, but instead a transition from a state of flexibility to a state of stable commitment. In contrast to the one-dimensional classic accumulator or leaky models, the dynamics of attractor networks are two-dimensional and can capture additional subtleties of the data. For example, like real neurons, they exhibit an initial unselective ramp up at target presentation and the later separation of activity into selected and unselected directions during motion discrimination. The classic accumulator models and attractor models also make some distinct predictions at the behavioral level. For instance, the attractor model of Wang (2002) produces longer response times in error trials than in correct trials, consistent with most experimental data. Such models also correctly predict that the influence of newly arriving inputs diminishes over time, as the network converges toward one of the attractor states representing the alternative choices. This is in agreement with studies that showed that the impact of a brief motion pulse in addition to the random-dot stimulus was greater with an earlier onset time (Huk & Shadlen, 2005; Wong et al., 2007).

Attractor models can also account for speed–accuracy adjustments during decision-making, depending on the contextual information provided to the subjects (for review, see Standage et al., 2014). Increasing the strength of recurrent dynamics shortens the effective time constant of the model (Standage et al., 2011; Wong & Wang, 2006), so the decision variable builds up more quickly, limiting the amount of integrated evidence. Decisions are consequently faster and less accurate. Lengthening and shortening the effective time constant of a decision circuit thus offers a principle for trading speed against accuracy, but it requires a way to increase and decrease the strength of recurrent dynamics under speed and accuracy conditions.

One example of network dynamics modulation comes from Standage et al. (2011), who used an urgency signal to differentially modulate decision dynamics under speed and accuracy conditions. The timing signal was an increasing function of time (for an example with a time-invariant signal, see Furman & Wang, 2008), building up more quickly with tighter temporal constraints and reaching a fixed maximum. The signal scaled the slope parameter of the interaction function, which in turn controlled the dynamics of the network as discussed previously in the context of Grossberg’s (1973) work. Consequently, network dynamics were weak at the start of each trial and were strengthened with elapsing time. This progression lengthened the time constant of the network prior to entry into the decision regime and then shortened it. Thus, decisions could thus be slower (faster) and more (less) accurate, depending on the speed–accuracy trade-off context.

Nevertheless, although these attractor models are more biologically realistic than the purely abstract formalisms discussed previously, and ultimately more promising as unifying theories of brain and behavior, we do not discuss them here in great depth. Instead, we return to the more abstract questions of the underlying computations, with the aim of distinguishing between different ways of integrating sensory evidence (short versus long time constants) and between different policies for determining when to commit (constant thresholds versus collapsing bounds).

Tasks With Changing Evidence

Several recent experiments have examined human and animal behavior during tasks with changing evidence. For example, many of our studies in both humans and nonhuman primates have used what we call the “tokens task”, in which subjects must guess which of two peripheral targets will receive the majority of 15 small tokens that randomly jump from the center every 200 ms but can decide at any time, thereby setting their own subjective criteria for committing to a choice (Cisek et al., 2009; Thura & Cisek, 2014; Thura et al., 2014). We can vary these criteria, encouraging subjects to be hastier in “fast blocks,” in which after the decision is made the tokens accelerate significantly to jump every 50 ms, than in “slow blocks,” in which tokens accelerate less to jump every 150 ms. Because the task allows us to precisely calculate the information in the stimulus at each moment, we can infer the level of confidence at the time of each choice. We found that it was decreasing over time, consistent with a rising urgency. Furthermore, commitment only depended on the token state at the time of commitment, and not on the profile of the token state earlier in the trial, consistent with a short time constant. In particular, decisions made in trials that contained a transient bias toward the chosen target were no faster than decisions made in trials with a transient bias toward the other target. In other words, subjects were only integrating the novel information provided by each token jump, and not the information, continuously present in the stimulus at each moment in time, about the evidence for or against a target.

To test the UGM at the neural level, we first recorded the spiking activity of individual neurons in monkey dorsal premotor (PMd) and primary motor cortex (M1) (Thura & Cisek, 2014, 2016a), two key nodes in the network controlling the selection and execution of reaching movements. In both regions, neurons active during the deliberation process exhibited activity patterns that clearly reflected how the sensory evidence provided by the tokens unfolded over time in different types of trials (Figure 5A, top rows). Critically, the sensory information was not integrated but instead low-pass-filtered with a short time constant (<200 ms). Furthermore, in addition to the sensory evidence, these same PMd and M1 neurons were also modulated by a signal that grew over time in exactly the block-dependent manner as the predicted urgency signal (Figure 5B). Finally, approximately 280 ms before movement onset, these same neurons reached approximately the same fixed firing rate threshold regardless of evidence or urgency. We then recorded the activity of neurons in the basal ganglia (BG), a set of subcortical nuclei long suspected of being involved in action selection (Mink, 1996; Redgrave et al., 1999), while monkeys performed the same tokens task (Thura & Cisek, 2017). We focused on the main output nucleus, the globus pallidus (GP), including both the external (GPe) and internal (GPi) segments. In contrast to the activity patterns we observed in PMd and M1, the evolution of changing evidence was only weakly reflected in the activity of GPe neurons and was effectively absent in GPi, the final output structure (Figure 5A, bottom rows). Instead, many neurons in both GPe and GPi exhibited time-dependent activities, either building up or decreasing as a function of time during deliberation (Figure 5C). Crucially, these time-dependent activity levels were also strongly modulated by the speed–accuracy trade-off condition in which the task was being performed: “Build-up” neurons were usually more active during fast blocks than in slow blocks, whereas “decreasing” cells showed the opposite pattern. Thus, the output activity of the BG appears to reflect the urgency signal as well as its adjustment across different speed–accuracy regimes. Taken together, our data suggest that during deliberation, cortical activity reflects a dynamic, biased competition between candidate actions, which is gradually amplified by an urgency signal from the BG that effectively determines the amount of evidence needed before the animal commits to the currently favored action target.

Open in new tab

Figure 5. Behavioral and neural data from the tokens task. (A) Neural activity of cells in dorsal premotor cortex (PMd), primary motor cortex (M1), and globus pallidus external (GPe) and internal (GPi) segments, aligned on the first token jump (left) or movement onset (right), during easy (blue), ambiguous (green), and misleading trials (red) in which the monkey chose a neuron’s preferred target (solid) or the opposite target (dashed). The inset shows the temporal evolution of evidence in those trials, and the gray bar indicates the estimated time of commitment—approximately 280 ms before movement onset. (B) The neural correlates of the urgency signal in PMd (left) and M1 (right) during fast (red) and slow blocks (blue), calculated as the average activity when the evidence for both targets was equal. (C) Neural activity of build-up cells in GPe (left) and decreasing cells in GPi (right) during fast (red) and slow blocks (blue).

Sources: Panels A and C modified with permission from Thura and Cisek (2017), and panel B from Thura and Cisek (2016a).

The tokens task is quite different than the types of tasks previously used to support the DDM, not just because the evidence information is changing. The token jumps are very salient and easy to detect, and the depleting pool of tokens in the center provides an obvious cue to elapsing time. Furthermore, because the tokens were always present and visible, perhaps integration was not necessary, and subjects were merely using the stimulus as an external stand-in for the accumulated evidence. A recent study by Ferrucci et al. (2021) tested this possibility by presenting human subjects with two versions of the task—one in which the tokens remained, as in our studies, and one in which they vanished within 200 ms. The behavior was again consistent with the UGM—that is, decisions were not any faster in trials that had an early bias toward the chosen target, as would be predicted by any model with a long time constant. In fact, decisions were faster when the early bias was in the opposite direction, as if subjects were accumulating the novel information provided by each token jump but with some additional leak. That is opposite to the predictions of the DDM.

Other tasks involving changing evidence lead to congruent conclusions. For example, Yang and Shadlen (2007) studied monkeys trained in a probabilistic inference task, in which they were presented with a sequence of four abstract symbols, each favoring one choice or another, and had to combine these to determine the best guess. These authors showed not only that the animals combined the information appropriately (by summing logLRs) but also that neural activity in parietal cortex reflected that summation process. This is consistent with the proposal that statistically independent samples, like their abstract symbols or our token jumps, should be summed.

In another test of evidence accumulation in both humans and rats, Brunton and colleagues (2013) developed behavioral tasks in which subjects were presented with two trains of auditory clicks, one containing left-labeled clicks and the other containing right-labeled clicks. At the end of the trial, subjects were instructed to report which of the trains had most of the clicks. Importantly, the timing of clicks was random, both within and between trials, and the duration of the stimulus presentation was controlled by the experimenter. Using inverse correlation, the authors estimated the extent to which click rates at each point in time influenced left and right decisions, and they found that all periods of the trial had a similar influence on the decision, consistent with a long time constant of integration. Although this may at first appear at odds with our proposals presented previously, it is in fact consistent given the nature of the stimulus. In the Brunton et al. task, as in our tokens task or the task of Yang and Shadlen (2007), the sequential samples of relevant information (individual clicks) are statistically independent of each other. Thus, according to Equation 6, each click’s contribution should be equal and related to the simple logLR, as is the case for individual token jumps. This is different than tasks such as random-dot motion discrimination, in which the motion signal across the frame at a given time is already partially predicted by motion signals earlier in the trial. In summary, the Brunton et al. (2013) studies support the proposal that what is integrated is novel evidence.

Noisy Tasks

In the tokens task, as in the auditory click task, the stimuli are unpredictable, but they are also unambiguous—that is, there is no uncertainty about the identity of each sample. However, the DDM has traditionally been used to explain decisions about noisy stimuli. Perhaps subjects use a longer time constant in tasks such as random-dot motion discrimination in order to filter out the sensory noise. To test this, we conducted a different experiment (now only with humans), in which the sensory evidence was provided by a changing random-dot motion stimulus (Thura et al., 2012). In this new task, subjects watched a field of dots that initially all moved randomly but then, every 200 ms, a small percentage of the random dots began to move coherently to the right or the left. This continued for a total of 15 steps of motion coherence changes. As in the tokens task, subjects were required to guess the direction in which the majority of the dots will be moving at the end of the trial, and after their choice the motion changes happened more quickly. If subjects were now using a long time constant of integration, they should show long-lasting effects of early motion states. However, that was not observed. Subject behavior was still consistent with a model with a short time constant—short enough that the effects of early motion steps leaked away within 200–300 ms. Perhaps this should not be surprising because a leaky integrator is equivalent to a low-pass filter, which is very good at suppressing the high-frequency noise (60 Hz) present in these kinds of stimuli.

Nevertheless, the tasks described previously still differ from classic perceptual decision-making tasks because subjects are being asked to make a guess about the future state of the stimulus and not just to make a judgment about what they are seeing now. Perhaps that knowledge of a changing stimulus is enough to motivate them to use a short time constant. To test this possibility, we conducted another study, again with humans, that was almost identical to the classic random-dot motion discrimination paradigm (Carland et al., 2016). As in all of those studies, subjects were shown a field of randomly moving dots and were asked to indicate, whenever they wanted, which way the dots were moving. In one condition, subjects performed trials in which the dots were moving either right or left with low, 3% coherence. Unknown to the subjects, on some trials we inserted brief 100-ms “pulses” of extra motion, for a total of 6% coherence in the same direction, at different times—either 100, 200, or 400 ms after the start of the trial. Any integrator model with a long time constant (e.g., DDM) will predict that these small pulses, even if not perceived, will accelerate the integration process and reduce reaction time, as long as the pulse happens before the threshold is reached. A leaky integrator (e.g., LCA and UGM) will make the same prediction, with the caveat that the effect of pulses can leak out over time so that if a decision is made long after the pulse, then the reaction time will be the same as if the pulse never happened. In other words, if subjects slow down their decision (by raising the threshold, lowering urgency, or any other mechanism), then any model with a long time constant (e.g., DDM) will predict that later pulses will now become as effective as early pulses in reducing RTs, whereas any model with a short time constant (e.g., LCA and UGM) will predict that late pulses will now become effective but the early pulses will become ineffective.

We found the latter: When subjects slowed down, the early pulses lost their efficacy. Consequently, in a condition in which fast decisions were encouraged, the 100- and 200-ms pulses had a significant effect on RT but the 400-ms pulse did not, whereas when slow decisions were encouraged, the 200- and 400-ms pulses had effects but the 100-ms pulse did not. In fact, comparison across subjects suggested that for any given subject and condition, there was always a small window of time, approximately 500 ms before a given individual’s mean RT in no-pulse trials, in which pulses were effective, but earlier pulses appeared to leak out. Because of variability inherent in such studies, it is not possible for us to estimate the time constant precisely, but it is possible to eliminate any model whose time constant is greater than 250 ms. Consequently, we have suggested that the data of the Carland et al. (2016) study are incompatible with any version of a model in which the time constant is long (>250 ms), regardless of any other parameter settings, and to our knowledge no one has shown otherwise.

Nevertheless, Winkel et al. (2014) came to a different conclusion on the basis of a behavioral study using a changing motion stimulus, suggesting that the UGM cannot explain the effects of early evidence during decision-making. These authors designed a task in which the evidence for or against the correct choice was presented either early (<200 ms) or later in the trial, and the data indicated that early evidence could influence late decisions. The authors attempted to fit these results with both the DDM and the UGM, concluding that the DDM significantly outperformed the UGM in predicting the effects of early information. However, the implementation of the UGM they used was incorrect because they did not include a low-pass filter at all. This is equivalent to assuming that the time constant is zero, and so it should be of no surprise that such a limited version of the UGM does not match the data from a noisy task. Indeed, we showed that a UGM that includes a low-pass filter with a 250-ms time constant can reproduce Winkel and colleagues’ RT distributions very well, and thus can explain the effects of early evidence (Carland et al., 2015), as long as subjects make decisions relatively quickly (Carland et al., 2016).

A similar omission was made in another study, which aimed to differentiate among decision-making models by analyzing neural response variance in monkey parietal cortex (Churchland et al., 2011). In this study, the time-variant model was defined as a time-dependent scaling of the momentary sensory evidence. But as in Winkel et al. (2014), it completely lacked the low-pass filter that would have allowed it to deal with the high-frequency noise in the stimulus. Adding a low-pass filter, as we and Ditterich (2006b) suggest, would make the model predict neural activity variance just as well as an integrator. This is because a low-pass filter and an integrator are mathematically equivalent with respect to the gain of their transfer function for frequency content above the filter’s cut-off frequency, like the noise in the stimulus and the variance in neural firing.

Evans et al. (2017) tested the DDM against the UGM and reasoned that slower decisions should qualitatively discriminate the two models better than fast decisions because the rapidly leaking evidence accumulation process of the UGM will make different predictions than the DDM over longer timescales. A large cohort (70) of human subjects made key press decisions about the net direction of a random-dot motion stimulus in which the early evidence (either favoring or opposing the correct choice) was presented for 500 ms. Then late evidence, always indicating the correct choice, was presented as an increase of the ongoing motion coherence at one of five ramp rates: none, slow, medium, fast, and very fast. Leaky models, including the UGM, predict decreasing effects of the early evidence with slower decisions, whereas perfect integrators, including the DDM, predict that the effects of early information will remain constant across changes in decision time. Contrary to the predictions of the leaky models, this study found that the DDM fit the data better than did the UGM. However, reverse correlation analyses of subjects’ conditional accuracy showed that choices were based on information from a smaller window of time than predicted by the DDM and more consistent with the UGM.

A recent study (Stine et al., 2020) addressed whether subjects actually integrate evidence at all when making decisions about noisy stimuli, even when they are encouraged to do so. Stine et al. tested an integration model against two non-integration models. In the “extrema detection” model, sensory evidence is sampled sequentially until any individual sample exceeds one of two bounds. In the “snapshot” model, the decision-maker acquires only one piece of information at a random time during stimulus presentation. This model thus lacks not only integration but also sequential sampling. By simulating data with the three models in various contexts (fixed stimulus duration, variable stimulus duration, and free response time), Stine and colleagues identified features that allowed non-integration strategies to mimic integration strategies and designed a novel variant of the random-dot motion discrimination task in which evidence integration and non-integration strategies are disentangled. Their results rule out non-integration strategies for all human subjects. However, the authors acknowledge that they tested two extreme non-integration models and that leaky integration models might provide more flexibility regarding how decision time and accuracy are linked. By estimating the duration of the leak of sensory processing in each subject, Stine and colleagues suggested long integration times in three subjects (out of five) but shorter integration durations in two subjects.

These and other studies demonstrate that determining the time constant of integration is not trivial and requires studies explicitly designed to constrain its values using changing evidence (Cisek et al., 2009; Trueblood et al., 2021). Furthermore, we must consider the possibility that the time constant might be context dependent and/or influenced by various top-down factors (Carrasco & McElree, 2001; Ossmy et al., 2013). Most studies have implicitly assumed that it is long and that the build-up of neural activity during decision-making is caused by perfect integration of sensory evidence. But most recent studies, explicitly designed to determine the time constant, suggest that in most cases it is much shorter than the decision process—no longer than 250 ms and perhaps significantly shorter. Consequently, neural activity build-up may be due to other factors, such as the growing urge to decide and act (Carland et al., 2019; Huk et al., 2017).

Evidence for Urgency in Nonhuman Primates

Although the question of the time constant remains under debate, a strong consensus is emerging in favor of the hypothesis that an urgency signal gradually pushes neural activity toward the decision threshold. This hypothesis makes two key predictions. First, it predicts that neural activity should grow with time independently of evidence. Second, it predicts that in a free response task with changing evidence, decisions made later in time will be made with less evidence. As discussed previously, our studies using the tokens task confirmed both these behavioral (Cisek et al., 2009) and neural predictions (Thura & Cisek, 2014, 2016a), but similar results have also been reported by others.

The earliest work interpreting neural activity in terms of an urgency signal was a study by Churchland et al. (2008) that investigated whether accumulation models could account for decisions beyond simple two-choice tasks. These authors measured behavioral and physiological responses of monkeys on a four-choice motion discrimination decision task and observed that parietal responses were lower at the beginning of the decision process compared to a two-choice task. This lower initial firing rate amounted to having a higher threshold for terminating the choice, which makes good sense when many options are available. But a larger excursion also comes at the expense of decision time, potentially reducing an animal’s global reward rate. Interestingly, the authors observed a time-dependent rise of neural firing rate in all trials, regardless of motion strength and direction. They interpreted this rising signal as an urgency component that limits the cost of accumulating evidence in difficult tasks.

With hindsight, one can interpret several previous results as having foreshadowed these findings. For example, build-up activity has been observed during a variety of tasks involving timing (Janssen & Shadlen, 2005; Jech et al., 2005; Leon & Shadlen, 2003; Renoult et al., 2006) and even during tasks without any decision-making component (Hanes & Schall, 1996; Ivry & Spencer, 2004; Lebedev et al., 2008; Munoz et al., 2000; Roesch & Olson, 2005; Tanaka, 2007; Thomas & Pare, 2007). Much of that activity could be related to the growing urge to respond, to motor preparation, or to other aspects of sensorimotor control (Costello et al., 2013; Huk et al., 2017).

For example, Bennur and Gold (2011) recorded LIP activity in a version of the motion discrimination task in which the monkeys were informed about the targets for reporting choices before, during, or after the motion stimulus. They found that the build-up of neural activity was much stronger after the targets were known and that neural selectivity for motion direction was approximately flat during the motion viewing interval. More recently, Kira et al. (2015) trained macaque monkeys to make decisions at any time based on sequential visual cues, which were statistically independent, and recorded firing rates of parietal neurons. They observed that at least for one of the monkeys, there was a gradual rise in firing rate apparent even when the accumulated evidence favored neither choice. Finally, several studies have shown that nonselective activity, sometimes growing over time, is related to changes in the animal’s speed–accuracy trade-off, either when instructed (Hanks et al., 2014; Heitz & Schall, 2012) or when freely determined by the animal (Thura & Cisek, 2016a), or even as a result of post-error slowing (Purcell & Kiani, 2016; Thura et al., 2017). As noted previously, our recordings in the output nuclei of the BG suggest that the urgency signal may project to the cerebral cortex from these subcortical regions (Thura & Cisek, 2017).

Evidence for Urgency in Human Decisions

It is important to note that a significant amount of support for time-variant models comes from studies in nonhuman primates. This raises the possibility that urgency-based decision policies are species-specific or a consequence of overtraining (Boehm et al., 2016; Hawkins et al., 2015). For instance, Hawkins and colleagues (2015) suggested that static threshold models provide a better fit to some human data sets compared to time-variant models, whereas time-variant models are better at fitting monkey behavior. It is indeed possible that human subjects instinctively overprioritize precision (perhaps because of pride) and thus set a low level of urgency to guarantee high percentages of correct responses (Bogacz, Hu, et al., 2010), even at a high cost of time. By contrast, monkeys perhaps make very rapid decisions, betting more on the overall success (and reward) rate instead of performance per se (Thura, 2016). Moreover, because monkeys are usually trained on a behavioral task over a long period of time, it is possible that the large amount of practice shapes their behavior, allowing them to explore more strategies than human subjects, who usually only experience a few experimental sessions.

However, the conclusions of recent studies do not support this hypothesis. For example, Thura (2020) showed that most human subjects instructed to perform a modified version of the tokens task in a test–retest design adopted an urgency-based decision policy (i.e., a dropping accuracy criterion) that was adjusted depending on the reward rate context of the task, irrespective of the session they performed. These observations are consistent with many other recent studies that have demonstrated that naive human decision-makers decrease their accuracy criterion as time is elapsing within a trial when making successive decisions between actions, whether these decisions are guided by sensory or value cues (Bhui, 2019; Derosiere et al., 2019, 2021; Farashahi et al., 2018; Gluth et al., 2012, 2013; Hauser et al., 2018; Malhotra et al., 2017, 2018; Miletić & van Maanen, 2019; Murphy et al., 2016; Palestro et al., 2018; Steinemann et al., 2018).

For instance, Gluth and collaborators conducted functional magnetic resonance imaging (fMRI; Gluth et al., 2012) and electroencephalography (EEG; Gluth et al., 2013) studies in humans during which participants were offered a stock and had to decide whether to buy or reject the offer or whether to pay for additional pieces of evidence about its value provided successively during each trial. In both studies, the authors found that the evidence required for deciding decreased linearly with time, similarly for buy and reject decisions. They also observed that participants required less evidence in trials in which successive information came at a higher cost. Model comparisons confirmed that a time-variant accumulation model was the most adequate for predicting value-based decisions compared to time-invariant models. In the fMRI study, the authors found significant activation in pre-supplementary motor area and in the right caudate nucleus in relation to early decisions, in agreement with the proposal that these areas could adjust the decision threshold by increasing baseline activity under elevated time pressure (Bogacz, Wagenmakers, et al., 2010; Forstmann et al., 2008; Ivanoff et al., 2008). Moreover, activity in the same areas steadily increased during ongoing trials, possibly in relation to the decrease of the decision threshold over time. In the EEG study, they found that the signal recorded at the mid-central electrode Cz reflected the different task-dependent influences on the timing of responses, including the manipulation of rating costs, the increased response pressure at later time points, and the accumulated evidence. In particular, the observed patterns of the readiness potential were compatible with an urgency signal that potentiates the developing decision variable during the accumulation process.

Murphy and colleagues (2016) designed a study aimed at challenging the view that human decision-makers may fail to implement a time-variant commitment policy that would yield higher reward rates, especially when fast decisions must be prioritized. These authors instructed 21 individuals to make perceptual decisions about the dominant direction of motion of a cloud of moving dots. Each subject performed this task under two levels of speed emphasis—a “free response” regime with no external speed pressure and a “deadline” regime with a penalty applied if a decision was not made by 1.4 s after motion onset. Through analyses of the observed behavior, computational modeling, and scalp electrophysiological data, the authors showed that subjects responded to deadline-induced speed pressure by lowering their accuracy criterion as the deadline approached. In the brain, this effect was reflected in an evidence-independent urgency signature in the lateral motor channels, pushing decision-related motor preparation signals closer to a fixed threshold.

Recent studies in which human subjects are asked to classify gradually changing facial expressions show that the evidence on which decisions are based can be decoded from the fusiform face area, long known to process visual information about faces, whereas the inverse of an urgency signal can be detected in the caudate nucleus of the BG (Yau et al., 2020). Furthermore, EEG data suggest that an evoked potential time-locked to the decision, the centroparietal positivity, reflects the product of evidence and urgency and correlates with differences in response rates among individuals (Yau et al., 2021).

Recently, Parés-Pujolràs et al. (2021) investigated in humans two neural signals, the P3 and the readiness potential, in the context of a decision-making task in which subjects had to monitor a stream of letters over time and decide whether or not an action is required, based on informative, ambiguous, frequent, or infrequent evidence. With computational modeling, the authors demonstrated that subjects’ behavior is best explained by a model combining a competition between options along with an urgency signal that multiplicatively modulates the integration of incoming evidence. In agreement with the simulations, the authors showed that the P3 component encodes a categorical decision variable and that the evolution of that variable appears modulated by a context-dependent urgency signal.

On the Global Nature of Urgency Signals

Urgency signals need not be strictly limited to the mechanism for making decisions about actions. Steinemann and colleagues (2018) noted that adjustments to time pressure have been almost exclusively examined within neural circuits involved in preparing actions and that the extent and nature of adjustments made at upstream and/or downstream processing levels have received much less attention. The authors designed a contrast-comparison paradigm and used scalp EEG to trace neural dynamics at multiple levels of the sensorimotor hierarchy in humans making decisions under varying response time constraints. Results indicated that speed pressure impacted multiple sensorimotor levels but in distinct ways. Crucially, an evidence-independent urgency was mostly visible in cortical action-preparation signals and downstream, probably impacting muscle activation, but not directly in upstream levels.

The possible impact of decision urgency on muscle activation is also supported by our studies involving the tokens task that revealed a strong interaction between decision speed and movement kinematics, both in humans and in monkeys (Thura, 2020; Thura et al., 2014). For instance, in blocks of trials favoring hasty decisions, the subjects’ reaching movements executed to report choices are faster than similar movements performed during blocks of trials encouraging slower, more accurate decisions. Moreover, within each block, hasty choices (based on strong sensory evidence combined with low urgency) are followed by slow movements, whereas long decisions (relying on comparatively weaker sensory evidence in combination with a higher level of urgency) are followed by faster arm movements. Consistent with this behavioral result, the speed-related neural activity in monkey PMd and M1 neurons is correlated with the current level of urgency at time of commitment (Thura & Cisek, 2016a). Together, these findings suggest that a shared signal controls the timing of decisions as well as the speed of the motor commands produced to express these decisions. Such a link between decision and action makes good sense in the context of reward rate maximization because reward rate is influenced not only by the time taken to decide but also by the effort and time spent executing the movement and obtaining the reward (Shadmehr & Ahmed, 2020; Shadmehr et al., 2016, 2019). More recent tests of the shared regulation hypothesis, however, indicate that decision urgency and movement vigor can be dissociated under certain circumstances. When human subjects are encouraged to execute slow and accurate movements to report their decisions in a variant of the tokens task, those decisions are generally faster compared to others made in blocks in which reaching movements are more vigorous. This observation suggests that urgent decisions can be computed from the prospect of time-consuming movements, and it supports a flexible and integrated mechanism in which one can trade decision time for movement time to possibly limit the loss of reward rate (Reynaud et al., 2020; Saleri Lunazzi et al., 2021).

Is the influence of decision urgency on motor behavior restricted to motor systems involved in reward rate maximization? In the tokens task studies mentioned previously, oculomotor behavior was never constrained. Consequently, frequency, speed, and accuracy of subjects’ saccades cannot directly impact their rate of reward at the session level. Nevertheless, we observed that decision urgency affects to some extent the speed of saccadic eye movements executed during deliberations, suggesting the influence of a broad and nonspecific signal invigorating motor systems beyond the unique goal of maximizing reward rate (Thura, 2020; Thura et al., 2014). Given this broad influence, an urgency signal would be expected to originate from a region that projects to a wide range of cortical areas and subcortical areas involved in various aspects of behavior. In support of such prediction, neurophysiological, pharmacological, and computational studies have recently suggested that the urgency to decide might be implemented at the neural level by a global gain under the control of the locus coeruleus–norepinephrine (LC-NE) system (Eckhoff et al., 2009; Hauser et al., 2018; Murphy et al., 2016). The LC projects to almost the entire cerebral cortex, and the release of NE could exert a multiplicative influence on neural dynamics related to decision-making and motor control (Aston-Jones & Cohen, 2005). Alternatively, or concurrently, the BG provide a good candidate for computing a shared invigoration signal as well. The BG have long been functionally associated with the regulation of motivated behavior for maximizing reward and are strongly implicated in both the control of movement vigor (Desmurget & Turner, 2010; Dudman & Krakauer, 2016; Horak & Anderson, 1984a, 1984b; Turner & Anderson, 1997; Turner & Desmurget, 2010; Yttri & Dudman, 2016) and the ability to motivate energy expenditure in the pursuit of potential rewards (Le Heron et al., 2018; Manohar et al., 2015; Mazzoni et al., 2007). Moreover, as described previously, monkey GP neurons appear to encode the urgency signal as well as its adjustment across different speed–accuracy regimes (Thura & Cisek, 2017). Finally, a major deficit of Parkinson’s disease is the inability to move rapidly, which has been proposed to be at least partly the result of a disorder in economic evaluation of the options (Mazzoni et al., 2007). The BG may thus be the central source of a global signal that energizes both the urgency of decisions and the vigor of the selected action depending on given reward and cost contingencies (Cisek & Thura, 2018). Importantly, if urgency is the mechanism that links together the timing of decisions and movements through projections from the BG to sensorimotor regions, then it might also influence many other aspects of motivated behavior through projections to other cortical regions, including prefrontal and limbic areas, explaining a wide variety of phenomena in both health and disease, ranging from personality traits such as impulsivity to some of the major symptom domains commonly observed in depression and Parkinson’s disease (Carland et al., 2019).