Motion Processing in Primates
Summary and Keywords
The ability to see motion is critical to survival in a dynamic world. Decades of physiological research have established that motion perception is a distinct sub-modality of vision supported by a network of specialized structures in the nervous system. These structures are arranged hierarchically according to the spatial scale of the calculations they perform, with more local operations preceding those that are more global. The different operations serve distinct purposes, from the interception of small moving objects to the calculation of self-motion from image motion spanning the entire visual field. Each cortical area in the hierarchy has an independent representation of visual motion. These representations, together with computational accounts of their roles, provide clues to the functions of each area. Comparisons between neural activity in these areas and psychophysical performance can identify which representations are sufficient to support motion perception. Experimental manipulation of this activity can also define which areas are necessary for motion-dependent behaviors like self-motion guidance.
Animals spend much of their waking hours in motion, and they must sense their own movements as well as the movements of objects or other animals (Gibson, 1950). The visual scene is rich with motion cues. Local translation can indicate the trajectory of an object of interest; expansion (or looming) can signal approaching danger (Schiff, Caviness, & Gibson, 1962), but can also be caused by the observer’s own motion through the environment (for a review, see Britten, 2008). As a result, highly specialized circuits for visual motion estimation have emerged across many taxa of visual animals, even when circuits for the recognition of objects or discrimination of color have not (Clifford & Ibbotson, 2002).
Through analysis of the time-varying firing rates of single neurons, much has been learned about the calculations performed in visual motion circuitry. Originally, studies used the classic tools of sensory systems neuroscience: relating single-neuron discharge to the properties of stimuli (Adrian, 1928). More recently, our understanding of these signals has been strengthened by considering populations of neurons, and by relating their discharges to perceptual or behavioral responses. Manipulation of normal circuit function with electrical stimulation and experimental lesions can build on these correlations and determine causal relationships between neural activity and perception or behavior. Focusing on cortical mechanisms and recent results from physiologic studies in nonhuman primates, this article will review the ways that visual motion information is processed and used to guide behavior.
Low-Level Motion Processing
Fundamentals of Visual Motion Estimation
The brain processes visual motion stimuli via a network of hierarchically arranged modules (Felleman & Van Essen, 1991; Nassi & Callaway, 2009). At the base of this hierarchy, local motion in the visual field must be computed from the raw set of photoreceptor responses. In principle, calculating velocity sounds simple: One determines an object’s motion by recording its spatial position at two successive points in time and dividing the difference in position by the elapsed time. In practice, the problem is harder because neurons do not “know” the identity of objects that affect their rates, so the identification of a single moving feature is ambiguous. Most of the time, however, as an object moves across the visual field, the pattern of light that it reflects into the eye stays relatively constant, but is simply displaced across the retina from one point in time to the next. Therefore, by combining neural responses that are displaced in space and delayed in time, one can create selectivity for motion in a specific direction and at a certain speed. This hypothesis formed the basis of two foundational models of biological motion detectors (Barlow & Levick, 1965; Reichardt, 1962): The first was designed to explain the responses of neurons in rabbit retina to moving stimuli, while Reichardt’s model laid the groundwork for understanding the link between motion estimation and self-motion guidance in insects.
Theoretical Motion Filter Circuits
To create direction-selective (DS) outputs from spatiotemporally offset, photoreceptor-like inputs, both models relied on nonlinear comparator units (Figure 1). In the Barlow-Levick model, an “and-not” logic gate signals only when one specific input is activated at any single time point. Reichardt’s original model uses two correlation units that each receive delayed and non-delayed outputs from the input pair. The outputs of these units are then subtracted to produce a signed final output: a negative response for motion in one direction and a positive response for motion in the other direction (Figure 1B). Despite their simplicity, motion filters that use point detectors (i.e., those that sample mean luminance over small regions of the visual field) at the input stage fail to capture key properties of their biological counterparts. The original Reichardt model has a spatial aliasing problem, in which certain textures “fool” the detector into reporting the wrong direction (van Santen & Sperling, 1984). In addition, the spatial response properties of neurons that project to cortical motion filters are not consistent with point detection. To overcome these shortfalls, an “Elaborated Reichardt Detector” was designed with spatial filters in the input stage and a biologically realistic unit in the correlation stage (van Santen & Sperling, 1985).
One can best see the advantage of using spatially filtered inputs by analyzing the retinal image in the frequency domain (Watson & Ahumada, 1983). In this framework, the time-varying retinal image is represented by the power and phase of its spatial and temporal frequency components (Figures 2A & 2B). Speed is represented as a ratio between spatial and temporal frequencies that determines the slope of a line in frequency space. Motion direction is indicated by the orientation of this constant-speed line. Simple linear filters can therefore be used to estimate these motion parameters by sampling from frequency space along these lines (Figure 2D). The response properties of some neurons in primary visual cortex are very similar to the spatiotemporal filter shown in Figure 2D (Carandini et al., 2005; DeAngelis, Ohzawa, & Freeman, 1993; De Valois, Albrecht, & Thorell, 1982). Representing motion processing in this framework also simplifies how one characterizes neuronal responses to visual stimuli: Images are represented as linear sums of frequency components, and motion filters perform linear operations on these components (DeValois & DeValois, 1988).
Building on this frequency-based approach, two groups built models that bypassed the correlation problem and instead used spatiotemporal filters as inputs (Adelson & Bergen, 1985; Watson & Ahumada, 1985). The Adelson-Bergen motion energy model is used as an example here, though its input-output relationship is equivalent to some versions of the Elaborated Reichardt Detector and Watson-Ahumada models. The motion energy model (Figure 1) is based on a quarter-phase offset of its input filters (quadrature pairing) and subsequent squaring of their outputs. These features produce an output that is constant in time and independent of contrast sign. Not only does this model closely approximate the responses of direction-selective neurons in cortex, but its method of estimating motion predicts the types of errors that are seen in perceptual judgments in psychophysical work (see Burr & Thompson, 2011, for an in-depth review). However, for a motion-selective unit to be truly tuned for speed, it must sample from many different spatial frequency-tuned inputs. The motion energy model samples only from local regions of frequency space along the constant-speed line (Figure 2C) and therefore fails for objects containing other frequencies. This limitation can be overcome by combining the outputs of multiple spatiotemporal filters that sample along the same speed line (see Section “Mid-level Motion Processing”; also, Priebe, Cassanello, & Lisberger, 2003).
Like most theory, these models can only indicate how such problems can be solved, and are indifferent as to how each operation might map onto neural circuitry. Recent advances in imaging, however, have in principle made such mapping feasible. Analysis of insect visual systems has made a good start in this direction (see Borst & Euler, 2011, for a review).
Neuronal Motion Filters: Peripheral Circuits
The retina is a rich source of information regarding how biological systems estimate motion. Direction-selective ganglion cells (DSGCs) in the vertebrate retina are the first direction-selective neurons in the visual pathway. These were initially discovered in the rabbit (Barlow & Levick, 1965), and have since been found in other species, including mouse, rat, and salamander (Dhande, Stafford, Lim, & Huberman, 2015). Evidence for DSGCs in the primate retina, however, remains elusive. These neurons act as primary afferents for the accessory optic system involved in image stabilization, but recent evidence suggests that they also project into image-forming pathways in the mouse (Cruz-Martin et al., 2014; Sun, Tan, Mensh, & Ji, 2016).
Ganglion cell-type-specific targeting of fluorescent markers in transgenic mouse lines has revealed at least seven subtypes of DSGC (Rivlin-Etzion et al., 2011; Sanes & Masland, 2015). On-off DSGCs subtypes are by far the best characterized. Their direction selectivity emerges from a combination of excitatory bipolar cells inputs and asymmetric inhibitory inputs from starburst amacrine cells (SACs) (Briggman, Helmstaedter, & Denk, 2011; Fried, Munch, & Werblin, 2002). This circuit is reminiscent of the “and not” logic gate in the Barlow-Levick model because of the delayed inhibitory input from the SACs. However, it is somewhat more complex because the SACs themselves have direction selectivity, preferring motion away from the soma, and systematically innervate DSCGs with matched preferred direction. Recent evidence suggests that the direction selectivity in SACs arises from a combination of circuit-based mechanisms (lateral inhibition from adjacent SACs) and a dendritic mechanism based on the separation of excitatory inputs and GABA release sites (Vlasits et al., 2016). For further discussion of the DSGC circuit, see Vaney, Sivyer, and Taylor (2012).
Neuronal Motion Filters: Central Circuits
Direction-selective units were first identified by Hubel and Wiesel in cat primary visual cortex (V1) (Hubel & Wiesel, 1959), and later in mouse and monkey cortex. Most evidence identifies V1 as the origin of direction selectivity in the primate image-forming system (Hirsch & Martinez, 2006; Priebe, Lampl, & Ferster, 2010). In primates, the primary input to V1 is from nondirectional magnocellular (M) and parvocellular (P) neurons of the lateral geniculate nucleus (LGN), which in turn are driven by parasol and midget ganglion cells in the retina, respectively. V1 is a laminar structure with elaborate microcircuitry that filters these thalamic inputs to estimate local features of the retinal image-like orientation, binocular disparity, and motion direction. For an in-depth review of visual processing in V1, see Nassi and Callaway (2009).
DS neurons in V1 are broadly divided into two classes: simple and complex cells, which are defined on the basis of their receptive field (RF) substructure. Simple cells respond maximally to a given stimulus within a subdivision of the receptive field (Figure 2D), while complex cells do not show this position (phase) dependence. In primates, spatiotemporal tuning in DS simple cells is likely built linearly from the feedforward inputs of nondirectional, orientation-tuned cells that have offset spatial receptive fields and different temporal latencies (De Valois & Cottaris, 1998). These offsets determine the DS cell’s spatial and temporal frequency tunings, respectively.
Both simple and complex DS cells in V1 have direction preferences that evenly sample all angles of motion, unlike DS cells in the retina, which are broadly tuned for motion in four cardinal directions (Briggman et al., 2011). The receptive fields of DS simple cells resemble spatiotemporally inseparable filters (Figure 2D; Livingstone & Conway, 2003), with a preference for moving contours oriented perpendicularly to the direction of motion. Neurons may inherit different temporal response profiles based on their thalamic inputs (Saul, Carras, & Humphrey, 2005), since P cell responses tend to lag M cells, or delays may be generated de novo within the V1 circuitry (Baker & Bair, 2012). Researchers differ on which explanation is better supported for primate V1.
While simple feedforward models are attractive, they cannot account for the response properties found in most DS cells in V1. Many DS simple cells exhibit greater direction-selectivity indices (Reid, Soodak, & Shapley, 1991) and narrower tuning for orientation (Gardner, Anzai, Ohzawa, & Freeman, 1999) than expected from a linear combination of inputs. Departures from linearity become more pronounced in the complex cells of the nongranular layers of monkey cortex (Kim & Freeman, 2016), suggesting that refinement of direction selectivity is a multistage, nonlinear process in V1 (Peterson, Li, & Freeman, 2004). Nonlinearities that can account for this refinement include spike thresholds (Priebe & Ferster, 2005) and divisive normalization (Carandini & Heeger, 2012). Spike thresholds can limit the responses of DS cells to only the most preferred directions associated with the greatest amount of input drive. Large suppression fields, broadly tuned for motion direction, can also enhance directionality (Monier, Chavane, Baudot, Graham, & Fregnac, 2003). Broadly tuned suppression is also a predicted consequence of normalization between DS units in V1. Suppressive inputs tuned for the null direction may also explain enhanced selectivity indices in many neurons (Rust, Schwartz, Movshon, & Simoncelli, 2005). All of these nonlinearities can be instantiated through lateral inhibition from other directional neurons within the same cortical layer, though other possibilities exist.
Lateral interactions between neurons in a given cortical area contribute to at least the proximal portions of the suppressive surrounds in V1 neurons (Angelucci & Bressloff, 2006). In macaques, the spread of these horizontal connections is substantial, reaching up to 7 mm in diameter and targeting neurons with similar stimulus preferences (Stettler, Das, Bennett, & Gilbert, 2002). Though it is unknown precisely to what extent lateral inputs (i.e., compared to the influence of feedback, see Section “Roles for Feedback in Motion Processing”) contribute to the nonlinear computations discussed in the previous paragraph, they likely play an important role especially for smaller stimuli (Nurminen & Angelucci, 2014). These connections in V1 could serve to contextualize local motion estimates to decrease noise in the population code and emphasize which local estimates should be bound together in extrastriate areas (Frégnac & Bathellier, 2015).
Cascades of these linear and nonlinear computation steps in V1 build progressively more abstract and useful estimators of local contour motion (Rust et al., 2005; Vintch, Movshon, & Simoncelli, 2015). From the input layers to output layers, the fraction of phase-invariant complex DS cells increases (Kim & Freeman, 2016). Despite this, DS projections from V1 to extrastriate cortex are primarily from a relatively homogenous group of complex cells in layer 4B, which is monosynaptically connected to the input layers (Felleman & Van Essen, 1991; Movshon & Newsome, 1996). Thus, an open question remains: What are the roles for the multiple output routes from V1? A few groups have found V1 DS neurons with properties classically attributed with mid-level motion areas, so it might be that V1 refines motion estimates more elaborately than once thought (Born & Bradley, 2005; Gur & Snodderly, 2007). Many DS neurons in layer 4B exhibit “end-stopped” suppression: They fire more vigorously when a contour terminates within their receptive fields as opposed to outside of it. End-stopped DS neurons are capable of reporting direction components parallel to the contour’s orientation, unlike other DS neurons that can only report components perpendicular to their preferred orientation (Pack, Livingstone, Duffy, & Born, 2003). There is also evidence that some populations of DS cells develop bona fide speed tuning within V1 (Priebe, Lisberger, & Movshon, 2006), which suggests this calculation is performed within striate cortex and refined in mid-level areas of the motion hierarchy.
Mid-Level Motion Processing
Limitations of Low-Level Motion Representations
The estimation of local motion across the visual field is a necessary first step in the development of full-fledged motion perception, but the local representation in V1 has limited utility for guiding perception or action. Most receptive fields at this level are less than a degree of visual angle wide and thus can process only small fractions of typically sized moving objects. The problem is compounded by orientation selectivity in V1, which limits neurons to representing motion orthogonal to their preferred orientation. These local measurements must be combined to determine true object motion. Spatial pooling would allow recovery of velocity (by pooling across different spatiotemporal frequencies) and reduce noise from individual estimates. But spatial pooling can be problematic if it crosses object boundaries. In such cases, local motion estimates must be segmented instead of pooled. Recent work has explored both of these conflicting operations (e.g., Huang, Albright, & Stoner, 2007), but the picture is far from complete at present.
Anatomy of Mid-Level Motion Processing
In primates, the visual motion hierarchy consists of a densely connected set of cortical areas embedded in the dorsal stream (Andersen, Snowden, Treue, & Graziano, 1990; Ungerleider & Mishkin, 1982). Directional neurons in V1 send monosynaptic projections to several areas (Figure 3), including the thick stripes of V2 and portions of V3, but the densest set of projections are sent to the middle temporal area (MT) (Ungerleider & Desimone, 1986). The parallel processing stream containing V2 and V3 eventually converges with MT, but what this path contributes to motion processing is unclear. There is evidence for maps of motion direction (Lu, Chen, Tanigawa, & Roe, 2010) and horizontal disparity (Chen, Lu, & Roe, 2008) in the thick stripes of V2, suggesting that this pathway may be responsible for most of the stereo tuning in MT. Indeed, inactivation of V2 and V3 appears to selectively disturb tuning for depth in MT neurons (Ponce, Lomber, & Born, 2008; Smolyanskaya, Haefner, Lomber, & Born, 2015).
V1 also projects to the little-studied area V6, which overlaps with portions of the parieto-occipital area (PO) (Colby, Gattass, Olson, & Gross, 1988). Other work suggests that PO should be subdivided into several functional subregions (including V6, V6Av, and V6Ad) based on anatomical and electrophysiological results (Galletti, Gamberini, Kutz, Baldinotti, & Fattori, 2005). In this scheme, V6 receives essentially the same inputs as MT (Fattori, Pitzalis, & Galletti, 2009), but has strikingly different response properties. V6 neurons are selective for optic flow, despite the relatively small size of their receptive fields (Fan, Liu, DeAngelis, & Angelaki, 2015). This region also contains cells that respond differentially to reafferent motion (i.e., motion due to retinal movement) and “real” motion (due to external object movement) (Pitzalis, Fattori, & Galletti, 2012). Neurons that discount reafferent motion are more commonly found in higher levels of the motion-processing hierarchy (see Section “Multimodal integration”). Given the reciprocal connections of V6 (and V6Ad/v) with these higher-level areas, selectivity for real motion may be the result of suppressive feedback from neurons sensitive to extraretinal information about eye movements (Wurtz, 2008).
Solutions to Low-Level Problems: Functional Properties of MT Neurons
Increasing the dimensionality of motion representation
Although the spatial receptive fields of MT neurons are around tenfold larger than those in V1, they appear to calculate displacements over a similar spatial range (Churchland, Priebe, & Lisberger, 2005; Livingstone, Pack, & Born, 2001). Thus, MT most likely inherits its direction selectivity from V1 and pools clusters of similarly tuned DS neurons, rather than calculating motion de novo.
Individual neurons in V1 have relatively narrow spatial and temporal bandwidths (Rust et al., 2005), but this tuning is wider in MT (Perrone & Thiele, 2001). Furthermore, in MT the joint tuning for spatial and temporal frequency is often slanted, approximating bona fide speed tuning. Some fraction of this tuning may be due to the selective projection of speed-tuned V1 neurons to MT (Priebe et al., 2003; Priebe et al., 2006). Neurons in MT, however, do tend to prefer higher speeds than in V1, which is suggested to be due to the influence of the indirect projections from V2/V3 (Ponce, Hunter, Pack, Lomber, & Born, 2011).
A long-standing question is whether MT explicitly encodes motion in depth. It is known that MT contains both velocity and depth representations, but it is less clear that these are combined to create true tuning for motion in depth (Huk, 2012; Maunsell & Van Essen, 1983; Zeki, 1974). Two groups now argue (Czuba, Huk, Cormack, & Kohn, 2014; Sanada & DeAngelis, 2014) that the most relevant cues for motion in depth—interocular velocity differences and changing disparity—were not appropriately tapped by previous stimuli. They also claim that the Maunsell and Van Essen definition for 3D motion selectivity as “a lack of maximal response for frontoparallel motion at the binocular disparity” is too limiting based on the possible stimulus space for motion energy with both depth and frontoparallel components. On the other hand, there is evidence for motion-disparity separability in MT (Smolyanskaya, Ruff, & Born, 2013), which is inconsistent with true 3D tuning and is not found in 3D motion coding areas like MST (Roy, Komatsu, & Wurtz, 1992).
Integration of local motion estimates
Even within the frontoparallel plane, MT response properties are more complex than those of its input areas. Objects include many different contour orientations and textures, and their movements often do not align with the motion of the object as a whole. Most neurons in V1, however, can only signal motion components perpendicular to these contours and are thus effectively blind to overall object motion (called the aperture problem; Figure 4A). Many neurons in MT are similar, and are termed component direction selective (CDS) cells. However, MT also contains pattern direction selective (PDS) cells that are sensitive to the overall motion within their RFs, rather than the individual motions of contours (Albright, 1984; Movshon, Adelson, Gizzi, & Newsome, 1985). Though there are other possible ways to recover object motion (e.g., tracking contour end points), most electrophysiological evidence suggests that these neurons integrate multiple motion components to recover pattern motion. This integration can be posed as finding an intersection of constraints: A given oriented motion component can be associated with an infinite number of object velocities whose vector endpoints form a constraint line in 2D velocity space (Figure 4C). The intersection of multiple constraint lines from the motion components present within a local region of space will reveal the velocity of the object. A good approximation to this computation can be accomplished through a linear-nonlinear cascade in which MT neurons pool the normalized outputs of a set of direction V1 neurons that evenly sample possible motion directions (Rust, Mante, Simoncelli, & Movshon, 2006; Simoncelli & Heeger, 1998; Wang & Movshon, 2016). Importantly, selectivity for the direction of pattern motion takes roughly 60ms to emerge in MT PDS neurons when either moving bar (Pack & Born, 2001) or overlapping drifting grating (or plaids) stimuli are used (Smith, Majaj, & Movshon, 2005). Prior to this, PDS neurons respond similarly to CDS neurons and report the direction of component motion. These temporal dynamics could reflect the additional synaptic delays present in lateral (see Section “Neuronal Motion Filters: Central Circuits”) or feedback (Section “Roles for Feedback in Motion Processing”) circuits in the visual motion pathway (Huang et al., 2007; Smith et al., 2005). A more recent study, however, contests these suggestions given their finding that MT neurons will only integrate temporally staggered plaid components when they are separated by less than 10ms (Kumbhani, El-Shamayleh, & Movshon, 2015). They argue that such a temporally precise operation would be difficult to achieve in anything but a feedforward architecture.
Segmentation of local motion estimates
Deciding which motion elements should be segregated is a complex problem that depends on the size and shape of objects, the depths of local motion estimates, and object texture. CDS cells probably contribute to the maintenance of segregation, and likely do so based on abrupt shifts in local image velocity that are associated with object boundaries. This can be accomplished with a center-surround RF organization. Indeed, MT neurons have motion-opponent surrounds with shapes ranging from uniform to polar (Bradley & Andersen, 1998; Xiao, Raiguel, Marcar, & Orban, 1997). Uniform surrounds support motion opponency (in which null-direction motion silences neurons), while asymmetric surrounds may contribute to the calculation of form from motion (Cui, Liu, Khawaja, Pack, & Butts, 2013), with each neuron identifying a different contour motif. Surrounds generally have broader motion tuning than the centers, which results in response suppression when local motion is surrounded by incoherent motion (e.g., a bird flying in front of tree leaves in a breeze) (Hunter & Born, 2011). Under different stimulus conditions, however, some cells that display surround suppression may instead integrate motion components (Huang et al., 2007). Thus, segmentation and integration operations may not necessarily tap distinct subpopulations of neurons.
Roles for Feedback in Motion Processing
Though this review focuses on feedforward processing of visual motion information, the primate visual system also contains an extensive network of lateral (Angelucci, Levitt, Walton, Hupé, Bullier, & Lund, 2002) and feedback connections (Markov et al., 2014). The emphasis on the feedforward flow of information is a useful simplification of the complex cortical circuitry given that feedforward, transthalamic connections most often serve as drivers for cortical neurons (Sherman & Guillery, 1998). Lateral connections via horizontal (Yoshimura, Sato, Imamura, & Watanabe, 2000) axons only produce subthreshold potentials in their neuronal targets, and feedback connections do not appear to drive neurons in the target area when the target’s primary afferents have been removed (Girard & Bullier, 1989). Cortical cooling experiments that have reversibly inactivated higher cortical areas, however, have demonstrated profound impacts on neural activity in response to motion in lower areas like V1 (Hupé et al., 1998, 2001).
Upon inactivating MT, Hupé and colleagues (Hupé et al., 1998) found that a third of recorded cells in V1, V2, and V3 showed significant reductions in activity in response to a moving bar over a textured background. These effects were most significant in V3 neurons when the contrast between the bar and the textured background was reduced. Additionally, when background motion was introduced in the same direction as the bar motion, responses increased during MT inactivation. These results were taken as evidence that feedback can increase the gain of V1 neuron suppressive surrounds. More recent inactivation studies using cortical cooling in V2 and V3 have supported the facilitative effects of feedback on V1 surround suppression (Nassi, Gomez-Laberge, Kreiman, & Born, 2014; Nassi, Lomber, & Born, 2013). Feedback inactivation was found to have little effect on direction selectivity, and a small decrease in orientation selectivity when drifting gratings were restricted to the receptive field center. They also found a significant reduction in surround suppression during cooling as the size of the stimulus was increased. Taken together, these results suggest that feedback plays a role in refining neurons’ receptive fields beyond what is possible with just feedforward connections alone.
Theoretical work has proposed a number of roles for feedback in visual cortex, including the integration of form and motion (Beck & Neumann, 2010), integration of local motion estimates (Bayerl & Neumann, 2004), and amplification of low-contrast stimuli (Angelucci & Bressloff, 2006). Neurons in higher levels of extrastriate cortex relay motion information from larger portions of the visual field than do those at lower levels and thus feedback projections (e.g., from MT) can effectively increase the size of the surrounds in their target neurons (e.g., V1, V2, or V3). As a result of this recurrent architecture, noisy motion estimates are reduced at the lower levels, enhancing the representation at the higher level. Additionally, units in a two-stage model of the V1-MT connection that incorporates normalization with feedback can solve the aperture problem with temporal dynamics similar that of real physiological data (Bayerl & Neumann, 2004; Pack & Born 2001). Finally, using a model that integrates form (e.g., T-junctions, and end-stop positions) and motion information via a bidirectional connection between a MT and a V2 stage, Beck and Neumann generated units that capture the percepts reported for the chopstick illusion (Anstis, 1990) and physiological findings from MT neurons in response to occluded motion (Pack, Gartland, & Born, 2004).
High-Level Motion Processing
While the mid-level is robust in its representation of object motion, it cannot capture motion that is more complex or spatially extensive. Optic flow refers to complex, global motion that is due to translational movement of the observer through space (Figure 5). Linear movement of the body through space produces a radial motion pattern centered on the direction of travel. Rotations of the eye produce circular motion patterns, centered on the axis of rotation. The retina will receive both, and the combined flow field is sometimes termed “retinal flow.” An excellent treatment of the transformations to the retinal image encountered in optic flow is given in Longuet-Higgins and Prazdny (1980). “Biological motion,” seen when observing other animals in motion, is generally considered a distinct problem and will not be discussed here (see Giese & Poggio, 2003, for a review).
Self-Motion: Neural Codes and Heading Perception
Gibson hypothesized that optic flow contains sufficient information to support the perception of heading, and psychophysical studies have supported this hypothesis. Observers are sensitive to small changes in optic flow (Warren & Hannon, 1988) and exploit this cue during navigation (Warren, Kay, Zosh, Duchon, & Sahuc, 2001). Monkeys have a similar degree of perceptual sensitivity (Britten & van Wezel, 2002), which is likely supported by neurons in the high-level areas of the dorsal stream hierarchy. Neurons in medial superior temporal area (MST) (Duffy & Wurtz, 1991; Tanaka & Saito, 1989) as well as ventral intraparietal sulcus (VIP) (Maciokas & Britten, 2010; Schlack, Hoffmann, & Bremmer, 2003) are selective for components of retinal flow (e.g., Figure 5C), with tuning preferences for expanding, contracting, spiraling patterns of motion that span large areas of the visual field. It is assumed that these regions build flow selectivity from their dense connections with cells from MT (Figure 4) (Boussaoud, Ungerleider, & Desimone, 1990), and models have suggested numerous possible underlying mechanisms (Mineault, Khawaja, Butts, & Pack, 2012; Perrone & Stone, 1998; Tanaka, Fukada, & Saito, 1989). Regardless, these responses are sufficiently sensitive to changes in these components to support perceptual sensitivity (Heuer & Britten, 2004; Zhang & Britten, 2010). Activity in the two regions has been causally linked to the perception of heading, as revealed by microstimulation studies that biased the heading perception in the direction of the stimulated neurons’ tunings (Britten & van Wezel, 2002; Zhang & Britten, 2011).
These studies test heading perception in a controlled fashion, presenting flow patterns corresponding to an unchanging heading direction while the eyes remain fixed. Normally, heading must be calculated while the eyes are moving. This shifts and distorts the optic flow pattern, even for the same heading. Despite the corruption of the flow signal, primates handle this problem quite well: During smooth pursuit eye movements, both humans and monkeys can perceptually discount the majority of reafferent motion (Crowell & Andersen, 2001; Royden, Banks, & Crowell, 1992). Neurons in MST are also able to partially correct for this pursuit-related motion by shifting their preferred heading directions in the direction of the change of the flow center anywhere from 30–51% of what is required for perfect compensation (Bradley, Maxwell, Andersen, Banks, & Shenoy, 1996; Shenoy, Crowell, & Andersen, 2002). Similar compensatory changes in single neurons also occur in VIP, and in fact may be more complete (Zhang, Heuer, & Britten, 2004).
There is still some debate as to how this compensation is accomplished. In theory, an optic flow field that represents self-motion in three dimensions contains sufficient information for the problem to be solved based on the retinal image alone (Heeger & Jepson, 1992; Longuet-Higgins & Prazdny, 1980; Warren & Hannon, 1990). The brain could also use extraretinal signals to solve the problem. These signals could arise either from eye-muscle proprioception (Wang, Zhang, Cohen, & Goldberg, 2007) or “efference copy” signals reflecting motor commands (Crapse & Sommer, 2008). To investigate whether retinal or extraretinal methods best account for behavior and neuronal activity, many groups have compared estimates of heading direction under real and simulated eye-movement conditions. In the latter condition, subjects are presented with optic flow representing a particular heading direction, but motion due to retinal slip is added to the screen while the eyes remain fixated. This keeps the retinal image the same between the two conditions, while removing any influence of efference copy signals.
Evidence from psychophysical studies suggests that extraretinal cues are necessary for heading perception based on the decreased precision of heading direction judgments during simulated eye movement (Royden, Crowell, & Banks, 1994). On the other hand, there is little physiological evidence that demonstrates how efference copy signals are combined with heading signals in MST or VIP. These neurons are modulated by eye position and speed (Squatrito & Maioli, 1997) and show overt extraretinal signals related to eye movements (Newsome, Wurtz, & Komatsu, 1988). In theory, efference copy would produce different neuronal responses during the real and simulated pursuit conditions. These differences could manifest as the simple addition of retinal and extraretinal inputs (Perrone & Krauzlis, 2008), or as more complicated nonlinear combinations of the two signals (Ben Hamed, Page, Duffy, & Pouget, 2003). One study has argued in favor of the purely retinal mechanism based on the responses of neurons in VIP during a simulated pursuit task (Sunkara, DeAngelis, & Angelaki, 2016). The authors find that the visual field deformations due to eye rotations are not simply discounted, but are instead jointly represented by VIP neurons alongside translational self-motion even in the simulated pursuit trials.
In primates, visual motion provides precise cues to self-motion direction. Its usefulness, however, decreases in conditions with sparse visual cues like darkness, fog, or snowstorm. Under these conditions, self-motion representation can be improved by combining these noisy visual cues with sensory information from other modalities related to self-motion (Stein & Stanford, 2008). Auditory cues can elicit illusions of self-motion in roughly 20–60% of those tested (Valjamae, 2009), but these percepts are generally weak, limited to rotational motion, and have an unknown neural basis. Proprioceptive cues alone are not very informative about self-motion relative to the world, but are critical to reframing the initial head or eye-centered representations in terms of whole body motion (Cullen, 2011). The semicircular canals and otolith organs of the vestibular system, however, have evolved specifically to sense rotational and translational acceleration, respectively. Vestibular perception is highly sensitive to changes in head velocity, though it cannot accurately report self-motion at slower (<0.1 Hz) angular velocities and can misinterpret linear acceleration as tilt (Benson, 1990). Given these perceptual findings, it is important to investigate potential neural substrates.
Flow-sensitive neurons in both MST and VIP receive strong vestibular projections alongside their visual inputs, and are selective for self-motion direction based on vestibular cues alone (Gu, Watkins, Angelaki, & DeAngelis, 2006). VIP neurons generally show congruent tunings for visual and vestibular self-motion direction (Chen, Deangelis, & Angelaki, 2013). This is similar to results from visual/auditory cue-combination experiments in the superior colliculus, where congruent tuning is the rule (Knudsen, 1982; Stein & Stanford, 2008). In MST neurons, however, the relationship between visual and vestibular preference for body translation and rotation is complex (Takahashi et al., 2007), with vestibular tuning often weaker and incongruent with tuning for optic flow. Bilateral, reversible inactivation of MST with muscimol leads to massive increases in perceptual thresholds for visual heading discrimination, with modest influences on discrimination based on vestibular stimuli (Gu, Deangelis, & Angelaki, 2012). Similar studies in VIP and the vestibular dominant regions parietoinsular vestibular cortex (PIVC) and visual posterior sylvian area (VPS) found increased vestibular-based discrimination thresholds only for PIVC/PVS (Chen, Gu, Liu, Deangelis, & Angelaki, 2016), with relatively small visual deficits. Whether these signals are more thoroughly integrated at another hierarchical level, or remain separated in a distributed code remains unknown.
Regardless of how extensively sensory modalities are integrated, it is also important to determine which operations are used to combine them. For optimal integration, the brain should weight the most reliable (i.e., highest signal-to-noise ratio) sensory signal the highest, lest it obscure an accurate reading of self-motion with a poor one. This is formalized using a framework of Bayesian inference (Ma, Beck, Latham, & Pouget, 2006) in which the probability of some state of self-motion is computed given observed activity from visual and vestibular inputs, where both have their own degrees of firing-rate variability. Thus, integration is maximally useful when both visual and vestibular cues are weak, but this utility decreases as either one becomes stronger (Meredith & Stein, 1986). Both behavioral and neuronal correlates (in MST) of this optimal weighting have been found in heading discrimination tasks, where the reliability of the vestibular and visual cues were manipulated throughout the experiments. (Fetsch, Pouget, DeAngelis, & Angelaki, 2012; Fetsch, Turner, DeAngelis, & Angelaki, 2009). Bayesian-like operations may be instantiated in neural circuits via a normalization scheme similar to those discussed in Sections “Neuronal Motion Filters: Central Circuits” and 3.3.2 (Ohshiro, Angelaki, & DeAngelis, 2011). Noisy, unreliable sensory signals are weak under a normalization regime, since no signal is strong enough to overcome the divisive influences of other neurons in the population. When the sensory inputs converge on a multisensory neuron, instead of summing linearly, they are combined in a weak winner-take-all fashion that favors the most reliable (stronger) stimulus unless the two inputs are roughly equally reliable.
Readout of Motion Representations
The study of visual motion representations has focused largely on encoding—how external stimuli are represented in the firing of neuronal populations. Considerable attention has been directed at the converse problem of decoding these representations to generate perception and observable behavior.
What is optimal for one problem (such as identifying the direction of a moving object) might not be optimal for another (such as detecting a weak signal in noise). And the familiar trade-off between speed and accuracy will have different outcomes depending on the problem. Traditional approaches to the problem, such as vector averaging of population responses, which are computationally simple and work reasonably well, do not take into account such constraints, and have largely been replaced by Bayesian approaches in modern work (Jazayeri & Movshon, 2006; Pouget, Dayan, & Zemel, 2003).
Bayesian-like decoding models can be built using biologically plausible circuits, and these models well predict psychophysical performance on detection and discrimination tasks. Like vector decoding, Bayesian decoding strategies have their own caveats. They largely assume that the responses of individual neurons in the population are independent, but this assumption is often violated. In cortex, spike-count correlations between neurons in the same area can be significant depending on where the area is situated in the processing hierarchy, the tuning similarity between two neurons, and whether or not the animal is anesthetized (Cohen & Kohn, 2011). Correlations between units in a probabilistic framework can reduce the amount of information the population can carry, decreasing the resolution at which stimuli are encoded. However, correlations are not always deleterious to population decoding, and may improve perceptual sensitivity in some circumstances (Averbeck, Latham, & Pouget, 2006; Zylberberg, Cafaro, Turner, Shea-Brown, & Rieke, 2016).
While the theory of readout is reasonably well-established, there remains a persistent problem of testing such hypotheses. One approach that has considerable traction is to use correlations between noise and performance, which in the context of decisions is termed choice probability (Britten, Newsome, Shadlen, Celebrini, & Movshon, 1996; Crapse & Basso, 2015). While useful and quantitative, these are still correlations and cannot prove causation. Stimulation methods are helpful here, and have a long and elegant history in the study of motor systems. However, the big unknowns with these methods have to do with defining the tissue being activated, and quantifying the degree of activation. Optogenetics offers considerable promise because of the ability to precisely control the cell types being activated, and which should result in substantial progress on defining readout in the near future.
This article has briefly summarized how visual motion is processed in the primate brain, and highlighted a selection of the most active areas of research in the field. An active research area uncovers more questions than it answers, so a few that are ripe for exploration are highlighted here:
1. What is the role of feedback in the motion pathway? This article has focused on the feedforward architecture because it suits theoretical accounts and the physiology well. But feedback likely performs important functions as well. In object vision, models suggest that feedback is helpful in assigning features to objects. Guided by this thought, it seems timely to look for influences of feedback on segmentation of motion in the scene.
2. Correlations in spiking activity are omnipresent but ill-understood. Theory suggests they carry potential for costs and benefits to computation and performance. More data is needed to understand the prevalence, statistics, and mechanisms of correlations in the motion system. High-density recording methods are already vastly improving our understanding, but more needs to be done.
3. How is direction selectivity calculated in cortex? While connectomics has pointed to a candidate circuit for the calculation in the insect lamina and in mammalian retina, progress in mammalian cortex is lagging. Advances in multiphoton calcium imaging and cell-type-specific labeling should make the question addressable.
4. How do the dorsal and ventral streams interact? Perceptually, object identities are informed by their motions, and object features such as color interact with motion percepts. For now, these are typically studied by different laboratories, but the two pathways are extensively interconnected. With the advent of large-scale, multi-area recording methods, such questions become addressable.
5. How is motion information used in closed-loop control of ongoing movement? Much of our study of motion perception and even action exploits open-loop design, yet most motion-guided behavior is continuously updated by new information. This is difficult to study because some loss of stimulus control is inevitable, but advances in control systems theory and decoding algorithms should make progress possible.
Borst, A., & Euler, T. (2011). Seeing things in motion: Models, circuits, and mechanisms. Neuron, 71(6), 974–994.Find this resource:
Burr, D., & Thompson, P. (2011). Motion psychophysics: 1985–2010. Vision Research, 51(13), 1431–1456.Find this resource:
Giese, M. A., & Poggio, T. (2003). Neural mechanisms for the recognition of biological movements. Nature Reviews Neuroscience, 4(3), 179–192.Find this resource:
Longuet-Higgins, H. C., & Prazdny, K. (1980). The interpretation of a moving retinal image. Proceedings of the Royal Society of London B: Biological Sciences, 208(1173), 385–397.Find this resource:
Vaney, D. I., Sivyer, B., & Taylor, W. R. (2012). Direction selectivity in the retina: Symmetry and asymmetry in structure and function. Nature Reviews Neuroscience, 13(3), 194–208.Find this resource:
Adelson, E. H., & Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Journal of the Optical Society of America A, 2(2), 284–299.Find this resource:
Adrian, E. D. (1928). The basis of sensation. New York: W. W. Norton & Co.Find this resource:
Albright, T. D. (1984). Direction and orientation selectivity of neurons in visual area MT of the macaque. Journal of Neurophysiology, 52(6), 1106–1130.Find this resource:
Andersen, R. A., Snowden, R. J., Treue, S., & Graziano, M. (1990). Hierarchical processing of motion in the visual cortex of monkey. Cold Spring Harbor Symposia on Quantitative Biology, 55, 741–748.Find this resource:
Angelucci, A., & Bressloff, P. C. (2006). Contribution of feedforward, lateral and feedback connections to the classical receptive field center and extra-classical receptive field surround of primate V1 neurons. Progress in Brain Research, 154, 93–120.Find this resource:
Angelucci, A., Levitt, J. B., Walton, E. J., Hupe, J. M., Bullier, J., & Lund, J. S. (2002). Circuits for local and global signal integration in primary visual cortex. Journal of Neuroscience, 22(19), 8633–8646.Find this resource:
Anstis, S. (1990). Imperceptible intersections: The chopstick illusion. In A. Blake & T. Troscianko (Eds.), AI and the eye (pp. 105–117). London: Wiley and Sons Ltd.Find this resource:
Averbeck, B. B., Latham, P. E., & Pouget, A. (2006). Neural correlations, population coding and computation. Nature Reviews Neuroscience, 7(5), 358–366.Find this resource:
Baker, P. M., & Bair, W. (2012). Inter-neuronal correlation distinguishes mechanisms of direction selectivity in cortical circuit models. Journal of Neuroscience, 32(26), 8800–8816.Find this resource:
Barlow, H. B., & Levick, W. R. (1965). The mechanism of directionally selective units in rabbit’s retina. Journal of Physiology, 178(3), 477–504.Find this resource:
Bayerl, P., & Neumann, H. (2004). Disambiguating visual motion through contextual feedback modulation. Neural Computation, 16(10), 2041–2066.Find this resource:
Beck, C., & Neumann, H. (2010). Interactions of motion and form in visual cortex—a neural model. Journal of Physiology-Paris, 104(1–2), 61–70.Find this resource:
Ben Hamed, S., Page, W., Duffy, C., & Pouget, A. (2003). MSTd neuronal basis functions for the population encoding of heading direction. Journal of Neurophysiology, 90(2), 549–558.Find this resource:
Benson, A. J. (1990). Sensory Functions and Limitations of the Vestibular System. In R. Warren & A. H. Wertheim (Eds.), Perception and control of self-motion (pp. 145–170). Lawrence Erlbaum Associates, Inc.Find this resource:
Born, R. T., & Bradley, D. C. (2005). Structure and function of visual area MT. Annual Review of Neuroscience, 28, 157–189.Find this resource:
Boussaoud, D., Ungerleider, L. G., & Desimone, R. (1990). Pathways for motion analysis: Cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque. Journal of Comparative Neurology, 296(3), 462–495.Find this resource:
Bradley, D. C., & Andersen, R. A. (1998). Center-surround antagonism based on disparity in primate area MT. Journal of Neuroscience, 18(18), 7552–7565.Find this resource:
Bradley, D. C., Maxwell, M., Andersen, R. A., Banks, M. S., & Shenoy, K. V. (1996). Mechanisms of heading perception in primate visual cortex. Science, 273(5281), 1544–1547.Find this resource:
Briggman, K. L., Helmstaedter, M., & Denk, W. (2011). Wiring specificity in the direction-selectivity circuit of the retina. Nature, 471(7337), 183–188.Find this resource:
Britten, K. H. (2008). Mechanisms of self-motion perception. Annu Rev Neurosci, 31, 389–410.Find this resource:
Britten, K. H., Newsome, W. T., Shadlen, M. N., Celebrini, S., & Movshon, J. A. (1996). A relationship between behavioral choice and the visual responses of neurons in macaque MT. Visual Neuroscience, 13(1), 87–100.Find this resource:
Britten, K. H., & van Wezel, R. J. (2002). Area MST and heading perception in macaque monkeys. Cerebral Cortex, 12(7), 692–701.Find this resource:
Carandini, M., Demb, J. B., Mante, V., Tolhurst, D. J., Dan, Y., Olshausen, B. A., … Rust, N. C. (2005). Do we know what the early visual system does? Journal of Neuroscience, 25(46), 10,577–10,597.Find this resource:
Carandini, M., & Heeger, D. J. (2012). Normalization as a canonical neural computation. Nature Reviews Neuroscience, 13(1), 51–62.Find this resource:
Chen, A., Deangelis, G. C., & Angelaki, D. E. (2013). Functional specializations of the ventral intraparietal area for multisensory heading discrimination. Journal of Neuroscience, 33(8), 3567–3581.Find this resource:
Chen, A., Gu, Y., Liu, S., Deangelis, & Angelaki, D. E. (2016). Evidence for a causal contribution of macaque vestibular, but not intraparietal, cortex to heading perception. Journal of Neuroscience, 36(13), 3789–3798.Find this resource:
Chen, G., Lu, H. D., & Roe, A. W. (2008). A map for horizontal disparity in monkey V2. Neuron, 58(3), 442–450.Find this resource:
Churchland, M. M., Priebe, N. J., & Lisberger, S. G. (2005). Comparison of the spatial limits on direction selectivity in visual areas MT and V1. J Neurophysiol, 93(3), 1235–1245.Find this resource:
Clifford, C. W. G., & Ibbotson, M. R. (2002). Fundamental mechanisms of visual motion detection: Models, cells and functions. Progress in Neurobiology, 68(6), 409–437.Find this resource:
Cohen, M. R., & Kohn, A. (2011). Measuring and interpreting neuronal correlations. Nature Neuroscience, 14(7), 811–819.Find this resource:
Colby, C. L., Gattass, R., Olson, C. R., & Gross, C. G. (1988). Topographical organization of cortical afferents to extrastriate visual area PO in the macaque: A dual tracer study. Journal of Comparative Neurology, 269(3), 392–413.Find this resource:
Crapse, T. B., & Basso, M. A. (2015). Insights into decision making using choice probability. Journal of Neurophysiology, 114(6), 3039–3049.Find this resource:
Crapse, T. B., & Sommer, M. A. (2008). Corollary discharge across the animal kingdom. Nature Reviews Neuroscience, 9(8), 587–600.Find this resource:
Crowell, J. A., & Andersen, R. A. (2001). Pursuit compensation during self-motion. Perception, 30(12), 1465–1488.Find this resource:
Cruz-Martin, A., El-Danaf, R. N., Osakada, F., Sriram, B., Dhande, O. S., Nguyen, P. L., … Huberman, A. D. (2014). A dedicated circuit links direction-selective retinal ganglion cells to the primary visual cortex. Nature, 507(7492), 358–361.Find this resource:
Cui, Y., Liu, L. D., Khawaja, F. A., Pack, C. C., & Butts, D. A. (2013). Diverse suppressive influences in area MT and selectivity to complex motion features. Journal of Neuroscience, 33(42), 16,715–16,728.Find this resource:
Cullen, K. E. (2011). The neural encoding of self-motion. Current Opinion in Neurobiology, 21(4), 587–595.Find this resource:
Czuba, T. B., Huk, A. C., Cormack, L. K., & Kohn, A. (2014). Area MT encodes three-dimensional motion. Journal of Neuroscience, 34(47), 15,522–15,533.Find this resource:
DeAngelis, G. C., Ohzawa, I., & Freeman, R. D. (1993). Spatiotemporal organization of simple-cell receptive fields in the cat’s striate cortex. II. Linearity of temporal and spatial summation. Journal of Neurophysiology, 69(4), 1118–1135.Find this resource:
De Valois, R. L., Albrecht, D. G., & Thorell, L. G. (1982). Spatial-frequency selectivity of sells in macaque visual-cortex. Vision Research, 22(5), 545–559.Find this resource:
De Valois, R. L., & Cottaris, N. P. (1998). Inputs to directionally selective simple cells in macaque striate cortex. Proceedings of the National Academy of Sciences of the United States of America, 95(24), 14,488–14,493.Find this resource:
DeValois, R. L., & DeValois, K. K. (1988). Linear systems analysis. Spatial vision. New York: Oxford University Press.Find this resource:
Dhande, O. S., Stafford, B. K., Lim, J.-H. A., & Huberman, A. D. (2015). Contributions of retinal ganglion cells to subcortical visual processing and behaviors. Annual Review of Vision Science, 1(1), 291–328.Find this resource:
Duffy, C. J., & Wurtz, R. H. (1991). Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. Journal of Neurophysiology, 65(6), 1329–1345.Find this resource:
Fan, R. H., Liu, S., DeAngelis, G. C., & Angelaki, D. E. (2015). Heading tuning in macaque area V6. Journal of Neuroscience, 35(50), 16,303–16,314.Find this resource:
Fattori, P., Pitzalis, S., & Galletti, C. (2009). The cortical visual area V6 in macaque and human brains. Journal of Physiology-Paris, 103(1–2), 88–97.Find this resource:
Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1), 1–47.Find this resource:
Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012). Neural correlates of reliability-based cue weighting during multisensory integration. Nature Neuroscience, 15(1), 146–154.Find this resource:
Fetsch, C. R., Turner, A. H., DeAngelis, G. C., & Angelaki, D. E. (2009). Dynamic reweighting of visual and vestibular cues during self-motion perception. Journal of Neuroscience, 29(49), 15601–15612.Find this resource:
Frégnac, Y., & Bathellier, B. (2015). Cortical correlates of low-level perception: From neural circuits to percepts. Neuron, 88(1), 110–126.Find this resource:
Fried, S. I., Munch, T. A., & Werblin, F. S. (2002). Mechanisms and circuitry underlying directional selectivity in the retina. Nature, 420(6914), 411–414.Find this resource:
Galletti, C., Gamberini, M., Kutz, D. F., Baldinotti, I., & Fattori, P. (2005). The relationship between V6 and PO in macaque extrastriate cortex. The European Journal of Neuroscience, 21(4), 959–970.Find this resource:
Gardner, J. L., Anzai, A., Ohzawa, I., & Freeman, R. D. (1999). Linear and nonlinear contributions to orientation tuning of simple cells in the cat’s striate cortex. Visual Neuroscience, 16(6), 1115–1121.Find this resource:
Gibson, J. J. (1950). The perception of the visual world. Oxford: Houghton Mifflin.Find this resource:
Girard, P., & Bullier, J. (1989). Visual activity in area V2 during reversible inactivation of area 17 in the macaque monkey. Journal of Neurophysiology, 62(6), 1287–1302.Find this resource:
Gu, Y., Deangelis, G. C., & Angelaki, D. E. (2012). Causal links between dorsal medial superior temporal area neurons and multisensory heading perception. Journal of Neuroscience, 32(7), 2299–2313.Find this resource:
Gu, Y., Watkins, P. V., Angelaki, D. E., & DeAngelis, G. C. (2006). Visual and nonvisual contributions to three-dimensional heading selectivity in the medial superior temporal area. Journal of Neuroscience, 26(1), 73–85.Find this resource:
Gur, M., & Snodderly, D. M. (2007). Direction selectivity in V1 of alert monkeys: Evidence for parallel pathways for motion processing. Journal of Physiology, 585(Pt. 2), 383–400.Find this resource:
Heeger, D. J., & Jepson, A. D. (1992). Subspace methods for recovering rigid motion .1. Algorithm and implementation. International Journal of Computer Vision, 7(2), 95–117.Find this resource:
Heuer, H. W., & Britten, K. H. (2004). Optic flow signals in extrastriate area MST: Comparison of perceptual and neuronal sensitivity. Journal of Neurophysiology, 91(3), 1314–1326.Find this resource:
Hirsch, J. A., & Martinez, L. M. (2006). Circuits that build visual cortical receptive fields. Trends in Neurosciences, 29(1), 30–39.Find this resource:
Huang, X., Albright, T. D., & Stoner, G. R. (2007). Adaptive surround modulation in cortical area MT. Neuron, 53(5), 761–770.Find this resource:
Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurons in the cat’s striate cortex. Journal of Physiology, 148(3), 574–591.Find this resource:
Huk, A. C. (2012). Multiplexing in the primate motion pathway. Vision Research, 62, 173–180.Find this resource:
Hunter, J. N., & Born, R. T. (2011). Stimulus-dependent modulation of suppressive influences in MT. Journal of Neuroscience, 31(2), 678–686.Find this resource:
Hupé, J. M., James, A. C., Girard, P., Lomber, S. G., Payne, B. R., & Bullier, J. (2001). Feedback connections act on the early part of the responses in monkey visual cortex. Journal of Neurophysiology, 85(1), 134–145.Find this resource:
Hupé, J. M., James, A. C., Payne, B. R., Lomber, S. G., Girard, P., & Bullier, J. (1998). Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature, 394(6695), 784–787.Find this resource:
Jazayeri, M., & Movshon, J. A. (2006). Optimal representation of sensory information by neural populations. Nature Neuroscience, 9(5), 690–696.Find this resource:
Kim, T., & Freeman, R. D. (2016). Direction selectivity of neurons in visual cortex is non-linear and laminar dependent. The European Journal of Neuroscience, 43(10), 1389–1399.Find this resource:
Knudsen, E. I. (1982). Auditory and visual maps of space in the optic tectum of the owl. Journal of Neuroscience, 2(9), 1177–1194.Find this resource:
Kumbhani, R. D., El-Shamayleh, Y., & Movshon, J. A. (2015). Temporal and spatial limits of pattern motion sensitivity in macaque MT neurons. Journal of Neurophysiology, 113(7), 1977–1988.Find this resource:
Livingstone, M. S., & Conway, B. R. (2003). Substructure of direction-selective receptive fields in macaque V1. Journal of Neurophysiology, 89(5), 2743–2759.Find this resource:
Livingstone, M. S., Pack, C. C., & Born, R. T. (2001). Two-dimensional substructure of MT receptive fields. Neuron, 30(3), 781–793.Find this resource:
Longuet-Higgins, H. C., & Prazdny, K. (1980). The interpretation of a moving retinal image. Proceedings of the Royal Society of London B: Biological Science, 208(1173), 385–397.Find this resource:
Lu, H. D., Chen, G., Tanigawa, H., & Roe, A. W. (2010). A motion direction map in macaque V2. Neuron, 68(5), 1002–1013.Find this resource:
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006). Bayesian inference with probabilistic population codes. Nature Neuroscience, 9(11), 1432–1438.Find this resource:
Maciokas, J. B., & Britten, K. H. (2010). Extrastriate area MST and parietal area VIP similarly represent forward headings. Journal of Neurophysiology, 104(1), 239–247.Find this resource:
Markov, N. T., Vezoli, J., Chameau, P., Falchier, A., Quilodran, R., Huissoud, C., … Kennedy, H. (2014). Anatomy of hierarchy: Feedforward and feedback pathways in macaque visual cortex. Journal of Comparative Neurology, 522(1), 225–259.Find this resource:
Maunsell, J. H., & Van Essen, D. C. (1983). Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. Journal of Neurophysiology, 49(5), 1148–1167.Find this resource:
Meredith, M. A., & Stein, B. E. (1986). Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. Journal of Neurophysiology, 56(3), 640–662.Find this resource:
Mineault, P. J., Khawaja, F. A., Butts, D. A., & Pack, C. C. (2012). Hierarchical processing of complex motion along the primate dorsal visual pathway. Proceedings of the National Academy of Sciences of the United States of America, 109(16), E972–E980.Find this resource:
Monier, C., Chavane, F., Baudot, P., Graham, L. J., & Fregnac, Y. (2003). Orientation and direction selectivity of synaptic inputs in visual cortical neurons: a diversity of combinations produces spike tuning. Neuron, 37(4), 663–680.Find this resource:
Movshon, J. A., Adelson, E. H., Gizzi, M. S., & Newsome, W. T. (1985). The analysis of moving visual patterns. Paper presented at the Pattern Recognition Mechanisms, Rome.Find this resource:
Movshon, J. A., & Newsome, W. T. (1996). Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys. Journal of Neuroscience, 16(23), 7733–7741.Find this resource:
Nassi, J. J., & Callaway, E. M. (2009). Parallel processing strategies of the primate visual system. Nature Reviews Neuroscience, 10(5), 360–372.Find this resource:
Nassi, J. J., Gomez-Laberge, C., Kreiman, G., & Born, R. T. (2014). Corticocortical feedback increases the spatial extent of normalization. Frontiers in systems neuroscience, 8, 105.Find this resource:
Nassi, J. J., Lomber, S. G., & Born, R. T. (2013). Corticocortical feedback contributes to surround suppression in V1 of the alert primate. Journal of Neuroscience, 33(19), 8504–8517.Find this resource:
Newsome, W. T., Wurtz, R. H., & Komatsu, H. (1988). Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs. Journal of Neurophysiology, 60(2), 604–620.Find this resource:
Nurminen, L., & Angelucci, A. (2014). Multiple components of surround modulation in primary visual cortex: Multiple neural circuits with multiple functions? Vision Research, 104, 47–56.Find this resource:
Ohshiro, T., Angelaki, D. E., & DeAngelis, G. C. (2011). A normalization model of multisensory integration. Nature Neuroscience, 14(6), 775–782.Find this resource:
Pack, C. C., & Born, R. T. (2001). Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain. Nature, 409(6823), 1040–1042.Find this resource:
Pack, C. C., Gartland, A. J., & Born, R. T. (2004). Integration of contour and terminator signals in visual area MT of alert macaque. Journal of Neuroscience, 24(13), 3268–3280.Find this resource:
Pack, C. C., Livingstone, M. S., Duffy, K. R., & Born, R. T. (2003). End-stopping and the aperture problem: Two-dimensional motion signals in macaque V1. Neuron, 39(4), 671–680.Find this resource:
Perrone, J. A., & Krauzlis, R. J. (2008). Vector subtraction using visual and extraretinal motion signals: A new look at efference copy and corollary discharge theories. Journal of Vision, 8(14), 24.Find this resource:
Perrone, J. A., & Stone, L. S. (1998). Emulating the visual receptive-field properties of MST neurons with a template model of heading estimation. Journal of Neuroscience, 18(15), 5958–5975.Find this resource:
Perrone, J. A., & Thiele, A. (2001). Speed skills: Measuring the visual speed analyzing properties of primate MT neurons. Nature Neuroscience, 4(5), 526–532.Find this resource:
Peterson, M. R., Li, B., & Freeman, R. D. (2004). The derivation of direction selectivity in the striate cortex. Journal of Neuroscience, 24(14), 3583–3591.Find this resource:
Pitzalis, S., Fattori, P., & Galletti, C. (2012). The functional role of the medial motion area V6. Frontiers in Behavioral Neuroscience, 6, 91.Find this resource:
Ponce, C. R., Hunter, J. N., Pack, C. C., Lomber, S. G., & Born, R. T. (2011). Contributions of indirect pathways to visual response properties in macaque middle temporal area MT. Journal of Neuroscience, 31(10), 3894–3903.Find this resource:
Ponce, C. R., Lomber, S. G., & Born, R. T. (2008). Integrating motion and depth via parallel pathways. Nature Neuroscience, 11(2), 216–223.Find this resource:
Pouget, A., Dayan, P., & Zemel, R. S. (2003). Inference and computation with population codes. Annual Review of Neuroscience, 26, 381–410.Find this resource:
Priebe, N. J., Cassanello, C. R., & Lisberger, S. G. (2003). The neural representation of speed in macaque area MT/V5. Journal of Neuroscience, 23(13), 5650–5661.Find this resource:
Priebe, N. J., & Ferster, D. (2005). Direction selectivity of excitation and inhibition in simple cells of the cat primary visual cortex. Neuron, 45(1), 133–145.Find this resource:
Priebe, N. J., Lampl, I., & Ferster, D. (2010). Mechanisms of direction selectivity in cat primary visual cortex as revealed by visual adaptation. Journal of Neurophysiology, 104(5), 2615–2623.Find this resource:
Priebe, N. J., Lisberger, S. G., & Movshon, J. A. (2006). Tuning for spatiotemporal frequency and speed in directionally selective neurons of macaque striate cortex. Journal of Neuroscience, 26(11), 2941–2950.Find this resource:
Reichardt, W. (1962). Nervous integration in the facet eye. Biophysical Journal, 2, 121–143.Find this resource:
Reid, R. C., Soodak, R. E., & Shapley, R. M. (1991). Direction selectivity and spatiotemporal structure of receptive fields of simple cells in cat striate cortex. Journal of Neurophysiology, 66(2), 505–529.Find this resource:
Rivlin-Etzion, M., Zhou, K., Wei, W., Elstrott, J., Nguyen, P. L., Barres, B. A., … Feller, M. B. (2011). Transgenic mice reveal unexpected diversity of on-off direction-selective retinal ganglion cell subtypes and brain structures involved in motion processing. Journal of Neuroscience, 31(24), 8760–8769.Find this resource:
Roy, J. P., Komatsu, H., & Wurtz, R. H. (1992). Disparity sensitivity of neurons in monkey extrastriate area MST. Journal of Neuroscience, 12(7), 2478–2492.Find this resource:
Royden, C. S., Banks, M. S., & Crowell, J. A. (1992). The perception of heading during eye movements. Nature, 360(6404), 583–585.Find this resource:
Royden, C. S., Crowell, J. A., & Banks, M. S. (1994). Estimating heading during eye movements. Vision Research, 34(23), 3197–3214.Find this resource:
Rust, N. C., Mante, V., Simoncelli, E. P., & Movshon, J. A. (2006). How MT cells analyze the motion of visual patterns. Nature Neuroscience, 9(11), 1421–1431.Find this resource:
Rust, N. C., Schwartz, O., Movshon, J. A., & Simoncelli, E. P. (2005). Spatiotemporal elements of macaque V1 receptive fields. Neuron, 46(6), 945–956.Find this resource:
Sanada, T. M., & DeAngelis, G. C. (2014). Neural representation of motion-in-depth in area MT. Journal of Neuroscience, 34(47), 15,508–15,521Find this resource:
Sanes, J. R., & Masland, R. H. (2015). The types of retinal ganglion cells: Current status and implications for neuronal classification. Annual Review of Neuroscience, 38, 221–246.Find this resource:
Saul, A. B., Carras, P. L., & Humphrey, A. L. (2005). Temporal properties of inputs to direction-selective neurons in monkey V1. Journal of Neurophysiology, 94(1), 282–294.Find this resource:
Schiff, W., Caviness, J. A., & Gibson, J. J. (1962). Persistent fear responses in rhesus monkeys to the optical stimulus of “looming.” Science, 136(3520), 982–983.Find this resource:
Schlack, A., Hoffmann, K. P., & Bremmer, F. (2003). Selectivity of macaque ventral intraparietal area (area VIP) for smooth pursuit eye movements. Journal of Physiology, 551(Pt. 2), 551–561.Find this resource:
Shenoy, K. V., Crowell, J. A., & Andersen, R. A. (2002). Pursuit speed compensation in cortical area MSTd. Journal of Neurophysiology, 88(5), 2630–2647.Find this resource:
Sherman, S. M., & Guillery, R. W. (1998). On the actions that one nerve cell can have on another: Distinguishing “drivers” from “modulators.” Proceedings of the National Academy of Sciences of the United States of America, 95(12), 7121–7126.Find this resource:
Simoncelli, E. P., & Heeger, D. J. (1998). A model of neuronal responses in visual area MT. Vision Research, 38(5), 743–761.Find this resource:
Smith, M. A., Majaj, N. J., & Movshon, J. A. (2005). Dynamics of motion signaling by neurons in macaque area MT. Nature Neuroscience, 8(2), 220–228.Find this resource:
Smolyanskaya, A., Haefner, R. M., Lomber, S. G., & Born, R. T. (2015). A modality-specific feedforward component of choice-related activity in MT. Neuron, 87(1), 208–219.Find this resource:
Smolyanskaya, A., Ruff, D. A., & Born, R. T. (2013). Joint tuning for direction of motion and binocular disparity in macaque MT is largely separable. Journal of Neurophysiology, 110(12), 2806–2816.Find this resource:
Squatrito, S., & Maioli, M. G. (1997). Encoding of smooth pursuit direction and eye position by neurons of area MSTd of macaque monkey. Journal of Neuroscience, 17(10), 3847–3860.Find this resource:
Stein, B. E., & Stanford, T. R. (2008). Multisensory integration: Current issues from the perspective of the single neuron. Nature Reviews Neuroscience, 9(4), 255–266.Find this resource:
Stettler, D. D., Das, A., Bennett, J., & Gilbert, C. D. (2002). Lateral connectivity and contextual interactions in macaque primary visual cortex. Neuron, 36(4), 739–750.Find this resource:
Sun, W., Tan, Z., Mensh, B. D., & Ji, N. (2016). Thalamus provides layer 4 of primary visual cortex with orientation- and direction-tuned inputs. Nature Neuroscience, 19(2), 308–315.Find this resource:
Sunkara, A., DeAngelis, G. C., & Angelaki, D. E. (2016). Joint representation of translational and rotational components of optic flow in parietal cortex. Proceedings of the National Academy of the Sciences of the United States of America, 113(18), 5077–5082.Find this resource:
Takahashi, K., Gu, Y., May, P. J., Newlands, S. D., DeAngelis, G. C., & Angelaki, D. E. (2007). Multimodal coding of three-dimensional rotation and translation in area MSTd: Comparison of visual and vestibular selectivity. Journal of Neuroscience, 27(36), 9742–9756.Find this resource:
Tanaka, K., Fukada, Y., & Saito, H. A. (1989). Underlying mechanisms of the response specificity of expansion/contraction and rotation cells in the dorsal part of the medial superior temporal area of the macaque monkey. Journal of Neurophysiology, 62(3), 642–656.Find this resource:
Tanaka, K., & Saito, H. (1989). Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. Journal of Neurophysiology, 62(3), 626–641.Find this resource:
Ungerleider, L. G., & Desimone, R. (1986). Cortical connections of visual area MT in the macaque. Journal of Comparative Neurology, 248(2), 190–222.Find this resource:
Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In D. J. Ingle, M. A. Goodale, & R. J. W. Mansfield (Eds.), Analysis of visual behavior (pp. 549–586). Cambridge, MA: MIT Press.Find this resource:
Valjamae, A. (2009). Auditorily-induced illusory self-motion: A review. Brain Research Reviews, 61(2), 240–255.Find this resource:
van Santen, J. P., & Sperling, G. (1984). Temporal covariance model of human motion perception. Journal of the Optical Society of America A, 1(5), 451–473.Find this resource:
van Santen, J. P., & Sperling, G. (1985). Elaborated Reichardt detectors. Journal of the Optical Society of America A, 2(2), 300–321.Find this resource:
Vintch, B., Movshon, J. A., & Simoncelli, E. P. (2015). A convolutional subunit model for neuronal responses in macaque V1. Journal of Neuroscience, 35(44), 14,829–14,841.Find this resource:
Vlasits, A. L., Morrie, R. D., Tran-Van-Minh, A., Bleckert, A., Gainer, C. F., DiGregorio, D. A., & Feller, M. B. (2016). Role for synaptic input distribution in a dendritic computation of motion direction in the retina. Neuron, 89, 1317–1330.Find this resource:
Wang, H. X., & Movshon, J. A. (2016). Properties of pattern and component direction-selective cells in area MT of the macaque. Journal of Neurophysiology, 115(6), 2705–2720.Find this resource:
Wang, X., Zhang, M., Cohen, I. S., & Goldberg, M. E. (2007). The proprioceptive representation of eye position in monkey primary somatosensory cortex. Nature Neuroscience, 10(5), 640–646.Find this resource:
Warren, W. H., & Hannon, D. J. (1988). Direction of self-motion is perceived from optical-flow. Nature, 336(6195), 162–163.Find this resource:
Warren, W. H. Jr., & Hannon, D. J. (1990). Eye movements and optical flow. Journal of the Optical Society of America A, 7(1), 160–169.Find this resource:
Warren, W. H. Jr., Kay, B. A., Zosh, W. D., Duchon, A. P., & Sahuc, S. (2001). Optic flow is used to control human walking. Nature Neuroscience, 4(2), 213–216.Find this resource:
Watson, A. B., & Ahumada, A. J. Jr. (1983). A look at motion in the frequency domain (NASA-TM-84352). Moffett Field, CA: NASA Ames Research Center. Retrieved from NASA Technical Reports Server website: https://ntrs.nasa.gov/search.jsp?R=19830015902.Find this resource:
Watson, A. B., & Ahumada, A. J. Jr. (1985). Model of human visual-motion sensing. Journal of the Optical Society of America A, 2(2), 322–341.Find this resource:
Wurtz, R. H. (2008). Neuronal mechanisms of visual stability. Vision research, 48(20), 2070–2089.Find this resource:
Xiao, D. K., Raiguel, S., Marcar, V., & Orban, G. A. (1997). The spatial distribution of the antagonistic surround of MT/V5 neurons. Cerebral Cortex, 7(7), 662–677.Find this resource:
Yoshimura, Y., Sato, H., Imamura, K., & Watanabe, Y. (2000). Properties of horizontal and vertical inputs to pyramidal cells in the superficial layers of the cat visual cortex. Journal of Neuroscience, 20(5), 1931–1940.Find this resource:
Zeki, S. M. (1974). Cells responding to changing image size and disparity in the cortex of the rhesus monkey. Journal of Physiology, 242(3), 827–841.Find this resource:
Zhang, T., & Britten, K. H. (2010). The responses of VIP neurons are sufficiently sensitive to support heading judgments. Journal of Neurophysiology, 103(4), 1865–1873.Find this resource:
Zhang, T., & Britten, K. H. (2011). Parietal area VIP causally influences heading perception during pursuit eye movements. Journal of Neuroscience, 31(7), 2569–2575.Find this resource:
Zhang, T., Heuer, H. W., & Britten, K. H. (2004). Parietal area VIP neuronal responses to heading stimuli are encoded in head-centered coordinates. Neuron, 42(6), 993–1001.Find this resource:
Zylberberg, J., Cafaro, J., Turner, M. H., Shea-Brown, E., & Rieke, F. (2016). Direction-selective circuits shape noise to ensure a precise population code. Neuron, 89(2), 369–383.Find this resource: