Accurate self-motion perception, which is critical for organisms to survive, is a process involving multiple sensory cues. The two most powerful cues are visual (optic flow) and vestibular (inertial motion). Psychophysical studies have indicated that humans and nonhuman primates integrate the two cues to improve the estimation of self-motion direction, often in a statistically Bayesian-optimal way. In the last decade, single-unit recordings in awake, behaving animals have provided valuable neurophysiological data with a high spatial and temporal resolution, giving insight into possible neural mechanisms underlying multisensory self-motion perception. Here, we review these findings, along with new evidence from the most recent studies focusing on the temporal dynamics of signals in different modalities. We show that, in light of new data, conventional thoughts about the cortical mechanisms underlying visuo-vestibular integration for linear self-motion are challenged. We propose that different temporal component signals may mediate different functions, a possibility that requires future studies.
Animals ; Humans ; Motion Perception/physiology* ; Bayes Theorem ; Optic Flow ; Cues ; Vestibule, Labyrinth/physiology* ; Photic Stimulation ; Visual Perception/physiology*

Evading or escaping from predators is one of the most crucial issues for survival across the animal kingdom. The timely detection of predators and the initiation of appropriate fight-or-flight responses are innate capabilities of the nervous system. Here we review recent progress in our understanding of innate visually-triggered defensive behaviors and the underlying neural circuit mechanisms, and a comparison among vinegar flies, zebrafish, and mice is included. This overview covers the anatomical and functional aspects of the neural circuits involved in this process, including visual threat processing and identification, the selection of appropriate behavioral responses, and the initiation of these innate defensive behaviors. The emphasis of this review is on the early stages of this pathway, namely, threat identification from complex visual inputs and how behavioral choices are influenced by differences in visual threats. We also briefly cover how the innate defensive response is processed centrally. Based on these summaries, we discuss coding strategies for visual threats and propose a common prototypical pathway for rapid innate defensive responses.
Mice ; Animals ; Zebrafish ; Neurons/physiology* ; Visual Perception/physiology*

Integrating multisensory inputs to generate accurate perception and guide behavior is among the most critical functions of the brain. Subcortical regions such as the amygdala are involved in sensory processing including vision and audition, yet their roles in multisensory integration remain unclear. In this study, we systematically investigated the function of neurons in the amygdala and adjacent regions in integrating audiovisual sensory inputs using a semi-chronic multi-electrode array and multiple combinations of audiovisual stimuli. From a sample of 332 neurons, we showed the diverse response patterns to audiovisual stimuli and the neural characteristics of bimodal over unimodal modulation, which could be classified into four types with differentiated regional origins. Using the hierarchical clustering method, neurons were further clustered into five groups and associated with different integrating functions and sub-regions. Finally, regions distinguishing congruent and incongruent bimodal sensory inputs were identified. Overall, visual processing dominates audiovisual integration in the amygdala and adjacent regions. Our findings shed new light on the neural mechanisms of multisensory integration in the primate brain.
Animals ; Macaca ; Acoustic Stimulation ; Auditory Perception/physiology* ; Visual Perception/physiology* ; Amygdala/physiology* ; Photic Stimulation

With inherent sparse spike-based coding and asynchronous event-driven computation, spiking neural network (SNN) is naturally suitable for processing event stream data of event cameras. In order to improve the feature extraction and classification performance of bio-inspired hierarchical SNNs, in this paper an event camera object recognition system based on biological synaptic plasticity is proposed. In our system input event streams were firstly segmented adaptively using spiking neuron potential to improve computational efficiency of the system. Multi-layer feature learning and classification are implemented by our bio-inspired hierarchical SNN with synaptic plasticity. After Gabor filter-based event-driven convolution layer which extracted primary visual features of event streams, we used a feature learning layer with unsupervised spiking timing dependent plasticity (STDP) rule to help the network extract frequent salient features, and a feature learning layer with reward-modulated STDP rule to help the network learn diagnostic features. The classification accuracies of the network proposed in this paper on the four benchmark event stream datasets were better than the existing bio-inspired hierarchical SNNs. Moreover, our method showed good classification ability for short event stream input data, and was robust to input event stream noise. The results show that our method can improve the feature extraction and classification performance of this kind of SNNs for event camera object recognition.
Visual Perception ; Learning ; Action Potentials ; Neural Networks, Computer ; Neuronal Plasticity

Neuroscientists have emphasized visceral influences on consciousness and attention, but the potential neurophysiological pathways remain under exploration. Here, we found two neurophysiological pathways of heart-brain interaction based on the relationship between oxygen-transport by red blood cells (RBCs) and consciousness/attention. To this end, we collected a dataset based on the routine physical examination, the breaking continuous flash suppression (b-CFS) paradigm, and an attention network test (ANT) in 140 immigrants under the hypoxic Tibetan environment. We combined electroencephalography and multilevel mediation analysis to investigate the relationship between RBC properties and consciousness/attention. The results showed that RBC function, via two independent neurophysiological pathways, not only triggered interoceptive re-representations in the insula and awareness connected to orienting attention but also induced an immune response corresponding to consciousness and executive control. Importantly, consciousness played a fundamental role in executive function which might be associated with the level of perceived stress. These results indicated the important role of oxygen-transport in heart-brain interactions, in which the related stress response affected consciousness and executive control. The findings provide new insights into the neurophysiological schema of heart-brain interactions.
Awareness ; Brain ; Consciousness ; Humans ; Oxygen ; Visual Perception

Visual object recognition in humans and nonhuman primates is achieved by the ventral visual pathway (ventral occipital-temporal cortex, VOTC), which shows a well-documented object domain structure. An on-going question is what type of information is processed in the higher-order VOTC that underlies such observations, with recent evidence suggesting effects of certain visual features. Combining computational vision models, fMRI experiment using a parametric-modulation approach, and natural image statistics of common objects, we depicted the neural distribution of a comprehensive set of visual features in the VOTC, identifying voxel sensitivities with specific feature sets across geometry/shape, Fourier power, and color. The visual feature combination pattern in the VOTC is significantly explained by their relationships to different types of response-action computation (fight-or-flight, navigation, and manipulation), as derived from behavioral ratings and natural image statistics. These results offer a comprehensive visual feature map in the VOTC and a plausible theoretical explanation as a mapping onto different types of downstream response-action systems.
Animals ; Brain Mapping ; Humans ; Magnetic Resonance Imaging ; Occipital Lobe ; Pattern Recognition, Visual ; Photic Stimulation ; Temporal Lobe ; Visual Pathways/diagnostic imaging* ; Visual Perception

To extract the temporal structure of sensory inputs is of great significance to our adaptive functioning in the dynamic environment. Here we characterize three types of temporal structure information, and review behavioral and neural evidence bearing on the encoding and utilization of such information in visual and auditory perception. The evidence together supports a functional view that the brain not only tracks but also makes use of temporal structure from diverse sources for a broad range of cognitive processes, such as perception, attention, and unconscious information processing. These functions are implemented by brain mechanisms including neural entrainment, predictive coding, as well as more specific mechanisms that vary with the type of temporal regularity and sensory modality. This framework enriches our understanding of how the human brain promotes dynamic information processing by exploiting regularities in ubiquitous temporal structures.
Attention ; Auditory Perception ; Brain ; physiology ; Humans ; Time Perception ; Visual Perception

Recent sensory history plays a critical role in the perception of event duration. For example, repetitive exposure to a particular duration leads to the distortion of subsequent duration perception. This phenomenon, termed duration adaptation, induces a robust repulsive duration aftereffect. In particular, adaptation to relatively long sensory events shortens the perceived duration of a subsequent event, while adaptation to relatively short sensory events lengthens the perception of subsequent event durations. This phenomenon implies the plasticity of duration perception and offers important clues for revealing the cognitive neural mechanism of duration perception. Duration aftereffect has received more and more attention in recent years. In this review, we introduce recent research advances in our understanding of duration aftereffect, especially with regards to its manifestations, origin, and cognitive neural mechanisms. We also propose possible directions for future research. In sum, we posit that studies on the duration aftereffect phenomenon are helpful in understanding general duration perception, and as such, should receive more attention in future.
Figural Aftereffect ; Humans ; Motion Perception ; Visual Perception

Visual memory, mainly composed of visual long-term memory (VLTM) and visual working memory (VWM), is an important mechanism of human information storage. Since Baddeley proposed the multicomponent working memory model, the idea that VWM is independent of the VLTM system has been widely accepted. However, the new theoretical evidence suggested a close connection between VLTM and VWM. For instance, the three embedded components model describes the VLTM and VWM in the same framework, which suggests that VWM is only a distinct state of VLTM. On the one hand, the operating function of VWM is supported by the persistence of VLTM. On the other hand, the evidence from neuroimaging studies shows that VWM and VLTM tasks activate some same brain areas. In addition, the whole visual memory system shows a trend of processing from early visual cortex to prefrontal cortex. The present article not only reviews the current studies about the relationship between VLTM and VWM but also gives some forecasts for future studies.
Brain ; physiology ; Humans ; Memory, Long-Term ; Memory, Short-Term ; Visual Cortex ; physiology ; Visual Perception

The core of visual processing is the identification and recognition of the objects relevant to cognitive behaviors. In natural environment, visual input is often comprised of highly complex 3-dimensional signals involving multiple visual objects. One critical determinant of object recognition is visual contour. Despite substantial insights on visual contour processing gained from previous findings, these studies have focused on limited aspects or particular stages of contour processing. So far, a systematic perspective of contour processing that comprehensively incorporates previous evidence is still missing. We therefore propose an integrated framework of the cognitive and neural mechanisms of contour processing, which involves three mutually interacting cognitive stages: contour detection, border ownership assignment and contour integration. For each stage, we provide an elaborated discussion of the neural properties, processing mechanism, and its functional interaction with the other stages by summarizing the relevant electrophysiological and human cognitive neuroscience evidence. Finally, we present the major challenges for further unraveling the mechanisms of visual contour processing.
Cognition ; Form Perception ; Humans ; Visual Cortex ; physiology ; Visual Perception