A research initiative at the College of Electronic Information and Optical Engineering at Nankai University involving flexible artificial synapse devices has led to the development of a neuromorphic motion perception system. The newly developed system understands the brain’s multisensory functions at the hardware level and demonstrates superior motion perception performance.
The paper, titled “Mammalian-brain-inspired neuromorphic motion-cognition nerve achieves cross-modal perceptual enhancement” is published in Nature Communications.
The design of the system is inspired by the macaque’s spatial perception and multisensory integration action. A macaque’s self-motion will trigger motion information such as optical flow signals and inertial signals in the retinal and the vestibule. The specified areas of the cerebral cortex will perform and detect the motion information encoded as spike pulse, and then comprehend spatial perception by combining information from various sensory modalities.
In the neuromorphic motion perception system, gyroscopes and accelerometers acquire angular velocity and acceleration signals respectively, which are encoded in two spike trains that are routed to high-performance synaptic transistors for computing. The linear relation between the two pulse sequences and their temporal relationship affects the device’s plasticity, which, in turn, affects the output of the device. Meanwhile, motion signals are grouped and detected by examining the average firing rate of the pulse and output current of the synaptic device.
Besides this, a vibrotactile sensor, an optical flow sensor, and an inertial sensor contain a sensing unit that can identify sensory information of tactile, visual, and acceleration modes. Information from various types of sensors can be combined effectively, thereby significantly enhancing the precision of motion recognition.
Importantly, the experimental outcomes are consistent with the brain’s perceptual enhancement effect. The significance of the system is in its ability to be attached to the human skin or arm on small drones to carry out complex tasks.
