French research lab CAE-Leti has developed a sonic event-driven object-localisation system using analogue in-memory neuromorphic computation, inspired by a barn owl’s brain.
“Real-world sensory-processing applications require compact, low-latency, and low-power computing systems,” according to CEA-Leti. “Enabled by their in-memory, event-driven computing abilities, hybrid memristive-CMOS neuromorphic architectures provide an ideal hardware substrate for such tasks.”
In the owl, the difference between arrival time of the same sound at each ear is processed to locate the sound source in azimuth – its nervous system has an array of neurons able to detect time correlations between signal spikes, and the neuron that fires indicates the time difference and therefore the target angle.
This auditory search is low-power and always-on and, when enough information has been received, it prompts the owl to start a more accurate but energy-hungry visual search.
Biological event-driven sensing and analogue in-memory computing are key.
“We drew inspiration from biology to incorporate these two aspects of computation into our hardware,” said CEA-Leti scientist Elisa Vianello. “In particular, we focused on the acoustic-based, object-localisation task. Owls efficiently solve this problem and thus we extrapolated their computational principles into our system.”
The electronic system combines 130nm CMOS with integrated hafnium-dioxide analogue resistive memory, which is matched to micromachined silicon piezoelectric ultrasound transducers under 1mm across – one 112kHz emitter, and a pair of receivers 100mm apart to model the owl’s ears.
The team developed a pre-processing pipeline for each ‘ear’. Each channel got a pre-amp, a band-pass filter, an envelope-detecting rectifier and a threshold detector – the latter emulating a particular type or neuron which delivers a spike-like pulse once certain time and amplitude dependencies have been met from the envelope detector.
If the sound-reflecting object is within reach of both ears, the pipelines delivers a spike on each channel whose time difference is related to target azimuth angle.
To detect time difference, each ear gets the equivalent of a tapped delay line into which its spike is fed.
Turn your imagination up a notch here dear reader, as we don’t have a diagram for this next bit.
A long row of coincidence detectors is wired to the taps of the delay lines. The first detector is wired to the first tap of left ear’s delay line and the last tap of the right ear. The next is wired to second tap of the left ear and the penultimate tap of the right ear, and so on.
The result, due to the different delays, is that for any particular time difference between the spikes from the left and right ears, only one coincidence detector get its spikes simultaneously and is therefore triggered – so the position of the triggered detector in the long row is directly related to the angle of the sound source to the ears – and more taps and detectors will increase the angular accuracy of the system. Should you need a diagram, look up ‘Jeffress model’.
In the CEA-Leti device, the delay lines and coincidence detectors are implemented using resistive-ram-based neuromorphic circuits. Non-volatile resistive ram stores the network’s synaptic weights for zero energy when the system is idle.
Energy savings come from the inherent way this arrangement computes asynchronously, and only when useful information arrives – compared with conventional processing where the detected signal is continuously sampled and processed to extract information.
The lab claims that this auditory processing system is five orders of magnitude more energy efficient “compared to conventional localisation systems” (see below).
CEA-Leti worked with CEA-List, University of Zurich, University of Tours and University of Udine.
The work is described in ‘Neuromorphic object localization using resistive memories and ultrasonic transducers‘, published in Nature Communications. This excellent paper is available in full without payment, and is clearly written despite being full of detail including the power assumptions that lead to the five orders of magnitude claim above.
