DOESciDAC ReviewOffice of Science
HARDWARE
Cortical Computing with MEMRISTIVE NANODEVICES
Brains require synapses, and lots of them—an estimated 1014. Although there are many stumbling blocks to achieving emulation of the brain's computational abilities using neuromorphic elements, a primary barrier is the lack of a small, cheap circuit that mimics the essential properties of these tiny synapses.

A back-of-the-envelope calculation is useful. A human cortex has a density of about 1010 synapses/cm2. Today's microprocessors pack roughly 109 transistors in 1 cm2 of complementary metal oxide semiconductor (CMOS). Thus, to build biological-scale neuromorphic circuits, electronic synapses will have to be about one-tenth the size of an average transistor. This is one important reason intelligent machines are not (yet) walking around on the street.
Research during the last decade uncovered nanoscale devices with unexpected properties. At first glance, many of these devices appear to be resistors, but experimentation has shown their resistance can be changed and the devices can remember the new resistance. Memristive nanodevices, a name meant to suggest a resistor with memory, act like tiny analog memories. Such devices, formed in the junctions of crossing nanowires separated by an appropriate material, may be the missing device for building synapses at a biological scale.
Hewlett-Packard Laboratories is exploring the combination of memristive nanodevices with conventional CMOS to implement circuits that emulate the functionality of cortex.

An Overly Simplistic Yet Useful First Model
The simple competitive network shown in figure 1 consists of "neurons" (shown as squares) that send signals to each other through wires (straight and curved lines). Some wires are interrupted with "synapses" (yellow circles) that can modulate the signals sent between neurons, in effect reducing signal strength to varying degrees. The input to the network is a set of binary signals that drive the lower neurons. The outputs are binary signals driven by the upper neurons.
If the neurons and synapses are designed correctly, this circuit can learn to do simple pattern recognition. When presented with a familiar input pattern seen many times before, the circuit will respond by activating one, and only one, of its output neurons to send a "1" with the others sending a "0." The winning neuron thus represents the "category" of the input pattern, giving a simple way to classify input data.
Memristive nanodevices, a name meant to suggest a resistor with memory, act like tiny analog memories.
Because there are only four output neurons, this network can only learn four categories. If "trained" by exposing it to a large number of input patterns consisting of only four categories, the network will learn to categorize the inputs.
To a logic designer, this may not seem interesting initially because it is easy to build logic circuits (decoders) that can do this function. What happens when the circuit is occasionally presented with an input pattern that does not fall cleanly into one of the four learned categories? The network then responds by activating the neuron for the category that is closest to it or most like the input. This property, called generalization, is very useful if inputs can be corrupted by noise or contain occasional outliers.
Learning in the competitive network of figure 1 requires both cooperation and competition between neurons. The neurons in the top layer compete by sending "inhibitory" signals to each other through the curved wires. Whenever one of these neurons becomes more excited (more likely to output a "1"), it tends to suppress its neighbors' excitement. This competition leads to the winner-take-all functionality required for categorization. The top layer neurons also cooperate with the bottom layer neurons in order to train the synapses between them to learn the necessary patterns.
Illustration: A. Tovey; Source: G. Snider, HP
Figure 1. Winner-take-all competitive network.
Training a synapse involves adjusting the strength or weight of the connection between the two neurons it connects: a large synaptic weight means signals are relayed between neurons unimpeded, and a small weight implies some serious reduction of the signal. At first glance, training might seem nearly impossible because synapses only have two connections, one to each of the two neurons that use it. How can signals be sent through the synapses and, at the same time, train the synapses with only two connections? The solution: have the signals do double-duty. The signals are not only used for communication between the neurons but also to internally adjust their own weight. To make things even more difficult, the synaptic weights are not merely binary but actually continuous. The synapses must adjust their weights in small steps in order to learn and not forget what they have learned—which is a lot of functionality to pack into a synaptic device or circuit with only two connections.

If the neurons and synapses are designed correctly, this circuit can learn to do simple pattern recognition.
Adding Memristive Nanodevices
When two metallic wires are separated with a few nanometers of memristive material (such as certain transition metal oxides), an electronic device is formed that acts much like a nonlinear resistor, but with a twist. The resistance varies over time as a function of the currents flowing through it. In other words, it is a resistor with memory.
The rate at which their resistance changes is extremely nonlinear in the voltage applied. Small voltages hardly perturb the resistance at all, while somewhat larger voltages can induce fast changes. A memristive device was subjected to a series of five positive triangle waves followed by three negative triangle waves (figure 2(c)). The positive waves induced counterclockwise hysteresis loops in the current versus voltage plots, with each wave causing an abrupt decrease in resistance when a large enough voltage was reached. The negative waves induced clockwise hysteresis loops, with a similar abruptness to increasing resistance.
Illustration: A. Tovey; Source: G. Snider, HP
Figure 2. Dynamical behavior of nanojunctions from experiments. Current-voltage curves are numbered sequentially and offset vertically for clarity. Positive voltage sweeps (1-5) are hysteresis loops of increasing conductivity; negative voltage sweeps (6-8) are hysteresis loops of decreasing conductivity.
These devices are (nearly) nonvolatile. When no voltage is applied, the resistance memory decays very slowly. Some devices fabricated at Hewlett-Packard Laboratories show less than 0.5% change in memory after sitting on the shelf for three years.
The analog device memory and nonvolatility are two synaptic properties being pursued. Another plus is the small size: 30 x 30 nm. The trick is figuring out how to exploit them for neuromorphic computing.

Learning Synapses
Biological neurons communicate with each other using "spikes," ~80 millivolt pulses each lasting about 1 ms. For electrical reasons, a spike is also needed in the electronic implementations. If memristive nanodevices are to be used as electronic synapses, implementation of both modulated signal transmission and learning in an environment of spikes must be demonstrated.
A "source" neuron may send a spike to a "sink" neuron through a synapse (figure 3) and have the strength of that spike modulated by the synapse's resistance memory. The input to the sink neuron is a "virtual ground" circuit (for example, a transimpedance amplifier) that allows it to collect and measure all incoming currents arriving through synapses. Because the source neuron must send its voltage spike through the synapse, the amount of current that arrives at the sink is dictated by Ohm's law (the lower the resistance of the synapse, the more current makes it to the sink). The result is a weighted signal transmission, where the synaptic weight is equal to the conductance (1/resistance) of the memristive synapse. In order to minimize the forward-going spike from perturbing the memory of the synapse, the spike's amplitude must be limited to below an effective "threshold," beyond which synaptic resistance changes quickly (figure 2).
For meaningful learning to take place, a voltage larger than that of a spike must be applied across the device, and it must be done at the appropriate times.
For meaningful learning to take place, a voltage larger than that of a spike must be applied across the device, and it must be done at the appropriate times. An algorithm for changing the synapse's weight in response to the spiking activity of the source and sink neurons connected to it is called a "learning law."
Hebbian learning is described as "when two neurons fire together, they wire together." The idea is simple—when two neurons have strongly correlated spiking activity, the weight of the synapses connecting them should increase (figure 4). Neurons are modified so when they spike they actually transmit information in two directions: forward with a positive pulse and backward with a negative pulse. The amplitudes of the spikes are small enough (just below the thresholds suggested in figure 2(c)) that neither spike alone will induce any change in the synaptic weight. But if the two spikes line up in time, the positive forward spike and negative backward spike create a voltage drop across the device equal to the sum of the amplitudes of the two spikes. The result is the synapse conductance is increased by a small amount. In other words, the synapse has learned (well, a little bit).
Illustration: A. Tovey; Source: G. Snider, HP
Figure 3. A source neuron sending a signal, a "spike," through a synapse to a destination neuron.
Illustration: A. Tovey; Source: G. Snider, HP
Figure 4. Correlational learning in memristive nanodevices used as synapses.
Building Silicon Neurons
Neurons collect weighted input signals (excitatory and inhibitory) from incoming synapses, process them, and generate output signals in response (figure 5). Neurons are synchronous and communicate by sending spikes (positive pulse/negative pulse sequence pairs) through attached synapses. Communication is bidirectional: output spikes are sent to both output synapses and excitatory and inhibitory input synapses, although spikes sent to inputs are phase delayed and asymmetrical compared to those sent to the output in order to implement learning.
Neurons (and some other circuits) are implemented in silicon cells. A neural silicon cell, shown in figure 6 (p62), has four pins on its surface that connect to circuitry within the cell and to nanowires imprinted on top. Two pins function as connection points to nanowire "dendrites" (inputs), and the other two pins function as connection points to nanowire "axons" (outputs). The nanowires implementing dendrites and axons are orthogonal. The small number of input pins still allows large "fan-in" due to the use of summing amplifiers on the input pins.
Illustration: A. Tovey; Source: G. Snider, HP
Figure 5. Synchronous spiking neuron. Neurons communicate by sending "forward spikes" (positive and negative pulse pairs) to outputs and "back spikes" to excitatory and inhibitory inputs. Back spikes are phase delayed relative to forward spikes. Coincidences of pulses from forward and back spikes on memristive synapses implement correlational learning.
Illustration: A. Tovey; Source: G. Snider, HP
Figure 6. Neuron cell architecture. (a) Despite having only four pins (discs), cell circuitry (b) can compute analog dot products of large numbers of input signals and synaptic weights, using summing amplifiers. Cell processing implements shunting dynamics. (c) Dendrites (horizontal nanowires) collect inputs from other neurons; axons (vertical nanowires) carry information to other neurons.


Taming the Wire Crossings
Converting a two-dimensional network representation (even a simple one like figure 1, p59) into a three-dimensional, real-life array of electronic neurons appears challenging. However, a little topological stretching makes the process relatively straightforward. Start with the two-dimensional, all-to-all bipartite graph of the original circuit (figure 7(a), p62). This diagram can first be reorganized into a much simpler crossbar (figure 7(b)), consisting of the silicon neurons (squares) interconnected with nanowires (pink and green lines). Synapses (yellow circles) are formed in the junctions of crossing nanowires. Shuffling and interleaving the neurons (figure 7(c)) makes the circuit more compact. This reorganization allows the physical implementation of three-dimensional silicon neurons (gray boxes) connecting through "pads" (black circles) on their top surfaces to nanowires (blue) that cross to form synapses (yellow; figure 7(d)).

Fabricating a Neuromorphic Circuit
Although these neuron arrays will be implemented in conventional CMOS, fabrication of nanowires and synapses will be done with nanoimprint lithography (figure 8, p63). The CMOS substrate is organized into square cells (figure 8(a)), and each cell implements a neuron or other critical circuitry. The top surface of each cell has four metallic "pins" used to interface the cell's neuron to metallic nanowires above. By convention, pins in the northwest and southeast corners of a cell are outputs from the silicon circuitry, and pins in the other two corners are inputs. Different neuron implementations will be required for each cortical layer (shown as shades of gray, one shade for each type of neuron).
Illustration: A. Tovey; Source: G. Snider, HP
Figure 7. (a) The two-dimensional, all-to-all bipartite graph of the original circuit. (b) The diagram reorganized into a much simpler crossbar, consisting of the silicon neurons (squares) interconnected with nanowires (pink and green lines). (c) Shuffling and interleaving the neurons makes the circuit more compact. (d) This reorganization allows the physical implementation of three-dimensional silicon neurons (gray boxes) connecting through "pads" (black circles) on their top surfaces to nanowires (blue) that cross to form synapses (yellow).
A first layer of nanowires (figure 8(b)) is imprinted on top of the silicon, with each nanowire making contact with a single, northwest pin. Since northwest pins are outputs, these nanowires serve as axons. The nanowires are not parallel to a cell edge but are angled slightly to allow each nanowire to span several cells. Also imprinted on the remaining pins are small "vias" that will carry electrical signals between the silicon and higher nanowire layers. An insulating material completes the layer by filling in the areas between the nanowires. The layer's surface is then flat, with the top surfaces of the nanowires flush with the surrounding insulator, ready for further imprinting.
A thin layer of memristive material is then laid down upon the first layer of nanowires, with embedded vias to propagate signals up to the next layer (figure 8(c)). On top of this memristive layer, a second layer of nanowires (also embedded in an insulating material) is imprinted orthogonally to the first (figure 8(d)). Because these connect to southwest (input) pins, they serve as dendrites. The memristive junctions formed by nanowires crossing over memristive material serve as synapses. This process of adding additional layers of memristive or insulating layers containing nanowires and vias is continued upward to create the desired synaptic interconnect between the neurons.
Source: G. Snider, HP
Figure 8. Conceptual fabrication. (a) Four silicon cells, each with four pins on the top surface. (b) First insulating layer with imprinted nanowires and vias (blue). (c) Second layer of memristive material and vias (blue). (d) Third insulating layer with imprinted nanowires and vias (blue).
Recreating the Cortex in Silicon
A biological brain cortex is made of neuron layers (about six) stacked vertically. Although silicon neurons cannot be similarly stacked, the same connectivity is achieved by interleaving the neurons of different cortical layers in silicon while stacking with multiple levels of imprinted nanowires that interconnect them. Positive and negative feedback is rampant; in fact, it is necessary to implement cortical algorithms. Simulations show the architecture is very tolerant of device variation and defective components.
Although silicon neurons cannot be stacked as they are in a biological brain cortex, the same connectivity is achieved by interleaving the neurons of different cortical layers in silicon while stacking with multiple levels of imprinted nanowires that interconnect them.
The basic idea is to emulate a laminar structure of the cortex by interleaving layers in CMOS (figure 9(a)). Neurons (gray boxes) are implemented in conventional CMOS; axons and dendrites (blue) in multiple layers of nanowires imprinted on top of the silicon; and synapses (yellow) in memristive (dynamical) junctions formed between selected adjacent layers of imprinted nanowires. CMOS neurons connect to the nanowires through metallic pads (black disks) on the top surface of the silicon. Nano vias (blue cylinders) allow neurons to connect to nanowires at several levels. Neurons in different cortical layers are represented by different shades of gray. Interconnections between and within cortical layers are accomplished with multiple levels of imprinted nanowires. Nanowires are rotated slightly relative to neuron edges to allow long-distance connections (figure 9(b)). Synaptic nanodevices are created wherever orthogonal nanowires, separated by memristive material, cross each other.
Source: G. Snider, HP
Figure 9. Nano/CMOS architecture for laminar, cortical circuits (left panel). Neurons are implemented in CMOS (gray), axons and dendrites in nanowires (blue). Synapses are implemented at the junctions of crossing wires separated by memristive material (yellow). Top view (right panel) shows how slight rotation of nanowires allows neurons to communicate via synapses to a neighborhood of other neurons. The small size of memristive nanodevices allows for a large ratio of synapses to neurons, necessary for neuromorphic computation; densities greater than 1010 devices/cm2 have already been achieved.
Example of Self-Organization
A classic problem in artificial neural networks is self-organization of orientation-sensitive "edge-detectors" of visual scenes (figure 10). Such detectors occur in the cortex's V1 region and are thought to be an early processing step in vision. Each detector (or neuron) receives inputs from a small region of the retina indirectly and responds by spiking on its output when the input pattern matches an "edge" oriented in a direction to which the detector is sensitive.
For this simulation, neurons were modeled as "integrate-and-fire" (integrating input spikes, then generating an output spike when the integral exceeds a threshold), while synapses were modeled after built memristive nanodevices. To induce self-organization, the network inputs were driven with bandpass-filtered noise. The simulation shows the development of each neuron's "receptive field" (the pattern to which the neuron responds most strongly) over time. Defects (shorts and opens) and variations among the modeled synapses impacted results very little. Neural architectures are inherently robust in the face of device "crumminess," a desirable property for economical manufacturing. This simulation shows that a network of memristive nanodevices can generate results expected from a network of biological neurons.
This simulation shows that a network of memristive nanodevices can generate results expected from a network of biological neurons.
Illustration: A. Tovey; Source: G. Snider, HP
Figure 10. Simulation of network evolution of a self-organized orientation map built from memristive nanodevices and silicon neurons.
Conclusion
Since the early days of digital computing, progress has been made in the neurophysiology and nonlinear dynamics needed to understand neural structures and interactions. Recent models have achieved impressive agreement with experiment. However, all neural models of interest are stiff, nonlinear dynamical systems that are very difficult to analyze mathematically. The nonlinear feedback precludes analytical solutions, and their stiffness makes them computationally expensive to solve numerically on digital computers. The outlined approach overcomes the stumbling block of synapse circuitry.
If successful, the market for such intelligent, adaptive systems would be staggering.
The potential applications of neuromorphic computing are stunning: intelligent adaptive control, pattern recognition, decision making, and intelligent-user interfaces with "common-sense" robotics. Because neuromorphic and digital computation have largely non-overlapping applications, future multi-core processors can be envisioned containing support for two cores. Digital cores would be used for number crunching and other conventional applications, and neuromorphic cores would be used for reasoning and adapting to a changing and uncertain world. Undoubtedly, cortical computing will require a series of many small, tentative steps and experiments, if it can be achieved at all in solid-state devices. However, if successful, the market for such intelligent, adaptive systems would be staggering.

Contributor: Greg S. Snider, Hewlett-Packard Laboratories