JP6501146B2 - Neural network circuit and learning method thereof - Google Patents

Description

  The present disclosure relates to a neural network circuit and a learning method thereof.

  Currently, research on computers imitating the information processing method of the brain of a living body is being conducted. The most basic of this processing model is a neural network. For example, Non-Patent Document 1 discloses a model (spiking neuron model) that represents information using pulse timing. According to Non-Patent Document 1, it is disclosed that the spiking neuron model has higher computing performance than the conventional model that does not use a pulse.

  Furthermore, as a neuron circuit which can realize the learning operation by pulse timing with the configuration of a smaller number of elements, for example, a configuration as disclosed in Patent Document 1 is disclosed.

Patent No. 5289647 gazette

W. Maass, "Networks of Spiking Neurons: The Third Generation of Neural Network Models," Neural Networks, vol. 10, no. 9, pp. 1659-1671, 1997.

  In the case of a neural network with a large number of neurons, the amount of computation in software computation is enormous, and the computation time becomes long. Therefore, dedicated hardware has been developed. However, the error back propagation learning can not be performed only by interconnecting the neuron circuits that realize the learning operation by the pulse timing as in Patent Document 1.

  Error back propagation learning is the most frequently used supervised learning method in hierarchical neural networks. In error back propagation learning, a teacher signal is input in addition to the input signal, and learning is performed so as to reduce an error between the output signal and the teacher signal. However, neither the method of calculating an error nor the method of reflecting the calculated error in the update of a load is disclosed by patent document 1. FIG.

Thus, the present disclosure provides a neural network circuit and a learning method thereof that can appropriately perform an error back propagation learning operation.
Additional benefits and advantages of the present disclosure will become apparent from the specification and the drawings. The benefits and / or advantages may be separately provided by the various aspects and features disclosed in the present specification and drawings, and not all are necessary to obtain one or more of them.

    A neural network circuit according to one aspect of the present disclosure includes a plurality of neural network circuit elements, an error calculation circuit, at least one input signal terminal, and at least one output signal terminal, and the at least one input. A neural network circuit for obtaining at least one output signal output from the at least one output signal terminal from an input signal input to a signal terminal, the error calculation circuit comprising: the at least one output signal; An error voltage which is a voltage signal having a magnitude corresponding to a time difference between the output signal and the teacher signal corresponding to the output signal, the teacher signal having a number equal to the number of the at least one output signal terminal being input. Configured to generate a signal, the neural network circuit element comprising at least one synapse The synapse circuit includes a resistance change element whose resistance value changes by application of a pulse voltage, and the neuron circuit changes from a reference value to a predetermined peak value with the passage of time. Finally, a load change pulse voltage signal having a predetermined first waveform that returns from the peak value to the reference value again, and a switching pulse voltage signal having a predetermined second waveform that defines a predetermined time width are generated. The load change pulse voltage signal is input to the synapse circuit of the neural network circuit element including the neuron circuit which has output the load change pulse voltage signal, and the switching pulse voltage signal is generated by the waveform generation circuit. The neural network including the neuron circuit that outputs a switching pulse voltage signal The neural network circuit element is input to the synapse circuit of the neural network circuit element other than the circuit element, and the neural network circuit element generates the amplitude of the load change pulse voltage signal based on the error voltage signal generated by the error calculation circuit. The synapse circuit is configured to change the period having the predetermined time width in the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the synapse circuit. The resistance value of the resistance change element in the synapse circuit is determined by a voltage according to the time difference between the switching pulse voltage signal and the load change pulse voltage signal generated by the neuron circuit of the neural network circuit element including the synapse circuit. Change Configured to

  According to the neural network circuit of the present disclosure, an error back propagation learning operation can be appropriately performed.

FIG. 1 is a block diagram showing a schematic configuration of neural network circuit elements that constitute a neural network circuit according to an embodiment of the present disclosure. FIG. 2A is a block diagram showing a configuration example of a neural network circuit configured using the neural network circuit element shown in FIG. FIG. 2B is a block diagram showing neural network circuit elements that constitute the neural network circuit according to an embodiment of the present disclosure. FIG. 3A is a diagram showing a waveform example of a pulse voltage used as an input signal in the neural network circuit element shown in FIG. FIG. 3B is a diagram showing a waveform example of an analog pulse voltage used as a teacher signal in the neural network circuit element shown in FIG. FIG. 4 is a circuit diagram showing a specific example of the integrating circuit in the neural network circuit element shown in FIG. FIG. 5A is a diagram showing a waveform example of a load change pulse voltage signal used in the neural network circuit element shown in FIG. FIG. 5B is a diagram showing a waveform example of a switching pulse voltage signal used in the neural network circuit element shown in FIG. FIG. 6 is a block diagram showing an example of a signal generation circuit in the neuron circuit of the neural network circuit element shown in FIG. FIG. 7A is a schematic cross-sectional view showing a specific example of the variable resistance element in the neural network circuit element shown in FIG. FIG. 7B is a diagram showing a circuit symbol of the variable resistance element shown in FIG. 7A. FIG. 8 is a circuit diagram showing a specific example of the first switch in the synapse circuit of the neural network circuit element shown in FIG. FIG. 9 is a graph showing the voltage applied to the third terminal of the variable resistance element according to the time difference between the load change pulse voltage signal shown in FIG. 5A and the switching pulse voltage signal shown in FIG. 5B. FIG. 10A is a block diagram showing a schematic configuration example of a time difference calculation circuit in the error calculation circuit of the neural network circuit element shown in FIG. FIG. 10B is a circuit diagram showing a specific example of the switch and the peak hold circuit in the time difference calculation circuit shown in FIG. 10A. FIG. 11 is a circuit diagram showing an example of a summing circuit in the error calculation circuit of the neural network circuit element shown in FIG. FIG. 12 is a block diagram showing a configuration example of a verification synapse circuit in the first embodiment of the present disclosure. FIG. 13 is a graph showing the result of verification using the verification synapse circuit shown in FIG. FIG. 14A is a block diagram showing a configuration example of a verification synapse circuit in the second embodiment of the present disclosure. FIG. 14B is a block diagram showing a configuration example of a verification neural network using the verification synapse circuit shown in FIG. 14A. FIG. 14C is a block diagram showing a configuration example of a verification neural network circuit using the verification neural network circuit element shown in FIG. 14B. FIG. 15 is a graph showing a change in error accompanying learning progress when learning of exclusive OR is performed in the neural network circuit shown in FIG. 14C. FIG. 16A is a schematic diagram showing a hierarchical neural network. FIG. 16B is a schematic view showing an interconnected neural network. FIG. 17 is a schematic view showing a general neuron. FIG. 18 is a graph showing temporal change of each value in a neuron unit having two input terminals. FIG. 19 is a circuit diagram showing a conventional neural network circuit element.

[Description of neural network]
First, the neural network which is the premise of the embodiment of the present disclosure and the conventional neural network circuit for realizing the neural network will be described in detail. As described above, the neural network is an imitation of the neural network of a living body. A neural network performs processing of information by arranging a plurality of neurons in a network, with a neuron imitating a neuron which is a functional unit in a neural network as a functional unit. For example, there is a hierarchical neural network in which the neurons 100 are connected in a hierarchical manner as shown in FIG. 16A, and an interconnected neural network (hop field network) in which the neurons 100 are mutually connected as shown in FIG. 16B.

  A neural network has two major functions. The first is a "processing" function that obtains an output from an input, and the second is a "learning" function that sets the input / output relationship of the entire neural network to a desired one.

Processing Function
Here, the operation of the information processing will be described by taking a hierarchical neural network as an example. The hierarchical neural network shown in FIG. 16A comprises three layers: an input layer 400, an intermediate layer 500, and an output layer 600. Each layer includes at least one neuron 100. Each neuron 100 of the input layer 400 is connected to each neuron 100 of the intermediate layer 500, and similarly, each neuron 100 of the intermediate layer 500 is connected to each neuron 100 of the output layer. The input signal 200 is input to the input layer 400, sequentially propagates to the intermediate layer 500 and the output layer 600, and is output from the output layer 600. In the neuron 100, a predetermined operation described later is performed on the input value, and the output value is propagated to the neuron of the next layer. Therefore, the output value from the output layer 600 is the final output 300 of the neural network. This series of operations is information processing of a neural network, and if the number of neurons included in the intermediate layer 500 is sufficiently increased, arbitrary input / output can be realized. The hierarchical neural network shown in FIG. 16A comprises three layers, but may comprise more than one intermediate layer 500.

Subsequently, a neuron which is a constituent unit of a neural network will be described. FIG. 17 is a schematic view showing a general neuron. The neuron 100 is a synapse (synapse)
) And the neuron unit 130. The number of synapses is equal to the number of neurons connected to the previous stage, that is, the number of input signals. The synapse unit 121 weights a plurality of external input signals 111. The synapse unit 122 weights the input signal 112 from the outside. Each of the weighting values (w ₁ , w ₂ ) is called a combined load. The neuron unit 130 calculates the sum of the input signals weighted by the synapse unit, and outputs a value obtained by performing a non-linear operation on the value of the sum. Here, external input signals are represented as x _i (1, 2,..., N), respectively. n is equal to the number of input signals. As shown in the following equation (1), the synapse units 121 and 122 multiply the respective input signals by the values w _i (1, 2,..., N) of corresponding coupling weights, and the neuron unit 130 Calculate their sum V _n .

V _n = Σw _i x _i (1)
Here, Σ is a sum symbol for i.

Further, the neuron unit sets an output value y as a result of performing the non-linear operation f on the obtained sum V _n . Therefore, the output y of the neuron is expressed as the following equation (2).

y = f (V _n ) (2)
As the non-linear function f, a monotonically increasing function having saturation characteristics is used. For example, a step function (step function) or a sigmoid function is used.

  The neural network circuit has parallel processability because a plurality of neuron units 130 can perform operations simultaneously. That is, unlike the sequential information processing of the conventional computer, it is characterized in that parallel information processing is possible.

[Learning function]
An important feature of the neural network is that it has a "learning" function as well as a "processing" function that obtains an output from an input as described above. The learning described here is setting the input / output relationship of the whole neural network circuit to a desired one by updating the above-described connection weight of the synapse.

  Learning is roughly divided into unsupervised learning and supervised learning. In unsupervised learning, the correlation between input signals to the neural network is learned by the network by inputting the input signal to the neural network. On the other hand, in supervised learning, a desired output signal corresponding to the input signal is given to the neural network together with the input signal. This desired output signal is called a teacher signal. The output signal from the neural network when the input signal is given to the neural network is learned so as to be equal to the teacher signal. In the hierarchical neural network shown in FIG. 16A, a learning method called error back propagation learning is generally used.

Error back propagation learning is the following learning method.
1. Provide the neural network with samples (input signal and teacher signal) for learning
2. Compare the actual network output caused by the input signal with the teacher signal and calculate its error
3. Adjust the coupling weight of each synapse to reduce the error,
4. Adjust the coupling weight in the order from the output layer to the neurons on the input layer side,
5. Repeat steps 1 to 4 above for all samples; The above processes 1 to 5 are repeated for all the samples until the error reaches a predetermined value.

  In error back propagation learning, as the name of this algorithm implies, errors (and learning) propagate from the output layer to the neurons on the input layer side.

Spiking neuron model
So far, processing and learning functions of neural networks have been described in detail. In the model used in the above description, the signal propagating between neurons was the value of current or potential represented by an analog value. On the other hand, it is known that neurons in the living body exchange pulses (spike pulses) of a substantially fixed shape. Therefore, there is known a model (spiking neuron model) that directly manipulates a pulse by imitating the neural circuit of a living body more faithfully. The spiking neuron model includes, for example, a model (a pulse density model) that expresses analog information using the number of pulses propagated at a certain time, and analog information using, for example, a time interval between pulses. Models (pulse timing models) etc. are included. These spiking neuron models can obtain higher computing performance than conventional neural networks using sigmoid functions.

  An integral firing model has been proposed as an operation model of a neuron unit that can be applied to information representation using pulses as described above. FIG. 18 is a graph showing temporal change of each value in a neuron unit having two input terminals.

As shown in FIG. 18, from the outside or other neurons unit, input to the synapse portion 121 pulses x _{1 (t)} is input, the input pulse x ₂ synaptic section 122 _(t) is input, a pulse is input At the specified timing, unimodal voltage changes appear in the respective synapse sections 121 and 122. Such a synapse potential is called post-synaptic potential (hereinafter abbreviated as "PSP"). FIG. 18 shows time changes P ₁ (t) and P ₂ (t) of PSP in the synapse section 121 and the synapse section 122. The height of PSP is proportional to the synaptic weight. Here, t represents time.

The neuron unit 130 calculates the sum of PSPs from all the synapse units 121 and 122 connected to the neuron unit 130. This sum is called an internal potential V _n (t) of the neuron unit 130. When there are two input terminals to the neuron unit 130, as shown in FIG. 18, the internal potential V _n (t) is the sum of P ₁ (t) and P ₂ (t). Generally speaking, the internal potential V _n (t) is represented by the following equation (3).

V _n (t) = ΣP _i (t) (3)
Here, P _i is the PSP at the ith synapse, and 和 is the sum symbol for i.

As shown in FIG. 18, when the internal potential V _n exceeds a predetermined threshold value V _th , the neuron unit outputs a pulse signal y (t). This is called "firing" of the neuron part. The pulse output y (t) is output from the neuron unit and is input to the other neuron units.

[Integrated circuit]
So far, the outline of the neural network has been described in detail, but when constructing the neural network, it becomes a problem how to realize the above-mentioned neuron. Heretofore, a conventional computer has often been used to implement a function of a neuron by software processing. However, in this case, since the CPU executes processing in a plurality of neurons in a time division manner, the original parallel information processing is not performed.
As mentioned above, it has been suggested that neural networks based on pulse timing information representation can achieve high performance. However, when the function of a neuron is realized by software processing, the operation time is enormous and the high computing ability which is the feature of spiking neural network can not be exhibited. Therefore, it is essential to configure and integrate a spiking neuron using hardware.

  Patent Document 1 discloses a specific example (neural network circuit element) in which a neuron operating based on a spiking neuron model is realized by hardware. FIG. 19 shows a conventional neural network circuit element. FIG. 19 shows the same configuration as FIG. 1 of Patent Document 1. As shown in FIG. The neural network circuit element 700 shown in FIG. 19 corresponds to the neuron 100 described above.

  As shown in FIG. 19, the neural network circuit element 700 comprises a synapse circuit 720 and a neuron circuit 730. The synapse circuit 720 corresponds to the synapse unit 120 described above, and the neuron circuit 730 corresponds to the neuron unit 130 described above. The synapse circuit 720 includes a variable resistance element 710, a selector circuit 711, and a switch circuit 712. The variable resistance element 710 has a function of storing the resistance value as a synapse connection weight.

The neuron circuit 730 includes an integration circuit 731, a waveform generation circuit 732, and a delay circuit 733. The waveform generation circuit 733 outputs the pulse voltage signal V _POST1 to another neural network circuit element. The waveform generation circuit 732 _{feeds back the} pulse voltage signal V _POST2 to the selector circuit 711 in the same neural network circuit element 700.

As described above, according to Patent Document 1, it is possible to control the selector circuit 711 using the pulse voltage signal V _{POST2 fed back} to the selector circuit 711 and to input the voltage pulse signal V _PRE1 to the gate electrode of the variable resistance element 710 The learning function by Spike-timing dependent synaptic plasticity (hereinafter abbreviated as "STDP") is realized by switching whether or not to.

  However, although the neural network circuit element disclosed in Patent Document 1 describes a method of learning a single neuron, it does not describe a method of learning when a plurality of neurons are connected and networked. In actual use of a neural network, it is necessary to interconnect neural network circuit elements to configure a network and to perform learning to realize an arithmetic function as a network. That is, only by connecting the neurons disclosed in Patent Document 1 as a hierarchical neural network, it is not possible to perform back propagation learning generally used in hierarchical neural networks. More specifically, in the error back propagation learning, a teacher signal is input in addition to the input signal, and learning is performed so as to reduce an error between the output signal and the teacher signal. Therefore, it is necessary to calculate an error and reflect the calculated error in the update of the load.

[Specific embodiment]
Therefore, a neural network circuit according to an aspect of the present disclosure includes a plurality of neural network circuit elements, an error operation circuit, at least one input signal terminal, and at least one output signal terminal, and the at least one neural network circuit A neural network circuit for obtaining at least one output signal output from the at least one output signal terminal from an input signal input to an input signal terminal. The error calculation circuit receives the at least one output signal and a number of teacher signals equal in number to the at least one output signal terminal, and outputs the output signal and the teacher signal corresponding to the output signal. To generate an error voltage signal that is a voltage signal having a magnitude corresponding to the time difference between The neural network circuit element comprises at least one synapse circuit and one neuron circuit. The synapse circuit includes a resistance change element whose resistance value is changed by application of a pulse voltage. The neuron circuit has a load changing pulse voltage signal having a predetermined first waveform that reaches a predetermined peak value from a reference value and returns from the peak value to the reference value again with the passage of time, and a predetermined time width. And a waveform generation circuit that generates a switching pulse voltage signal having a predetermined second waveform. The load change pulse voltage signal is input to the synapse circuit of the neural network circuit element including the neuron circuit that has output the load change pulse voltage signal, and the switching pulse voltage signal outputs the switching pulse voltage signal. It is configured to be input to the synapse circuit of the neural network circuit element other than the neural network circuit element including the neuron circuit. The neural network circuit element is configured to change the amplitude of the load change pulse voltage signal based on the error voltage signal generated by the error calculation circuit. The synapse circuit is a period during which the switching pulse voltage signal inputted from the neural network circuit element other than the neural network circuit element including the synapse circuit has the predetermined time width, The resistance value of the resistance change element in the synapse circuit is changed by a voltage according to a time difference between the neural network circuit element including the synapse circuit and the load change pulse voltage signal generated by the neuron circuit. .

  According to the above configuration, by converting the time difference between the output signal and the teacher signal into a voltage signal, the magnitude of the error between the output signal and the teacher signal is reflected in the magnitude of the voltage signal for changing the synapse load. can do. Therefore, the load can be changed so as to reduce the error between the output signal and the teacher signal. Therefore, error back propagation learning can be appropriately realized.

  The teacher signal serves as a reference potential when the time difference from the output signal is 0, and when the time difference from the output signal is within a predetermined range, the time difference is large with dipolarity with the reference potential as the central potential. The larger the potential difference with the central potential is, the signal may be such that when the time difference is outside the predetermined range, the maximum value of the potential difference within the predetermined range is held. Thus, the time difference between the output signal and the teacher signal can be efficiently converted to a voltage signal.

  The error calculation circuit includes the same number of time difference calculation circuits as the output signal terminals and one addition circuit, and the time difference calculation circuits respectively output the output signals output from the corresponding output signal terminals. And the error voltage signal according to the time difference between the output signal and the teacher signal corresponding to the output signal, the error voltage signal being the output signal being the output signal output from the output signal terminal. The neuron circuit included in the neural network circuit element is input, and the summing circuit generates a summing voltage signal by summing the error voltage signals generated in each of the time difference arithmetic circuits, and the summing voltage The number of neurons included in the neural network circuit element which is not the output signal whose output signal is the output signal output from the output signal terminal It may be configured to enter into. Thus, for the neural network circuit element (neural network circuit element included in the output layer) the output signal is an output signal output from the output signal terminal of the neural network circuit, the corresponding output signal and teacher signal The load can be changed to reduce the error based on the error of. Furthermore, for neural network circuit elements (neural network circuit elements included in the intermediate layer) whose output signal is not an output signal output from the output signal terminal of the neural network circuit, the generated error voltage signal is summed The load can be changed based on the calculated summed voltage signal. Thereby, error back propagation learning can be realized not only for the neural network circuit elements included in the output layer but also for all neural network circuit elements.

  The variable resistance element includes a first terminal, a second terminal, and a third terminal, and the neural network circuit element includes the variable resistance element between the first terminal and the second terminal. The constant voltage based on the switching pulse voltage signal input from another neural network circuit element is applied, and the neural network includes the resistance change element between the first terminal and the third terminal. A period having the predetermined time width in the switching pulse voltage signal input from the neural network circuit element other than the network circuit element, the switching pulse voltage signal and the variable resistance element are included in the neural network circuit element Voltage according to the time difference from the load change pulse voltage signal generated by the neuron circuit Is applied, it may be configured so that the resistance value is changed between the first terminal and the second terminal according to the potential difference between the first terminal and the third terminal.

  The synapse circuit is between the third terminal of the resistance change element and a terminal to which the load change pulse voltage signal generated by the neuron circuit of the neural network circuit element including the resistance change element is input. And a first switch for switching connection or disconnection, the first switch based on the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the variable resistance element. The connection or disconnection may be configured to be switched.

  The variable resistance element may be a ferroelectric memory element.

  The ferroelectric memory has a control electrode formed on a substrate, a ferroelectric layer provided to abut the control electrode, a semiconductor layer formed on the ferroelectric layer, and a semiconductor layer. A first electrode and a second electrode are provided, and the resistance value between the first electrode and the second electrode is changed in accordance with the potential difference between the first electrode and the control electrode. It may be done.

  The neuron circuit includes an integration circuit that integrates a current value flowing through the resistance change element of the synapse circuit, and a waveform that generates the first waveform and the second waveform according to the current value integrated by the integration circuit. A generation circuit may be included, and the waveform generation circuit may include a multiplication circuit that multiplies the magnitude of the first waveform by the magnitude of the error voltage signal.

  The synapse circuit includes a second switch having one end connected to the first reference voltage source and the other end connected to the first terminal of the resistance change element, and the second switch is the other neural network circuit The first reference voltage source may be connected to the first terminal during a period having the predetermined time width of the switching pulse voltage signal input from an element.

  A learning method of a neural network circuit according to another aspect of the present disclosure is a learning method of a neural network circuit having the above configuration, wherein an output signal is the output signal output from the output signal terminal. The amplitude of the load change pulse voltage signal output from the neuron circuit of the first neural network circuit element which is the neural network circuit element is changed based on the error voltage signal, and the output signal is the output. The amplitude of the load change pulse voltage signal output from the neuron circuit of the second neural network circuit element that is the neural network circuit element that is not the output signal output from the signal terminal is based on the error voltage signal It is something to change.

  According to the above method, first, the load change is performed on the first neural network circuit element based on the error back propagation learning. Thereafter, the second neural network circuit element is subjected to load change based on error back propagation learning. For this reason, it is possible to change the load so that the error is efficiently reduced with respect to the synapse circuit of each neural network circuit element. Therefore, error back propagation learning can be appropriately realized.

  In the learning method, the synapse circuit of the second neural network circuit element receives a signal whose potential is equal to a reference potential instead of the load change pulse voltage signal, thereby causing the second neural network circuit element to receive the signal. A first step of changing a resistance value of the variable resistance element of the first neural network circuit element in a state where the resistance value of the variable resistance element is not changed, and the synapse of the second neural network circuit element The resistance of the second neural network circuit element is generated by generating the load change pulse voltage signal having the first waveform generated by the neuron circuit of the neural network circuit element including the synapse circuit in a circuit. Change the resistance value of the change element, 2nd step If, hints, time difference between the output signal corresponding to the teacher signal may be repeatedly executed the processing of the first step and the second step until the specified value or less.

A neural network circuit according to an aspect of the present disclosure receives an output signal and a teacher signal, generates an error signal having a voltage value according to a time difference between the output signal and the teacher signal, and the neural network A first one or more neural network circuit elements included in an intermediate layer of the circuit, and a second one or more neural network circuit elements included in an output layer of the neural network circuit, the output layer including the output signal Outputting, each of said first one or more neural network circuit elements comprising a first one or more synapse circuit and a first neuron circuit, each of said second one or more neural network circuit elements being a second Said first one or more synaptic circuits, said second one or more Each of the naps circuits includes a variable resistance element whose resistance value changes according to the voltage value of the applied pulse voltage, and each of the first neuron circuit and the second neuron circuit has a peak value from a reference value over time And a waveform generation circuit for generating a switching pulse voltage signal having a waveform that returns from the peak value to the reference value again, and the switching pulse signal generated by the first neuron circuit. The first load change pulse voltage signal which is a voltage signal is input to each of the first one or more synapse circuits, and the second load change pulse which is the load change pulse voltage signal generated by the second neuron circuit The voltage signal is input to each of the second one or more synapse circuits, and the switching pulse generated in the first neuron circuit The first switching pulse signal, which is a pressure signal, is input to each of the second one or more synapse circuits, and the amplitude of the first load change pulse voltage signal and the amplitude of the second load change pulse voltage signal are A voltage determined based on the error signal and represented by a time width indicated by the first switching pulse voltage signal, and a voltage corresponding to a time difference between the first switching pulse voltage signal and the second load change pulse voltage signal , The resistance value of the variable resistance element included in each of the second one or more synapse circuits is changed.
The switching pulse voltage signal generated by the second neuron circuit may be the output signal.

  Hereinafter, a neural network circuit and a learning method of the neural network circuit according to an embodiment of the present disclosure will be described with reference to the drawings.

Embodiment
An embodiment of the present disclosure will be described. FIG. 1 is a block diagram showing a schematic configuration of neural network circuit elements that constitute a neural network circuit according to an embodiment of the present disclosure. FIG. 2A is a block diagram showing a configuration example of a neural network circuit configured using the neural network circuit element shown in FIG.

As shown in FIG. 2A, the neural network circuit 1 is configured by connecting an input layer 2, an intermediate layer 3, an output layer 4 and an error calculation circuit 5 in series.
At least one input signal V _in is input to the input layer 2 of the neural network circuit 1. In FIG. 2A, at least one input signal V _in is a plurality of input signals V _in ^A , V _in ^B ,.
The input layer 2 of the neural network circuit 1 comprises at least one input signal terminal 6. In FIG. 2A, at least one input signal terminal 6 is the input signal terminals 6A, 6B,.
At least one input signal V _in is input from at least one input signal terminal 6. In FIG. 2A, the input signal V _in ^A is input from the input signal terminal 6A, and the input signal V _in ^B is input from the input signal terminal 6B.
At least one teacher signal V _t is input to the input layer 2 of the neural network circuit 1. Incidentally, in FIG. 2A, at least one teacher signal _{V t,} a plurality of teacher signal _V ^t _{a, V} ^{t _b,} a ....
The input layer 2 of the neural network circuit 1 comprises at least one teacher signal terminal 7. In FIG. 2A, at least one teacher signal terminal 7 is teacher signal terminals 7a, 7b,.
At least one teacher signal Vt is input from at least one teacher signal terminal 7. In FIG. 2A, the teacher signal Vta is input from the teacher signal terminal 7a, and the teacher signal Vtb is input from the teacher signal terminal 7b.
The neural network circuit 1 is a circuit that obtains at least one output signal V _out output from at least one output signal terminal 8 based on an input signal V _in input to at least one input signal terminal 6.
At least one output signal V _out is output from the output layer 4 of the neural network circuit 1. In FIG. 2A, at least one output signal V _out is a plurality of output signals V _out ^A , V _out ^B _,.
The output layer 4 of the neural network circuit 1 comprises at least one output signal terminal 8. In FIG. 2A, at least one output signal terminal 8 is the output signal terminals 8a, 8b,.
At least one output signal V _out is output from at least one output signal terminal 8. In FIG. 2A, the output signal V _out ^A is output from the output signal terminal 8 a, and the output signal V _out ^B is output from the output signal terminal 8 b.

FIG. 3A is a diagram showing a waveform example of a pulse voltage used as an input signal in the neural network circuit shown in FIG. 2A. FIG. 3B is a diagram showing a waveform example of an analog pulse voltage used as a teacher signal in the neural network circuit shown in FIG. 2A. Input signal V _in is input from the input signal terminal 6, for example, as shown in FIG. 3A, configured as a switching pulse voltage signal having a predetermined waveform to determine the predetermined time width. Teacher signal V _t which is input from the teacher signal terminal 7 is configured as a load change pulse voltage signal having a waveform as shown in FIG. 3B, for example. Teacher signal V _t illustrated in FIG. 3B, reference when the reference potential (e.g., 0 potential) and when the time difference between the output signal V _out is zero, the time difference between the output signal V _out is within a predetermined range The potential difference with the central potential increases as the time difference in the bipolar increases with the potential as the central potential, and a signal that holds the maximum value of the potential difference within the predetermined range when the time difference is outside the predetermined range is there. In FIG. 3B, positive polarity (positive value) occurs when the time is advanced from time t _{t at} which the reference potential as a signal to be bipolar is reached (the arrival time of output signal V _out is earlier than teacher signal V _t ). In the case where the time is later than the time t _t (when the arrival time of the output signal V _out is later than the teacher signal V _t ), a signal which is negative polarity (negative value) is illustrated. Note that a signal in which the positive polarity and the negative polarity are opposite to each other may be used.

Furthermore, the neural network circuit 1 comprises at least one output signal terminal 8 for obtaining at least one output signal V _out for at least one input signal V _in . In FIG. 2A, each output signal is represented as V _out ^a , V _out ^b ,... As a plurality of output signals, and each output signal terminal is represented as 8 a, 8 b,. . The number of all teacher signal terminals (7a, 7b, ...) of the neural network circuit 1 is equal to the number of all output signal terminals (8a, 8b, ...) of the neural network circuit 1. The number of all input signal terminals (6A, 6B,...) Of the neural network circuit 1 and the number of all output signal terminals (8a, 8b,...) Of the neural network circuit 1 may not be equal.

A plurality of neural network circuit elements 40 as shown in FIG. 1 are provided in each of the intermediate layer 3 and the output layer 4 shown in FIG. 2A. As shown in FIG. 1, the neural network circuit element 40 includes at least one synapse circuit 20 and one neuron circuit 30. The input terminal 51 of the neural network circuit element 40 included in the intermediate layer 3 is connected to the input signal terminal of the input layer 6, and an input signal is input. The neural network circuit element 40 included in the intermediate layer 3 outputs the switching pulse voltage signal V _POST2 generated by the neuron circuit 30 from the output terminal 53. The input terminal 51 of the neural network circuit element 40 included in the output layer 4 is connected to the output terminal 53 of the neural network circuit element 40 included in the intermediate layer 3 and is output from the neural network circuit element 40 included in the intermediate layer 3 Switching pulse voltage signal V _POST2 is input. The output terminal 53 of the neural network circuit element 40 included in the output layer 4 is connected to the output signal terminal 8 and outputs the switching pulse voltage signal V _POST2 generated by the neuron circuit 30 as the output signal V _out .

  In the present embodiment, the neural network circuit elements 40 included in the intermediate layer 3 are connected in parallel, but a plurality of neural network circuit elements 40 may be connected in series in the intermediate layer 3. In other words, a plurality of intermediate layers 3 shown in FIG. 1 may be provided. In this case, the input terminal 51 of the neural network circuit element 40 included in the intermediate layer 3 is connected to the input signal terminal of the input layer 2 or the output terminal 53 of the neural network circuit element 40 of the previous stage, and the output terminal 53 is It is connected to the input terminal 51 of the neural network circuit element 40 or the output signal terminal 8 of the neural network circuit 1.

Further, the neural network circuit element 40 is provided with an error input terminal 52 to which a signal based on an error voltage signal V _error described later is input.

  The synapse circuit 20 includes a first input terminal 41 connected to the input terminal 51 of the neural network circuit element 40, a second input terminal 42 connected to a first output terminal 47 of the neuron circuit 30 described later, and an output terminal 43. And. The neuron circuit 30 has a first input terminal 44 connected to the output terminal 43 of the synapse circuit 20, a second input terminal 45 connected to the error input terminal 52 of the neural network circuit element 40, and a second input terminal A first output terminal 47 connected to the input terminal 42 and a second output terminal 46 connected to the output terminal 53 of the neural network circuit element 40 are provided. Although only one neuron circuit 30 and one synapse circuit 20 are shown in FIG. 1 for easy viewing of the drawing, actually, as shown in FIG. 2B, one neuron circuit 30 is shown. , And a plurality of synapse circuits 20 may be connected.

[Neuron circuit]
The neuron circuit 30 includes an integration circuit 31, a comparison circuit 32, and a waveform generation circuit 33.

  The first input terminal 44 of the neuron circuit 30 is connected to the integration circuit 31. The integration circuit 31 calculates the sum of the currents output from at least one synapse circuit 20 connected to the neuron circuit 30. FIG. 4 is a circuit diagram showing a specific example of the integrating circuit in the neural network circuit element shown in FIG. In the specific example shown in FIG. 4, the integration circuit 31 is configured as an analog integration circuit including an operational amplifier 35, a capacitor 36 and a resistance element 37. The capacitance value of the capacitor 36 is, for example, 1 pF, and the resistance value of the resistance element 37 is, for example, 1 MΩ. The positive input terminal (+) of the operational amplifier 35 is configured to be equal to the ground voltage, and the negative input terminal (−) of the operational amplifier 35 is connected to the input terminal 44 of the neuron circuit 30. A capacitor 36 and a resistor 37 are connected in parallel between the negative input terminal (−) of the operational amplifier 35 and the output terminal.

The integration circuit 31 charges the capacitor 36 using the current input from the synapse circuit 20 to the neuron circuit 30. By this operation, the calculation result of the time integration of the current is output as the integral voltage signal V _n . Further, the operational amplifier 35 is configured to feed back the integrated voltage signal V _n , which is an output signal, to the negative input terminal (−) of the operational amplifier 35 via the resistance element 37. As a result, the negative polarity input terminal (-) of the operational amplifier 35 is in a virtual ground state.

Negative input terminal of the operational amplifier 35 (-) because the a state of the virtual ground, regardless of the integration potential V _n of the number and the capacitor 36 of the synapse circuit 20, the resistance change between the first reference voltage V ₁ of the current voltage source 23 A constant current determined by the resistance value of the element 10 is input to the neuron circuit 30 and accumulated.

The integral potential V _n of the integration circuit 31 is input to the comparison circuit 32. Comparison circuit 32, when the calculated value exceeds a predetermined threshold V _TH, and outputs an output signal (trigger signal) V _c to the waveform generating circuit 33.

Signal generating circuit 33 as a trigger the output signal V _c of comparator 32 reaches the reference value with time to a predetermined peak value, a predetermined first waveform as returns to the reference value from the peak value A load changing pulse voltage signal V _POST1 and a switching pulse voltage signal V _POST2 having a predetermined second waveform defining a predetermined time width are generated. The load change pulse voltage signal V _POST1 is output from the first output terminal 47, and the switching pulse voltage signal V _POST2 is output from the second output terminal 46.

  FIG. 5A is a diagram showing a waveform example of a load change pulse voltage signal used in the neural network circuit element shown in FIG. 1, and FIG. 5B is a waveform example of a switching pulse voltage signal used in the neural network circuit element shown in FIG. FIG.

As the first waveform, the load change pulse voltage signal V _POST1 illustrated in FIG. 5A rises from the zero potential, which is the reference value, to a predetermined positive peak voltage value with the passage of time, and then again from the positive peak voltage value It is an analog pulse voltage signal having a triangular waveform falling to 0 potential. In addition to the waveform shown in FIG. 5A, for example, a waveform obtained by inverting the sign of the waveform in FIG. 5A can be used as the load change pulse voltage signal V _POST1 . Furthermore, although the example in FIG. 5A describes a waveform in which the rise and fall of the potential change linearly with time, the potential may change non-linearly with time.

The switching pulse voltage signal V _POST2 illustrated in FIG. 5B has a square wave as a second waveform. More specifically, switching pulse voltage signal V _POST2 illustrated in FIG. 5B has a second waveform which is higher than a predetermined voltage level (HI state) during a period having a predetermined time width (input allowable period). There is. The switching pulse voltage signal V _POST2 is not limited to the waveform shown in FIG. 5B, and may be any waveform that can determine the time width of either the HI state or the LO state. Therefore, the switching pulse voltage signal V _POST2 may be a digital waveform or an analog waveform. As will be described later, the switching pulse voltage signal V _PRE input to the first input terminal 51 of the neural network circuit element 40 also has the same waveform as the switching pulse voltage signal V _POST2 .

The signal generation circuit 33 generates the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _POST2 at the same timing. Specifically, the time at which the potential change of the load change pulse voltage signal V _POST1 occurs and the time center point of the switching pulse voltage signal V _POST2 (shown as time t ₀ in FIGS. 5A and 4B) coincide. Output.

FIG. 6 is a block diagram showing an example of a signal generation circuit in the neuron circuit of the neural network circuit element shown in FIG. As shown in FIG. 6, the signal generation circuit 33 drives a digital / analog converter 332 driven by the output signal V _C from the comparison circuit 32 as a trigger, and digital in which digital waveform data for generating an analog pulse voltage is stored. A memory 331 and an analog multiplication circuit 333 are provided. When the output signal V _C from the comparison circuit 32 is input, the digital / analog converter 332 reads digital waveform data of the first waveform from the digital memory 331, converts this to an analog waveform, and outputs it to the analog multiplication circuit 333. input. The analog multiplication circuit 333 multiplies the magnitude of the first waveform of the digital memory 331 by the magnitude of the error voltage signal V _error input from the second input terminal 45, and changes the weight of the analog waveform having the amplitude after multiplication. A pulse voltage signal V _POST1 is generated. The generated load change pulse voltage signal V _POST1 is output from the first output terminal 47. As shown in FIG. 1, the load change pulse voltage signal V _POST1 output from the first output terminal 47 is a feedback input to all the synapse circuits 20 in the neural network circuit element 40 including the neuron circuit 30 which has output this. (See FIG. 2B).

The signal generation circuit 33 also generates the switching pulse voltage signal V _POST2 and outputs it to the second output terminal 46. As described above, the second output terminal 46 of the neuron circuit 30 is connected to the output signal terminal 53 of the neural network circuit element 40. Therefore, the switching pulse voltage signal V _POST2 is an output signal of the neural network circuit element 40. An output signal of the neural network circuit element 40 included in the intermediate layer 3 is an input terminal of a neural network circuit element 40 (a neural network circuit element 40 included in the output layer 4) different from the neuron circuit 30 that outputs the output signal. 51 is input as a first input signal (that is, a switching pulse voltage signal V _PRE described later). The output signal of the neural network circuit element 40 included in the output layer 4 is the output signal _Vout output from the output signal terminal 8 of the neural network circuit 1.

[Synapse circuit]
Next, the synapse circuit 20 in the present embodiment will be described. As shown in FIG. 1, the synapse circuit 20 includes a resistance change element 10 whose resistance value changes by application of a pulse voltage. The synapse circuit 20 has a predetermined time width in the switching pulse voltage signal V _PRE input to the first input terminal 41, and the load changing pulse voltage input to the switching pulse voltage signal V _PRE and the second input terminal 42. The resistance value of the resistance change element 10 in the synapse circuit 20 is changed by a voltage corresponding to the time difference from the signal V _POST1 . Here, the switching pulse voltage signal V _PRE is a switching pulse voltage signal input from a neural network circuit element 40 different from the neural network circuit element 40 including the synapse circuit 20 to which the switching pulse voltage signal V _PRE is input. V _{POST 2} or the input signal V _in input from the input signal terminal 6. Further, the load change pulse voltage signal _{V POST1} is a load change pulse voltage signal _{V POST1} generated by the neuron circuit 30 of the neural network circuit element 40 to which the load change pulse voltage signal _{V POST1} is included synapse circuit 20 is inputted is there.

The resistance change element 10 includes a first terminal 13, a second terminal 14, and a third terminal 15. A constant voltage based on the switching pulse voltage signal V _PRE is applied between the first terminal 13 and the second terminal 14. A period (input allowable period) having a predetermined time width in the switching pulse voltage signal V _PRE between the first terminal 13 and the third terminal 15, the switching pulse voltage signal V _PRE and the load change pulse voltage signal V _POST1 Voltage according to the time difference between In the resistance change element 10, the resistance value between the first terminal 13 and the second terminal 14 changes in accordance with the potential difference between the first terminal 13 and the third terminal 15 generated in this manner.

  The variable resistance element 10 can use a non-volatile variable resistance element. The non-volatility means the property that the changed resistance value is maintained even after the application of the voltage is stopped after the resistance value is changed by the application of the voltage to the resistance change element 10. By using such a variable resistance element 10, it is possible to maintain the resistance value even after the voltage supply to the variable resistance element 10 is cut off.

The variable resistance element 10 in the present embodiment is a ferroelectric memristor.
7A is a schematic cross-sectional view showing a specific example of the variable resistance element in the neural network circuit element shown in FIG. 1, and FIG. 7B is a diagram showing a circuit symbol of the variable resistance element shown in FIG. 7A.

  As shown in FIG. 7A, the ferroelectric memory element has a field effect transistor structure using the ferroelectric layer 71 as a gate insulating layer (also referred to as a ferroelectric gate transistor). The ferroelectric memory has a control electrode (gate electrode) 73 formed on a substrate 72, a ferroelectric layer 71 provided so as to abut the control electrode 73, and a semiconductor formed on the ferroelectric layer 71. A layer 74 and a first electrode (source electrode) 75 and a second electrode (drain electrode) 76 provided on the semiconductor layer 74 are provided. The first electrode 75 is connected to the first terminal 13, the second electrode 76 is connected to the second terminal 14, and the control electrode 73 is connected to the third terminal 15. In the variable resistance element 10 having such a configuration, the resistance value between the first electrode 73 and the second electrode 74 changes in accordance with the potential difference between the first electrode 75 and the control electrode 73.

The semiconductor layer 74 is formed of, for example, ZnO, GaN, InGaZnO, or the like. The ferroelectric layer 71 is, for _{example, Pb (Zr, Ti) O} 3, Sr (Bi, Ta) O, or is formed by _Bi 12 _{TiO 20,} and the like. The first electrode 75, the second electrode 76, and the control electrode 73 are formed of, for example, a laminate including a platinum layer and a titanium layer.

  In such a ferroelectric memory element, when a voltage is applied between the control electrode 73 and the first electrode 75 and / or the second electrode 76, the polarization direction of the ferroelectric layer 71 (arrows in FIG. 7A) Depending on Q), the resistance between the first electrode 75 and the second electrode 76 changes to non-volatile.

  It will be described more specifically. Hereinafter, the direction from the control electrode 73 toward the semiconductor layer 74 is referred to as the upward direction, and the direction from the semiconductor layer 74 toward the control electrode 73 is referred to as the downward direction. As shown in FIG. 7A, in the case where a part of the ferroelectric layer 71 has an upward polarization (polarization in the direction indicated by the arrow Q in FIG. 7A), above the portion where the polarization of the ferroelectric layer 71 occurs. The stacked semiconductor layer 74 has a low resistance value. On the other hand, when part of the ferroelectric layer 71 has downward polarization (polarization in the direction opposite to the direction indicated by the arrow Q in FIG. 7A), lamination is made above the portion where polarization of the ferroelectric layer 71 occurs. The semiconductor layer 74 having a high resistance value. The resistance value between the first electrode 75 and the second electrode 76 is the resistance value of the region of the semiconductor layer 74 sandwiched between the first electrode 75 and the second electrode 76. Therefore, the resistance value between the first electrode 75 and the second electrode 76 changes continuously depending on the ratio of polarization in the ferroelectric layer 71 located below the region of the semiconductor layer 74.

  In such a ferroelectric memory, in order to control the resistance value between the first electrode 75 and the second electrode 76, the potential difference between the first electrode 75 and / or the second electrode 76 and the control electrode 73. The polarization direction of the ferroelectric layer 71 is changed by changing. For example, when a positive voltage is applied to the control electrode 73 with respect to the first electrode 75 and / or the second electrode 76, the direction of the electric field due to the polarization of the ferroelectric layer 71 is upward (toward the semiconductor layer 74). It will be easier. On the contrary, when a negative voltage is applied to the control electrode 73, the direction of the electric field due to the polarization of the ferroelectric layer 71 tends to be directed downward (the control electrode 73 side). Also, as the magnitude (absolute value) of the voltage to be applied is larger, the amount of change in polarization in the ferroelectric layer 71 is larger. Therefore, when a positive voltage is applied to control electrode 73, the resistance between first electrode 75 and second electrode 76 decreases, and when a negative voltage is applied to control electrode 73, first electrode 75 and second electrode 76 The resistance between them increases. Further, the change in the resistance value between the first electrode 75 and the second electrode 76 becomes more remarkable as the absolute value of the voltage applied to the control electrode 73 is larger. As described above, the ferroelectric memory element can be employed as the above-described resistance change element 10.

Here, a method of manufacturing the above-described ferroelectric memristor will be illustrated. First, for example, on a (001) -oriented single crystal substrate 72 made of strontium titanate (SrTiO ₃ ), an oxide conductor layer made of strontium ruthenate (SrRuO ₃ ), for example, is formed by pulse laser deposition (hereinafter PLD) Deposited by The thickness of the oxide conductor layer is, for example, 30 nm. The temperature of the substrate 72 at the time of deposition is 700.degree. After deposition of the oxide conductor layer, a control electrode 73 is formed by photolithography and ion milling.

Furthermore, with the temperature of the substrate 72 being 700 ° C., a ferroelectric layer 71 made of, for example, lead zirconate titanate (Pb (Zr, Ti) O ₃ ) is deposited on the control electrode 73 using PLD . The thickness of the ferroelectric layer 71 is, for example, 450 nm. Thereafter, the temperature of the substrate 72 is lowered to 400 ° C., and a semiconductor layer 74 made of, for example, zinc oxide (ZnO) is deposited on the ferroelectric layer 71. The thickness of the semiconductor layer 74 is, for example, 450 nm.

  Next, a patterned resist is formed on the deposited semiconductor layer 74. Thereafter, for example, a titanium layer and a platinum layer are deposited by electron beam evaporation at room temperature to form a laminate composed of the titanium layer and the platinum layer. The thickness of the titanium layer is, for example, 5 nm, and the thickness of the platinum layer is, for example, 30 nm. After the formation of the laminate, the first electrode 75 and the second electrode 76 are formed by a lift-off method. The first electrode 75 and the second electrode 76 are formed in contact with the main surface of the semiconductor layer 74 so as to be separated from each other by a predetermined distance. Thus, the resistance change element 10 which is a ferroelectric memory element is obtained.

As a specific configuration for changing the resistance value of the resistance change element 10, the synapse circuit 20 is connected between the third terminal 15 of the resistance change element 10 and the second input terminal 42 to which the load change pulse voltage signal V _POST1 is input. And a first switch 21 that switches connection or disconnection of the switch. The first switch 21 switches connection or disconnection between the third terminal 15 and the second input terminal 42 of the variable resistance element 10 based on the switching pulse voltage signal _VPRE input from the first input terminal 51.

For example, the switching pulse voltage signal V _PRE has a waveform which is equal to or higher than a predetermined voltage level (HI state) during a period (input allowable period) having a predetermined time width. While the switching pulse voltage signal V _PRE is in the HI state, the first switch 21 connects the third terminal 15 of the variable resistance element 10 and the second input terminal 42 of the synapse circuit 20. That is, when the first switch 21 is closed while the switching pulse voltage signal V _PRE is in the HI state, conduction is established between the second input terminal 42 of the synapse circuit 20 and the third terminal 15 of the resistance change element 10. It becomes possible. Therefore, between the third terminal 15 of the variable resistance element 10 and the second output terminal 46 for outputting the load change pulse voltage signal V _POST1 of the neuron circuit 30 in the input allowable period determined based on the switching pulse voltage signal V _PRE . The connection is made, and the connection is cut off during the other period.

Furthermore, the synapse circuit 20 is connected to the first reference voltage source 23 for generating a predetermined first reference voltage V ₁ (for example, power supply voltage V _DD ), one end is connected to the first reference voltage source 23, and the other end is resistance change A second switch 22 connected to the first terminal 13 of the element 10 is provided. The second switch 22 is a period (input allowable period) in which the switching pulse voltage signal V _PRE input from the other neural network circuit element 40 or from the input signal terminal 6 has a predetermined time width, for example, a predetermined voltage level As described above (HI state), the first reference voltage source 23 and the first terminal 13 are connected. The synapse circuit 20 further includes a delay circuit 29 that delays the switching pulse voltage signal V _PRE for a predetermined time.

  One end of the delay circuit 29 is connected to the first input terminal 41 of the synapse circuit 20, and the other end of the delay circuit 29 is connected to the control terminal 26 of the first switch 21 and the control terminal 18 of the second switch 22. . One terminal 27 of the pair of main terminals of the first switch 21 is connected to the second input terminal 42, and the other terminal 28 of the pair of main terminals of the first switch 21 is of the variable resistance element 10. It is connected to the third terminal 15. The first switch 21 is configured to connect the pair of main terminals 27 and 28 when a voltage equal to or higher than a predetermined voltage is applied to the control terminal 26. One terminal 16 of the pair of main terminals of the second switch 22 is connected to the first reference voltage source 23, and the other terminal 17 of the pair of main terminals of the second switch 22 is the resistance change element 10. Is connected to the first terminal 13 of the The second switch 22 is configured to connect the pair of main terminals 16 and 17 when a voltage equal to or higher than a predetermined voltage is applied to the control terminal 18. The second terminal 14 of the variable resistance element 10 is connected to the output terminal 43.

In such a configuration, the switching pulse voltage signal V _PRE input from the first input terminal 51 of the neural network circuit element 40 is input to the control terminal 26 of the first switch 21 and the control terminal 18 of the second switch 22. Therefore, the first switch 21 connects between the pair of main terminals 27 and 28 in a period having a predetermined time width in the switching pulse voltage signal V _PRE (period of the HI state), and the other period (period of the LO state). Between the pair of main terminals 27 and 28 are cut off. Similarly, the second switch 22 connects between the pair of main terminals 16 and 17 in a period having a predetermined time width in the switching pulse voltage signal V _PRE (period of HI state), and the other period (period of LO state). Between the pair of main terminals 16 and 17).

  FIG. 8 is a circuit diagram showing a specific example of the first switch in the synapse circuit of the neural network circuit element shown in FIG. In the embodiment shown in FIG. 8, the first switch 21 is configured using at least two transistors operating in a complementary manner. In the example of FIG. 8, the first switch 21 includes two n-type MOSFETs 211 and 212 and one inverter 213. A predetermined reference voltage (for example, ground voltage) is applied to the source terminal of one n-type MOSFET 211, the drain terminal is connected to the source terminal of the other n-type MOSFET 212, and the gate terminal is connected to the output terminal of the inverter 213 There is. The drain terminal of the other n-type MOSFET 212 is connected to one terminal 27 of the pair of main terminals, and the gate terminal is connected to the control terminal 26. The input terminal of the inverter 213 is also connected to the control terminal 26. The common terminal between the two n-type MOSFETs 211 and 212 is connected to the other terminal 28 of the pair of main terminals. Note that instead of using the inverter circuit 213, one (211) of the two n-type MOSFETs 211 and 212 may be a p-type MOSFET.

As described above, at one terminal 27 of the pair of main terminals, an output from the second output terminal 46 of the neuron circuit 30 provided in the neural network circuit element 40 in which the first switch 21 is provided The load change pulse voltage signal V _POST1 is input. Further, a switching pulse voltage signal V _PRE input from the outside of the neural network circuit element 40 provided with the first switch 21 is input to the control terminal 26.

In such a configuration, while the switching pulse voltage signal V _PRE applied to the control terminal 26 is in the HI state, the n-type MOSFET 211 is in the open state (cutoff state) and the n-type MOSFET 212 is in the closed state (connection state). Therefore, the voltage value corresponding to the voltage value of the load change pulse voltage signal V _POST1 is the output voltage at the other terminal 28 of the pair of main terminals. While the switching pulse voltage signal V _PRE is in the LO state, the n-type MOSFET 211 is closed (connected) and the n-type MOSFET 212 is opened (cut-off). The potential is substantially the same as the ground potential applied to the source terminal of the n-type MOSFET 211.

  The second switch 22 can be realized by, for example, a field effect transistor (FET) or the like. In this case, the gate terminal of the FET functions as the control terminal 18.

In such a synapse circuit 20, a switching pulse voltage signal V _POST2 output from another neural network circuit element 40 is applied to the first input terminal 41 as a first input signal, that is, a switching pulse voltage signal V _PRE. Ru. Further, the load change pulse voltage signal V _POST1 output from the neuron circuit 30 in the same neural network circuit element 40 is applied to the second input terminal 42 of the synapse circuit 20.

[Processing Function and Learning Function of Neural Network Circuit Element]
The variable resistance element 10 has a characteristic that the resistance value between the first terminal 13 and the second terminal 14 is variable as described above. By closing the second switch 22, the first reference voltage V ₁ is applied between the first terminal 13 and the second terminal 14 of the variable resistance element 10. Thereby, a current proportional to the conductivity (reciprocal of resistance value) at the current time of the variable resistance element 10 flows from the DC voltage source 23 to the variable resistance element 10. This current is input to the neuron circuit 30. The magnitude of the current input to the neuron circuit 30 is proportional to the synapse connection weight w, and corresponds to PSP (P ₁ (t), P ₂ (t)) as shown in FIG. As described above, in the present embodiment, the conductivity (reciprocal number of resistance value) of the resistance change element 10 corresponds to the synapse coupling load w.

The current input from the plurality of synapse circuits 20 to the neuron circuit 30 is asynchronously supplied from each of the plurality of synapse circuits 20 included in the neural network circuit element 40 in which the neuron circuit 30 is included. The integration circuit 31 performs space-time addition of input currents from the plurality of synapse circuits 20. The integrated voltage generated by space-time addition can be regarded as the above-mentioned internal potential V _{n of the} neuron. When the internal potential V _n exceeds a predetermined threshold voltage V _TH , the waveform generation circuit 33 generates two pulse voltages (V _POST1 , V _POST2 ). The switching pulse voltage signal V _POST2 output from the neuron circuit 30 of the neural network circuit element 40 included in the intermediate layer 3 is switched to the first input terminal 51 of the synapse circuit 20 of the neural network circuit element 40 included in the output layer 4 It is applied as a pulse voltage signal V _PRE . The switching pulse voltage signal V _POST2 output from the neuron circuit 30 of the neural network circuit element 40 included in the output layer 4 is output from the output signal terminal 8 as the output signal V _out of the neural network circuit.

Switching pulse voltage signal V _POST2 output from neural network circuit element 40 included in intermediate layer 3 is input as a first input signal (switching pulse voltage signal V _PRE ) to neural network circuit element 40 included in output layer 4 In this case, the synapse circuit 20 of the neural network circuit element 40 included in the output layer 4 switches the first switch 21 based on the value of the switching pulse voltage signal V _PRE . In the switching pulse voltage signal V _PRE , the first switch 21 connects the third terminal 15 of the variable resistance element 10 and the second input terminal 42 of the synapse circuit 20 in a period having a predetermined time width. As a result, an input allowable period in which the load change pulse voltage signal V _POST1 can be input to the third terminal 15 of the resistance change element 10 is obtained. The load change pulse voltage signal V _POST1 input to the second input terminal 42 in this input allowable period is applied to the third terminal 15 of the variable resistance element 10. Thus, a pulse voltage having a waveform that temporally overlaps with the switching pulse voltage signal V _PRE among the analog waveform pulse voltage signal V _POST1 (ie, having a predetermined time width) is transmitted to the third terminal 15 of the variable resistance element 10. Applied to the The resistance value of the variable resistance element 10 is changed by the pulse voltage having the predetermined time width. As described above, in the present embodiment, since the inverse number (conductivity) of the resistance value of the resistance change element 10 represents a synaptic connection load, the synaptic connection strength of the synaptic circuit 20 is updated by the change of the resistance value. "Learning" operation becomes possible.

The waveform of the pulse voltage applied to the third terminal 15 of the variable resistance element 10 changes according to the timing (time difference) at which the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE are applied. As described above, the degree of change in resistance value of the variable resistance element 10 depends on the magnitude of the applied voltage, that is, the voltage waveform. The waveform of the pulse voltage applied to the third terminal 15 of the variable resistance element 10 changes in accordance with the timing at which the two pulse voltages are applied. Therefore, the degree of resistance value change of the variable resistance element 10 also changes in accordance with the timing at which the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE are applied.

FIG. 9 is a graph showing the voltage applied to the third terminal of the variable resistance element according to the time difference between the load change pulse voltage signal shown in FIG. 5A and the switching pulse voltage signal shown in FIG. 5B. As described above, the load change pulse voltage signal V _POST1 is applied to the third terminal 15 of the resistance change element 10 only while the switching pulse voltage signal V _PRE is in the HI state. In FIG. 9, a voltage applied to the third terminal 15 of the variable resistance element 10 (hereinafter referred to as a gate input voltage) in the input allowable period by the operation of the first switch 21 is set to V _sample . Also, the input timing difference t _POST1 -t _PRE between the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE is the time when the potential change of the load change pulse voltage signal V _POST1 occurs and the time of the switching pulse voltage signal V _POST2 With the central point (time t ₀ in FIGS. 5A and 5B) as the reference, the time difference of the reference point is used. In FIG. 9, the magnitude of the difference between the two pulse voltage input timings is indicated by an arrow.

As shown in FIG. 9, when the input timing difference t _POST1 -t _PRE between the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE changes, the waveform of the gate input voltage signal V _sample changes. For example, when t _POST1 -t _PRE <0 (when V _POST1 is input earlier than V _PRE ), when the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE overlap in time, the gate input voltage signal V _sample is a voltage in the negative direction, and the magnitude of the gate input voltage signal V _sample increases as the input timing difference decreases until the input timing difference decreases to some extent. As the magnitude of the gate input voltage signal V _sample increases in the negative direction, the conductivity (reciprocal of the resistance value) of the variable resistance element 10 decreases significantly. On the other hand, when t _POST1 -t _PRE > 0 (when V _POST1 is input later than V _PRE ), when the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE overlap in time, the gate input The voltage signal V _sample is a voltage in the positive direction, and the magnitude of the gate input voltage signal V _sample increases as the input timing difference decreases until the input timing difference decreases to some extent. As the magnitude of the gate input voltage signal V _sample increases in the positive direction, the conductivity (reciprocal of the resistance value) of the variable resistance element 10 increases significantly. In a predetermined region where the input timing difference t _POST1 -t _PRE is close to 0, the gate input voltage signal V _sample has an average value close to approximately 0.

Thus, the analog pulse voltage V _POST1 generated in the same neural network circuit element 40 and the other neural network circuit element 40 are input by switching the first switch 21 using the switching pulse voltage signal V _PRE . The gate input voltage signal V _sample dependent on the input timing difference is applied to the third terminal 15 of the variable resistance element 10 based on the input timing difference of the switching pulse voltage V _PRE . The modulation of the resistance value depending on the input timing difference can be realized in the variable resistance element 10 by the gate input voltage signal V _sample applied in this manner. According to this embodiment, since the resistance is changed according to the pulse timing of the neural network circuit element 40 of the previous stage and the amplitude of the pulse which is changed by the feedback signal from the error calculation circuit 5 described later To be realized.

Furthermore, the neural network circuit element 40 changes the amplitude of the load change pulse voltage signal V _POST1 output from the waveform generation circuit 33 of the neuron circuit 30 based on the error voltage signal V _error generated by the error calculation circuit 5 Configured

According to this structure, by converting the time difference between the output signal _{V out} and the teacher signal _{V t} into a voltage signal (error voltage signal _{V error),} the magnitude of the error between the output signal _{V out} and the teacher signal _{V t} Can be reflected in the magnitude of the voltage signal (load change pulse voltage signal V _POST1 ) for changing the synapse load. Therefore, it is possible to change the load so as to reduce the error between the output signal V _out and the teacher signal V _t. Therefore, error back propagation learning can be appropriately realized.

[Error operation circuit]
Next, the error calculation circuit 5 in the present embodiment will be described. The error calculating circuit 5, and at least one output signal V _out, at least one teacher signal V _t is input. Includes error calculating circuit 5 is at least one output signal terminal 8, the number of the at least one teacher signal V _t is input to the number of error calculating circuit 5 of the at least one output signal terminal 8 are equal. Error calculating circuit 5 generates an output signal _{V out,} the magnitude of the voltage signal corresponding to the time difference between the teacher signal _{V t} corresponding to the output signal _{V out} as the error voltage signal _{V error.} Specifically, the error calculation circuit 5 includes the same number of time difference calculation circuits 90 as the output signal terminal 8 and one addition circuit 80. In FIG. 2A, at least one output signal terminal 8 is a plurality of output signal terminals 8a, 8b,. Further, in FIG. 2A, time difference calculation circuits corresponding to the plurality of output signal terminals 8a, 8b,... Are represented by 90a, 90b,. That is, the error calculation circuit 5 includes at least one time difference calculation circuit 90, and in FIG. 2A, at least one time difference calculation circuit 90 is a time difference calculation circuit 90a, 90b,.

Time difference calculating circuit 90, respectively, 8a, 8b, corresponding to the time difference between the output signal _{V out} output from the corresponding output signal terminal of ..., the teacher signal _{V t} corresponding to the output signal _{V out} The error voltage signal V _error is generated, and the error voltage signal V _error is configured to be input to the neuron circuit 30 included in the neural network circuit element 40 of the output layer 4. Specifically, the time difference calculation circuit 90 is connected to the corresponding teacher signal terminal among the first input terminal 93 connected to the corresponding output signal terminal of the neural network circuit 1, 7a, 7b,. A second input terminal 94 and an output terminal 95 for outputting an error voltage signal V _error are provided. The output terminal 95 is connected to the second input terminal 52 of the neural network circuit element 40 of the output layer 4.

The summing circuit 80 sums up the error voltage signal V _error generated in each of the time difference computing circuits 90 to generate a summed voltage signal V _sam , and the summed voltage signal V _sam is included in the intermediate layer 3. It is configured to be input to the neuron circuit 30 of the neural network circuit element 40. Specifically, the summing circuit 80 has the same number of input terminals 84 as the time difference computing circuit 90 connected to the output terminal 95 of the time difference computing circuit 90 and one output terminal 85 for outputting the _summed voltage signal V _sam. And. The output terminal 85 of the summing circuit 80 is fed back to the error input terminal 52 of all the neural network circuit elements 40 included in the intermediate layer 3.

  FIG. 10A is a block diagram showing a schematic configuration example of a time difference calculation circuit in the error calculation circuit of the neural network circuit element shown in FIG. In FIG. 10A, the time difference calculation circuit 90 includes a switch 91 and a peak hold circuit 92. The switch 91 includes a pair of main terminals and a control terminal. One of the pair of main terminals is connected to the second input terminal 94 of the time difference calculation circuit 90, and the other of the pair of main terminals is connected to the input terminal of the peak hold circuit 92 of the time difference calculation circuit 90, The control terminal is connected to the first input terminal 93. The output terminal of the peak hold circuit 92 is connected to the output terminal 95 of the time difference calculation circuit 90.

FIG. 10B is a circuit diagram showing a specific example of the switch and the peak hold circuit in the time difference calculation circuit shown in FIG. 10A. In the embodiment shown in FIG. 10B, the switch 91 is composed of at least two transistors operating in a complementary manner. In the example of FIG. 10B, the switch 91 includes two n-type MOSFETs 911, 912 and one inverter 913. The second reference voltage source 920 is connected to the source terminal of one of the two n-type MOSFETs 911, and the second reference voltage V ₂ is applied to the source terminal. The source terminal of the other n-type MOSFET 912 is connected to the drain terminal of the n-type MOSFET 911 and the gate terminal is connected to the output terminal of the inverter 913.

The drain terminal of the other n-type MOSFET is connected to the second input terminal 94, a teacher signal V _t input to the teacher signal terminal 7 of the input layer 2 is input. The gate terminal of the n-type MOSFET 912 and the input terminal of the inverter 913 are connected to the first input terminal 93, and the switching pulse voltage signal V _POST2 output from the neural network circuit element 40 of the output layer 4 is input. The common terminal between the two n-type MOSFETs 911 and 912 is connected to the input terminal 914 of the peak hold circuit 92. Note that one of the two n-type MOSFETs 911 and 912 (for example, 911) may be used as a p-type MOSFET instead of using the inverter 913.

In such a configuration of the switch 91, the n-type MOSFET 911 is open (cut-off state) and the n-type MOSFET 912 is closed (off state) while the switching pulse voltage signal V _POST2 applied to the first input terminal 93 is in the HI state since the connection state) and a voltage value corresponding to the teacher signal V _t is the output voltage of the switch 91. While the switching pulse voltage signal V _POST2 is in the LO state, the n-type MOSFET 911 is closed (connected) and the n-type MOSFET 912 is opened (cut-off), so the output voltage is the second reference voltage V ₂ . The second reference voltage V _2, for example the minimum potential of the teacher signal V _t is set to (potential difference between the reference potential is the largest negative potential) potential below.

  In the example of FIG. 10B, the peak hold circuit 92 includes two operational amplifiers 915 and 916, a diode 917, a capacitor 918, and a third reference voltage source 919. The capacitance value of the capacitor 918 is, for example, 100 pF. The input terminal 914 of the peak hold circuit 92 is connected to the negative input terminal (−) of the first operational amplifier 915. The positive input terminal (+) of the first operational amplifier 915 is connected to the positive input terminal (+) of the second operational amplifier 916. The anode of the diode 917 is connected to the output terminal of the first operational amplifier 915, and the cathode of the diode 917 is connected to the positive input terminal (+) of the first operational amplifier 915 and the positive input terminal Connected to +).

Furthermore, the cathode of the diode 917 and the positive input terminal (+) of the first and second operational amplifiers 915 and 916 are connected to one end of the capacitor 918. A third reference voltage source 919 is connected to the other end of the capacitor 918. The negative input terminal (−) of the second operational amplifier 916 is connected to the output terminal of the second operational amplifier 916. The output terminal of the second operational amplifier 916 is connected to the output terminal 95 of the time difference calculation circuit 90. The third reference voltage V ₃ at the third reference voltage source 919, a potential lower than the reference potential of the teacher signal V _t (for example, 0 potential), for example, is set to the minimum potential below the potential of the teacher signal V _t. The second reference voltage V ₂ is set to the third reference voltage V ₃ potential below.

According to such a configuration, the time difference calculation circuit 90 outputs a DC voltage according to the time difference between the switching pulse voltage signal V _POST2 and the teacher signal V _t to the output terminal 95 of the error calculation circuit 5. In a state (LO state) before switching pulse voltage signal V _POST2 is input to first input terminal 93, capacitor 918 is charged to the same potential as third reference voltage V _3, and output terminal 95 potential becomes the third reference voltage _{V 3.} When the pulse waveform of the switching pulse voltage signal V _POST2 is input to the first input terminal 93, the switch 91 outputs the voltage of the teacher signal V _t while the switching pulse voltage signal V _POST2 is in the HI state. That is, when the time difference between the switching pulse voltage signal V _POST2 and the teacher signal V _t is within a predetermined range, the switching pulse voltage signal V _POST2 is input earlier than the teacher signal V _t (t _o −t _t > In the case of 0), the output voltage has a negative polarity and a larger potential difference with the reference potential as the time difference becomes larger, and the switching pulse voltage signal V _POST2 is input later than the teacher signal Vt (t _o − In the case of t _t <0), the output voltage has a larger potential difference with the reference potential as the polarity and time difference become larger, and the time difference between the switching pulse voltage signal V _POST2 and the teacher signal V _t is 0 case, the voltage output is substantially equal to the reference potential of the teacher signal V _t.

When the voltage output from the switch 91 is input to the input terminal 914 of the peak hold circuit 92, the capacitor 918 is charged by the output voltage. Therefore, the potential of the output terminal 95 of the time difference calculation circuit 90 becomes equal to that of the input terminal 914. The output of the switch 91 is returned to the third reference voltage V _2, the diode 917 is reverse biased, the capacitor 918 does not occur becomes charged and discharged in an open state. Therefore, the potential of the output terminal 95 is held at the maximum potential output from the switch 91. With this configuration, peak hold circuit 92 can output a DC voltage having a different polarity and magnitude depending on the time difference between switching pulse voltage signal V _POST2 and teacher signal V _t to output terminal 95 as error voltage signal V _error. It becomes possible.

The peak hold circuit 92 is connected to a positive input terminal (+) of the second operational amplifier 916 with a switch (not shown) for switching whether or not to connect to the ground potential. When the pulse of the switching pulse voltage signal V _POST2 is input to the time difference calculation circuit 90 and the switch is closed at a predetermined timing after the error voltage signal V _error based on that is output, the capacitor 918 is charged. charge is discharged and is reset to its original state (state in which the potential of the output terminal 95 becomes the third reference voltage V _3).

FIG. 11 is a circuit diagram showing an example of a summing circuit in the error calculation circuit of the neural network circuit element shown in FIG. The summing circuit 80 illustrated in FIG. 11 includes a resistive element 81, an operational amplifier 83, and a feedback resistive element 82 according to the number of time difference computing circuits 90 (ie, the number of error voltage signals V _error to be input). Are configured as an analog summing circuit having the The resistance value of each of the resistive element 81 and the feedback resistive element 82 is, for example, 1 kΩ. The positive input terminal (+) of the operational amplifier 83 is configured to be at the ground potential, and the negative input terminal (−) of the operational amplifier 83 is connected to the input terminal 84 of the summing circuit 80 via the resistance element 81 . The output terminal of the operational amplifier 83 is connected to the output terminal 85 of the summing circuit 80.

By such a summing circuit 80, one error voltage signal V _error according to the time difference between the switching pulse voltage signal V _POST2 generated in each of the neural network circuit elements 40 included in the output layer 4 and the teacher signal V _t And output as a sum voltage signal V _sam .

Thus, for the neural network circuit element 40 included in the output layer 4, it is possible to change the load so as to reduce the error based on the error between the corresponding output signal V _out and the teacher signal V _t . Furthermore, for the neural network circuit element 40 contained in the intermediate layer 3, it is possible to change the load on the basis of the generated error voltage signal V _error in summing the voltage signal V _sam was summing. Thereby, error back propagation learning can be realized not only for the neural network circuit elements 40 included in the output layer 4 but also for all the neural network circuit elements 40.

[Back propagation learning]
Hereinafter, an aspect of back propagation learning using the neural network circuit 1 will be described in more detail.

First, the case of learning one pattern will be described. As shown _in FIG. 3B according to the desired output signal V _out , the predetermined input signal V _in as shown in FIG. 3A is input to the input signal terminal 6 to perform error back propagation learning. Input a proper teacher signal V _t . As described later, two inputs are performed for one pattern (hereinafter, the first input is referred to as a first step, and the second input is referred to as a second step).

Here, the information of each signal is expressed by the pulse input timing. For example, when "0" is input, the pulse voltage is input at time t = 0 ms, and when "1" is input, the pulse voltage is input at time t = 1 ms. Here, the input timing is defined as an intermediate point in time in the pulse waveform, such as times t _in and t _t shown in FIGS. 3A and 3B.

The switching pulse voltage signal V _POST2 output from the output terminal 53 of the neural network circuit element 40 included in the output layer 4 is output as the output signal V _out of the neural network circuit 1. The timing at which the output signal V _out is output, like the input signal V _in , is output information. For example, when "0" is output, the switching pulse voltage signal V _POST2 is output at time t = 10 ms, and when "1" is output, the switching pulse voltage V _POST2 is output at time t = 11 _ms .

In the learning method in the present embodiment, as the first step, the amplitude of the load change pulse voltage signal V _POST1 output from the neuron circuit 30 of the neural network circuit element 40 included in the output layer 4 is based on the error voltage signal V _error . As a second step, the amplitude of the load change pulse voltage signal V _POST1 output from the neuron circuit 30 of the neural network circuit element 40 included in the intermediate layer 3 is changed based on the error voltage signal V _error. .

Specifically, in the first step, the synapse circuit 20 of the neural network circuit element 40 included in the intermediate layer 3 inputs a signal whose potential is equal to the reference potential (for example, ground voltage) instead of the load change pulse voltage signal. Thus, the resistance value of the variable resistance element 10 of the neural network circuit element 40 included in the output layer 4 is set in a state in which the resistance value of the variable resistance element 10 of the neural network circuit element 40 included in the intermediate layer 3 is not changed. Change. The second step is a load change pulse voltage having a first waveform generated by the neuron circuit 30 of the neural network circuit element 40 in which the synapse circuit 20 is included in the synapse circuit 20 of the neural network circuit element 40 included in the intermediate layer 3 By inputting the signal V _POST1 , the resistance value of the variable resistance element 10 of the neural network circuit element 40 included in the intermediate layer 3 is changed. Learning including such first and second steps is repeatedly performed until the time difference between the teacher signal V _t and the corresponding output signal V _out becomes less than or equal to a prescribed value.

  According to the above method, first, the load change is performed on the neural network circuit element 40 included in the output layer 4 based on the error back propagation learning. Thereafter, the load change is performed on the neural network circuit element 40 included in the intermediate layer 3 based on the error back propagation learning. For this reason, it is possible to change the load so that the error is efficiently reduced with respect to the synapse circuit 20 of each neural network circuit element 40. Therefore, error back propagation learning can be appropriately realized.

  Each step will be described in more detail below.

[First step]
As described above, the switching pulse voltage signal V _PRE is input from the input signal terminal 6 as the input signal V _in to the first input terminal 51 of the neural network circuit element 40 included in the intermediate layer 3. Therefore, the first switch 21 and the second switch 22 are closed in a predetermined time width _in which the input signal Vin is in the HI state. When the second switch 22 is closed, the DC voltage source 23 is connected to the first terminal 13 of the variable resistance element 10, and the first reference is applied between the first terminal 13 and the second terminal 14 of the variable resistance element 10. voltage _{V 1} is applied.

Here, in the first step, in the synapse circuit 20 of the neural network circuit element 40 included in the intermediate layer 3, the potential is the reference instead of the load change pulse voltage signal V _POST1 having the first waveform as shown in FIG. 5A. A signal equal to the potential (eg, ground voltage) is input. Specifically, for example, the neuron circuit 30 of the neural network circuit element 40 included in the intermediate layer 3 is configured to always output the reference potential from the first output terminal 47 in the first step. For this purpose, for example, the digital memory 331 of the waveform generation circuit 33 is configured to output the reference potential in the first step and output the first waveform in the second step. The configuration for determining which of the first step and the second step is used may be, for example, a configuration of inputting to the waveform generation circuit 33 a signal for identifying which of the first step and the second step. For example, an identification signal input terminal connected to the waveform generation circuit 33 is provided in the input layer 2 and an identification signal for determining whether the user of this neural network circuit is the first step or the second step May be input to the identification signal terminal), and the waveform generation circuit 33 has a memory capable of storing how many times the switching pulse voltage _V.sub.PRE has been input to the synapse circuit 20 from the predetermined start time. According to this, either the first step or the second step may be determined.

Thus, even if the first switch 21 and the second switch 22 are closed, the voltage causing the resistance change of the variable resistance element 10 is not applied to the third terminal 15 of the variable resistance element 10 ). Therefore, the resistance value of the variable resistance element 10 of the neural network circuit element 40 included in the intermediate layer 3 does not change. That is, in the first step, the learning operation does not occur in the neuron circuit 30 of the neural network circuit element 40 included in the intermediate layer 3. Therefore, the switching pulse voltage V _POST2 is generated at a timing according to the current resistance value of the variable resistance element 10 and is sent to the neural network circuit element 40 of the output layer 4.

The switching pulse voltage V _POST2 output from the neural network circuit 40 of the intermediate layer 3 is input to the first input terminal 51 of the neural network circuit element 40 included in the output layer 4. Furthermore, in the first step, a load change pulse voltage signal V _POST1 having a first waveform as shown in FIG. 5A is feedback input to the synapse circuit 20 of the neural network circuit element 40 included in the output layer 4. Therefore, in the neural network circuit element 40 included in the output layer 4 in the first step, the resistance value of the variable resistance element 10 changes based on the load change pulse voltage signal V _POST1 .

Furthermore, the switching pulse voltage signal V _POST2 (output signal V _out ) output from the neural network circuit element 40 included in the output layer 4 is also input to the first input terminal 93 of the time difference calculation circuit 90 in the error calculation circuit 5. Ru. The time difference calculation circuit 90 outputs an error voltage signal V _error having an amplitude (a polarity and a magnitude) corresponding to the time difference between the output signal V _out and the teacher signal V _t .

The error voltage signal V _error is fed back to the second input terminal 52 of the neural network circuit element 40 of the output layer 4. The error voltage signal V _error input to the second input terminal 52 becomes a multiplication coefficient of the analog multiplier 333 of the waveform generation circuit 30, and is multiplied by the second waveform. As a result, the load change pulse voltage signal V _POST1 generated by the neuron circuit 30 of the neural network circuit element 40 of the output layer 4 is a signal having an amplitude proportional to the amplitude of the error voltage signal V _error .

As described above, the waveform generation circuit 33 of the neuron circuit 30 generates the load change pulse voltage signal V _POST1 at the same timing as the switching pulse voltage signal V _POST2 . Since the input timing of the error voltage signal V _error is sufficiently shorter than the pulse width of the switching pulse voltage V _PRE , the load change pulse voltage signal V _POST1 is a switching pulse in which the feedback input of the voltage error signal V _error is reflected. It can be considered that the voltage signal V _POST2 is output at the same timing.

As described above, in the neural network circuit element 40 included in the output layer 4, the load change pulse voltage signal V _POST1 having an amplitude corresponding to the time difference between the output signal V _out and the teacher signal V _t , and the switching pulse voltage signal A learning operation is performed by changing the resistance value of the variable resistance element 10 based on the time difference from V _PRE . That is, in the learning operation of the neural network circuit element 40 included in the output layer 4, the error between the corresponding output signal V _out and the teacher signal V _t is reflected.

The error voltage signal V _error generated by each time difference calculation circuit 90 based on the switching pulse voltage signal V _POST2 (output signal V _out ) output from the neural network circuit element 40 included in the output layer 4 in the first step Is also input to the summing circuit 80. Summing circuit 80 generates a summing voltage signal _{V sam} by summing all the error voltage signal _{V error} input. The summing voltage signal V _sam is used in the second step.

[The second step]
Next, as a second step, it is input again to the input signal V _in is the input signal terminal 6 of the same pattern, the teacher signal V _t corresponding to the input signal V _in is input to the teacher signal terminal 7.

Here, in the second step, as shown in FIG. 5A generated by the neuron circuit 30 of the neural network circuit element 40 including the synapse circuit in the synapse circuit 20 of the neural network circuit element 40 included in the intermediate layer 3. The load change pulse voltage signal V _POST1 having the first waveform is input. Therefore, in the neural network circuit element 40 included in the intermediate layer 3 in the second step, the resistance value of the variable resistance element 10 changes based on the load change pulse voltage signal V _POST1 .

As described above, in the first step, the _summing circuit 80 calculates each time difference calculating circuit 90 based on the switching pulse voltage signal V _POST2 (output signal V _out ) output from the neural network circuit element 40 included in the output layer 4. A sum voltage signal V _sam is generated by _summing up all the error voltage signals V _error generated in the above, and is fed back to the second input terminal 52 of the neural network circuit element 40 included in the intermediate layer 3. The error voltage signal V _error input to the second input terminal 52 becomes a multiplication coefficient of the analog multiplier 333 of the waveform generation circuit 30, and is multiplied by the second waveform. As a result, the load change pulse voltage signal V _POST1 generated by the neuron circuit 30 of the neural network circuit element 40 of the intermediate layer 3 is a signal having an amplitude proportional to the amplitude of the _summing voltage signal V _sam .

Accordingly, in a second step, the neural network circuit element 40 contained in the intermediate layer 3, a load change pulses having an amplitude corresponding to the sum of the time difference between the output signal V _out and the teacher signal V _t produced in the first step A learning operation is performed by changing the resistance value of the variable resistance element 10 based on the time difference between the voltage signal V _POST1 and the switching pulse voltage signal V _PRE . That is, in the learning operation of the neural network circuit element 40 contained in the intermediate layer 3, the error between the corresponding output signal V _out and the teacher signal V _t is reflected. Thus, in all of the neural network circuit element 40 constituting the neural network circuit 1, so that the error between the output signal V _out and the teacher signal V _t is decreased, the synapse circuit 20 of each neural network circuit element 40, the It is possible to appropriately change the load in accordance with the degree of contribution to the output signal V _out of the synapse circuit 20.

In the second step, the neural network circuit element 40 included in the output layer 4 may not perform the learning operation. In this case, in the second step, the synapse circuit 20 of the neural network circuit element 40 included in the output layer 4 is referenced to a potential instead of the load change pulse voltage signal V _POST1 having the first waveform as shown in FIG. A signal equal to the potential (eg, ground voltage) is input. Instead of this, the neural network circuit element 40 included in the output layer 4 may perform the learning operation in any of the first and second steps.

As described above, _in the first step in which the first input of the input signal V _in and the teacher signal V _t is performed, in the second step in which the learning operation is performed in the output layer 4 and the second input is performed. The learning operation is performed in the middle layer 3. In this way, error back propagation learning can be realized as the learning operation is performed from the layer close to the output signal terminal 8 to the layer distant from the layer.

When the learning operation is performed for one pattern, one learning including the first step and the second step is repeated until the time difference between the teacher signal V _t and the corresponding output signal V _out becomes less than or equal to a specified value. Run. When the learning operation is performed for a plurality of different patterns, the learning of the first step and the second step is performed for the first pattern, and then the learning of the first step and the second step is performed for the second pattern. As described above, the patterns are changed to perform learning in the first step and the second step. In this way, the operation of repeating the learning in the first step and the second step by the number of patterns is one learning operation. The single learning operation is repeatedly executed until the time difference between the output signal V _out corresponding to the teacher signal V _t in each pattern becomes less than a specified value.

[Example]
Examples will be described below.

[Evaluation of synapse circuit]
First, as a first embodiment, a verification synapse circuit 20A using the ferroelectric memory element described with reference to FIG. 7A as the resistance change element 10 is configured, and whether the switching operation shown in FIG. 9 can be actually realized Verified.

FIG. 12 is a block diagram showing a configuration example of a verification synapse circuit in the first embodiment of the present disclosure. In the verification synapse circuit 20A shown in FIG. 12, the ferroelectric memory element shown in FIG. 7A is used as the resistance change element 10 (in FIG. 12, the resistance change element 10 is indicated by the circuit symbol of FIG. Furthermore, the configuration of the verification synapse circuit 20A other than the verification of the switching operation of the synapse circuit 20 shown in FIG. 1 is omitted. Specifically, in the verification synapse circuit 20A, the second switch 22 shown in FIG. 1 is omitted. Therefore, the DC voltage source 23 is connected to the first terminal 13 of the variable resistance element 10 via the ammeter 39. The output terminal 43 of the verification synapse circuit 20A is grounded. In addition, as the first switch 21, the configuration shown in FIG. 8 is used. The first reference voltage _{V 1} of the DC voltage source 23 is 0.1V, for example.

As described above, in the first switch 21, the load change pulse voltage signal V _POST1 is applied to the control electrode 73 of the variable resistance element 10 in a period (input allowable period) of the switching pulse voltage signal V _PRE in the HI state. In other periods, the control electrode 73 of the resistance change element 10 is grounded.

In this verification, the load change pulse voltage signal V _POST1 shown in FIG. 5A and the switching pulse voltage signal V _PRE shown in FIG. 5B are input. Note that the cycle of the load change pulse voltage signal V _POST1 used in this verification is 32 ms (the period from a certain voltage 0 to the time when the voltage is maximum is 7 ms, and from the time when the voltage is maximum The period until the point of voltage 0 is 25 ms). In this verification, according to the error voltage signal V _error , using two load change pulse voltage signals V _POST1 that the maximum value V _MAX of the voltage of the load change pulse voltage signal V _POST1 is 1.0 V and 0.5 V. The influence of the change in amplitude of the load change pulse voltage signal V _POST1 was verified. The pulse width of the switching pulse voltage signal V _PRE used in the present verification is 1 ms, and the maximum values of the positive voltage and the negative voltage are 5 V, respectively. Conductivity before and after applying the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE while changing the timing of the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE to be applied using such a waveform The rate (the current value obtained by the ammeter 39) was measured.

FIG. 13 is a graph showing the result of verification using the verification synapse circuit shown in FIG. FIG. 13 shows the amount of change in the conductivity of the variable resistance element 1 with respect to the time difference between the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE obtained from the verification synapse circuit 20A shown in FIG. In FIG. 13, a pulse voltage according to the time difference between the load change pulse voltage signal V _POST1 whose maximum value V _{MAX of} voltage is 0.5 V and the switching pulse voltage signal V _PRE is represented by V _{pulse1. MAX} represents a pulse voltage corresponding to the time difference between the load change pulse voltage signal _{V POST1} and the switching pulse voltage signal _{V PRE} is 1.0V at _{V Pulse2.}

As shown in FIG. 13, by using a ferroelectric memory element as the resistance change element 10, the conductance of the resistance change element 10 is determined according to the input timing difference between the load change pulse voltage signal V _POST1 and the switching pulse voltage signal V _PRE. Pulse voltages V _pulse1 and V _{pulse2 to} be _changed were obtained. Furthermore, by changing the amplitude of the load change pulse voltage signal V _POST1 , pulse voltages V _pulse1 and V _pulse2 having different amounts for changing the conductance with the same time difference were obtained. Thus, the amplitude of the load changes the pulse voltage signal _{V POST1} changed by the error voltage signal _{V error} fed back from the input timing difference and the error calculation circuit 5 with the load change pulse voltage signal _{V POST1} and the switching pulse voltage signal _{V PRE} It has been shown that the learning operation can be appropriately realized accordingly.

[Evaluation of error back propagation learning]
Next, as a second embodiment, an error back propagation learning operation is actually performed in a neural network circuit in which a plurality of verification synapse circuits 29B using the ferroelectric memory element described with reference to FIG. 7A as the resistance change element 10 are connected. We verified that it could be realized.

  FIG. 14A is a block diagram showing a configuration example of a verification synapse circuit in the second embodiment of the present disclosure. FIG. 14B is a block diagram showing a configuration example of a verification neural network using the verification synapse circuit shown in FIG. 14A. FIG. 14C is a block diagram showing a configuration example of a verification neural network circuit using the verification neural network circuit element shown in FIG. 14B.

In FIG. 14B, only one switching pulse voltage signal V _PRE is shown to be input to one neural network circuit element 40B for ease of illustration, but as shown in FIG. The switching pulse voltage signal V _PRE of can be input to each neural network circuit element 40B.

In the present embodiment, the neural network circuit element 40B is provided with sixteen synapse circuits 20B having different delay times in the delay circuit 29. The delay time of the delay circuit 29 in each synapse circuit 20B is 1, 2, 3, ..., 16 ms. In FIG. 14B, for ease of illustration, the wiring of the load change pulse voltage signal V _POST1 that is fed back from the neuron circuit 30 to the synapse circuit 20B is omitted. For the integration circuit 31 of the neuron circuit 30, a circuit as shown in FIG. 4 was used. The resistance value of the resistance element 37 is 500 kΩ, and the capacitance value of the capacitor 36 is 1 nF. The threshold value V _TH of the comparison circuit 32 is 5.0V.

The input layer 2B of the neural network circuit 1B has three input signal terminals 6BA, 6BB, 6BC and one teacher signal terminal 7B. Input signals V _in ^A , V _in ^B and V _in ^C are input to the input signal terminals 6BA, 6BB and 6BC, respectively. The teacher signal terminal 7B, a teacher signal V _t is input. The middle layer 3B includes five neural network circuit elements 40B, and the output layer 4B includes one neural network circuit element 40B.

  The input signal terminals 6BA, 6BB, 6BC of the input layer 2B are connected to the first input terminals 51 of all the neural network circuit elements 40B included in the intermediate layer 3B. The output terminal 53 of the neural network circuit element 40B included in the intermediate layer 3B is connected to the first input terminal 51 of the neural network circuit element 40B of the output layer 4B. The output terminal 53 of the neural network circuit element 40B of the output layer 4B is connected to the output terminal 8B of the neural network circuit 1B.

  In the present embodiment, as the evaluation of back propagation learning, learning of exclusive OR (XOR) is performed. Exclusive-OR learning is a learning pattern that is generally used when evaluating the learning performance of error back propagation learning. Table 1 shows four patterns obtained by combining two inputs and one output of exclusive OR.

As described above, in the spiking neuron model, information is expressed by pulse timing. Therefore, in the present embodiment, “0” and “1” in the input and output are expressed as the input time and output time of the pulse as shown in Table 2 below. It should be noted, it shows the input time of the input signal _{V in t in,} the input time of the teacher signal _{V t} as _{t t.}

As described above, the input signal corresponding to the input 1 is V _in ^A , the input signal corresponding to the input 2 is V _in ^B , and the input signal V _in ^C always receives a signal indicating “1”.

The input signal _{V in,} the pulse width is a rectangular pulse of 7 ms. The maximum value of the rectangular pulse is +5 V, and the minimum value is -5 V. As a load change pulse voltage signal V _POST1 output from the neuron circuit 30, a triangular pulse-shaped first waveform as shown in FIG. 5B was used. The cycle of the load change pulse voltage signal V _POST1 is 32 ms (A period from a certain voltage 0 time to a maximum voltage time is 7 ms, and a period from the maximum voltage time to the next voltage 0 time is 25 ms). Also in the present verification, two load change pulse voltage signals V _POST1 are used in which the maximum value V _MAX of the voltage of the load change pulse voltage signal V _POST1 is 1.0 V and 0.5 V. As the switching pulse V _POST2 output from the neuron circuit 30, a rectangular pulse as shown in FIG. 5A was used. The pulse width of the switching pulse V _POST2 was 1 ms, the maximum value of the voltage was +5 V, and the minimum value was −5 V.

FIG. 15 is a graph showing a change in error accompanying learning progress when learning of exclusive OR is performed in the neural network circuit shown in FIG. 14C. The error shown in FIG. 15 is calculated as 1⁄2 of the square of the time difference between the output signal V _out and the teacher signal V _t . As shown in FIG. 15, the errors of all four patterns finally become 0.001 ms ² or less, which indicates that the learning is successful. An error of 0.001 ms ² or less means that a time difference between the output signal V _out and the teacher signal V _t is 0.7% or less.

  From the above description, many modifications and other embodiments of the present disclosure will be apparent to those skilled in the art. Accordingly, the above description should be taken as exemplary only, and is provided to teach those skilled in the art the best mode of carrying out the present disclosure. The structural and / or functional details may be substantially altered without departing from the spirit of the present disclosure.

  For example, although the example in which the neuron circuit 30 and the error calculation circuit 5 are configured as analog circuits has been described in the above embodiment, the present invention is not limited to this, and these may be configured as digital circuits.

  The present disclosure is useful for appropriately performing error back propagation learning operation in a neural network circuit and its learning method.

DESCRIPTION OF SYMBOLS 1 neural network circuit 3 middle layer 4 output layer 5 error calculation circuit 6, 6A, 6B, ... input signal terminal 7, 7a, 7b, ... teacher signal terminal 8, 8a, 8b, ... output signal terminal 10 resistance change element 13th 1 terminal 14 second terminal 15 third terminal 20 synapse circuit 21 first switch 22 second switch 23 first reference voltage source 30 neuron circuit 31 integration circuit 33 waveform generation circuit 40 neural network circuit element 71 ferroelectric layer 72 substrate 73 Control electrode 74 Semiconductor layer 75 First electrode 76 Second electrode 80 Sum circuit 90 Time difference calculation circuit

Claims (13)

A plurality of neural network circuit elements,
An error calculation circuit,
At least one input signal terminal,
And at least one output signal terminal;
A neural network circuit for obtaining at least one output signal output from the at least one output signal terminal from an input signal input to the at least one input signal terminal, wherein
The error calculation circuit receives the at least one output signal and a number of teacher signals equal in number to the at least one output signal terminal, and outputs the output signal and the teacher signal corresponding to the output signal. Configured to generate an error voltage signal that is a voltage signal having a magnitude corresponding to the time difference between
The neural network circuit element comprises at least one synapse circuit and one neuron circuit,
The synapse circuit includes a resistance change element whose resistance value is changed by application of a pulse voltage,
The neuron circuit has a load changing pulse voltage signal having a predetermined first waveform that reaches a predetermined peak value from a reference value and returns from the peak value to the reference value again with the passage of time, and a predetermined time width. And a waveform generation circuit for generating a switching pulse voltage signal having a predetermined second waveform.
The load change pulse voltage signal is input to the synapse circuit of the neural network circuit element including the neuron circuit that has output the load change pulse voltage signal.
The switching pulse voltage signal is input to the synapse circuit of the neural network circuit element other than the neural network circuit element including the neuron circuit which has output the switching pulse voltage signal.
The neural network circuit element is configured to change an amplitude of the load change pulse voltage signal based on the error voltage signal generated by the error calculation circuit.
The synapse circuit is a period during which the switching pulse voltage signal inputted from the neural network circuit element other than the neural network circuit element including the synapse circuit has the predetermined time width, The resistance value of the resistance change element in the synapse circuit is changed by a voltage according to a time difference between the neural network circuit element including the synapse circuit and the load change pulse voltage signal generated by the neuron circuit. , Neural network circuit.   The teacher signal serves as a reference potential when the time difference from the output signal is 0, and when the time difference from the output signal is within a predetermined range, the time difference is large with dipolarity with the reference potential as the central potential. 2. The signal according to claim 1, wherein the potential difference with the central potential becomes larger and the maximum value of the potential difference within the predetermined range is held when the time difference is outside the predetermined range. Neural network circuit. The error calculation circuit includes the same number of time difference calculation circuits as the output signal terminals, and one addition circuit.
The time difference calculation circuit generates the error voltage signal according to the time difference between the output signal output from the corresponding output signal terminal and the teacher signal corresponding to the output signal, and the error voltage signal Are input to the neuron circuit included in the neural network circuit element where the output signal is the output signal output from the output signal terminal,
The summing circuit generates a summing voltage signal by summing the error voltage signals generated in each of the time difference arithmetic circuits, and the signal to be output is output from the output signal terminal. The neural network circuit according to claim 1 or 2, configured to input to the neuron circuit included in the neural network circuit element that is not the output signal to be output. The variable resistance element is
Comprising a first terminal, a second terminal, and a third terminal,
A constant voltage based on the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the variable resistance element between the first terminal and the second terminal. Is applied,
The predetermined time width in a switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the variable resistance element between the first terminal and the third terminal A voltage corresponding to a time difference between the switching pulse voltage signal and the load change pulse voltage signal generated by the neuron circuit of the neural network circuit element including the resistance change element,
The neural network circuit according to any one of claims 1 to 3, wherein a resistance value between the first terminal and the second terminal changes in accordance with a potential difference between the first terminal and the third terminal. . The synapse circuit is between the third terminal of the resistance change element and a terminal to which the load change pulse voltage signal generated by the neuron circuit of the neural network circuit element including the resistance change element is input. It has a first switch that switches connection or disconnection,
The first switch switches the connection or the disconnection based on the switching pulse voltage signal input from the neural network circuit element other than the neural network circuit element including the variable resistance element. Neural network circuit as described.   The neural network circuit according to claim 4, wherein the resistance change element is a ferroelectric memory element. The ferroelectric memory star is
A control electrode formed on a substrate, a ferroelectric layer provided so as to abut the control electrode, a semiconductor layer formed on the ferroelectric layer, a first electrode provided on the semiconductor layer, and And a second electrode,
The neural network circuit according to claim 6, wherein a resistance value between the first electrode and the second electrode changes in accordance with a potential difference between the first electrode and the control electrode. The neuron circuit is
An integrating circuit that integrates a current value flowing through the resistance change element of the synapse circuit;
And a waveform generation circuit that generates the first waveform and the second waveform according to the current value integrated by the integration circuit.
The neural network circuit according to any one of claims 1 to 7, wherein the waveform generation circuit comprises a multiplication circuit which multiplies the magnitude of the first waveform and the magnitude of the error voltage signal. The synapse circuit includes a second switch having one end connected to the first reference voltage source and the other end connected to the first terminal of the resistance change element.
The second switch is configured to connect the first reference voltage source and the first terminal during a period having the predetermined time width in the switching pulse voltage signal input from the another neural network circuit element. The neural network circuit according to any one of claims 4 to 7, which is The learning method of a neural network circuit according to any one of claims 1 to 9,
The amplitude of the load change pulse voltage signal output from the neuron circuit of the first neural network circuit element which is the neural network circuit element, wherein the output signal is the output signal output from the output signal terminal, After changing based on the error voltage signal, an output signal is output from the neuron circuit of the second neural network circuit element that is the neural network circuit element that is not the output signal output from the output signal terminal. A learning method of a neural network circuit, wherein an amplitude of the load change pulse voltage signal is changed based on the error voltage signal. The resistance of the resistance change element of the second neural network circuit element is input by inputting a signal whose potential is equal to the reference potential instead of the load change pulse voltage signal to the synapse circuit of the second neural network circuit element. Changing the resistance value of the variable resistance element of the first neural network circuit element while not changing the value;
By causing the synapse circuit of the second neural network circuit element to generate the load change pulse voltage signal having the first waveform generated by the neuron circuit of the neural network circuit element including the synapse circuit, And V. changing the resistance value of the variable resistance element of the second neural network circuit element.
The neural network circuit learning method according to claim 10, wherein the processing of the first step and the second step is repeatedly executed until the time difference between the teacher signal and the corresponding output signal becomes less than or equal to a specified value. A neural network circuit,
An error calculation circuit which receives an output signal and a teacher signal and generates an error signal having a voltage value corresponding to a time difference between the output signal and the teacher signal;
A first one or more neural network circuit elements included in an intermediate layer of the neural network circuit;
A second one or more neural network circuit elements included in an output layer of the neural network circuit, the output layer outputting the output signal;
Each of the first one or more neural network circuit elements includes a first one or more synapse circuits and a first neuron circuit,
Each of the second one or more neural network circuit elements includes a second one or more synapse circuits and a second neuron circuit,
Each of the first one or more synapse circuits and the second one or more synapse circuits includes a resistance change element whose resistance value changes according to the voltage value of the applied pulse voltage,
Each of the first neuron circuit and the second neuron circuit has a load change pulse voltage signal having a waveform that returns from the reference value to the peak value over time and returns from the peak value to the reference value again, and a switching pulse voltage A waveform generation circuit that generates a signal and
A first load change pulse voltage signal, which is the load change pulse voltage signal generated by the first neuron circuit, is input to each of the first one or more synapse circuits,
A second load change pulse voltage signal, which is the load change pulse voltage signal generated by the second neuron circuit, is input to each of the second one or more synapse circuits,
A first switching pulse signal, which is the switching pulse voltage signal generated by the first neuron circuit, is input to each of the second one or more synapse circuits,
The amplitude of the first load change pulse voltage signal and the amplitude of the second load change pulse voltage signal are determined based on the error signal,
The second one is based on a time period indicated by the first switching pulse voltage signal and a voltage corresponding to a time difference between the first switching pulse voltage signal and the second load change pulse voltage signal. A neural network circuit, wherein a resistance value of the variable resistance element included in each of the synapse circuits described above is changed.   The neural network circuit of claim 12 wherein said switching pulse voltage signal generated by said second neuron circuit is said output signal.

Images