The present invention relates to a neural network and neural coupling of neurons which constitute the neural network.
A proposed prior art type of the computer includes a neural network operating in a mode similar to information processing in the human brain.
This type of device processes mathematically random matter such as pattern recognition and associative memorization, which could not be satisfactorily processed by the sequential processing mode Neumann type computer.
In FIG. 2 is shown an elementary prior art neuron unit of a neural network. The neuron 1 receives n input signals x.sub.1, x.sub.2, . . . x.sub.i, . . . x.sub.n and multiplies each by a synaptic coupling weight w.sub.1, w.sub.2, . . . w.sub.i, . . . w.sub.n. Neuron 1 calculates the sum of the multiplication results. The neuron 1 adds a specific threshold value h to the sum and operates on the added result with a nonlinear function to thereby derives an output signal y. Such operation is mathematically represented by the following relation: EQU y=f(.SIGMA..sub.1 w.sub.i x.sub.i +h) (1)
In the operation, if the synaptic coupling weight w.sub.i is positive, it provides an excitatory synaptic coupling, while if the synaptic coupling weight w.sub.i is negative, it provides an inhibitatory synaptic coupling. When simulating a neuron in a mathematical form, the nonlinear function is normally represented by the following sigmoid function: EQU f(x)=1;{1+exp(-x)} (2)
FIG. 3 is a block diagram of the neuron performance. The neuron is comprised of a multiplication and summation unit 2 for multiplying synaptic coupling weights w.sub.i with the corresponding input signals x.sub.i to obtain the sum of the multiplication results. Nonlinear operating unit 3 adds a specific threshold value to the sum and nonlinearly processes the added result. The value of the synaptic coupling weight w.sub.i is determined according to an adaptive learning function unit (not shown) based on back-propagation method.
FIG. 4 is a block diagram of an example of a prior art neural network. In the Figure, the neural network includes input terminals 4.sub.1, 4.sub.2. . . 4.sub.i, . . . 4.sub.l, primary neurons 5.sub.1, 5.sub.2, . . . 5.sub.j, . . . 5.sub.m, secondary neurons 6.sub.1, 6.sub.2, . . . 6.sub.k, . . . 6.sub.n, and output terminals 7.sub.1, 7.sub.2, . . . 7.sub.k, . . . 7.sub.n. The input terminals 4.sub.1, 4.sub.2, . . . 4.sub.i, . . . 4.sub.l receive each of l number of input signals a.sub.1, a.sub.2, . . . a.sub.i, . . . a.sub.l. These input signals are distributed to and supplied to m neurons 5.sub.1, 5.sub.2, . . . 5.sub.j, . . . 5.sub.m. Each of the neurons r.sub.2, 5.sub.2, . . . 5.sub.j, . . . 5.sub.m carries out the before-mentioned multiplication and summation operations and nonlinear processing. Further, output signals of the primary neurons 5.sub.1, 5.sub.2, . . . 5.sub.j, . . . 5.sub.m are distributed and supplied into respective one of n secondary neurons 6.sub.1, 6.sub.2, . . . 6.sub.k, . . . 6.sub.n. Each of the neurons 6.sub.1, 6.sub.2, . . . 6.sub.k, . . . 6.sub.n also carries out the before-mentioned multiplication and summation operation and the subsequent nonlinear processing, such that respective output signals b.sub.1, b.sub.2 , . . . b.sub.k, . . . b.sub.n are derived from n output terminals 7.sub.1, 7.sub.2, . . . 7.sub.k, . . . 7.sub.n. The values of the synaptic coupling weights of respective neurons 5.sub.1 - 5.sub.m and 6.sub.1 - 6.sub.n are determined by a controlling learning function unit (not shown). The neural network is supplied with a set of input signals a.sub.1 - a.sub.l, the processed results of which are known as a set of output signals b.sub.1 - b.sub.n. A current set of output signals is compared with the known output signals to determine the value of synaptic coupling weights by the learning function unit through the back-propagation method. By repeating such procedure a appropriate number of times, the final synaptic coupling weights are adaptively adjusted to produce optimum output signals when the input signals a.sub.1 - a.sub.l are supplied to the neural network.
As described above, according to adaptive learning by the learning function unit, the neural network can organize itself so as to obtain correct output signals b.sub.1 - b.sub.n from input signals a.sub.1 - a.sub.l. Therefore, the neural network does not need any specific algorithm or program for processing the signals. Moreover, the neural network can process of multiple of data concurrently in parallel so as to provide high speed processing of mathematically random data such as derived during pattern recognition and associative memorization.
The performance of the neural network depends the number of neurons, and performance is more effective as the number of neurons increases . However, when increasing the number of neurons, there is a corresponding increase in the number of couplings between the input terminals and the neurons and between the neurons with each other. Generally, when the number of neurons is multiplied by n-, the number of couplings is increased on the order of n.sup.2 .noteq.times.
The currently available integrated circuit technology is only applicable to a two-dimensional substrate, and the number of multi-layered patterns is limited to a certain degree. Therefore, in order to construct a neural network as shown in FIG. 4 in the form of an integrated circuit, the number of available neurons is considerably restricted, thereby causing a problem. There has been experimentally produced various types of the neural networks. For example, patterns are formed to constitute couplings between input terminals and neurons and among the neurons. The patterns are commonly used in the time-sharing mode for a plurality of input terminals and neurous so as to reduce the actual number of neural couplings. In another conventional neural network, photoemitters, photodetecters, optical fibers and spatial photomodulators are used to provide effective spatial couplings to transmit optical signals between input terminals and neurons and among neurons. Coupling weights can be also optically set in the neural network.
However, in the prior art, perfect parallel processing cannot be performed. In the latter type of the prior art, presently available optical technology is not sufficiently sophisticated to enable device size reduction, and it would be difficult to continuously change coupling weight according to learning results. Either of the prior art structures has drawbacks to impair essential neural network features.
USPTO Backgrounds
USPTO Backgrounds