As a method of enabling pattern recognition of a new category, a method using a neural network such as a multilayer perceptron or radial basis function network is known well. Especially, of models capable of reducing the influence of noise, a method of executing hierarchical template matching of local features (e.g., non-patent reference 1 (Yanagawa, Fukushima, and Yoshida, “Additional Learnable Neocognitron”, Technical Report of IEICE, NC2001-176, pp. 63-70, 2002)) is one of models capable of additional learning of a new category.
In patent reference 1 (Japanese Patent Registration No. 2780454), the threshold value of an inhibitory unit is corrected or a new excitatory unit is added in an intermediate layer, or the threshold value of an excitatory unit is corrected or a new inhibitory unit is added in an intermediate layer calculation unit in accordance with a recognition error signal, thereby enabling additional learning.
In patent reference 2 (Japanese Patent Registration No. 2779119), the weight of the sub-categorization unit is corrected on the basis of a signal obtained by weighting a learning control signal reflecting the degree of progress of learning by the degree of group membership output from the categorization unit. Since the category adding unit adds a new category to the sub-categorization unit as needed, additional learning can effectively be performed.
In the arrangement disclosed in patent reference 3 (Japanese Patent Laid-Open No. 9-62648), learning of the pattern validation unit is controlled by using recognition error pattern data and pattern recognition internal state data at the time of recognition error. Additional learning is repeated without relearning, thereby decreasing recognition errors.
Patent reference 4 (Japanese Patent Laid-Open No. 5-274455) comprises a learning control means, having already learned connections, for allowing a user to rewrite an arbitrary synapse connection, and a mechanism which causes an intermediate layer neuron to present a supervisory signal for additional learning.
Patent reference 5 (Japanese Patent Laid-Open No. 2002-42107) discloses a method, in which the learning coefficient between the intermediate layer and the output layer is made larger than that between the input layer and the intermediate layer, thereby enabling high-speed learning without any large change in the structure of the loose connection module which has already finished learning.
Patent reference 6 (Japanese Patent Laid-Open No. 9-138785) discloses a method of properly identifying an unlearned pattern. In this method, learning is done such that when a boundary pattern outside the category range is input to the input unit group, each unit of the output unit group outputs 0 or a small value.
In a hierarchical neural network which executes learning by back propagation, a constructive back propagation (to be abbreviated as CBP hereinafter) is known, which adds a unit of a hidden layer (e.g., non-patent reference 2 ((M. Lehtokangas, “Modeling with constructive backpropagation,” Neural Networks, vol. 12, pp. 707-716, 1999)). In CBP, an already learned connection updates only a connection from a permanently added unit to the output layer in accordance with BP.
In the above-described prior arts, it is difficult to efficiently execute learning (adjustment of internal parameters such as the number of modules of the neural network and the connection weight distribution) for recognition of an arbitrary new pattern (unlearned feature pattern). In addition, the type of a feature useful for internal representation of an arbitrary pattern cannot be known in advance. The connection (so-called receptor field) between operation elements suitable for detecting a useful feature cannot efficiently be learned.
For example, in non-patent reference 1, when a predetermined S-layer neuron having random connection outputs a larger value than an S-layer neuron having selectivity for another feature class, a class for detection of a new feature category is added, thereby enabling additional learning. However, it is not guaranteed that the feature class learned at this time is a new and effective feature category. It is not guaranteed either that the feature class is effective for recognizing an object of another class.
In non-patent reference 2, it is not always guaranteed that the newly added unit executes learning to minimize errors, i.e., learns a connection suitable for detecting a feature class useful for a given pattern recognition problem and, more particularly, detection of a new category. For this reason, it is difficult to execute efficient learning as a whole in recognizing a complex pattern.
In patent reference 2, when the category to which an input pattern signal to be learned belongs does not belong to the sub-categorization unit to execute learning, the currently input pattern signal is input to the category dictionary as a reference pattern, thereby enabling additional learning to recognize the new category. However, since the reference pattern itself is input as the new category, it is not guaranteed that proper recognition can be done even when the pattern is deformed. For this reason, it is difficult to execute efficient recognition and learning operation using a few hardware resources (circuit elements or memory space).
In patent reference 4, especially synapse connections on the input layer side are learned, and connections on the output layer side are fixed. However, no new processing module corresponding to a new category is added. Hence, it is difficult to sufficiently implement adaptation to environment and versatility.
USPTO Backgrounds
USPTO Backgrounds