Convergence Properties of a Learning Algorithm
in memory. In the learning process described by the algorithm, observations are made on individuals one at a time and the current estimate of the required partitioning may be adjusted after each observation, on the basis of knowledge of the category to which the individual observed belongs. At any given time, the current estimate of the partitioning is all that is held in memory; past history is lost except insofar as it has been incorporated into the present estimate. The learning process of perceptrons, as well as that of other artificial intelligences, is of this general form. 2. Notation and assumptions. It is assumed that each individual is a member of one and only one of two categories. The results obtained are applicable to the more general case, however, for they may be applied to appropriate partitions of a set of three or more categories into two subsets. Each individual in the population is characterized by an attribute vector X in m-dimensional Euclidean space; let S,, S2 be the sets of vectors attributed to members of the first and second categories, respectively. We shall suppose that this characterization is sufficiently rich with respect to the given classification problem, that is to say that the regions S, and S2 are separable by a hyperplane (except for a set of probability measure zero). This terminology is appropriate to situations for which in the case of failure of the condition of sufficient richness, a re-examination of the world of individuals and the subsequent increasing of the number of components of the characterizing vectors can be expected to yield a new description for which this condition is satisfied. The question of whether, in a particular case, a sufficiently rich characterization can be achieved is obviously crucial but beyond the scope of this paper.