We advocate the position that unsupervised learning of rich representations requires careful consideration of an issue that usually receives only cursory attention: The definition of a statistical 'event', or 'sample'. Data sets are presumed to have been generated by sampling from some probability distribution that is to be estimated, but there is no general canonical way to select a model for a given data set and define the correspondence between the various components of its joint random variable and particular subsets, or more generally, features, of the data. Any attempt to automate this choice must confront the fact that without a definition of 'event', this exercise cannot be formulated as a statistical learning problem. We introduce two supplementary criteria, information at a distance and information contrast, in order to clear this impasse, and show anecdotal results from using each. We argue that this issue also arises (whether recognized or not) in automated learning of feature hierarchies to form a rich representations, because distinct events are selected at one level of the hierarchy and bound together to form joint events at the next level.
['Richard James Rohwer']
Events, interest, segmentation, binding and hierarchy