27853952
A non-greedy approach to tree-structured clustering
Abstract We propose a new interdisciplinary approach for the hard optimization problem of tree-structured clustering, wherein the imposition of structural constraints on the solution drastically reduces the complexity of classifying data. The method, derived with analogy to statistical physics, performs a global optimization over the entire tree. Experimentation with non-trivial examples shows that our approach consistently outperforms greedy methods and avoids local minima that may trap them.
11019674
A Deterministic Annealing Approach for Parsimonious Design of Piecewise Regression Models
A new learning algorithm is proposed for piecewise regression modeling. It employs the technique of deterministic annealing to design space partition regression functions. While the performance of traditional space partition regression functions such as CART and MARS is limited by a simple tree-structured partition and by a hierarchical approach for design, the deterministic annealing algorithm enables the joint optimization of a more powerful piecewise structure based on a Voronoi partition. The new method is demonstrated to achieve consistent performance improvements over regular CART as well as over its extension to allow arbitrary hyperplane boundaries. Comparison tests, on several benchmark data sets from the regression literature, are provided.
45307048
Image decoding over noisy channels using minimum mean-squared estimation and a Markov mesh
Recently, we developed a sequence-based minimum mean-squared error (MMSE) estimator for decoding quantized data transmitted over noisy channels. The method effectively views the encoder and noisy channel tandem as a discrete hidden Markov model (HMM), with transmitted indices the unknown states and received indices the observable symbols. Here, we extend this 1D approach to images, using a Markov mesh random field to model the encoded image. Our decoder is based on an approximate forward/backward algorithm for calculating pixel "label probabilities" in Markov meshes which may also have application to image labeling and segmentation. For a DPCM-based image coding system and a high error-rate channel, the new decoder obtains significant performance gains, both objective and visually discernable, over the standard decoder, as well as over several other competing techniques.
17425751
A Mixture of Experts Classifier with Learning Based on Both Labelled and Unlabelled Data
We address statistical classifier design given a mixed training set consisting of a small labelled feature set and a (generally larger) set of unlabelled features. This situation arises, e.g., for medical images, where although training features may be plentiful, expensive expertise is required to extract their class labels. We propose a classifier structure and learning algorithm that make effective use of unlabelled data to improve performance. The learning is based on maximization of the total data likelihood, i.e. over both the labelled and unlabelled data subsets. Two distinct EM learning algorithms are proposed, differing in the EM formalism applied for unlabelled data. The classifier, based on a joint probability model for features and labels, is a "mixture of experts" structure that is equivalent to the radial basis function (RBF) classifier, but unlike RBFs, is amenable to likelihood-based training. The scope of application for the new method is greatly extended by the observation that test data, or any new data to classify, is in fact additional, unlabelled data - thus, a combined learning/classification operation - much akin to what is done in image segmentation - can be invoked whenever there is new data to classify. Experiments with data sets from the UC Irvine database demonstrate that the new learning algorithms and structure achieve substantial performance gains over alternative approaches.
16945889
A Proof Theoretic Approach to Operational Semantics
Abstract Proof theory can be applied to the problem of specifying and reasoning about the operational semantics of process calculi. We overview some recent research in which λ-tree syntax is used to encode expressions containing bindings and sequent calculus is used to reason about operational semantics. There are various benefits of this proof theoretic approach for the π-calculus: the treatment of bindings can be captured with no side conditions; bisimulation has a simple and natural specification in which the difference between bound input and bound output is characterized using difference quantifiers; various modal logics for mobility can be specified declaratively; and simple logic programming-like deduction involving subsets of second-order unification provides immediate implementations of symbolic bisimulation. These benefits should extend to other process calculi as well. As partial evidence of this, a simple λ-tree syntax extension to the tyft/tyxt rule format for name-binding and name-passing is possible that allows one to conclude that (open) bisimilarity is a congruence.
2634305
Deadlines, Travel Time, and Robot Problem Solving
1 Abstract This paper describes some extensions to the reductionist planning paradigm typified by Sacerdoti's NOAH program. Certain inadequacies of the partial ordering scheme used in NOAH are pointed out and a new architecture is detailed which circumvents these problems. An example from the semi-automated factory domain is used to illustrate features of the new planner. Techniques for eliminating unnecessary travel time by the robot and avoiding backtracking due to deadline failures are discussed and their incorporation in the planner is described. 2 Introduction Most planners since NOAH [8] have represented a plan for a set of tasks as a partial (ie., non-linear) ordering of the steps required for carrying out those tasks. NOAH and its successors (eg., NONLIN [9] and DEVISER [10]) employ a partially ordered network of tasks to avoid early and unnecessary commitment to task orderings. The motivation for this is to eliminate backtrack-ing. However, maintaining a consistent partial order is difficult in domains where the fact that tasks actually take time plays an important role: domains in which deadlines or robot travel time are serious considerations. That a partial order leads unavoidably to a deadline failure usually cannot be discovered until an attempt is made to linearize the ordering. However, failure to notice a deadline violation early will require backtracking later. In addition, planning with a partial order is not well suited to efficiently managing factors like travel or machine running time. It's not difficult to represent that moving from one workstation to another takes time proportional to the distance separating the two workstations. However, generating a plan that eliminates unnecessary travel between workstations requires exploring some of the linearized task orderings. It is not until tasks are completely ordered that the source and destination workstations of each movement can be known and the travel time computed with any accuracy. In this paper we describe an approach to planning that combines the use of a partial order with a method for exploring the possible repercussions of that partial order. This approach has been implemented in the FORBIN planner (First Order RoBot INtender). FORBIN is a planner capable of solving a significantly wider class of problems than any of its predecessors. 2.1 The Factory Domain One problem domain that has been used for exploring our approach to planning is what we refer to as the semi-automated factory. In this domain a mobile robot operator wanders about the …
18852816
Higher-Order Logic Programming
In this paper we consider the problem of extending Prolog to include predicate and function variables and typed λ-terms. For this purpose, we use a higher-order logic to describe a generalization to first-order Horn clauses. We show that this extension possesses certain desirable computational properties. Specifically, we show that the familiar operational and least fixpoint semantics can be given to these clauses. A language, λProlog that is based on this generalization is then presented, and several examples of its use are provided. We also discuss an interpreter for this language in which new sources of branching and backtracking must be accommodated. An experimental interpreter has been constructed for the language, and all the examples in this paper have been tested using it.
711002
Counting Distinct Strings
Abstract. This paper discusses how to count and generate strings that are ``distinct'' in two senses: p -distinct and b -distinct. Two strings x on alphabet A and x' on alphabet A' are said to be p -distinct iff they represent distinct ``patterns''; that is, iff there exists no one—one mapping from A to A' that transforms x into x' . Thus aab and baa are p -distinct while aab and ddc are p -equivalent. On the other hand, x and x' are said to be b -distinct iff they give rise to distinct border (failure function) arrays: thus aab with border array 010 is b -distinct from aba with border array 001 . The number of p -distinct (resp. b -distinct) strings of length n formed using exactly k different letters is the [k,n] entry in an infinite p' (resp. b' ) array. Column sums p[n] and b[n] in these arrays give the number of distinct strings of length n . We present algorithms to compute, in constant time per string, all p -distinct (resp. b -distinct) strings of length n formed using exactly k letters, and we also show how to compute all elements p'[k,n] and b'[k,n] . These ideas and results have application to the efficient generation of appropriate test data sets for many string algorithms.
15440056
Unification Under a Mixed Prefix
Abstract Unification problems are identified with conjunctions of equations between simply typed λ-terms where free variables in the equations can be universally or existentially quantified. Tow schemes for simplifying quantifier alternation, called Skolemization and raising (a dual of Skolemization), are presented. In this setting where variables of functional type can be quantified and not all types contain closed terms, the naive generalization of first-order Skolemization has several technical problems that are addressed. The method of searching for pre-unifiers described by Huet is easily extended to the mixed prefix setting, although solving flexible-flexible unification problems is undecidable since types may be empty. Unification problems may have numerous incomparable unifiers. Occasionally, unifiers share common factors and general of these are presented. Various optimizations on the general unification search problem are as discussed.