1499410
On Prediction by Data Compression
Traditional wisdom has it that the better a theory compresses the learning data concerning some phenomenon under investigation, the better we learn, generalize, and the better the theory predicts unknown data. This belief is vindicated in practice but apparently has not been rigorously proved in a general setting. Making these ideas rigorous involves the length of the shortest effective description of an individual object: its Kolmogorov complexity. In a previous paper we have shown that optimal compression is almost always a best strategy in hypotheses identification (an ideal form of the minimum description length (MDL) principle). Whereas the single best hypothesis does not necessarily give the best prediction, we demonstrate that nonetheless compression is almost always the best strategy in prediction methods in the style of R. Solomonoff. 
 
Paul Vitanyi is also affiliated with the University of Amsterdam. He was supported by NSERC through International Scientific Exchange Award ISE0125663, and by the European Union through NeuroCOLT ESPRIT Working Group Nr. 8556, and by NWO through NFI Project ALADDIN under Contract number NF 62-376. Ming Li was supported in part by NSERC operating grant OGP-046506, ITRC, and a CGAT grant and the Steacie Fellowship. On sabbatical leave from: Department of Computer Science, University of Waterloo; Email: mli@math.uwaterloo.ca.
14168735
Inductive Reasoning and Kolmogorov Complexity
Abstract Reasoning to obtain the “truth” about reality, from external data, is an important, controversial, and complicated issue in man's effort to understand nature. (Yet, today, we try to make machines do this.) There have been old useful principles, new exciting models, and intricate theories scattered in vastly different areas including philosophy of science, statistics, computer science, and psychology. We focus on inductive reasoning in correspondence with ideas of R. J. Solomonoff. While his proposals result in perfect procedures, they involve the noncomputable notion of Kolmogorov complexity. In this paper we develop the thesis that Solomonoff's method is fundamental in the sense that many other induction principles can be viewed as particular ways to obtain computable approximations to it. We demonstrate this explicitly in the cases of Gold's paradigm for inductive inference, Rissanen's minimum description length (MDL) principle, Fisher's maximum likelihood principle, and Jaynes' maximum entropy principle. We present several new theorems and derivations to this effect. We also delimit what can be learned and what cannot be learned in terms of Kolmogorov complexity, and we describe an experiment in machine learning of handwritten characters. We also give an application of Kolmogorov complexity in Valiant style learning, where we want to learn a concept probably approximately correct in feasible time and examples.
553776
Linear approximation of shortest superstrings
We consider the following problem: given a collection of strings s1, ••• ,sm, find the shortest string s such that each s; appears as a substring (a consecutive block) of s. Although this problem is known to be NP-hard, a simple greedy procedure appears to do quite well and is routinely used in DNA sequencing and data compression practice, namely: repeatedly merge the pair of (distinct) strings with maximum overlap until only one string remains. Let n denote the A. Blum was supported by a National Science Foundation (NSF) Graduate Fellowship. Part of this work was done while A. Blum was visiting AT & T Bell Labs. T. Jiang was supported in part by a grant from SERB, McMaster University and NSERC Operating Grant OGP0046613. M. Li was supported in part by the NSERC Operating Grants OGP0036747 and OGP0046506. J. Tromp was supported in part by NSERC Grant OGP0036747 while the author was visiting at the University of Waterloo. Authors' present addresses: A. Blum, School of Computer Science, Carnegie-Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213. E-mail: avrim@theory.cs.cmu.edu; T. Jiang, Department of Computer Science, McMaster University, Hamilton, Ont., Canada L8S 4Kl. E-mail: jiang<Q•maccs. mcmaster.ca; M. Li, Department of Computer Science, University of Waterloo, Waterloo, Ont., Canada N3L 30 I. E-mail: mli@watmath.waterloo.edu; J. Tromp, CWI, P.O. Box 4079, 1009 AB Amsterdam, The Netherlands. E-mail: tromp@cwi.nl; M. Yannakakis, Room 2C-319, AT&T Bell Labs, 600 Mountain Avenue, Murray Hill, NJ 07974. E-mail: mihalis@research.att.com. Permission to copy without fee all or part of this material is granted provided that the copies arc not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. © 1994 ACM 0004-5411/94/0700-0630$03.50 Journal of the Association for Computing Machinery, Vol. 41, No. 4, July 1994, pages 630-647. Linear Approximation of Shortest Superstrings 631 length of the optimal superstring. A common conjecture states that the above greedy procedure produces a superstring of length O(n) (in fact, 2n), yet the only previous nontrivial bound known for any polynomial-time algorithm is a recent O(n log n) result. We show that the greedy algorithm does in fact achieve a constant factor approximation, proving an upper bound of 4n. Furthermore, we present a simple modified version of the greedy algorithm that we show produces a superstring of length at most 3n. We also show the superstring problem to be MAX SNP-hard, which implies that a polynomial-time approximation scheme for this problem is unlikely.
871253
Kolmogorov Complexity Arguments in Combinatorics
Abstract The utility of a Kolmogorov complexity method in combinatorial theory is demonstrated by several examples.
9397376
Inferring a DNA Sequence from Erroneous Copies
Abstract We suggest a novel approach for efficiently reconstructing an original DNA sequence from erroneous copies.
43310555
Twist-Rotation Transformations of Binary Trees and Arithmetic Expressions
The paper studies the computational complexity and efficient algorithms for the twist?rotation transformations of binary trees, which is equivalent to the transformation of arithmetic expressions over an associative and commutative binary operation. The main results are (1) a full binary tree with n labeled leaves can be transformed into any other in at most 3nlogn+2n twist and rotation operations, (2) deciding the twist?rotation distance between two binary trees is NP-complete, and (3) the twist?rotation transformation can be approximated with ratio 6logn+4 in polynomial time for full binary trees with n uniquely labeled leaves.
10964754
SETRED: Self-training with Editing
Self-training is a semi-supervised learning algorithm in which a learner keeps on labeling unlabeled examples and retraining itself on an enlarged labeled training set. Since the self-training process may erroneously label some unlabeled examples, sometimes the learned hypothesis does not perform well. In this paper, a new algorithm named Setred is proposed, which utilizes a specific data editing method to identify and remove the mislabeled examples from the self-labeled data. In detail, in each iteration of the self-training process, the local cut edge weight statistic is used to help estimate whether a newly labeled example is reliable or not, and only the reliable self-labeled examples are used to enlarge the labeled training set. Experiments show that the introduction of data editing is beneficial, and the learned hypotheses of Setred outperform those learned by the standard self-training algorithm.
3009418
The hamiltonian index of a graph and its branch-bonds
Let $G$ be an undirected and loopless finite graph that is not a path. The minimum $m$ such that the iterated line graph $L^m(G)$ is hamiltonian is called the hamiltonian index of $G,$ denoted by $h(G).$ A reduction method to determine the hamiltonian index of a graph $G$ with $h(G)\geq 2$ is given here. With it we will establish a sharp lower bound and a sharp upper bound for $h(G)$, respectively, which improves some known results of P.A. Catlin et al. [J. Graph Theory 14 (1990)] and H.-J. Lai [Discrete Mathematics 69 (1988)]. Examples show that $h(G)$ may reach all integers between the lower bound and the upper bound.
4489692
Inferring Queue Sizes in Access Networks by Active Measurement
Router queues can impact both round-trip times and throughput. Yet little is publicly known about queue provisioning employed by Internet services providers for the routers that control the access links to home computers. This paper proposes QFind, a black-box measurement technique, as a simple method to approximate the size of the access queue at last mile router. We evaluate QFind through simulation, emulation, and measurement. Although precise access queue results are limited by receiver window sizes and other system events, we find there are distinct difference between DSL and cable access queue sizes.
46138782
Leveraging legacy codes to distributed problem-solving environments: a Web services approach
This paper presents WSOWG, a Web-services-oriented wrapper generator for automatically wrapping non-networked legacy codes as Web services for reuse in distributed problem-solving environments. Using WSOWG, a finite element based computational fluid dynamics (CFD) legacy code has been wrapped as a Web service. A problem-solving environment for simulating incompressible Navier-Stokes flows has also been implemented. A user makes use of the CFD service through a Web page without knowing the exact implementation of the service. In this way, a user's computing environment can be extended to a heterogeneous distributed computing environment. Performance evaluation shows that the overhead to invoke the CFD Web service generated by WSOWG using Simple Object Access Protocol (SOAP) and CORBA Internet Inter-ORB Protocol (IIOP) is reasonable compared with that of invoking another CFD Web service manually wrapped from the CFD legacy code using SOAP only.
9175149
Generation of Comprehensible Hypotheses from Gene Expression Data
Machine learning techniques have been recognized as powerful tools for the analysis of gene expression data. However, most learning techniques used in class prediction in gene expression analysis during the past years generate black-box models. Although the prediction accuracy of these models could be very well, they provide little insight into the biological facts. This paper holds the recognition that a more reasonable role for machine learning techniques is to generate hypotheses that can be verified or refined by human experts instead of making decisions for human experts. Based on this recognition, a general approach to generate comprehensible hypotheses from gene expression data is described and applied to human acute leukemias as a test case. The results demonstrate the feasibility of using machine learning techniques to help form hypotheses on the relationship between genes and certain diseases.