5971084
Minimum description length induction, Bayesianism, and Kolmogorov complexity
The relationship between the Bayesian approach and the minimum description length approach is established. We sharpen and clarify the general modeling principles minimum description length (MDL) and minimum message length (MML), abstracted as the ideal MDL principle and defined from Bayes's rule by means of Kolmogorov complexity. The basic condition under which the ideal principle should be applied is encapsulated as the fundamental inequality, which in broad terms states that the principle is valid when the data are random, relative to every contemplated hypothesis and also these hypotheses are random relative to the (universal) prior. The ideal principle states that the prior probability associated with the hypothesis should be given by the algorithmic universal probability, and the sum of the log universal probability of the model plus the log of the probability of the data given the model should be minimized. If we restrict the model class to finite sets then application of the ideal principle turns into Kolmogorov's minimal sufficient statistic. In general, we show that data compression is almost always the best strategy, both in model selection and prediction.
553776
Linear approximation of shortest superstrings
We consider the following problem: given a collection of strings s1, ••• ,sm, find the shortest string s such that each s; appears as a substring (a consecutive block) of s. Although this problem is known to be NP-hard, a simple greedy procedure appears to do quite well and is routinely used in DNA sequencing and data compression practice, namely: repeatedly merge the pair of (distinct) strings with maximum overlap until only one string remains. Let n denote the A. Blum was supported by a National Science Foundation (NSF) Graduate Fellowship. Part of this work was done while A. Blum was visiting AT & T Bell Labs. T. Jiang was supported in part by a grant from SERB, McMaster University and NSERC Operating Grant OGP0046613. M. Li was supported in part by the NSERC Operating Grants OGP0036747 and OGP0046506. J. Tromp was supported in part by NSERC Grant OGP0036747 while the author was visiting at the University of Waterloo. Authors' present addresses: A. Blum, School of Computer Science, Carnegie-Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213. E-mail: avrim@theory.cs.cmu.edu; T. Jiang, Department of Computer Science, McMaster University, Hamilton, Ont., Canada L8S 4Kl. E-mail: jiang<Q•maccs. mcmaster.ca; M. Li, Department of Computer Science, University of Waterloo, Waterloo, Ont., Canada N3L 30 I. E-mail: mli@watmath.waterloo.edu; J. Tromp, CWI, P.O. Box 4079, 1009 AB Amsterdam, The Netherlands. E-mail: tromp@cwi.nl; M. Yannakakis, Room 2C-319, AT&T Bell Labs, 600 Mountain Avenue, Murray Hill, NJ 07974. E-mail: mihalis@research.att.com. Permission to copy without fee all or part of this material is granted provided that the copies arc not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. © 1994 ACM 0004-5411/94/0700-0630$03.50 Journal of the Association for Computing Machinery, Vol. 41, No. 4, July 1994, pages 630-647. Linear Approximation of Shortest Superstrings 631 length of the optimal superstring. A common conjecture states that the above greedy procedure produces a superstring of length O(n) (in fact, 2n), yet the only previous nontrivial bound known for any polynomial-time algorithm is a recent O(n log n) result. We show that the greedy algorithm does in fact achieve a constant factor approximation, proving an upper bound of 4n. Furthermore, we present a simple modified version of the greedy algorithm that we show produces a superstring of length at most 3n. We also show the superstring problem to be MAX SNP-hard, which implies that a polynomial-time approximation scheme for this problem is unlikely.
871253
Kolmogorov Complexity Arguments in Combinatorics
Abstract The utility of a Kolmogorov complexity method in combinatorial theory is demonstrated by several examples.
43310555
Twist-Rotation Transformations of Binary Trees and Arithmetic Expressions
The paper studies the computational complexity and efficient algorithms for the twist?rotation transformations of binary trees, which is equivalent to the transformation of arithmetic expressions over an associative and commutative binary operation. The main results are (1) a full binary tree with n labeled leaves can be transformed into any other in at most 3nlogn+2n twist and rotation operations, (2) deciding the twist?rotation distance between two binary trees is NP-complete, and (3) the twist?rotation transformation can be approximated with ratio 6logn+4 in polynomial time for full binary trees with n uniquely labeled leaves.
14168735
Inductive Reasoning and Kolmogorov Complexity
Abstract Reasoning to obtain the “truth” about reality, from external data, is an important, controversial, and complicated issue in man's effort to understand nature. (Yet, today, we try to make machines do this.) There have been old useful principles, new exciting models, and intricate theories scattered in vastly different areas including philosophy of science, statistics, computer science, and psychology. We focus on inductive reasoning in correspondence with ideas of R. J. Solomonoff. While his proposals result in perfect procedures, they involve the noncomputable notion of Kolmogorov complexity. In this paper we develop the thesis that Solomonoff's method is fundamental in the sense that many other induction principles can be viewed as particular ways to obtain computable approximations to it. We demonstrate this explicitly in the cases of Gold's paradigm for inductive inference, Rissanen's minimum description length (MDL) principle, Fisher's maximum likelihood principle, and Jaynes' maximum entropy principle. We present several new theorems and derivations to this effect. We also delimit what can be learned and what cannot be learned in terms of Kolmogorov complexity, and we describe an experiment in machine learning of handwritten characters. We also give an application of Kolmogorov complexity in Valiant style learning, where we want to learn a concept probably approximately correct in feasible time and examples.
9397376
Inferring a DNA Sequence from Erroneous Copies
Abstract We suggest a novel approach for efficiently reconstructing an original DNA sequence from erroneous copies.
1558162
Monitoring Abnormal Patterns with Complex Semantics over ICU Data Streams
Monitoring abnormal patterns in data streams is an important research area for many applications. In this paper we present a new approach MAPS(Monitoring Abnormal Patterns over data Streams) to model and identify the abnormal patterns over the massive data streams. Compared with other data streams, ICU streaming data have their own features: pseudo-periodicity and polymorphism. MAPS first extracts patterns from the online arriving data streams and then normalizes them according to their pseudo-periodic semantics. Abnormal patterns will be detected if they are satisfied the predicates defined in the clinician-specifying normal patterns. At last, a real application demonstrates that MAPS is efficient and effective in several important aspects.
30409598
Underground structure monitoring with wireless sensor networks
9175149
Generation of Comprehensible Hypotheses from Gene Expression Data
Machine learning techniques have been recognized as powerful tools for the analysis of gene expression data. However, most learning techniques used in class prediction in gene expression analysis during the past years generate black-box models. Although the prediction accuracy of these models could be very well, they provide little insight into the biological facts. This paper holds the recognition that a more reasonable role for machine learning techniques is to generate hypotheses that can be verified or refined by human experts instead of making decisions for human experts. Based on this recognition, a general approach to generate comprehensible hypotheses from gene expression data is described and applied to human acute leukemias as a test case. The results demonstrate the feasibility of using machine learning techniques to help form hypotheses on the relationship between genes and certain diseases.
16584711
PeerWindow: an efficient, heterogeneous, and autonomic node collection protocol
Nodes in peer-to-peer systems need to know the information about others to optimize neighbor selection, resource exchanging, replica placement, load balancing, query optimization, and other collaborative operations. However, how to collect this information effectively is still an open issue. In this paper, we propose a novel information collection protocol, PeerWindow, with which each node can collect a large amount of pointers to other nodes at a very low cost. Compared to existing protocols, PeerWindow is 1) efficient, the cost of collecting 1,000 pointers being less than 1 kbps in a common system environment, 2) heterogeneous, nodes with different capacities collecting different amounts of information, and 3) autonomic, nodes determining their bandwidth cost for node collection by themselves and adjusting it dynamically. PeerWindow can be used in many existing peer-to-peer systems and has tremendous potential for future expansions.
206858561
Hamiltonian Cycles in Almost Claw-Free Graphs
Abstract. Some known results on claw-free graphs are generalized to the larger class of almost claw-free graphs. In this paper, we prove the following two results and conjecture that every 5-connected almost claw-free graph is hamiltonian. (1). Every 2-connected almost claw-free graph G∉J on n≤ 4 δ vertices is hamiltonian, where J is the set of all graphs defined as follows: any graph G in J can be decomposed into three disjoint connected subgraphs G1, G2 and G3 such that EG(Gi, Gj) = {ui, uj, vivj} for i≠j and i,j = 1, 2, 3 (where ui≠vi∈V(Gi) for i = 1, 2, 3). Moreover the bound 4δ is best possible, thereby fully generalizing several previous results. (2). Every 3-connected almost claw-free graph on at most 5δ−5 vertices is hamiltonian, hereby fully generalizing the corresponding result on claw-free graphs.