34458462
Learning in the Presence of Malicious Errors
In this paper an extension of the distribution-free model of learning introduced by Valiant [Comm. ACM, 27(1984), pp. 1134–1142] that allows the presence of malicious errors in the examples given to a learning algorithm is studied. Such errors are generated by an adversary with unbounded computational power and access to the entire history of the learning algorithm’s computation. Thus, a worst-case model of errors is studied.The results of this research include general methods for bounding the rate of error tolerable by any learning algorithm, efficient algorithms tolerating nontrivial rates of malicious errors, and equivalences between problems of learning with errors and standard combinatorial optimization problems.
2607374
Information distance
While Kolmogorov complexity is the accepted absolute measure of information content in an individual finite object, a similarly absolute notion is needed for the information distance between two individual objects, for example, two pictures. We give several natural definitions of a universal information metric, based on length of shortest programs for either ordinary computations or reversible (dissipationless) computations. It turns out that these definitions are equivalent up to an additive logarithmic term. We show that the information distance is a universal cognitive similarity distance. We investigate the maximal correlation of the shortest programs involved, the maximal uncorrelation of programs (a generalization of the Slepian–Wolf theorem of classical information theory), and the density properties of the discrete metric spaces induced by the information distances. A related distance measures the amount of nonreversibility of a computation. Using the physical theory of reversible computation, we give an appropriate (universal, antisymmetric, and transitive) measure of the thermodynamic work required to transform one object in another object by the most efficient process. Information distance between individual objects is needed in pattern recognition where one wants to express effective notions of “pattern similarity” or “cognitive similarity” between individual objects and in thermodynamics of computation where one wants to analyze the energy dissipation of a computation from a particular input to a particular output.
14168735
Inductive Reasoning and Kolmogorov Complexity
Abstract Reasoning to obtain the “truth” about reality, from external data, is an important, controversial, and complicated issue in man's effort to understand nature. (Yet, today, we try to make machines do this.) There have been old useful principles, new exciting models, and intricate theories scattered in vastly different areas including philosophy of science, statistics, computer science, and psychology. We focus on inductive reasoning in correspondence with ideas of R. J. Solomonoff. While his proposals result in perfect procedures, they involve the noncomputable notion of Kolmogorov complexity. In this paper we develop the thesis that Solomonoff's method is fundamental in the sense that many other induction principles can be viewed as particular ways to obtain computable approximations to it. We demonstrate this explicitly in the cases of Gold's paradigm for inductive inference, Rissanen's minimum description length (MDL) principle, Fisher's maximum likelihood principle, and Jaynes' maximum entropy principle. We present several new theorems and derivations to this effect. We also delimit what can be learned and what cannot be learned in terms of Kolmogorov complexity, and we describe an experiment in machine learning of handwritten characters. We also give an application of Kolmogorov complexity in Valiant style learning, where we want to learn a concept probably approximately correct in feasible time and examples.
871253
Kolmogorov Complexity Arguments in Combinatorics
Abstract The utility of a Kolmogorov complexity method in combinatorial theory is demonstrated by several examples.
43310555
Twist-Rotation Transformations of Binary Trees and Arithmetic Expressions
The paper studies the computational complexity and efficient algorithms for the twist?rotation transformations of binary trees, which is equivalent to the transformation of arithmetic expressions over an associative and commutative binary operation. The main results are (1) a full binary tree with n labeled leaves can be transformed into any other in at most 3nlogn+2n twist and rotation operations, (2) deciding the twist?rotation distance between two binary trees is NP-complete, and (3) the twist?rotation transformation can be approximated with ratio 6logn+4 in polynomial time for full binary trees with n uniquely labeled leaves.
553776
Linear approximation of shortest superstrings
We consider the following problem: given a collection of strings s1, ••• ,sm, find the shortest string s such that each s; appears as a substring (a consecutive block) of s. Although this problem is known to be NP-hard, a simple greedy procedure appears to do quite well and is routinely used in DNA sequencing and data compression practice, namely: repeatedly merge the pair of (distinct) strings with maximum overlap until only one string remains. Let n denote the A. Blum was supported by a National Science Foundation (NSF) Graduate Fellowship. Part of this work was done while A. Blum was visiting AT & T Bell Labs. T. Jiang was supported in part by a grant from SERB, McMaster University and NSERC Operating Grant OGP0046613. M. Li was supported in part by the NSERC Operating Grants OGP0036747 and OGP0046506. J. Tromp was supported in part by NSERC Grant OGP0036747 while the author was visiting at the University of Waterloo. Authors' present addresses: A. Blum, School of Computer Science, Carnegie-Mellon University, 5000 Forbes Avenue, Pittsburgh, PA 15213. E-mail: avrim@theory.cs.cmu.edu; T. Jiang, Department of Computer Science, McMaster University, Hamilton, Ont., Canada L8S 4Kl. E-mail: jiang<Q•maccs. mcmaster.ca; M. Li, Department of Computer Science, University of Waterloo, Waterloo, Ont., Canada N3L 30 I. E-mail: mli@watmath.waterloo.edu; J. Tromp, CWI, P.O. Box 4079, 1009 AB Amsterdam, The Netherlands. E-mail: tromp@cwi.nl; M. Yannakakis, Room 2C-319, AT&T Bell Labs, 600 Mountain Avenue, Murray Hill, NJ 07974. E-mail: mihalis@research.att.com. Permission to copy without fee all or part of this material is granted provided that the copies arc not made or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Association for Computing Machinery. To copy otherwise, or to republish, requires a fee and/or specific permission. © 1994 ACM 0004-5411/94/0700-0630$03.50 Journal of the Association for Computing Machinery, Vol. 41, No. 4, July 1994, pages 630-647. Linear Approximation of Shortest Superstrings 631 length of the optimal superstring. A common conjecture states that the above greedy procedure produces a superstring of length O(n) (in fact, 2n), yet the only previous nontrivial bound known for any polynomial-time algorithm is a recent O(n log n) result. We show that the greedy algorithm does in fact achieve a constant factor approximation, proving an upper bound of 4n. Furthermore, we present a simple modified version of the greedy algorithm that we show produces a superstring of length at most 3n. We also show the superstring problem to be MAX SNP-hard, which implies that a polynomial-time approximation scheme for this problem is unlikely.
20792301
An Efficient Algorithm for Computing Optimal Discrete Voltage Schedules
We consider the problem of job scheduling on a variable voltage processor with d discrete voltage/speed levels. We give an algorithm which constructs a minimum energy schedule for n jobs in O(dnlog n) time. Previous approaches solve this problem by first computing the optimal continuous solution in O(n3) time and then adjusting the speed to discrete levels. In our approach, the optimal discrete solution is characterized and computed directly from the inputs. We also show that O(nlog n) time is required, hence the algorithm is optimal for fixed d.
9576587
Non-Threshold based Event Detection for 3D Environment Monitoring in Sensor Networks
Event detection is a crucial task for wireless sensor network applications, especially environment monitoring. Existing approaches for event detection are mainly based on some predefined threshold values, and thus are often inaccurate and incapable of capturing complex events. For example, in coal mine monitoring scenarios, gas leakage or water osmosis can hardly be described by the overrun of specified attribute thresholds, but some complex pattern in the full-scale view of the environmental data. To address this issue, we propose a non-threshold based approach for the real 3D sensor monitoring environment. We employ energy-efficient methods to collect a time series of data maps from the sensor network and detect complex events through matching the gathered data to spatio-temporal data patterns. Finally, we conduct trace driven simulations to prove the efficacy and efficiency of this approach on detecting events of complex phenomena from real-life records.
1974621
Engineering high-performance legacy codes as CORBA components for problem-solving environments
This paper describes techniques used to leverage high-performance legacy codes as CORBA components to a distributed problem-solving environment. It first briefly introduces the software architecture adopted by the environment. Then it presents a CORBA oriented wrapper generator (COWG) which can be used to automatically wrap high-performance legacy codes as CORBA components. Two legacy codes have been wrapped with COWG. One is an MPI-based molecular dynamic simulation (MDS) code, the other is a finite element-based computational fluid dynamics (CFD) code for simulating incompressible Navier-Stokes flows. Performance comparisons between runs of the MDS CORBA component and the original MDS legacy code on a cluster of workstations and on a parallel computer are also presented. Wrapped as CORBA components, these legacy codes can be reused in a distributed computing environment. The first case shows that high-performance can be maintained with the wrapped MDS component. The second case shows that a Web user can submit a task to the wrapped CFD component through a Web page without knowing the exact implementation of the component. In this way, a user's desktop computing environment can be extended to a high-performance computing environment using a cluster of workstations or a parallel computer.
9175149
Generation of Comprehensible Hypotheses from Gene Expression Data
Machine learning techniques have been recognized as powerful tools for the analysis of gene expression data. However, most learning techniques used in class prediction in gene expression analysis during the past years generate black-box models. Although the prediction accuracy of these models could be very well, they provide little insight into the biological facts. This paper holds the recognition that a more reasonable role for machine learning techniques is to generate hypotheses that can be verified or refined by human experts instead of making decisions for human experts. Based on this recognition, a general approach to generate comprehensible hypotheses from gene expression data is described and applied to human acute leukemias as a test case. The results demonstrate the feasibility of using machine learning techniques to help form hypotheses on the relationship between genes and certain diseases.
10855228
Generating Symbolic Interpolants for Scattered Data with Normal Vectors
Algorithms to generate a triangular or a quadrilateral interpolant with G1-continuity are given in this paper for arbitrary scattered data with associated normal vectors over a prescribed triangular or quadrilateral decomposition. The interpolants are constructed with a general method to generate surfaces from moving Bézier curves under geometric constraints. With the algorithm, we may obtain interpolants in complete symbolic parametric forms, leading to a fast computation of the interpolant. A dynamic interpolation solid modelling software package DISM is implemented based on the algorithm which can be used to generate and manipulate solid objects in an interactive way.