4964831
Information-theoretic tools for mining database structure from large data sets
Data design has been characterized as a process of arriving at a design that maximizes the information content of each piece of data (or equivalently, one that minimizes redundancy). Information content (or redundancy) is measured with respect to a prescribed model for the data, a model that is often expressed as a set of constraints. In this work, we consider the problem of doing data redesign in an environment where the prescribed model is unknown or incomplete. Specifically, we consider the problem of finding structural clues in an instance of data, an instance which may contain errors, missing values, and duplicate records. We propose a set of information-theoretic tools for finding structural summaries that are useful in characterizing the information content of the data, and ultimately useful in data design. We provide algorithms for creating these summaries over large, categorical data sets. We study the use of these summaries in one specific physical design task, that of ranking functional dependencies based on their data redundancy. We show how our ranking can be used by a physical data-design tool to find good vertical decompositions of a relation (decompositions that improve the information content of the design). We present an evaluation of the approach on real data sets.
18318781
Schema Discovery
Structured data is distinguished from unstructured data by the presence of a schema describing the logical structure and semantics of the data. The schema is th e means through which we understand and query the underlying data. The schema permits the more sophi ticated structured queries that are not possible over schema-less data. Most systems assume that th e schema is predefined and is an accurate reflection of the data. This assumption is often not valid in n etworked databases that may contain data originating from many sources and may not be valid within leg acy databases where the semantics of data have evolved over time. As a result, querying and tasks that d epend on structured queries (including data integration and schema mapping) may not be effective. I n this paper, we consider the problem of discovering schemas from data. We focus on discovering prop erties of data that can be exploited in querying and transforming data. Finally, we very briefly con sider the suitability of mining approaches to the task of schema discovery.
12403884
Preserving mapping consistency under schema changes
Abstract.In dynamic environments like the Web, data sources may change not only their data but also their schemas, their semantics, and their query capabilities. When a mapping is left inconsistent by a schema change, it has to be detected and updated. We present a novel framework and a tool (ToMAS) for automatically adapting (rewriting) mappings as schemas evolve. Our approach considers not only local changes to a schema but also changes that may affect and transform many components of a schema. Our algorithm detects mappings affected by structural or constraint changes and generates all the rewritings that are consistent with the semantics of the changed schemas. Our approach explicitly models mapping choices made by a user and maintains these choices, whenever possible, as the schemas and mappings evolve. When there is more than one candidate rewriting, the algorithm may rank them based on how close they are to the semantics of the existing mappings.
914575
Building a speech interface to a medical diagnostic system
A description is given of the design of an interface for QMR-DT, an evolving decision-theoretic version of Quick Medical Reference, which performs diagnostic reasoning about diseases in internal medicine. QMR-DT encompasses the part of QMR's functionality that provides differential diagnoses for a set of patient characteristics, but it uses a different algorithm to compute diagnoses. The work described includes two programs that integrate off-the-shelf speech technology with programs that manipulate medical terminology. The term identifier program uses an isolated-word, speaker-dependent speech product to provide an interface for entering medical findings into QMR-DT. Frame browser is an auxiliary program used by the developers of term identifier to examine frame structures. Frame browser was also used to experiment with a continuous-speech system.<<ETX>>
8719734
Association rules over interval data
We consider the problem of mining association rules over interval data (that is, ordered data for which the separation between data points has meaning). We show that the measures of what rules are most important (also called rule interest) that are used for mining nominal and ordinal data do not capture the semantics of interval data. In the presence of interval data, support and confidence are no longer intuitive measures of the interest of a rule. We propose a new definition of interest for association rules that takes into account the semantics of interval data. We developed an algorithm for mining association rules under the new definition and overview our experience using the algorithm on large real-life datasets.
641485
Mapping XML and relational schemas with Clio
Merging and coalescing data from multiple and diverse sources into different data formats continues to be an important problem in modern information systems. Schema matching (the process of matching elements of a source schema with elements of a target schema) and schema mapping (the process of creating a query that maps between two disparate schemas) are at the heart of data integration systems. We demonstrate Clio, a semi-automatic schema mapping tool developed at the IBM Almaden Research Center. In this paper, we showcase Clio's mapping engine which allows mapping to and from relational and XML schemas, and takes advantage of data constraints in order to preserve data associations.
1321953
Data-driven understanding and refinement of schema mappings
At the heart of many data-intensive applications is the problem of quickly and accurately transforming data into a new form. Database researchers have long advocated the use of declarative queries for this process. Yet tools for creating, managing and understanding the complex queries necessary for data transformation are still too primitive to permit widespread adoption of this approach. We present a new framework that uses data examples as the basis for understanding and refining declarative schema mappings. We identify a small set of intuitive operators for manipulating examples. These operators permit a user to follow and refine an example by walking through a data source. We show that our operators are powerful enough both to identify a large class of schema mappings and to distinguish effectively between alternative schema mappings. These operators permit a user to quickly and intuitively build and refine complex data transformation queries that map one data source into another.