2826488
Computing Iceberg Queries Efficiently
Many applications compute aggregate functions (such as COUNT, SUM) over an attribute (or set of attributes) to find aggregate values above some specified threshold. We call such queries iceberg queries because the number of above-threshold results is often very small (the tip of an iceberg), relative to the large amount of input data (the iceberg). Such iceberg queries are common in many applications, including data warehousing, information-retrieval, market basket analysis in data mining, clustering and copy detection. We propose efficient algorithms to evaluate iceberg queries using very little memory and significantly fewer passes over data, as compared to current techniques that use sorting or hashing. We present an experimental case study using over three gigabytes of Web data to illustrate the savings obtained by our algorithms.
7969252
The role of theory today
The work of Hartmanis and Stearns, for which they rightly and belatedly won the Turing award, was both amazingly insightful and widely useful. It led to a long sequence of important innovation.s, such as work on cryptographic complexity, and it is a principal justification for theoretical research today. Further, theory has had triumphs outside the computational complexity area. Without wishing to be exhaustive, I could cite work on programming-language semantics that led to real advances in language design, the use of automaton-theoretic ideas in practical protocol verifiers, and many specific algorithms that have been used in practice. Yet looking back at the 1960s, I see a naive optimism that was based on too little evidence. The work on O(nlogn) sorting algorithms or Strassen’s matrix-multiplication algorithm, for example, caused us to believe that all problems would soon yield to algorithms that wene significantly more efficient than the obvious ones. LR(k ) parsing—a wonderful idea deeply rooted in language and automata theory—suggested that all aspects of compiling and other important systems software would soon yield to formalization and theoretical analysis. The theory community was not alone in its naivete. AI was similarly taken in by early successes, such as Samuels’s checker-playing program, to believe that very hard problems were in fact easy. This view has led to periodic, overly optimistic predictions of success that continue to trouble AI. Thus, despite the successes mentioned above, as the theory field grew there was a tendency to attack problems obliquely, assuming that we were contributing to an important goal. Papers would be written to address the issues raised by other papers. Then another round of work would address these papers, and so on. In the process, the real problems that spawned the original paper were forgotten, and the community began to imagine that the research embodied in the papers was the ultimate goal. (I confess to being as guilty of this behavior as anyone in the field.) Is the theory community therefore doomed to become ineffective within the broader discipline of computer science? I do not believe so for a moment. However, we need to reinvent theory for the next generation if we are to have the same impact now as a generation ago. Not everything needs to change. First, there is a core of theory, roughly the complexity theory outlined in the Hartmanis article, that needs to be pre-
209399936
Answering Queries using Templates with Binding Patterns
When integrating heterogeneous information resources, it is often the case that the source is rather limited in the kinds of queries it can answer. If a query is asked of the entire system, we have a new kind of optimization problem, in which we must try to express the given query in terms of the limited query templates that this source can answer. For the case of conjunctive queries, we show how to decide with a nondeterministic polynomial-time algorithm whether the given query can be answered. We then extend our results to allow arithmetic comparisons in the given query and in the templates.
9277235
Index selection for OLAP
On-line analytical processing (OLAP) is a recent and important application of database systems. Typically, OLAP data is presented as a multidimensional "data cube." OLAP queries are complex and can take many hours or even days to run, if executed directly on the raw data. The most common method of reducing execution time is to precompute some of the queries into summary tables (subcubes of the data cube) and then to build indexes on these summary tables. In most commercial OLAP systems today, the summary tables that are to be precomputed are picked first, followed by the selection of the appropriate indexes on them. A trial-and-error approach is used to divide the space available between the summary tables and the indexes. This two-step process can perform very poorly. Since both summary tables and indexes consume the same resource-space-their selection should be done together for the most efficient use of space. The authors give algorithms that automate the selection of summary tables and indexes. In particular, they present a family of algorithms of increasing time complexities, and prove strong performance bounds for them. The algorithms with higher complexities have better performance bounds. However, the increase in the performance bound is diminishing, and they show that an algorithm of moderate complexity can perform fairly close to the optimal.
220902
MedMaker: a mediation system based on declarative specifications
Mediators are used for integration of heterogeneous information sources. We present a system for declaratively specifying mediators. It is targeted for integration of sources with unstructured or semi-structured data and/or sources with changing schemas. We illustrate the main features of the Mediator Specification Language (MSL), show how they facilitate integration, and describe the implementation of the system that interprets the MSL specifications.
636164
Querying Semistructured Heterogeneous Information
Semistructured data has no absolute schema fixed in advance and its structure may be irregular or incomplete. Such data commonly arises in sources that do not impose a rigid structure (such as the World-Wide Web) and when data is combined from several heterogeneous sources. Data models and query languages designed for well structured data are inappropriate in such environments. Starting with a “lightweight” object model adopted for the TSIMMIS project at Stanford, in this paper we describe a query language and object repository designed specifically for semistructured data. Our language provides meaningful query results in cases where conventional models and languages do not: when some data is absent, when data does not have regular structure, when similar concepts are represented using different types, when heterogeneous sets are present, and when object structure is not fully known. This paper motivates the key concepts behind our approach, describes the language through a series of examples (a complete semantics is available in an accompanying technical report [QRS+94]), and describes the basic architecture and query processing strategy of the “lightweight” object repository we have developed.
209399085
Answering Queries Using Limited External Processors.