970550
Realizing parallelism in database operations: insights from a massively multithreaded architecture
A new trend in processor design is increased on-chip support for multithreading in the form of both chip multiprocessors and simultaneous multithreading. Recent research in database systems has begun to explore increased thread-level parallelism made possible by these new multicore and multithreaded processors. The question of how best to use this new technology remains open, particularly as the number of cores per chip and threads per core increase. In this paper we use an existing massively multithreaded architecture, the Cray MTA-2, to explore the implications that a larger degree of on-chip multithreading may have for parallelism in database operations. We find that parallelism in database operations is easy to achieve on the MTA-2 and that, with little effort, parallelism can be made to scale linearly with the number of available processor cores.
9401708
Complex Aggregation at Multiple Granularities
Datacube queries compute simple aggregates at multiple granularities. In this paper we examine the more general and useful problem of computing a complex subquery involving multiple dependent aggregates at multiple granularities. We call such queries multi-feature cubes. An example is Broken down by all combinations of month and customer, find the fraction of the total sales in 1996 of a particular item due to suppliers supplying within 10% of the minimum price (within the group), showing all subtotals across each dimension. We classify multi-feature cubes based on the extent to which fine granularity results can be used to compute coarse granularity results; this classification includes distributive, algebraic and holistic multi-feature cubes. We provide syntactic sufficient conditions to determine when a multi-feature cube is either distributive or algebraic. This distinction is important because, as we show, existing datacube evaluation algorithms can be used to compute multi-feature cubes that are distributive or algebraic, without any increase in I/O complexity. We evaluate the CPU performance of computing multi-feature cubes using the datacube evaluation algorithm of Ross and Srivastava. Using a variety of synthetic, benchmark and real-world data sets, we demonstrate that the CPU cost of evaluating distributive multi-feature cubes is comparable to that of evaluating simple datacubes. We also show that a variety of holistic multi-feature cubes can be evaluated with a manageable overhead compared to the distributive case.
16656102
Groupwise Processing of Relational Queries
In this paper, we define and examine a particular class of queries called group queries. Group queries are natural queries in many decisionsupport applications. The main characteristic of a group query is that it can be executed in a groupby-group fashion. In other words, the underlying relation(s) can be partitioned (based on some set of attributes) into disjoint groups, and each group can be processed separately. We give a syntactic criterion to identify these queries and prove its sufficiency. We also prove the strong result that every group query has an equivalent formulation that satisfies our syntactic criterion. We describe a general evaluation technique for group queries, and demonstrate how an optimizer can determine this plan. We then consider more complex queries whose components are group queries with potentially different partitioning attributes. We give two methods to identify group query components within such a query. We also give some performance results for group queries expressed in standard SQL, comparing a commercial database system with our optimized plan on top of the same commercial system. These results indicate that there are significant potential performance improvements.
7567374
Foundations of Aggregation Constraints
We introduce a new constraint domain, aggregation constraints, which is useful in database query languages, and in constraint logic programming languages that incorporate aggregate functions. We study the fundamental problem of checking if a conjunction of aggregation constraints is solvable, and present undecidability results for many different classes of aggregation constraints. We describe a complete and minimal axiomatization of the class of aggregation constraints over finite multisets of reals, which permits a natural reduction from the class of aggregation constraints to the class of mixed integer/real, non-linear arithmetic constraints. We then present a polynomial-time algorithm that directly checks for solvability of a useful class of aggregation constraints, where the reduction-based approach does not lead to efficient checks for solvability.
38852147
On Negation in HiLog
Abstract The logic HiLog of Chen, Kifer, and Warren has a second order syntax, while its semantics is first order. HiLog programs with negative literals in the body are considered. A stable-model semantics and a well-founded semantics for this class of program are defined, and it is shown that these semantics generalize the stable-model semantics and the well-founded semantics, respectively, for range-restricted normal programs. A second order property called preservation under extensions is proposed and investigated. Preservation under extensions ensures that the semantics of a program is not changed when rules having no symbols in common with the program are appended to the program. It is shown that for normal programs, domain independence and preservation under extensions are equivalent, while for HiLog programs, preservation under extensions is strictly stronger. Range restrictedness is generalized to HiLog programs in two ways, and it is shown that range restricted HiLog programs are preserved under extensions with respect to the well-founded semantics. Conditions under which the well-founded semantics is two-valued are investigated, and the class of modularly stratified programs is generalized to HiLog. An extension of magic-set techniques to modularly stratified HiLog programs is described.
31185043
Inferring negative information from disjunctive databases
We propose criteria that any rule for inferring negative information from disjunctive databases should satisfy, and examine existing rules from this viewpoint. We then present a new inference rule, the ‘disjunctive database rule’ (DDR), and compare it to the existing rules with respect to the criteria. In particular, the DDR is equivalent to the CWA for definite databases, it infers no more negative information than the GCWA, and it interprets disjunction inclusively rather than exclusively. We generalize the DDR to a class of layered databases, describe an implementation of the DDR, ‘negation as positive failure’, and study its soundness and completeness properties.
10455339
Pollution in P2P file sharing systems
One way to combat P2P file sharing of copyrighted content is to deposit into the file sharing systems large volumes of polluted files. Without taking sides in the file sharing debate, in this paper we undertake a measurement study of the nature and magnitude of pollution in the FastTrack P2P network, currently the most popular P2P file sharing system. We develop a crawling platform which crawls the majority of the FastTrack Network's 20,000+ supernodes in less than 60 minutes. From the raw data gathered by the crawler for popular audio content, we obtain statistics on the number of unique versions and copies available in a 24-hour period. We develop an automated procedure to detect whether a given version is polluted or not, and we show that the probabilities of false positives and negatives of the detection procedure are very small. We use the data from the crawler and our pollution detection algorithm to determine the fraction of versions and fraction of copies that are polluted for several recent and old songs. We observe that pollution is pervasive for recent popular songs. We also identify and describe a number of anti-pollution mechanisms.
10446891
Periodic broadcasting with VBR-encoded video
Abstract.We consider designing near video on demand (NVOD) systems that minimize startup latency while maintaining high image quality. Recently nonuniform segmentation has been used to develop periodic broadcasting techniques for NVOD. These techniques give significant reductions in startup latency as compared with more conventional uniform segmentation. Essentially all of these schemes assume, however, that the videos are CBR-encoded. Since a CBR-encoded video has a larger average rate than an open-loop VBR encoding with the same image quality, there is potential to obtain further performance improvements by using VBR video. In this paper we develop a series of multiplexing schemes for the periodic broadcasting of VBR-encoded video that are based on smoothing, server buffering, and client prefetching. Two key but conflicting performance measures exist when using VBR video: latency and packet loss. By introducing small additional delays in our multiplexing schemes, our trace-based numerical work shows that the schemes can achieve nearly 100% link utilization with negligible packet loss. When the ratio of the CBR rate to the VBR average rate is a modest 1.8, startup latency can be reduced by a factor of four or more for common scenarios.
7927284
Hierarchical Peer-to-Peer Systems
Structured peer-to-peer (P2P) lookup services organize peers into a flat overlay network and offer distributed hash table (DHT) functionality. Data is associated with keys and each peer is responsible for a subset of the keys. In hierarchical DHTs, peers are organized into groups, and each group has its autonomous intra-group overlay network and lookup service. Groups are organized in a top-level overlay network. To find a peer that is responsible for a key, the top-level overlay first determines the group responsible for the key; the responsible group then uses its intra-group overlay to determine the specific peer that is responsible for the key. We provide a general framework and a scalable hierarchical overlay management. We study a two-tier hierarchy using Chord for the top level. Our analysis shows that by using the most reliable peers in the top level, the hierarchical design significantly reduces the expected number of hops.
18166886
On Routing in ATM Networks
Alternative routing of calls has long been acknowledged as a means of increasing call throughput and robustness in the telephone network. Given these performance gains, it is natural to consider dynamic routing of Virtual Circuits (VCs) in Asynchronous Transfer Mode (ATM) networks. Unlike telephone networks, VCs have heterogeneous bandwidth characteristics and Quality of Service requirements that the routing algorithm must take into account in order to optimize performance. In this paper we attempt to classify ATM networks according to the network characteristics that have the greatest bearing on the performance of dynamic routing algorithms; we also discuss appropriate routing algorithms for diierent classes of ATM network. The classiication is based on the degree of heterogeneity of the sources and the degree of statistical multiplexing.
15587034
Packet Scheduling Across Networks of Switches
Recent developments in computer and communication networks require scheduling decisions to be made under increasingly complex system dynamics. We model and analyze the problem of packet transmissions through an arbitrary network of buffered queues, and provide a framework for describing routing and migration. This paper introduces an intuitive geometric description of stability for these networks and describes some simple algorithms which lead to maximal throughput. We show how coordination over sequential timeslots by algorithms such as those based on a round robin can provide considerable advantages over a randomized scheme.