5068634
Efficient exploitation of similar subexpressions for query processing
Complex queries often contain common or similar subexpressions, either within a single query or among multiple queries submitted as a batch. If so, query execution time can be improved by evaluating a common subexpression once and reusing the result in multiple places. However, current query optimizers do not recognize and exploit similar subexpressions, even within the same query.
 We present an efficient, scalable, and principled solution to this long-standing optimization problem. We introduce a light-weight and effective mechanism to detect potential sharing opportunities among expressions. Candidate covering subexpressions are constructed and optimization is resumed to determine which, if any, such subexpressions to include in the final query plan. The chosen subexpression(s) are computed only once and the results are reused to answer other parts of queries. Our solution automatically applies to optimization of query batches, nested queries, and maintenance of multiple materialized views. It is the first comprehensive solution covering all aspects of the problem: detection, construction, and cost-based optimization. Experiments on Microsoft SQL Server show significant performance improvements with minimal overhead.
10635154
Buffering Accesses to Memory-Resident Index Structures
Recent studies have shown that cache-conscious indexes outperform conventional main memory indexes. Cache-conscious indexes focus on better utilization of each cache line for improving search performance of a single lookup. None has exploited cache spatial and temporal locality between consecutive lookups. We show that conventional indexes, even "cache-conscious" ones, suffer from significant cache thrashing between accesses. Such thrashing can impact the performance of applications such as stream processing and query operations such as index-nested-loops join. 
 
We propose techniques to buffer accesses to memory-resident tree-structured indexes to avoid cache thrashing. We study several alternative designs of the buffering technique, including whether to use fixed-size or variable-sized buffers, whether to buffer at each tree level or only at some of the levels, how to support bulk access while there are concurrent updates happening to the index, and how to preserve the order of the incoming lookups in the output results. Our methods improve cache performance for both cache-conscious and conventional index structures. Our experiments show that buffering techniques enable a probe throughput that is two to three times higher than traditional methods.
6466084
Architecture Sensitive Database Design: Examples from the Columbia Group
In this article, we discuss how different aspects of modern CPU architecture can be effectively used by database systems. While we shall try to cover most important aspects of modern architectures, we shall not aim to provide an exhaustive survey of the literature. Rather, we will use examples from our own recent work to illustrate the principles involved. While there is related work on using other components of modern machines such as graphics cards and intelligent disk controllers for database work, we limit our discussion here to just a single CPU and the memory hierarchy (excluding disks). In recent years the primary focus of database performance research has fundamentally shifted from disks to main memory. Systems with a very large main memory allow a database’s working set to fit in RAM. At the same time processors have continued to double in performance roughly every eighteen months in line with “Moore’s Law,” while memory speed improvements have been marginal. The CPU now spends many hundreds of cycles waiting for data to arrive from main memory, and database workloads are often memory-bound. Because of this bottleneck, database research at Columbia and elsewhere has focused on effectively using the cache hierarchy to improve database performance. Changes in processor technology have also influenced database research. Modern processors feature a very long pipeline, which helps to provide better performance. It also results in a higher branch misprediction penalty, because the pipeline must be flushed and restarted when a branch is mispredicted. Modern CPUs provide the database programmer with a rich instruction set. Using specialized instructions, such as SIMD instructions, database performance can be improved by allowing a degree of parallelism and eliminating some conditional branch instructions, thus reducing the impact of branch mispredictions.
17064783
Implementing database operations using SIMD instructions
Modern CPUs have instructions that allow basic operations to be performed on several data elements in parallel. These instructions are called SIMD instructions, since they apply a single instruction to multiple data elements. SIMD technology was initially built into commodity processors in order to accelerate the performance of multimedia applications. SIMD instructions provide new opportunities for database engine design and implementation. We study various kinds of operations in a database context, and show how the inner loop of the operations can be accelerated using SIMD instructions. The use of SIMD instructions has two immediate performance benefits: It allows a degree of parallelism, so that many operands can be processed at once. It also often leads to the elimination of conditional branch instructions, reducing branch mispredictions.We consider the most important database operations, including sequential scans, aggregation, index operations, and joins. We present techniques for implementing these using SIMD instructions. We show that there are significant benefits in redesigning traditional query processing algorithms so that they can make better use of SIMD technology. Our study shows that using a SIMD parallelism of four, the CPU time for the new algorithms is from 10% to more than four times less than for the traditional algorithms. Superlinear speedups are obtained as a result of the elimination of branch misprediction effects.
18692556
Transparent mid-tier database caching in SQL server
Many applications today run in a multi-tier environment with browser-based clients, mid-tier (application) servers and a backend database server. Mid-tier database caching attempts to improve system throughput and scalability by offloading part of the database workload to intermediate database servers that partially replicate data from the backend server. The fact that some queries are offloaded to an intermediate server should be completely transparent to applications one of the key distinctions between caching and replication. MTCache is a prototype mid-tier database caching solution for SQL Server that achieves this transparency. It builds on SQL Server's support for materialized views, distributed queries and replication. This paper describes MTCache and reports experimental results on the TPC-W benchmark. The experiments show that a significant part of the query workload can be offloaded to cache servers, resulting in greatly improved scale-out on the read-dominated workloads of the benchmark. Replication overhead was small with an average replication delay of less than two seconds.
626554
Buffering databse operations for enhanced instruction cache performance
As more and more query processing work can be done in main memory access is becoming a significant cost component of database operations. Recent database research has shown that most of the memory stalls are due to second-level cache data misses and first-level instruction cache misses. While a lot of research has focused on reducing the data cache misses, relatively little research has been done on improving the instruction cache performance of database systems.We first answer the question "Why does a database system incur so many instruction cache misses?" We demonstrate that current demand-pull pipelined query execution engines suffer from significant instruction cache thrashing between different operators. We propose techniques to buffer database operations during query execution to avoid instruction cache thrashing. We implement a new light-weight "buffer" operator and study various factors which may affect the cache performance. We also introduce a plan refinement algorithm that considers the query plan and decides whether it is beneficial to add additional "buffer" operators and where to put them. The benefit is mainly from better instruction locality and better hardware branch prediction. Our techniques can be easily integrated into current database systems without significant changes. Our experiments in a memory-resident PostgreSQL database system show that buffering techniques can reduce the number of instruction cache misses by up to 80% and improve query performance by up to 15%.
3004014
Intermediary non-repudiation protocols
In commercial transactions, an intermediary might be involved to help transacting parties to conduct their business. Nevertheless, the intermediary may not be fully trusted. We introduce the concept of intermediary (or agent) in a non-repudiation protocol, define the aims of intermediary non-repudiation protocols, and analyze their security requirements. We present a simple scenario with only one recipient, followed by a more complicated framework where multiple recipients are involved and collusion between them is possible.
15147736
Exploiting medium access diversity in rate adaptive wireless LANs
Recent years have seen the growing popularity of multi-rate wireless network devices (e.g., 802.11a cards) that can exploit variations in channel conditions and improve overall network throughput. Concurrently, rate adaptation schemes have been developed that selectively increase data transmissions on a link when it offers good channel quality. In this paper, we propose a Medium Access Diversity (MAD) scheme that leverages the benefits of rate adaptation schemes by aggressively exploiting multiuser diversity. The basic mechanism of MAD is to obtain instantaneous channel condition information from multiple receivers and selectively transmit data to a receiver that improves the overall throughput of the network, while maintaining temporal fairness among multiple data flows. We identify and address the challenges in the design and implementation of MAD's three phases: channel probing, data transmission, and receiver scheduling. We also use analytical models to examine the tradeoff between network performance improvement and overhead of channel probing, and derive an asymptotic performance bound for the receiver scheduling algorithms used by MAD. Results from the analysis and the extensive simulations demonstrate that, on average, MAD can improve the overall throughput of IEEE 802.11 wireless LANs by 50% as compared with the best existing rate adaptation scheme.
2288037
An Overlay Subscription Network for Live Internet TV Broadcast
We propose a framework, called overlay subscription network (OSN), for live Internet TV broadcast, where a subscriber can choose to watch at any time. This framework allows the source server to incrementally build a topology graph that contains the network connections not only from the server to each subscriber, but also among the subscribers themselves. With such a topology graph in place, we consider efficient overlay multicast for scalable OSN services. We first show that idling nodes, which do not receive video data for their own playback, can actually be used for data forwarding to significantly reduce the cost of overlay multicast. In light of this observation, we then propose a novel overlay multicast technique that distinguishes itself from existing schemes with these three aspects. First, the proposed technique is centered on the topology graph and can take advantage of the actual network connections among the subscribing nodes. Second, the new scheme is able to find and incorporate appropriate idling nodes in multicast to reduce network traffic. Third, with our approach, a node can be used in multiple multicast trees for data forwarding to improve the overall system performance. We evaluate the performance of the proposed technique through simulation. Our extensive studies show that the proposed framework has the potential to enable the Internet, a vehicle up to date mainly for transferring text and image data, for large-scale and cost-effective TV broadcast
11738299
Optimizing parallel execution of detailed wireless network simulation
With parallel and discrete event simulation (PDES) techniques, the runtime performance of detailed wireless network simulation can be improved significantly without compromising fidelity of the simulation results. However, modelling characteristics of wireless communications such as signal propagation and interference may severely hinder the potential speedup yielded by PDES. This paper proposes various optimization techniques to address three major concerns in achieving efficient parallel execution of wireless network simulation: i.e., (1) reducing communication and computation overhead of simulating signal propagation across multiple logical processes; (2) reducing synchronization overhead among logical processes; (3) minimizing event scheduling overhead within individual logical processes. These techniques have been implemented in a parallel version of GloMoSim and QualNet. The experimental results with mobile ad hoc networking scenarios demonstrate that the proposed optimization techniques can improve the performance of parallel wireless network simulation by up to an order of magnitude.
1909548
An intensive survey of fair non-repudiation protocols
With the phenomenal growth of the Internet and open networks in general, security services, such as non-repudiation, become crucial to many applications. Non-repudiation services must ensure that when Alice sends some information to Bob over a network, neither Alice nor Bob can deny having participated in a part or the whole of this communication. Therefore a fair non-repudiation protocol has to generate non-repudiation of origin evidences intended to Bob, and non-repudiation of receipt evidences destined to Alice. In this paper, we clearly define the properties a fair non-repudiation protocol must respect, and give a survey of the most important non-repudiation protocols without and with trusted third party (TTP). For the later ones we discuss the evolution of the TTP's involvement and, between others, describe the most recent protocol using a transparent TTP. We also discuss some ad-hoc problems related to the management of non-repudiation evidences.