11499161
Minimal replication cost for availability
Today, the utility of many replicated Internet services is limited by availability rather than raw performance. To better understand the effects of replica placement on availability, we propose the problem of minimal replication cost for availability. Let replication cost be the cost associated with replica deployment, dynamic replica creation and teardown at n candidate locations. Given client access patterns, replica failure patterns, network partition patterns, a required consistency level and a target level of availability, the minimal replication cost is the lower bound on a system's replication cost. Solving this problem also answers the dual question of optimal availability given a constraint on replication cost.We design the first algorithm we are aware of to solve the problem, through reduction to integer linear programming and enumeration of pruned serialization orders. Using practical faultloads and workloads, we demonstrate that the exponential complexity of our algorithm is tractable for practical problems with hundreds of candidate locations. The lower bound computed by our algorithm is tight, but the tightness can be sacrificed by a proposed optimization for large problems. We also show that with low replica creation and teardown costs, the bound is close to tight in practical problems even with the optimization.
26402360
Signed quorum systems
With n servers that independently fail with probability of p < 0.5, it is well known that the majority quorum system achieves the best availability among all quorum systems. However, even this optimal construction requires (n+1)/2 functioning servers out of n. Furthermore, the number of probes needed to acquire a quorum is also lower bounded by (n+1)/2.Motivated by the need for a highly available and low probe complexity quorum system in the Internet, this paper proposes signed quorum systems (SQS) that can be available as long as any O(1) servers are available, and simultaneously have O(1) probe complexity. SQS provides probabilistic intersection guarantees and exploits the property of independent mismatches in today’s Internet. Such key property has been validated previously under multiple Internet measurement traces. This paper then extensively studies the availability, probe complexity, and load of SQS, derives lower bounds for all three metrics, and constructs matching upper bounds. We show that in addition to the qualitatively superior availability and probe complexity, SQS also decouples availability from load and probe complexity, so that optimal availability can be achieved under most probe complexity and load values.
13464702
Building replicated Internet services using TACT: a toolkit for tunable availability and consistency tradeoffs
An ultimate goal for modern Internet services is the development of scalable, high-performance, highly-available and fault-tolerant systems. Replication is an important approach to achieve this goal. However replication introduces the issue of consistency among replicas, which is further complicated by network partitions. Generally, higher consistency levels result in lower system availability in the presence of network partitions. Thus, there is a fundamental tradeoff between consistency and availability in building replicated Internet services. We argue that Internet services can benefit from dynamically choosing availability/consistency tradeoffs. With three consistency metrics, Unseen Writes, Uncommitted Writes and Staleness, we show how consistency can be meaningfully quantified for many Internet services. We present the design of the TACT (Tunable Availability and Consistency Tradeoffs) toolkit that allows Internet services to flexibly and dynamically choose their own availability/consistency, tradeoffs, enabling differentiated availability/consistency quality of service. Further TACT makes it possible for Internet services to dynamically trade consistency for performance.
8892885
The costs and limits of availability for replicated services
As raw system and network performance continues to improve at exponential rates, the utility of many services is increasingly limited by availability rather than performance. A key approach to improving availability involves replicating the service across multiple, wide-area sites. However, replication introduces well-known tradeoffs between service consistency and availability. Thus, this paper explores the benefits of dynamically trading consistency for availability using a continuous consistency model. In this model, applications specify a maximum deviation from strong consistency on a per-replica basis. In this paper, we: i) evaluate availability of a prototype replication system running across the Internet as a function of consistency level, consistency protocol, and failure characteristics, ii) demonstrate that simple optimizations to existing consistency protocols result in significant availability improvements (more than an order of magnitude in some scenarios), iii) use our experience with these optimizations to prove tight upper bounds on the availability of services, and iv) show that maximizing availability typically entails remaining as close to strong consistency as possible during times of good connectivity, resulting in a communication versus availability trade-off.
5638908
Invalidation Clues for Database Scalability Services
For their scalability needs, data-intensive Web applications can use a database scalability service (DBSS), which caches applications' query results and answers queries on their behalf. One way for applications to address their security/privacy concerns when using a DBSS is to encrypt all data that passes through the DBSS. Doing so, however, causes the DBSS to invalidate large regions of its cache when data updates occur. To invalidate more precisely, the DBSS needs help in order to know which results to invalidate; such help inevitably reveals some properties about the data. In this paper, we present invalidation clues, a general technique that enables applications to reveal little data to the DBSS, yet limit the number of unnecessary invalidations. Compared with previous approaches, invalidation clues provide applications significantly improved tradeoffs between security/privacy and scalability. Our experiments using three Web application benchmarks, on a prototype DBSS we have built, confirm that invalidation clues are indeed a low-overhead, effective, and general technique for applications to balance their privacy and scalability needs.
7547507
Efficient Numerical Error Bounding for Replicated Network Services
The goal of this work is to support replicated network services that accept updates to numerical records from multiple wide-area locations. Given the high overhead of maintaining strong consistency, many replicated services can tolerate divergence of their shared data, as long as the numerical error is bounded. Target distributed services include replicated stock quotes services, online auctions, distributed sensor systems, wide-area resource accounting and load balancing for replicated servers. We present two algorithms to e ciently bound absolute error using only local information. Split-Weight AE separately bounds increases and decreases, while Compound-Weight AE bounds them together. The two algorithms can be combined to provide good performance and low space overhead. Our Inductive RE algorithm transforms relative error to absolute error solely based on local knowledge, taking advantage of the fact that the divergence was properly bounded prior to each invocation of the algorithm. We also discuss two optimizations that reduce the space and computational overheads in the algorithms. This work is supported in part by the National Science Foundation (EIA-99772879). Vahdat is also supported by an NSF CAREER award (CCR-9984328). Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, requires a fee and/or special permission from the Endowment. Proceedings of the 26th VLDB Conference, Cairo, Egypt, 2000.
15030041
Adaptive reduction parallelization techniques
In this paper, we propose to adapt parallelizing transformations, more specifically, reduction parallelizations, to the actual reference pattern executed by a loop, i.e., to the particular input data and dynamic phase of a program. More precisely we will show how, after validating a reduction at run-time (when this is not possible at compile time) we can dynamically characterize its reference pattern and choose the most appropriate method for parallelizing it. For this purpose, we develop a library of parallel reduction algorithms, including both previously known and novel schemes, which includes algorithms specialized for different classes of access behavior. In particular, each algorithm in our library has identified strengths related to specific reference pattern characteristics, which are matched, at run-time, with measured characteristics of the actual reference pattern. The matching of algorithm to reference pattern is performed using a decision-tree based selection scheme. The contribution of this work consists in new optimizations for reduction parallelization and in the introduction of a new approach to the optimization of irregular applications: Characteristic based customization.
7168030
Minimal skew clock embedding considering time variant temperature gradient
The existing temperature-aware clock embedding assumes a time-invariant temperature gradient. However, it is not solved how to find the worst-case temperature gradient leading to the worst case skew. In this paper, we develop a PErturbation based Clock Optimization(PECO) considering the time-variant temperature gradient. For a given clock topology, we minimizethe worst case skew without asking for the worst case temperature map. We decide the merging point level by level based on the sensitivity of the skew with respect to the change of merging point. Such sensitivity is calculated using a parameterized model, which is compressed by a singular-value-decomposition (SVD) and K-meansbased clustering considering the temperature correlation. The experimental results show that our algorithm reduces worst-case skew by up to 5X compared to the existing zero skew based ZST/DME method with small (up to 1%) wirelength overhead.
12258821
Graph Matching Using Manifold Embedding
This paper describes how graph-spectral methods can be used to transform the node correspondence problem into one of point-set alignment. We commence by using a heat kernel analysis to compute geodesic distances between nodes in the graphs. With geodesic distances to hand, we use the ISOMAP algorithm to embed the nodes of a graph in a low-dimensional Euclidean space. With the nodes in the graph transformed to points in a metric space, we can recast the problem of graph-matching into that of aligning the points. Here we use a variant of the Scott and Longuet-Higgins algorithm to find point correspondences. We experiment with the resulting algorithm on a number of real-world problems.
39372956
An Inequality on Connected Domination Parameters
46532464
Lagrangian dynamics and analysis of a hybrid linkage system