2044757
Consistent and automatic replica regeneration
Reducing management costs and improving the availability of large-scale distributed systems require automatic replica regeneration, that is, creating new replicas in response to replica failures. A major challenge to regeneration is maintaining consistency when the replica group changes. Doing so is particularly difficult across the wide area where failure detection is complicated by network congestion and node overload.In this context, this article presents Om, the first read/write peer-to-peer, wide-area storage system that achieves high availability and manageability through online automatic regeneration while still preserving consistency guarantees. We achieve these properties through the following techniques. First, by utilizing the limited view divergence property in today's Internet and by adopting the witness model, Om is able to regenerate from any single replica, rather than requiring a majority quorum, at the cost of a small (10−6 in our experiments) probability of violating consistency during each regeneration. As a result, Om can deliver high availability with a small number of replicas, while traditional designs would significantly increase the number of replicas. Next, we distinguish failure-free reconfigurations from failure-induced ones, enabling common reconfigurations to proceed with a single round of communication. Finally, we use a lease graph among the replicas and a two-phase write protocol to optimize for reads, so that reads in Om can be processed by any single replica. Experiments on PlanetLab show that consistent regeneration in Om completes in approximately 20 seconds.
13464702
Building replicated Internet services using TACT: a toolkit for tunable availability and consistency tradeoffs
An ultimate goal for modern Internet services is the development of scalable, high-performance, highly-available and fault-tolerant systems. Replication is an important approach to achieve this goal. However replication introduces the issue of consistency among replicas, which is further complicated by network partitions. Generally, higher consistency levels result in lower system availability in the presence of network partitions. Thus, there is a fundamental tradeoff between consistency and availability in building replicated Internet services. We argue that Internet services can benefit from dynamically choosing availability/consistency tradeoffs. With three consistency metrics, Unseen Writes, Uncommitted Writes and Staleness, we show how consistency can be meaningfully quantified for many Internet services. We present the design of the TACT (Tunable Availability and Consistency Tradeoffs) toolkit that allows Internet services to flexibly and dynamically choose their own availability/consistency, tradeoffs, enabling differentiated availability/consistency quality of service. Further TACT makes it possible for Internet services to dynamically trade consistency for performance.
208060001
The costs and limits of availability for replicated services
As raw system performance continues to improve at exponential rates, the utility of many services is increasingly limited by availability rather than performance. A key approach to improving availability involves replicating the service across multiple, wide-area sites. However, replication introduces well-known trade-offs between service consistency and availability. Thus, this article explores the benefits of dynamically trading consistency for availability using a continuous consistency model. In this model, applications specify a maximum deviation from strong consistency on a per-replica basis. In this article, we: i) evaluate the availability of a prototype replication system running across the Internet as a function of consistency level, consistency protocol, and failure characteristics, ii) demonstrate that simple optimizations to existing consistency protocols result in significant availability improvements (more than an order of magnitude in some scenarios), iii) use our experience with these optimizations to prove tight upper bound on the availability of services, and iv) show that maximizing availability typically entails remaining as close to strong consistency as possible during times of good connectivity, resulting in a communication versus availability trade-off.
7547507
Efficient Numerical Error Bounding for Replicated Network Services
The goal of this work is to support replicated network services that accept updates to numerical records from multiple wide-area locations. Given the high overhead of maintaining strong consistency, many replicated services can tolerate divergence of their shared data, as long as the numerical error is bounded. Target distributed services include replicated stock quotes services, online auctions, distributed sensor systems, wide-area resource accounting and load balancing for replicated servers. We present two algorithms to e ciently bound absolute error using only local information. Split-Weight AE separately bounds increases and decreases, while Compound-Weight AE bounds them together. The two algorithms can be combined to provide good performance and low space overhead. Our Inductive RE algorithm transforms relative error to absolute error solely based on local knowledge, taking advantage of the fact that the divergence was properly bounded prior to each invocation of the algorithm. We also discuss two optimizations that reduce the space and computational overheads in the algorithms. This work is supported in part by the National Science Foundation (EIA-99772879). Vahdat is also supported by an NSF CAREER award (CCR-9984328). Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the VLDB copyright notice and the title of the publication and its date appear, and notice is given that copying is by permission of the Very Large Data Base Endowment. To copy otherwise, or to republish, requires a fee and/or special permission from the Endowment. Proceedings of the 26th VLDB Conference, Cairo, Egypt, 2000.
30353816
SybilGuard: defending against sybil attacks via social networks
846192
Design and evaluation of a conit-based continuous consistency model for replicated services
The tradeoffs between consistency, performance, and availability are well understood. Traditionally, however, designers of replicated systems have been forced to choose from either strong consistency guarantees or none at all. This paper explores the semantic space between traditional strong and optimistic consistency models for replicated services. We argue that an important class of applications can tolerate relaxed consistency, but benefit from bounding the maximum rate of inconsistent access in an application-specific manner. Thus, we develop a conit-based continuous consistency model to capture the consistency spectrum using three application-independent metrics, numerical error, order error, and staleness. We then present the design and implementation of TACT, a middleware layer that enforces arbitrary consistency bounds among replicas using these metrics. We argue that the TACT consistency model can simultaneously achieve the often conflicting goals of generality and practicality by describing how a broad range of applications can express their consistency semantics using TACT and by demonstrating that application-independent algorithms can efficiently enforce target consistency levels. Finally, we show that three replicated applications running across the Internet demonstrate significant semantic and performance benefits from using our framework.
31039691
Hybrid checkpointing protocol based on selective-sender-based message logging
This paper presents a hybrid checkpointing protocol-an asynchronous checkpointing protocol using a message sending/receiving state change for reducing the overhead of failure-free operation combined with a selective sender-based message logging protocol for reducing the cascade rollback of asynchronous checkpointing protocol. The selective sender-based message logging protocol records only potential orphan messages when taking a checkpoint. And this paper presents a message dependency tree recording the inter-process message sending/receiving information on a volatile storage for reducing the search time of inter-process information during the failure recovery.
1171693
A direct LDA algorithm for high-dimensional data - with application to face recognition
However, for a task with very high dimensional data such as images, the traditional LDA algorithm encounters several difficulties. Consider face recognition for example. A low-definition face image of size 64 by 64 implies a feature space of 64 × 64 = 4096 dimensions, and therefore scatter matrices of size 4096 × 4096 = 16M. First, it is computationally challenging to handle big matrices (such as computing eigenvalues). Second, those matrices are almost always singular, as the number of training images needs to be at least 16M for them to be non-degenerate. Due to these difficulties, it is commonly believed that a direct LDA solution for such high-dimensional data is infeasible. Thus, ironically, before LDA can be used to reduce dimensionality, another procedure has to be first applied for dimensionality reduction. In face recognition, many techniques have been proposed (For a good review, see [2]). Among them, the most notable is a two-stage PCA+LDA approach [4,1]:
2884857
A genetic approach to planning in heterogeneous computing environments
Planning is an artificial intelligence problem with a wide range of real-world applications. Genetic algorithms, neural networks, and simulated annealing are heuristic search methods often used to solve complex optimization problems. In this paper, we propose a genetic approach to planning in the context of workflow management and process coordination on a heterogenous grid. We report results for two planning problems, the Towers of Hanoi and the Sliding-tile puzzle.
28810285
Workflow management and resource discovery for an intelligent grid
A computational grid provides coordinated and transparent access to computing resources for grid users. Workflow management and resource discovery are two important functions of an intelligent grid. Workflow management refers to automatic workflow creation and coordinated workflow execution, and resource discovery facilitates resource allocation and claiming. In this paper we discuss workflow management and resource discovery in an intelligent grid environment.
46532464
Lagrangian dynamics and analysis of a hybrid linkage system