7603120
Defragmenting DHT-based Distributed File Systems
Existing DHT-based file systems use consistent hashing to assign file blocks to random machines. As a result, a user task accessing an entire file or multiple files needs to retrieve blocks from many different machines. This paper demonstrates that significant availability and performance gains can be achieved if instead, users are able to retrieve all the data needed for a given task from only a few DHT nodes. We explore the design and implications of such a "defragmented" DHT-based distributed file system, called D2, that also maintains important DHT properties like storage load balance. We show using real-world file system traces that a simple key encoding scheme is sufficient to maintain good defragmentation for most user tasks. Using both simulation and an actual 1,000 node deployment, we show that D2 increases availability by over an order of magnitude and improves user-perceived latency by 30- 100% compared to a traditional design.
8892885
The costs and limits of availability for replicated services
As raw system and network performance continues to improve at exponential rates, the utility of many services is increasingly limited by availability rather than performance. A key approach to improving availability involves replicating the service across multiple, wide-area sites. However, replication introduces well-known tradeoffs between service consistency and availability. Thus, this paper explores the benefits of dynamically trading consistency for availability using a continuous consistency model. In this model, applications specify a maximum deviation from strong consistency on a per-replica basis. In this paper, we: i) evaluate availability of a prototype replication system running across the Internet as a function of consistency level, consistency protocol, and failure characteristics, ii) demonstrate that simple optimizations to existing consistency protocols result in significant availability improvements (more than an order of magnitude in some scenarios), iii) use our experience with these optimizations to prove tight upper bounds on the availability of services, and iv) show that maximizing availability typically entails remaining as close to strong consistency as possible during times of good connectivity, resulting in a communication versus availability trade-off.
2617895
Overcoming the Majority Barrier in Large-Scale Systems
In asynchronous environments, the majority barrier prevents us from achieving consensus when more than half of the nodes fail. We argue that the majority barrier significantly limits the availability of practical systems such as peer-to-peer systems. To overcome the barrier realistically, this paper proposes a witness model that probabilistically strengthens the traditional asynchronous model for large-scale systems. The model is motivated by the prevalence of fuzzy partitions in today’s Internet, namely, reachability is far from transitive. Our assumptions in the model are strictly weaker than those in previous approaches. To show that the model is realistic, we carefully use real Internet measurements to validate our strengthening assumptions. On the other hand, we design three consensus protocols under the witness model that can tolerate n-1 failures, showing that the model is strong enough to overcome the majority barrier.
2044757
Consistent and automatic replica regeneration
Reducing management costs and improving the availability of large-scale distributed systems require automatic replica regeneration, that is, creating new replicas in response to replica failures. A major challenge to regeneration is maintaining consistency when the replica group changes. Doing so is particularly difficult across the wide area where failure detection is complicated by network congestion and node overload.In this context, this article presents Om, the first read/write peer-to-peer, wide-area storage system that achieves high availability and manageability through online automatic regeneration while still preserving consistency guarantees. We achieve these properties through the following techniques. First, by utilizing the limited view divergence property in today's Internet and by adopting the witness model, Om is able to regenerate from any single replica, rather than requiring a majority quorum, at the cost of a small (10−6 in our experiments) probability of violating consistency during each regeneration. As a result, Om can deliver high availability with a small number of replicas, while traditional designs would significantly increase the number of replicas. Next, we distinguish failure-free reconfigurations from failure-induced ones, enabling common reconfigurations to proceed with a single round of communication. Finally, we use a lease graph among the replicas and a two-phase write protocol to optimize for reads, so that reads in Om can be processed by any single replica. Experiments on PlanetLab show that consistent regeneration in Om completes in approximately 20 seconds.
11499161
Minimal replication cost for availability
Today, the utility of many replicated Internet services is limited by availability rather than raw performance. To better understand the effects of replica placement on availability, we propose the problem of minimal replication cost for availability. Let replication cost be the cost associated with replica deployment, dynamic replica creation and teardown at n candidate locations. Given client access patterns, replica failure patterns, network partition patterns, a required consistency level and a target level of availability, the minimal replication cost is the lower bound on a system's replication cost. Solving this problem also answers the dual question of optimal availability given a constraint on replication cost.We design the first algorithm we are aware of to solve the problem, through reduction to integer linear programming and enumeration of pruned serialization orders. Using practical faultloads and workloads, we demonstrate that the exponential complexity of our algorithm is tractable for practical problems with hundreds of candidate locations. The lower bound computed by our algorithm is tight, but the tightness can be sacrificed by a proposed optimization for large problems. We also show that with low replica creation and teardown costs, the bound is close to tight in practical problems even with the optimization.
52804588
Cryptanalysis of the Full HAVAL with 4 and 5 Passes
HAVAL is a cryptographic hash function with variable digest size proposed by Zheng, Pieprzyk and Seberry in 1992. It has three variants, 3-, 4-, and 5-pass HAVAL. Previous results on HAVAL suggested only practical collision attacks for 3-pass HAVAL. In this paper, we present collision attacks for 4 and 5 pass HAVAL. For 4-pass HAVAL, we describe two practical attacks for finding 2-block collisions, one with 243 computations and the other with 2 36 computations. In addition, we show that collisions for 5-pass HAVAL can be found with about 2 123 computations, which is the first attack more efficient than the birthday attack.
10203274
Adaptation of Length in a Nonstationary Environment
In this paper, we examine the behavior of a variable length GA in a nonstationary problem environment. Results indicate that a variable length GA is better able to adapt to changes than a fixed length GA. Closer examination of the evolutionary dynamics reveals that a variable length GA can in fact take advantage of its variable length representation to exploit good quality building blocks after a change in the problem environment.
120152362
Graph matching using spectral embedding and alignment
This paper describes how graph-spectral methods can be used to transform the node correspondence problem into one of point-set alignment. We commence by using the ISOMAP algorithm to embed the nodes of a graph in a low-dimensional Euclidean space. With the nodes in the graph transformed to points in a metric space, we can recast the problem of graph matching into that of aligning the points. Here, we use a variant of the Scott and Longuet-Higgins algorithm to find point correspondences. We experiment with the resulting algorithm on a number of real-world problems.
5492949
Resource Matching and a Matchmaking Service for an Intelligent Grid
We discuss the application of matching in the area of resource discovery and resource allocation in grid computing. We present a formal definition of matchmaking, overview algorithms to evaluate different matchmaking expressions, and develop a matchmaking service for an intelligent grid environment. Keywords—Grid, Matchmaking, Ontology
12258821
Graph Matching Using Manifold Embedding
This paper describes how graph-spectral methods can be used to transform the node correspondence problem into one of point-set alignment. We commence by using a heat kernel analysis to compute geodesic distances between nodes in the graphs. With geodesic distances to hand, we use the ISOMAP algorithm to embed the nodes of a graph in a low-dimensional Euclidean space. With the nodes in the graph transformed to points in a metric space, we can recast the problem of graph-matching into that of aligning the points. Here we use a variant of the Scott and Longuet-Higgins algorithm to find point correspondences. We experiment with the resulting algorithm on a number of real-world problems.