5775560
Hierarchical Bloom filter arrays (HBA): a novel, scalable metadata management system for large cluster-based storage
An efficient and distributed scheme for file mapping or file lookup scheme is critical in decentralizing metadata management within a group of metadata servers. This work presents a technique called HBA (hierarchical Bloom filter arrays) to map file names to the servers holding their metadata. Two levels of probabilistic arrays, i.e., Bloom filter arrays, with different accuracies are used on each metadata server. One array, with lower accuracy and representing the distribution of the entire metadata, trades accuracy for significantly reduced memory overhead, while the other array, with higher accuracy, caches partial distribution information and exploits the temporal locality of file access patterns. Extensive trace-driven simulations have shown our HBA design to be highly effective and efficient in improving performance and scalability of file systems in clusters with 1,000 to 10,000 nodes (or superclusters).
536806
CEFT: A cost-effective, fault-tolerant parallel virtual file system
The vulnerability of computer nodes due to component failures is a critical issue for cluster-based file systems. This paper studies the development and deployment of mirroring in cluster-based parallel virtual file systems to provide fault tolerance and analyzes the tradeoffs between the performance and the reliability in the mirroring scheme. It presents the design and implementation of CEFT, a scalable RAID-10 style file system based on PVFS, and proposes four novel mirroring protocols depending on whether the mirroring operations are server-driven or client-driven, whether they are asynchronous or synchronous. The comparisons of their write performances, measured in a real cluster, and their reliability and availability, obtained through analytical modeling, show that these protocols strike different tradeoffs between the reliability and performance. Protocols with higher peak write performance are less reliable than those with lower peak write performance, and vice versa. A hybrid protocol is proposed to optimize this tradeoff.
20068677
A case study of parallel I/O for biological sequence search on Linux clusters
In this paper we analyze the I/O access patterns of a widely-used biological sequence search tool and implement two variations that employ parallel-I/O for data access based on PVFS (Parallel Virtual File System) and CEFT-PVFS (cost-effective fault-tolerant PVFS). Experiments show that the two variations outperform the original tool when equal or even fewer storage devices are used in the former. It is also found that although the performance of the two variations improves consistently when initially increasing the number of servers, this performance gain from parallel I/O becomes insignificant with further increase in server number. We examine the effectiveness of two read performance optimization techniques in CEFT-PVFS by using this tool as a benchmark. Performance results indicate: (1) doubling the degree of parallelism boosts the read performance to approach that of PVFS; and (2) skipping hot-spots can substantially improve the I/O performance when the load on data servers is highly imbalanced. The I/O resource contention due to the sharing of server nodes by multiple applications in a cluster has been shown to degrade the performance of the original tool and the variation based on PVFS by up to 10 and 21 folds, respectively; whereas, the variation based on CEFT-PVFS only suffered a two-fold performance degradation.
10945986
Improving the performance of I/O-intensive applications on clusters of workstations
Load balancing in a workstation-based cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies can cause system performance to decrease substantially. In this paper, two I/O-aware load-balancing schemes, referred to as IOCM and WAL-PM, are presented to improve the overall performance of a cluster system with a general and practical workload including I/O activities. The proposed schemes dynamically detect I/O load imbalance of nodes in a cluster, and determine whether to migrate some I/O load from overloaded nodes to other less- or under-loaded nodes. The current running jobs are eligible to be migrated in WAL-PM only if overall performance improves. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in the system, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Extensive trace-driven simulations for both synthetic and real I/O-intensive applications show that: (1) Compared with existing schemes that only consider CPU and memory, the proposed schemes improve the performance with respect to mean slowdown by up to a factor of 20; (2) When compared to the existing approaches that only consider I/O with non-preemptive job migrations, the proposed schemes achieve improvements in mean slowdown by up to a factor of 10; (3) Under CPU-memory intensive workloads, our scheme improves the performance over the existing approaches that only consider I/O by up to 47.5%.
16064075
Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system
Load balancing in a cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may cause the system performance to decrease substantially. To solve this problem, a new I/O-aware load-balancing scheme with preemptive job migration is presented to sustain the high performance of a cluster with a diverse set of workload conditions. The proposed scheme dynamically detects I/O load imbalance on nodes of a cluster, and determines whether to preempt some running jobs on overloaded nodes and migrate them to other less- or under-loaded nodes. Besides balancing I/O load, the scheme takes into account both CPU and memory load sharing in clusters, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation show that, compared to the existing approaches that only consider I/O with nonpreemptive job migrations, the proposed schemes achieve the improvement in mean slowdown by up to a factor of 10.
8902892
Dynamic Load Balancing for I/O- and Memory-Intensive Workload in clusters Using a Feedback Control Mechanism
One common assumption of the existing models of load balancing is that the weights of resources and I/O buffer size are statically configured. Though the static configuration of these parameters performs well in a cluster where the workload can be predicted, its performance is poor in dynamic systems where the workload is unknown. In this paper, a new feedback control mechanism is proposed to improve the overall performance of a cluster with I/O-intensive and memory-intensive workload. The mechanism dynamically adjusts the resource weights as well as the I/O buffer size. Results from a trace-driven simulation show that this mechanism is effective in enhancing the performance of a number of existing load-balancing schemes.
17203024
A channel-aware adaptive control to the MAC protocol in rate adaptive wireless LANs
Due to the low efficiency of legacy IEEE 802.11 MAC protocol, it has been the main bottleneck of wireless local area networks (WLANs). In this paper, we focus on multi-rate WLANs and propose an adaptive transmission control scheme, which adapts the transmission probability according to the number of active stations, transmission rates, and channel conditions. At first, an analytical model is built in terms of the proposed scheme to explore the throughput of the network. Then, by means of the characteristics of practical system, a heuristic algorithm is developed to approach the maximum throughput on-line. Extensive numerical calculations and simulations based on NS-2 are implemented to evaluate the performance of the proposed algorithm and the impact of inaccurate estimation of parameters. The results show that our algorithm outperforms the existing adaptive algorithms in imperfect channel, and insensitive to inaccurate estimation of parameters. Copyright © 2005 John Wiley & Sons, Ltd.
32403517
Multi-resolution LOD Volume Rendering in Medicine
This paper presents a level of detail (LOD) selection algorithm for multi-resolution volume rendering using 3D texture mapping. It uses an adaptive scheme that renders the volume in a region-of-interest at a high resolution and the volume away from this region at progressively lower resolutions. The algorithm is based on several important criteria, rendering is done adaptively by selecting high-resolution cells close to a center of attention and low-resolution cells away from this area. In addition, our hierarchical level-of-detail representation guarantees consistent interpolation between different resolution levels. Experiments have been applied to a number of large medical data and produced high quality images at interactive frame rates using standard PC hardware.
42202570
Improving ASM Search Using Mixture Models for Grey-Level Profiles
The use of Active Shape Models (ASM) has been shown to be an efficient approach to image interpretation and pattern recognition. In ASM, grey-level profiles at landmarks are modelled as a Gaussian distribution. Mahalanobis distance from a sample profile to the model mean is used to locate the best position of a given landmark during ASM search. We present an improved ASM methodology, in which the profiles are modelled as a mixture of Gaussians, and the probability that a sample is from the distribution is calculated using the probability density function (pdf) of the mixture model. Both improved and original ASM methods were tested on synthetic and real data. The performance comparison demonstrates that the improved ASM method is more generic and robust than the original approach.
18332258
Simulation and visualization of knee joint contact using deformable model
We introduce a new method for simulating and visualizing femur-menisci contact in human knee joint using computer generated 3D models. Knee joint contact analysis is important for understanding or solving clinical problems. Traditional knee biomechanical analysis is mostly conducted on a cadaver or based on mathematical descriptions. The lack of 3D graphical visualization makes it difficult to understand the analysis and integrate it into virtual medical simulation systems. Our method addresses this problem by using patient specific knee models and motion data to obtain the contact area information. Our method employs a new physics-based algorithm for menisci deformation and can calculate and visualize knee-menisci contact areas during gait cycles.
2274256
Overlay Networks with Linear Capacity Constraints
Overlay networks are virtual networks residing over the IP network; consequently, overlay links may share hidden lower level bottlenecks. Previous work has assumed an independent overlay model: a graph with independent link capacities. We introduce a model of overlays that incorporates correlated link capacities and linear capacity constraints (LCC) to formulate hidden shared bottlenecks; we refer to these as LCC-overlays. We define metrics to qualitatively measure overlay quality in terms of its accuracy (in representing the true network topology) and efficiency (that is, performance). Through analysis and simulations, we show that LCC-overlay is perfectly accurate and, hence, enjoys much higher efficiency than the inaccurate independent overlay. We discover that even a highly restricted LCC class-node-based LCC-yields near-optimal accuracy and significantly higher efficiency. We study two network flow problems in the context of LCC-graphs: widest path and maximum flow. We prove that Widest Path with LCC is NP-complete. We formulate Maximum Flow with LCC as a linear program and propose an efficient distributed algorithm to solve it. Based on the LCC model, we further study the problem of optimizing delay while still maintaining optimal or near-optimal bandwidth. We also outline a distributed algorithm to efficiently construct an overlay with node-based LCC.