2340130
Improved read performance in a cost-effective, fault-tolerant parallel virtual file system (CEFT-PVFS)
Due to the ever-widening performance gap between processors and disks, I/O operations tend to become the major performance bottleneck of data-intensive applications on modern clusters. If all the existing disks on the nodes of a cluster are connected together to establish high performance parallel storage systems, the cluster's overall performance can be boosted at no additional cost. CEFT-PVFS (a RAID 10 style parallel file system that extends the original PVFS), as one such system, divides the cluster nodes into two groups, stripes the data across one group in a round-robin fashion, and then duplicates the same data to the other group to provide storage service of high performance and high reliability. Previous research has shown that the system reliability is improved by a factor of more than 40 with mirroring while maintaining a comparable write performance. This paper presents another benefit of CEFT-PVFS in which the aggregate peak read performance can be improved by as much as 100% over that of the original PVFS by exploiting the increased parallelism. Additionally, when the data servers, which typically are also computational nodes in a cluster environment, are loaded in an unbalanced way by applications running in the cluster, the read performance of PVFS will be degraded significantly. On the contrary, in the CEFT-PVFS, a heavily loaded data server can be skipped and all the desired data is read from its mirroring node. Thus the performance will not be affected unless both the server node and its mirroring node are heavily loaded.
16064075
Boosting performance of I/O-intensive workload by preemptive job migrations in a cluster system
Load balancing in a cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies that focus on CPU and memory usage may cause the system performance to decrease substantially. To solve this problem, a new I/O-aware load-balancing scheme with preemptive job migration is presented to sustain the high performance of a cluster with a diverse set of workload conditions. The proposed scheme dynamically detects I/O load imbalance on nodes of a cluster, and determines whether to preempt some running jobs on overloaded nodes and migrate them to other less- or under-loaded nodes. Besides balancing I/O load, the scheme takes into account both CPU and memory load sharing in clusters, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation show that, compared to the existing approaches that only consider I/O with nonpreemptive job migrations, the proposed schemes achieve the improvement in mean slowdown by up to a factor of 10.
10945986
Improving the performance of I/O-intensive applications on clusters of workstations
Load balancing in a workstation-based cluster system has been investigated extensively, mainly focusing on the effective usage of global CPU and memory resources. However, if a significant portion of applications running in the system is I/O-intensive, traditional load balancing policies can cause system performance to decrease substantially. In this paper, two I/O-aware load-balancing schemes, referred to as IOCM and WAL-PM, are presented to improve the overall performance of a cluster system with a general and practical workload including I/O activities. The proposed schemes dynamically detect I/O load imbalance of nodes in a cluster, and determine whether to migrate some I/O load from overloaded nodes to other less- or under-loaded nodes. The current running jobs are eligible to be migrated in WAL-PM only if overall performance improves. Besides balancing I/O load, the scheme judiciously takes into account both CPU and memory load sharing in the system, thereby maintaining the same level of performance as existing schemes when I/O load is low or well balanced. Extensive trace-driven simulations for both synthetic and real I/O-intensive applications show that: (1) Compared with existing schemes that only consider CPU and memory, the proposed schemes improve the performance with respect to mean slowdown by up to a factor of 20; (2) When compared to the existing approaches that only consider I/O with non-preemptive job migrations, the proposed schemes achieve improvements in mean slowdown by up to a factor of 10; (3) Under CPU-memory intensive workloads, our scheme improves the performance over the existing approaches that only consider I/O by up to 47.5%.
7586381
Dynamic Load Balancing for I/O-Intensive Tasks on Heterogeneous Clusters
Since I/O-intensive tasks running on a heterogeneous cluster need a highly effective usage of global I/O resources, previous CPU- or memory-centric load balancing schemes suffer significant performance drop under I/O-intensive workload due to the imbalance of I/O load. To solve this problem, we develop two I/O-aware load-balancing schemes, which consider system heterogeneity and migrate more I/O-intensive tasks from a node with high I/O utilization to those with low I/O utilization. If the workload is memory-intensive in nature, the new method applies a memory-based load balancing policy to assign the tasks. Likewise, when the workload becomes CPU-intensive, our scheme leverages a CPU-based policy as an efficient means to balance the system load. In doing so, the proposed approach maintains the same level of performance as the existing schemes when I/O load is low or well balanced. Results from a trace-driven simulation study show that, when a workload is I/O-intensive, the proposed schemes improve the performance with respect to mean slowdown over the existing schemes by up to a factor of 8. In addition, the slowdowns of almost all the policies increase consistently with the system heterogeneity.
20068677
A case study of parallel I/O for biological sequence search on Linux clusters
In this paper we analyze the I/O access patterns of a widely-used biological sequence search tool and implement two variations that employ parallel-I/O for data access based on PVFS (Parallel Virtual File System) and CEFT-PVFS (cost-effective fault-tolerant PVFS). Experiments show that the two variations outperform the original tool when equal or even fewer storage devices are used in the former. It is also found that although the performance of the two variations improves consistently when initially increasing the number of servers, this performance gain from parallel I/O becomes insignificant with further increase in server number. We examine the effectiveness of two read performance optimization techniques in CEFT-PVFS by using this tool as a benchmark. Performance results indicate: (1) doubling the degree of parallelism boosts the read performance to approach that of PVFS; and (2) skipping hot-spots can substantially improve the I/O performance when the load on data servers is highly imbalanced. The I/O resource contention due to the sharing of server nodes by multiple applications in a cluster has been shown to degrade the performance of the original tool and the variation based on PVFS by up to 10 and 21 folds, respectively; whereas, the variation based on CEFT-PVFS only suffered a two-fold performance degradation.
10155852
Nexus: a novel weighted-graph-based prefetching algorithm for metadata servers in petabyte-scale storage systems
An efficient, accurate and distributed metadata-oriented prefetching scheme is critical to the overall performance in large distributed storage systems. In this paper, we present a novel weighted-graph-based prefetching technique, built on successor relationship, to gain performance benefit from prefetching specifically for clustered metadata servers, an arrangement envisioned necessary for petabyte-scale distributed storage systems. Extensive trace-driven simulations show that by adopting our new prefetching algorithm, the hit rate for metadata access on the client site can be increased by up to 13%, while the average response time of metadata operations can be reduced by up to 67%, compared with LRU and an existing state of the art prefetching algorithm.
42202570
Improving ASM Search Using Mixture Models for Grey-Level Profiles
The use of Active Shape Models (ASM) has been shown to be an efficient approach to image interpretation and pattern recognition. In ASM, grey-level profiles at landmarks are modelled as a Gaussian distribution. Mahalanobis distance from a sample profile to the model mean is used to locate the best position of a given landmark during ASM search. We present an improved ASM methodology, in which the profiles are modelled as a mixture of Gaussians, and the probability that a sample is from the distribution is calculated using the probability density function (pdf) of the mixture model. Both improved and original ASM methods were tested on synthetic and real data. The performance comparison demonstrates that the improved ASM method is more generic and robust than the original approach.
41212780
Real-time rendering of 3D medical data sets
Interactive exploration of three dimension (3D) medical data sets is required by many applications, but the huge amount of computational time and storage space needed for rendering do not allow the visualization of large medical data sets by now. In this paper we present a new algorithm for rendering large medical data sets at interactive frame rates on standard PC hardware. The input data is converted into a compressed hierarchical wavelet representation in a preprocessing step. During rendering, the wavelet representation is decompressed on-the-fly and rendered using hardware texture mapping. The level of detail used for rendering is adapted to the local frequency spectrum of the data and its position relative to the viewer. Using a prototype implementation of the algorithm we were able to perform an interactive walkthrough of large medical data sets on a single of-the-shelf PC.
20128924
Long path connectivity of regular graphs
Abstract Any pair of vertices in a 4-connected non-bipartite k -regular graph are joined by a Hamilton path or a path of length at least 3 k −6.
16266615
NeuroQL: A Domain-Specific Query Language for Neuroscience Data
In this paper, we propose a domain-specific query language called NeuroQL for the neuroscience domain. NeuroQL is designed primarily for neuroinformatics database users and aims to enable users to directly interact with neuroscience databases in their professional concepts and terms with the help of a conceptual data model. NeuroQL is DBMS independent and can be translated into traditional query language such as SQL, OQL and XQuery. It integrates some object-oriented features, and supports neuron domain-specific data types and query operators, which can dynamically evolve when the underlying database schema evolves.
2014328
A 3D reconstruction algorithm based on 3D deformable atlas
Reconstructing 3D models from 2D images is an important problem in visualization, particularly in medical and biological visualization. We present a new algorithm for 3D model reconstruction from 2D-images. In our approach, a 3D atlas is deformed to fit the feature data set in the target images. Compared with existing 3D reconstruction techniques, our approach is more efficient and easier to control. The resultant model has good inherent connectivity and smoothness. The resolution of the final model is determined by the resolution of reference model rather than the dataset. The model generates highly detailed view with a limited number of images. This algorithm has been successfully applied to medical simulation.