15157401
Improving parallel data transfer times using predicted variances in shared networks
It is increasingly common to use multiple distributed storage systems as a single data store within which large datasets may be replicated. Thus, we face the problem of how to access replicated data efficiently. Multiple-source parallel transfers can reduce access times by transferring data from several replicas in parallel. However, we then face the problem of deciding which data to fetch from which replicas. We propose a Tuned Conservative scheduling technique that uses predicted means and variances for network performance to make data selection decisions. This stochastic scheduling technique adjusts the amount of data fetched on a link according to not only the link performance but the expected variance in that performance. We incorporate our technique into the striped GridFTP server from the Globus Toolkit, and demonstrate that the technique can produce data transfer times that are significantly faster and less variable than those of other techniques.
10496869
Conservative Scheduling: Using Predicted Variance to Improve Scheduling Decisions in Dynamic Environments
In heterogeneous and dynamic environments, efficient execution of parallel computations can require mappings of tasks to processors whose performance is both irregular (because of heterogeneity) and time-varying (because of dynamicity). While adaptive domain decomposition techniques have been used to address heterogeneous resource capabilities, temporal variations in those capabilities have seldom been considered. We propose a conservative scheduling policy that uses information about expected future variance in resource capabilities to produce more efficient data mapping decisions. We first present techniques, based on time series predictors that we developed in previous work, for predicting CPU load at some future time point, average CPU load for some future time interval, and variation of CPU load over some future time interval. We then present a family of stochastic scheduling algorithms that exploit such predictions of future availability and variability when making data mapping decisions. Finally, we describe experiments in which we apply our techniques to an astrophysics application. The results of these experiments demonstrate that conservative scheduling can produce execution times that are both significantly faster and less variable than other techniques.
2104066
Design and evaluation of a resource selection framework for Grid applications
While distributed, heterogeneous collections of computers ("Grids") can in principle be used as a computing platform, in practice the problems of first discovering and then organizing resources to meet application requirements are difficult. We present a general-purpose resource selection framework that addresses these problems by defining a resource selection service for locating Grid resources that match application requirements. At the heart of this framework is a simple, but powerful, declarative language based on a technique called set matching, which extends the Condor matchmaking framework to support both single-resource and multiple-resource selection. This framework also provides an open interface for loading application-specific mapping modules to personalize the resource selector. We present results obtained when this framework is applied in the context of a computational astrophysics application, Cactus. These results demonstrate the effectiveness of our technique.
26409739
Guest Editors' Introduction
5761634
High-performance operating system controlled memory compression
This article describes a new software-based on-line memory compression algorithm for embedded systems and presents a method of adaptively managing the uncompressed and compressed memory regions during application execution. The primary goal of this work is to save memory in disk-less embedded systems, resulting in greater functionality, smaller size, and lower overall cost, without modifying applications or hardware. In comparison with algorithms that are commonly used in on-line memory compression, our new algorithm has a comparable compression ratio but is twice as fast. The adaptive memory management scheme effectively responds to the predicted needs of applications and prevents on-line memory compression deadlock, permitting reliable and efficient compression for a wide range of applications. We have evaluated our technique on an embedded portable device and have found that the memory available to applications can be increased by 150%, allowing the execution of applications with larger working data sets, or allowing existing applications to run with less physical memory
3393163
Dynamic Integration Strategy for Mediation Framework
The trend in the information area is to provide integrated access to diverse and dynamic information sources. We present a modeling and architectural solution to the problem of data integration from heterogeneous data sources based on the use of wrappers and mediators. The mediator supports integrated query processing by the semantic mapping between the global schema type and the local schema types. However, mediation poses extensive security problems. Protecting proprietary data from unauthorized access is recognized as one of the most significant barriers to the mediation systems. Our approach benefits in both flexible strategy of data integration based on semantic mappings and dynamic access control in mediation systems.
6650910
A predictive, decentralized load balancing approach
The growth of load balancing system raises the issue of scalability, and decentralized load balancing architecture has been proposed to address this issue. In this paper, we investigate how a load balancing architecture can be built on decentralized policies based on CORBA and enhanced by predictive algorithm. The L/sub 2/E predictive filtering model was used to supply workstations with robust cluster load information, which allows them to make more accurate independent allocation decisions. Experimental results showed that our decentralized load balancing approach was able to suppress thrashing and oscillations compared to other load monitoring and prediction techniques, and it was able to achieve a highly balanced system than Sun Grid Engine.
3410538
A Flexible Context-Aware Authorization Framework for Mediation Systems
Security is a critical concern for mediator-based data integration among heterogeneous data sources. This paper provides a modeling and architectural solution to the problem of mediation security that addresses the security challenges including context-awareness, semantic heterogeneity, and multiple security policy specification. A generic, extensible modeling method for the security policies in mediation systems is presented. A series of authorization constraints are identified based on the relationship on the different security components in the mediation systems. Moreover, we enforce the flexible access control to mediation systems while providing uniform access for heterogeneous data sources.