12672425
Support for data-intensive, variable-granularity grid applications via distributed file system virtualization - a case study of light scattering spectroscopy
A key challenge faced by large-scale, distributed applications in grid environments is efficient, seamless data management. In particular, for applications that can benefit from access to data at variable granularities, data management can pose additional programming burdens to an application developer. This work presents a case for the use of virtualized distributed file systems as a basis for data management for data-intensive, variable-granularity applications. The approach leverages on-demand transfer mechanisms of existing, de-facto network file system clients and servers that support transfers of partial data sets in an application-transparent fashion, and complement them with user-level performance and functionality enhancements such as caching and encrypted communication channels. The paper uses a nascent application from the medical imaging field (light scattering spectroscopy -LSS) as a motivation for the approach, and as a basis for evaluating its performance. Results from performance experiments that consider the 16-processor parallel execution of LSS analysis and database generation programs show that, in the presence of data locality, a virtualized wide-area distributed file system setup and configured by grid middleware can achieve performance levels close (13% overhead or less) to that of a local disk, and superior (up to 680% speedup) to nonvirtualized distributed file systems.
15313560
A user-level secure grid file system
A grid-wide distributed file system provides convenient data access interfaces that facilitate fine-grained cross-domain data sharing and collaboration. However, existing widely-adopted distributed file systems do not meet the security requirements for grid systems. This paper presents a Secure Grid File System (SGFS) which supports GSI-based authentication and access control, end-to-end message privacy, and integrity. It employs user-level virtualization of NFS to provide transparent grid data access leveraging existing, unmodified clients and servers. It supports user and application-tailored security customization per SGFS session, and leverages secure management services to control and configure the sessions. The system conforms to the GSI grid security infrastructure and allows for seamless integration with other grid middleware. A SGFS prototype is evaluated with both file system benchmarks and typical applications, which demonstrates that it can achieve strong security with an acceptable overhead, and substantially outperform native NFS in wide-area environments by using disk caching.
12453165
Supporting application-tailored grid file system sessions with WSRF-based services
This paper presents novel service-based grid data management middleware that leverages standards defined by WSRF specifications to create and manage dynamic grid file system sessions. A unique aspect of the service is that the sessions it creates can be customized to address application data transfer needs. Application-tailored configurations enable selection of both performance-related features (block-based partial file transfers and/or whole-file transfers, cache parameters and consistency models) and reliability features (file system copy-on-write checkpointing to aid recovery of client-side failures; replication, autonomous failure detection and data access redirection for server-side failures). These enhancements, in addition to cross-domain user identity mapping and encrypted communication, are implemented via user level proxies managed by the service, requiring no changes to existing kernels. Sessions established using the service is mounted as distributed file systems and can be used transparently by unmodified binary applications. The paper analyzes the use of the service to support virtual machine based grid systems and workflow execution, and also reports on the performance and reliability of service managed wide-area file system sessions with experiments based on scientific applications (NanoMOS/Matlab, CHID, GAUSS and SPECseis).
18032989
On the Use of Virtualization and Service Technologies to Enable Grid-Computing
The In-VIGO approach to Grid-computing relies on the dynamic establishment of virtual grids on which application services are instantiated. In-VIGO was conceived to enable computational science to take place In Virtual Information Grid Organizations. Having its first version deployed on July of 2003, In-VIGO middleware is currently used by scientists from various disciplines, a noteworthy example being the computational nanoelectronics research community (http://www.nanohub.org). All components of an In-VIGO-generated virtual grid – machines, networks, applications and data – are themselves virtual and services are provided for their dynamic creation. This article reviews the In-VIGO approach to Grid-computing and overviews the associated middleware techniques and architectures for virtualizing Grid components, using services for creation of virtual grids and automatically Grid-enabling unmodified applications. The In-VIGO approach to the implementation of virtual networks and virtual application services are discussed as examples of Grid-motivated approaches to resource virtualization and Web-service creation.
6728240
Autonomic resource management in virtualized data centers using fuzzy logic-based approaches
Data centers, as resource providers, are expected to deliver on performance guarantees while optimizing resource utilization to reduce cost. Virtualization techniques provide the opportunity of consolidating multiple separately managed containers of virtual resources on underutilized physical servers. A key challenge that comes with virtualization is the simultaneous on-demand provisioning of shared physical resources to virtual containers and the management of their capacities to meet service-quality targets at the least cost. This paper proposes a two-level resource management system to dynamically allocate resources to individual virtual containers. It uses local controllers at the virtual-container level and a global controller at the resource-pool level. An important advantage of this two-level control architecture is that it allows independent controller designs for separately optimizing the performance of applications and the use of resources. Autonomic resource allocation is realized through the interaction of the local and global controllers. A novelty of the local controller designs is their use of fuzzy logic-based approaches to efficiently and robustly deal with the complexity and uncertainties of dynamically changing workloads and resource usage. The global controller determines the resource allocation based on a proposed profit model, with the goal of maximizing the total profit of the data center. Experimental results obtained through a prototype implementation demonstrate that, for the scenarios under consideration, the proposed resource management system can significantly reduce resource consumption while still achieving application performance targets.
7356729
Application-Tailored Cache Consistency for Wide-Area File Systems
The inability to perform optimizations based on application-specific information presents a hurdle to the deployment of pervasive LAN file systems across WAN environments. This paper proposes a novel approach addressing this problem through application-tailored caching and consistency in widearea file systems. It leverages widely available Network File System (NFS) deployments without any modifications to kernels nor applications, and employs middleware to dynamically establish Grid-wide Virtual File System (GVFS) sessions with application-tailored cache consistency. Two consistency models are discussed in this paper: a relaxed model based on invalidation polling, and a stronger model based on delegation and callback. Experimental evaluation based on microbenchmarks and scientific applications show that with application-tailored cache consistency, GVFS is able to both improve application runtimes and reduce server load significantly, compared to kernel-level NFS in WAN.
6353584
A CoS-enabled Request Scheduling Policy
As a lightweight SLA between service providers and consumers, class of service (CoS) can avoid the runtime overhead brought by multiple negotiation of service agreements. This is especially important in grid environment where lots of transient services exist. We bring CoS into grid environment and propose a QoS-enabled framework named CQEF for grid service. In this paper, we present a CoSenabled request scheduling policy for CoS deployment in CQEF. According to this policy, incoming requests can be identified and classified based on the CoS information attached to them and subsequently be assigned to different waiting queues with different priorities. Besides, queuing scheduling algorithm is used to achieve the CoS enabling purpose. A prototype has been developed over the Globus platform. The test results show that, by using this policy, service providers are able to offer differentiated services for different CoS and QoS of premium CoSs will be guaranteed preferentially.
6920619
On the achievable outage probability of opportunistic relaying with decode-and-forward
Based on decode-and-forward strategy, we investigate the achievable outage probability of opportunistic relaying, in which there is a single source-destination transmission pair with the help of one of multiple relays. In two-hop scenario, based on adaptive decode-and forward strategy, the achievable diversity-multiplexing tradeoff with the knowledge of channel state amplitudes is analyzed. We show that opportunistic relaying has the same diversity-multiplexing tradeoff as the relaying with distributed space-time coding. We then present the expression in a closed form and find a suboptimal power allocation for the achievable outage probability of opportunistic relaying with decode-and-forward. Simulation results are provided to demonstrate the validity of our analysis and the applicability of the optimum results to select a single ‘good’ relay.
1953336
Tour the world: Building a web-scale landmark recognition engine
Modeling and recognizing landmarks at world-scale is a useful yet challenging task. There exists no readily available list of worldwide landmarks. Obtaining reliable visual models for each landmark can also pose problems, and efficiency is another challenge for such a large scale system. This paper leverages the vast amount of multimedia data on the Web, the availability of an Internet image search engine, and advances in object recognition and clustering techniques, to address these issues. First, a comprehensive list of landmarks is mined from two sources: (1) ~20 million GPS-tagged photos and (2) online tour guide Web pages. Candidate images for each landmark are then obtained from photo sharing Websites or by querying an image search engine. Second, landmark visual models are built by pruning candidate images using efficient image matching and unsupervised clustering techniques. Finally, the landmarks and their visual models are validated by checking authorship of their member images. The resulting landmark recognition engine incorporates 5312 landmarks from 1259 cities in 144 countries. The experiments demonstrate that the engine can deliver satisfactory recognition performance with high efficiency.
122893722
Iterative Turbo Channel Estimation for OFDM System over Rapid Dispersive Fading Channel
Coherent OFDM detection requires accurate channel state information (CSI). Mobile radio channels are both time and frequency dispersive, especially at high vehicular speeds, which makes channel estimation a challenging problem in system design. Conventional preamble-based and pilot-aided channel estimation require numerous reference signals, which significantly compromises the system throughput. This paper proposes a novel low complexity iterative turbo channel estimation technique which makes use of preamble, pilots and soft decoded data information in an iterative fashion to improve the system performance over the time and frequency selective fading channel while maintaining the system throughput. The numerical and analytical results show that the proposed technique can approach the performance of systems with perfect CSI with much fewer preamble and pilots symbols compared to existing channel estimation methods.
10078589
SNCC: A Selective Network-Coded Cooperation Scheme in Wireless Networks
Network-coded cooperation is a new communication paradigm that exploits spatial diversity by pooling distributed "communication" (relay) and also the distributed "computation" resources (coding) of different nodes in a network so as to increase wireless networks system performance. In this paper, we develop a selective network-coded cooperation (SNCC) scheme suitable for multiple unicast transmission pairs in a single-cell wireless network. The relay node performs network coding on received information from multiple sources before forwarding the network-coded data towards the corresponding destinations, which can decode their desired data from the network-coded data and the data they overhear from the other sources. The performance analysis and evaluation in terms of the single-pair outage probability and diversity-multiplexing tradeoff show that under SNCC any transmission pair achieves a full diversity order of M+1 (M is the number of available relays) while holding a better diversity-multiplexing tradeoff performance than existing cooperative cooperation schemes. Moreover, due to the mechanism that the relay adapts the data on which to perform network coding according to the observed source-to-relay channel quality, SNCC smartly avoids error propagation due to incorporating erroneous data into network-coded data, which is verified by numerical results of the single-pair outage probability.