11127837
Nomad: migrating OS-bypass networks in virtual machines
Virtual machine (VM) technology is experiencing a resurgence due to various benefits including ease of management, security and resource consolidation. Live migration of virtual machines allows transparent movement of OS instances and hosted applications across physical machines. It is one of the most useful features of VM technology because it provides a powerful tool for effective administration of modern cluster environments. Migrating network resources is one of the key problems that need to be addressed in the VM migration process. Existing studies of VM migration have focused on traditional I/O interfaces such as Ethernet. However, modern high-speed interconnects with intelligent NICs pose significantly more challenges as they have additional features including hardware level reliable services and direct I/O accesses. In this paper we present Nomad, a design for migrating modern interconnects with the aforementioned features, focusing on cluster environments running VMs. We introduce a thin namespace virtualization layer to efficiently address location dependent resource handles and a handshake protocol which transparently maintains reliable service semantics during migration. We demonstrate our design by implementing a prototype based on the Xen virtual machine monitor and InfiniBand. Our performance analysis shows that Nomad can achieve efficient migration of network resources, even in environments with stringent communication performance requirements.
2376785
Designing Efficient Asynchronous Memory Operations Using Hardware Copy Engine: A Case Study with I/OAT
Memory copies for bulk data transport incur large overheads due to CPU stalling, small register-size data movement, etc. Intel's I/O Acceleration Technology offers an asynchronous memory copy engine in kernel space which alleviates such overheads. In this paper, we propose a set of designs for asynchronous memory operations in user space for both single process (as an offloaded memcpy()) and lPC using the copy engine. We analyze our design based on overlap efficiency, performance and cache utilization. Our microbenchmark results show that using the copy engine for performing memory copies can achieve close to 87% overlap with computation. Further, the copy engine improves the copy latency of bulk memory data transfers by 50% and avoids cache pollution effects. With the emergence of multi-core architectures, the support for asynchronous memory operations holds a lot of promise in reducing the gap between the memory and processor performance.
1603873
Application-Transparent Checkpoint/Restart for MPI Programs over InfiniBand
Ultra-scale computer clusters with high speed interconnects, such as InfiniBand, are being widely deployed for their excellent performance and cost effectiveness. However, the failure rate on these clusters also increases along with their augmented number of components. Thus, it becomes critical for such systems to be equipped with fault tolerance support. In this paper, we present our design and implementation of checkpoint/restart framework for MPI programs running over InfiniBand clusters. Our design enables low-overhead, application-transparent checkpointing. It uses coordinated protocol to save the current state of the whole MPI job to reliable storage, which allows users to perform rollback recovery if the system runs into faulty states later. Our solution has been incorporated into MVAPICH2, an open-source high performance MPI-2 implementation over InfiniBand. Performance evaluation of this implementation has been carried out using NAS benchmarks, HPL benchmark, and a real-world application called GROMACS. Experimental results indicate that in our design, the overhead to take checkpoints is low, and the performance impact for checkpointing applications periodically is insignificant. For example, time for checkpointing GROMACS is less than 0.3% of the execution time, and its performance only decreases by 4% with checkpoints taken every minute. To the best of our knowledge, this work is the first report of checkpoint/restart support for MPI over InfiniBand clusters in the literature
11481204
Scheduling of MPI-2 one sided operations over InfiniBand
MPI-2 provides interfaces for one sided communication, which is becoming increasingly important in scientific applications. MPI-2 semantics provide the flexibility to reorder the one sided operations within an access epoch. Based on this flexibility, in this paper we try to improve the performance of one sided communication by scheduling one sided operations. We have come up with several re-ordering and aggregating schemes to achieve better network utilization. We have evaluated these schemes on both PCI-X and PCI-Express platforms. With re-ordering scheme, we see an improvement in the throughput up to 76%, latency up to 40%. With aggregation scheme, we observe an improvement of 44% and 42% for MPI/spl I.bar/Put and MPI/spl I.bar/Get latency respectively on PCI-Express platform.
2763282
Can memory-less network adapters benefit next-generation infiniband systems?
InfiniBand is emerging as a high-performance interconnect. It is gaining popularity because of its high performance and open standard. Recently, PCI-Express, which is the third generation high-performance I/O bus used to interconnect peripheral devices, has been released. The third generation of InfiniBand adapters allow applications to take advantage of PCI-Express. PCI-Express offers very low latency access of the host memory by network interface cards (NICs). Earlier generation InfiniBand adapters used to have an external DIMM attached as local NIC memory. This memory was used to store internal information. This memory increases the overall cost of the NIC. In this paper we design experiments, analyze the performance of various communication patterns and end applications on PCI-Express based systems, whose adapters can be chosen to run with or without local NIC memory. Our investigations reveal that on these systems, the memory fetch latency is the same for both local NIC memory and host memory. Under heavy I/O bus usage, the latency of a scatter operation increased only by 10% and only for message sizes IB -4 KB. These memory-less adapters allow more efficient use of overall system memory and show practically no performance impact (less than 0.1%) for the NAS parallel benchmarks on 8 processes. These results indicate that memory-less network adapters can benefit next generation InfiniBand systems.
29243423
Design Alternatives and Performance Trade-Offs for Implementing MPI-2 over InfiniBand
MPICH2 provides a layered architecture to achieve both portability and performance. For implementations of MPI-2 over InfiniBand, it provides the flexibility for researchers at the RDMA channel, CH3 or ADI3 layer. In this paper we analyze the performance and complexity trade-offs associated with implementations at these layers. We describe our designs and implementations, as well as optimizations at each layer. To show the performance impacts of these design schemes and optimizations, we evaluate our implementations with different micro-benchmarks, HPCC and NAS test suite. Our experiments show that although the ADI3 layers adds complexity in implementation, the benefits achieved through optimizations justify moving to the ADI layer to extract the best performance.
33589927
Towards Context-Aware Knowledge Management in e-Enterprises
13283914
ConKMeL: a contextual knowledge management framework to support multimedia e-Learning
As the latest stage of learning and training evolution, e-Learning is supposed to provide intelligent functionalities not only in processing multi-media education resources but also in supporting context-sensitive pedagogical education processes. Towards providing an integrated solution for intelligent multimedia e-Learning, this paper presents a context-aware knowledge management framework named ConKMeL. Proposed framework features a semantic context-based approach for representing and integrating information and knowledge in e-Learning. Requirement analysis in university e-Learning environments shows that knowledge communications are usually in a hybrid mode across different conceptual levels. Based on the fact, a multi-layer contextual knowledge representation model called KG (knowledge graph) is presented. Corresponding key issues in development such as context-based knowledge retrieval and logical knowledge interpretation are discussed. On the application side, a scenario-based learning case study is shown to demonstrate the concepts and techniques developed in the ConKMeL framework.
17604511
POINT: a database for the prediction of protein-protein interactions based on the orthologous interactome
One possible path towards understanding the biological function of a target protein is through the discovery of how it interfaces within protein-protein interaction networks. The goal of this study was to create a virtual protein-protein interaction model using the concepts of orthologous conservation (or interologs) to elucidate the interacting networks of a particular target protein. POINT (the prediction of interactome database) is a functional database for the prediction of the human protein-protein interactome based on available orthologous interactome datasets. POINT integrates several publicly accessible databases, with emphasis placed on the extraction of a large quantity of mouse, fruit fly, worm and yeast protein-protein interactions datasets from the Database of Interacting Proteins (DIP), followed by conversion of them into a predicted human interactome. In addition, protein-protein interactions require both temporal synchronicity and precise spatial proximity. POINT therefore also incorporates correlated mRNA expression clusters obtained from cell cycle microarray databases and subcellular localization from Gene Ontology to further pinpoint the likelihood of biological relevance of each predicted interacting sets of protein partners.
1537043
Topological Map Merging
When multiple robots cooperatively explore an environment, maps from individual robots must be merged to produce a single globally consistent map. This is a challenging problem when the robots do not have a common reference frame or global positioning. In this paper, we describe an algorithm for merging embedded topological maps. Topological maps provide a concise description of the navigability of an environment, and, with measurements easily collected during exploration, the vertices of the map can be embedded in a metric space. Our algorithm uses both the structure and the geometry of topological maps to determine the best correspondence between maps with single or multiple overlapping regions. Experiments with simulated and real-world data demonstrate the efficacy of our algorithm.
21933462
Multi-media semantics contextualisation for knowledge-oriented e-learning
Existing e-learning technical solutions are more focused on content packaging rather than semantics management, which is one of the keys to knowledge-oriented e-learning. This paper presents a context-based approach to address the challenging semantics management issue by contextualising the semantics of multimedia learning resources at different levels. A prototype of semi-automatic multimedia semantics generation and editing tool - CONMAN shows the basic idea of the approach in practice.