7089503
Supporting MPI-2 One Sided Communication on Multi-rail InfiniBand Clusters: Design Challenges and Performance Benefits
In cluster computing, InfiniBand has emerged as a popular high performance interconnect with MPI as the de facto programming model. However, even with InfiniBand, bandwidth can become a bottleneck for clusters executing communication intensive applications. Multi-rail cluster configurations with MPI-1 are being proposed to alleviate this problem. Recently, MPI-2 with support for one-sided communication is gaining significance. In this paper, we take the challenge of designing high performance MPI-2 one-sided communication on multi-rail InfiniBand clusters. We propose a unified MPI-2 design for different configurations of multi-rail networks (multiple ports, multiple HCAs and combinations). We present various issues associated with one-sided communication such as multiple synchronization messages, scheduling of RDMA (Read, Write) operations, ordering relaxation and discuss their implications on our design. Our performance results show that multi-rail networks can significantly improve MPI-2 one-sided communication performance. Using PCI-Express with two-ports, we can achieve a peak MPI_Put bidirectional bandwidth of 2620 Million Bytes/s, compared to 1910 MB/s for single-rail implementation. For PCI-X with two HCAs, we can almost double the throughput and reduce the latency to half for large messages.
2763282
Can memory-less network adapters benefit next-generation infiniband systems?
InfiniBand is emerging as a high-performance interconnect. It is gaining popularity because of its high performance and open standard. Recently, PCI-Express, which is the third generation high-performance I/O bus used to interconnect peripheral devices, has been released. The third generation of InfiniBand adapters allow applications to take advantage of PCI-Express. PCI-Express offers very low latency access of the host memory by network interface cards (NICs). Earlier generation InfiniBand adapters used to have an external DIMM attached as local NIC memory. This memory was used to store internal information. This memory increases the overall cost of the NIC. In this paper we design experiments, analyze the performance of various communication patterns and end applications on PCI-Express based systems, whose adapters can be chosen to run with or without local NIC memory. Our investigations reveal that on these systems, the memory fetch latency is the same for both local NIC memory and host memory. Under heavy I/O bus usage, the latency of a scatter operation increased only by 10% and only for message sizes IB -4 KB. These memory-less adapters allow more efficient use of overall system memory and show practically no performance impact (less than 0.1%) for the NAS parallel benchmarks on 8 processes. These results indicate that memory-less network adapters can benefit next generation InfiniBand systems.
2376785
Designing Efficient Asynchronous Memory Operations Using Hardware Copy Engine: A Case Study with I/OAT
Memory copies for bulk data transport incur large overheads due to CPU stalling, small register-size data movement, etc. Intel's I/O Acceleration Technology offers an asynchronous memory copy engine in kernel space which alleviates such overheads. In this paper, we propose a set of designs for asynchronous memory operations in user space for both single process (as an offloaded memcpy()) and lPC using the copy engine. We analyze our design based on overlap efficiency, performance and cache utilization. Our microbenchmark results show that using the copy engine for performing memory copies can achieve close to 87% overlap with computation. Further, the copy engine improves the copy latency of bulk memory data transfers by 50% and avoids cache pollution effects. With the emergence of multi-core architectures, the support for asynchronous memory operations holds a lot of promise in reducing the gap between the memory and processor performance.
11481204
Scheduling of MPI-2 one sided operations over InfiniBand
MPI-2 provides interfaces for one sided communication, which is becoming increasingly important in scientific applications. MPI-2 semantics provide the flexibility to reorder the one sided operations within an access epoch. Based on this flexibility, in this paper we try to improve the performance of one sided communication by scheduling one sided operations. We have come up with several re-ordering and aggregating schemes to achieve better network utilization. We have evaluated these schemes on both PCI-X and PCI-Express platforms. With re-ordering scheme, we see an improvement in the throughput up to 76%, latency up to 40%. With aggregation scheme, we observe an improvement of 44% and 42% for MPI/spl I.bar/Put and MPI/spl I.bar/Get latency respectively on PCI-Express platform.
29243423
Design Alternatives and Performance Trade-Offs for Implementing MPI-2 over InfiniBand
MPICH2 provides a layered architecture to achieve both portability and performance. For implementations of MPI-2 over InfiniBand, it provides the flexibility for researchers at the RDMA channel, CH3 or ADI3 layer. In this paper we analyze the performance and complexity trade-offs associated with implementations at these layers. We describe our designs and implementations, as well as optimizations at each layer. To show the performance impacts of these design schemes and optimizations, we evaluate our implementations with different micro-benchmarks, HPCC and NAS test suite. Our experiments show that although the ADI3 layers adds complexity in implementation, the benefits achieved through optimizations justify moving to the ADI layer to extract the best performance.
11127837
Nomad: migrating OS-bypass networks in virtual machines
Virtual machine (VM) technology is experiencing a resurgence due to various benefits including ease of management, security and resource consolidation. Live migration of virtual machines allows transparent movement of OS instances and hosted applications across physical machines. It is one of the most useful features of VM technology because it provides a powerful tool for effective administration of modern cluster environments. Migrating network resources is one of the key problems that need to be addressed in the VM migration process. Existing studies of VM migration have focused on traditional I/O interfaces such as Ethernet. However, modern high-speed interconnects with intelligent NICs pose significantly more challenges as they have additional features including hardware level reliable services and direct I/O accesses. In this paper we present Nomad, a design for migrating modern interconnects with the aforementioned features, focusing on cluster environments running VMs. We introduce a thin namespace virtualization layer to efficiently address location dependent resource handles and a handshake protocol which transparently maintains reliable service semantics during migration. We demonstrate our design by implementing a prototype based on the Xen virtual machine monitor and InfiniBand. Our performance analysis shows that Nomad can achieve efficient migration of network resources, even in environments with stringent communication performance requirements.
1537043
Topological Map Merging
When multiple robots cooperatively explore an environment, maps from individual robots must be merged to produce a single globally consistent map. This is a challenging problem when the robots do not have a common reference frame or global positioning. In this paper, we describe an algorithm for merging embedded topological maps. Topological maps provide a concise description of the navigability of an environment, and, with measurements easily collected during exploration, the vertices of the map can be embedded in a metric space. Our algorithm uses both the structure and the geometry of topological maps to determine the best correspondence between maps with single or multiple overlapping regions. Experiments with simulated and real-world data demonstrate the efficacy of our algorithm.
62635072
Kernel autoassociator with applications to visual classification
Autoassociator is an important issue in concept learning, and the learned concept of a particular class can be used to distinguish the class from the others. For nonlinear autoassociation, This work presents a new model referred to as kernel autoassociator. Using kernel feature space as a potential nonlinear manifold, the model formulates the autoassociation as a special reconstruction problem from kernel feature space to input space. Two methods are developed to solve the problem. We evaluate the autoassociator with artificial data, and apply it to handwritten digit recognition and multiview face recognition, yielding positive experimental results.
7757717
A kernel autoassociator approach to pattern classification
Autoassociators are a special type of neural networks which, by learning to reproduce a given set of patterns, grasp the underlying concept that is useful for pattern classification. In this paper, we present a novel nonlinear model referred to as kernel autoassociators based on kernel methods. While conventional non-linear autoassociation models emphasize searching for the non-linear representations of input patterns, a kernel autoassociator takes a kernel feature space as the nonlinear manifold, and places emphasis on the reconstruction of input patterns from the kernel feature space. Two methods are proposed to address the reconstruction problem, using linear and multivariate polynomial functions, respectively. We apply the proposed model to novelty detection with or without novelty examples and study it on the promoter detection and sonar target recognition problems. We also apply the model to mclass classification problems including wine recognition, glass recognition, handwritten digit recognition, and face recognition. The experimental results show that, compared with conventional autoassociators and other recognition systems, kernel autoassociators can provide better or comparable performance for concept learning and recognition in various domains.
17604511
POINT: a database for the prediction of protein-protein interactions based on the orthologous interactome
One possible path towards understanding the biological function of a target protein is through the discovery of how it interfaces within protein-protein interaction networks. The goal of this study was to create a virtual protein-protein interaction model using the concepts of orthologous conservation (or interologs) to elucidate the interacting networks of a particular target protein. POINT (the prediction of interactome database) is a functional database for the prediction of the human protein-protein interactome based on available orthologous interactome datasets. POINT integrates several publicly accessible databases, with emphasis placed on the extraction of a large quantity of mouse, fruit fly, worm and yeast protein-protein interactions datasets from the Database of Interacting Proteins (DIP), followed by conversion of them into a predicted human interactome. In addition, protein-protein interactions require both temporal synchronicity and precise spatial proximity. POINT therefore also incorporates correlated mRNA expression clusters obtained from cell cycle microarray databases and subcellular localization from Gene Ontology to further pinpoint the likelihood of biological relevance of each predicted interacting sets of protein partners.
12648516
Experiments in impulsive manipulation
In this paper, we present the results of our experimental effort in one form of impulsive manipulation: tapping. Our previous work studied the mechanics of tapping a planar object which then slides on a support surface, coming to rest due to friction. This work addresses the practical issues in creating a system which uses this mode of manipulation. We begin with the design of tapping devices-end effecters designed to deliver an impulse to an object, and report some of the issues we have found to be important in their design. Our next step was to perform single-tap experiments in order to fit and evaluate the models of impact and sliding. These experiments have shown that objects rotate less than predicted; we have found that the addition of a scaling factor for the torque due to friction enables the models to predict object motion reasonably well. In order to do positioning experiments, we developed a number of planning methods (or feedback control strategies) to compensate for errors in modeling, parameters, and actuation. These planning methods were successfully used to demonstrate a positioning task. We also have experimentally demonstrated that tapping can be used to position an object more precisely than the manipulator can position the tapping device. We offer some sensitivity analysis in support of this result.