graph processing
218489492
Subway: minimizing data transfer during out of GPU memory graph processing
In many graph based applications, the graphs tend to grow, imposing a great challenge for GPU based graph processing. When the graph size exceeds the device memory capacity (i.e. GPU memory oversubscription) the performance of graph processing often degrades dramatically, due to the sheer amount of data transfer between CPU and GPU. To reduce the volume of data transfer, existing approaches track the activeness of graph partitions and only load the ones that need to be processed. In fact, the recent advances of unified memory implements this optimization implicitly by loading memory pages on demand. However, either way, the benefits are limited by the coarse granularity activeness tracking each loaded partition or memory page may still carry a large ratio of inactive edges. In this work, we present, to the best of our knowledge, the first solution that only loads active edges of the graph to the GPU memory. To achieve this, we design a fast subgraph generation algorithm with a simple yet efficient subgraph representation and a GPU accelerated implementation. They allow the subgraph generation to be applied in almost every iteration of the vertex centric graph processing. Furthermore, we bring asynchrony to the subgraph processing, delaying the synchronization between a subgraph in the GPU memory and the rest of the graph in the CPU memory. This can safely reduce the needs of generating and loading subgraphs for many common graph algorithms. Our prototyped system, Subway (subgraph processing with asynchrony) yields over 4X speedup on average comparing with existing out of GPU memory solutions and the unified memory based approach, based on an evaluation with six common graph algorithms.
EuroSys
Amir Hossein Nodehi Sabet Zhijia Zhao Rajiv Gupta
219571237
G3: When Graph Neural Networks Meet Parallel Graph Processing Systems on GPUs
This paper demonstrates G, a programming framework for Graph Neural Network (GNN) training, tailored from Graph processing systems onGraphics processing units (GPUs) G aims at improving the efficiency of GNN training by supporting graph structured operations using parallel graph processing systems. G enables users to leverage the massive parallelism and other architectural features of GPUs in the following two ways: building GNN layers by writing sequential C/C+ code with a set of flexible APIs (Application Programming Interfaces) creating GNN models with essential GNN operations and layers provided in G. The runtime system of G automatically executes the user defined GNNs on the GPU, with a series of graph centric optimizations enabled. We demonstrate the steps of developing some common GNN structures with G, and the superior performance of G against existing GNN training systems, i.e. PyTorch and TensorFlow. PVLDB Reference Format: Husong Liu, Shengliang Lu, Xinyu Chen, and Bingsheng He. G3: When Graph Neural Networks Meet Parallel Graph Processing Systems on GPUs. PVLDB, 12(xxx) xxxx yyyy, 2019. DOI: https:/doi.org/10.14778/xxxxxxx.xxxxxxx
Proc. VLDB Endow.
Husong Liu Shengliang Lu Xinyu Chen Bingsheng He
208303216
Cost Aware Partitioning for Efficient Large Graph Processing in Geo Distributed Datacenters
Graph processing is an emerging computation model for a wide range of applications and graph partitioning is important for optimizing the cost and performance of graph processing jobs. Recently, many graph applications store their data on geo distributed datacenters (DCs) to provide services worldwide with low latency. This raises new challenges to existing graph partitioning methods, due to the multi level heterogeneities in network bandwidth and communication prices in geo distributed DCs. In this article, we propose an efficient graph partitioning method named Geo Cut, which takes both the cost and performance objectives into consideration for large graph processing in geo distributed DCs. Geo Cut adopts two optimization stages. First, we propose a cost aware streaming heuristic and utilize the one pass streaming graph partitioning method to quickly assign edges to different DCs while minimizing inter DC data communication cost. Second, we propose two partition refinement heuristics which identify the performance bottlenecks of geo distributed graph processing and refine the partitioning result obtained in the first stage to reduce the inter DC data transfer time while satisfying the budget constraint. Geo Cut can be also applied to partition dynamic graphs thanks to its lightweight runtime overhead. We evaluate the effectiveness and efficiency of Geo Cut using real world graphs with both real geo distributed DCs and simulations. Evaluation results show that Geo Cut can reduce the inter DC data transfer time by up to 79 percent (42 percent as the median) and reduce the monetary cost by up to 75 percent (26 percent as the median) compared to state of the art graph partitioning methods with a low overhead.
IEEE Transactions on Parallel and Distributed Systems
Amelie Chi Zhou B Shen Yao Xiao Shadi Ibrahim Bingsheng He
208222374
GPOP: A Scalable Cache and Memory efficient Framework for Graph Processing over Parts
Past decade has seen the development of many shared memory graph processing frameworks, intended to reduce the effort of developing high performance parallel applications. However many of these frameworks, based on Vertex centric or Edge centric paradigms suffer from several issues, such as poor cache utilization, irregular memory accesses, heavy use of synchronization primitives and theoretical inefficiency, that deteriorate overall performance and scalability. Recently, we proposed a cache and memory efficient partition centric paradigm for computing PageRank [26] In this paper, we generalize this approach to develop a novel Graph Processing Over Partitions (GPOP) framework that is cache efficient, scalable and work efficient. GPOP induces locality in memory accesses by increasing granularity of execution to vertex subsets called 'partitions' thereby dramatically improving the cache performance of a variety of graph algorithms. It achieves high scalability by enabling completely lock and atomic free computation. GPOP's built in analytical performance model enables it to use a hybrid of source and partitioncentric communication modes in a way that ensures work efficiency each iteration, while simultaneously boosting high bandwidth sequential memory accesses. Finally, the GPOP framework is designed with programmability in mind. It completely abstracts away underlying parallelism and programming model details from the user, and provides an easy to program set of APIs with the ability to selectively continue the active vertex set across iterations. Such functionality is useful for many graph algorithms but not intrinsically supported by the current frameworks. We extensively evaluate the performance of GPOP for a variety of graph algorithms, using several large datasets. We observe that GPOP incurs up to 9x, 6.8x and 5.5x less L2 cache misses compared to Ligra, GraphMat and Galois, respectively. In terms of execution time, GPOP is upto 19x, 9.3x and 3.6x faster than Ligra, GraphMat and Galois respectively.
ACM Trans. Parallel Comput.
Kartik Lakhotia Rajgopal Kannan Sourav Pati Viktor K Prasanna
212735029
GRETA: HARDWARE OPTIMIZED GRAPH PROCESSING FOR GNNS
Graph Neural Networks (GNNs) are a type of deep neural network that can learn directly from irregular graphstructured data. However, GNN inference relies on sparse operations that are difficult to implement on accelerator architectures optimized for dense, fixed sized computation. This paper proposes GReTA (Gather, Reduce, Transform, Activate) a graph processing abstraction designed to be efficient for an accelerator to execute and flexible enough to implement GNN inference. We demonstrate GReTA's advantages by designing and synthesizing a custom accelerator ASIC for GReTA and implementing several GNN models (GCN, GraphSage, G GCN, and GIN. Across several benchmark graphs, our implementation reduces 97th percentile latency by a geometric mean of 15x and 21x compared to a CPU and GPU baseline respectively.
Kevin Kiningham
221107223
Balancing Graph Processing Workloads Using Work Stealing on Heterogeneous CPU FPGA Systems
We propose, implement and evaluate a work stealing based scheduler, called HWS, for graph processing on heterogeneous CPU FPGA systems that tightly couple the CPU and the FPGA to share system memory. HWS addresses unique concerns that arise with work stealing in the context of our target system. Our evaluation is conducted on the Intel Heterogeneous Architecture Research Platform (HARPv2) using three key processing kernels and seven real world graphs. We show that HWS effectively balances workloads. Further, the use of HWS results in better graph processing performance compared to static scheduling and a representative of existing adaptive partitioning techniques, called HAP. Improvements vary by graph processing application, input graph and number of threads, and can be up to 100% over static scheduling, and up to 17% over HAP. We also compare to an oracle chunk self scheduler, in which the best chunk size is known a priori for each number of threads and each input graph. HWS performs no worse than 1 3% in most cases. Finally, our graph processing throughput scales well with increasing threads. These results collectively demonstrate the effectiveness of work stealing for graph processing on our heterogeneous target platform.
ICPP
Matthew Agostini Francis O'Brien Tarek S Abdelrahman
214751453
Large scale graph processing systems: a survey
Graph is a significant data structure that describes the relationship between entries. Many application domains in the real world are heavily dependent on graph data. However, graph applications are vastly different from traditional applications. It is inefficient to use general purpose platforms for graph applications, thus contributing to the research of specific graph processing platforms. In this survey, we systematically categorize the graph workloads and applications, and provide a detailed review of existing graph processing platforms by dividing them into general purpose and specialized systems. We thoroughly analyze the implementation technologies including programming models, partitioning strategies, communication models, execution models, and fault tolerance strategies. Finally, we analyze recent advances and present four open problems for future research.
Frontiers of Information Technology Electronic Engineering
Ning Liu Dong-sheng Li Yiming Zhang Xiong-lve Li
219057282
An Analysis of Distributed Programming Models and Frameworks for Large scale Graph Processing
In recent years, processing and analysing large graphs has become a major need in many research areas. Distributed graph processing programming models and frameworks arised as a natural solution to.
Alejandro Corbellini Daniela Godoy Cristian Mateos Silvia N Schiaffino Alejandro Zunino
220605953
A Heterogeneous PIM Hardware Software Co Design for Energy Efficient Graph Processing
Processing In Memory (PIM) is an emerging technology that addresses the memory bottleneck of graph processing. In general, analog memristor based PIM promises high parallelism provided that the underlying matrix structured crossbar can be fully utilized while digital CMOS based PIM has a faster single edge execution but its parallelism can be low. In this paper, we observe that there is no absolute winner between these two representative PIM technologies for graph applications, which often exhibit irregular workloads. To reap the best of both worlds, we introduce a new heterogeneous PIM hardware, called Hetraph, to facilitate energy efficient graph processing. Hetraph incorporates memristor based analog computation units (for high parallelism computing) and CMOS based digital computation cores (for efficient computing) on the same logic layer of a 3D die stacked memory device. To maximize the hardware utilization, our software design offers a hardware heterogeneity aware execution model and a workload offloading mechanism. For performance speedups, such a hardware software co design outperforms the state of the art by 7.54 x(CPU) 1.56 x(GPU) 4.13x (memristor based PIM) and 3.05x (CMOS based PIM) on average. For energy savings, Hetraph reduces the energy consumption by 57.58x (CPU) 19.93x (GPU) 14.02 x(memristor based PIM) and 10.48 x(CMOS based PIM) on average.
2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
Yu Huang Long Zheng Pengcheng Yao Jieshan Zhao Xiaofei Liao Hai Jin Jingling Xue
221737276
GraphPulse: An Event Driven Hardware Accelerator for Asynchronous Graph Processing
Graph processing workloads are memory intensive with irregular access patterns and large memory footprint resulting in low data locality. Their popular software implementations typically employ either Push or Pull style propagation of changes through the graph over multiple iterations that follow the Bulk Synchronous Model. The performance of these algorithms on traditional computing systems is limited by random reads/writes of vertex values, synchronization overheads, and additional overheads for tracking active sets of vertices or edges across iterations. In this paper, we present GraphPulse, a hardware framework for asynchronous graph processing with event driven scheduling that overcomes the performance limitations of software frameworks. Event driven computation model enables a parallel dataflow style execution where atomic updates and active sets tracking are inherent to the model; thus, scheduling complexity is reduced and scalability is enhanced. The dataflow nature of the architecture also reduces random reads of vertex values by carrying the values in the events themselves. We capitalize on the update properties commonly present in graph algorithms to coalesce in flight events and substantially reduce the event storage requirement and the processing overheads incurred. GraphPulse event model naturally supports asynchronous graph processing, enabling substantially faster convergence by exploiting available parallelism, reducing work, and eliminating synchronization at iteration boundaries. The framework provides easy to use programming interface for faster development of hardware graph accelerators. A single GraphPulse accelerator achieves up to 74x speedup (28x on average) over Ligra, a state of the art software framework, running on a 12 core CPU. It also achieves an average of 6.2x speedup over Graphicionado, a state of the art graph processing accelerator.
2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
Shafiur Rahman Nael B Abu-Ghazaleh Rajiv Gupta