16197603
Parallel and distributed methods for incremental frequent itemset mining
Traditional methods for data mining typically make the assumption that the data is centralized, memory-resident, and static. This assumption is no longer tenable. Such methods waste computational and input/output (I/O) resources when data is dynamic, and they impose excessive communication overhead when data is distributed. Efficient implementation of incremental data mining methods is, thus, becoming crucial for ensuring system scalability and facilitating knowledge discovery when data is dynamic and distributed. In this paper, we address this issue in the context of the important task of frequent itemset mining. We first present an efficient algorithm which dynamically maintains the required information even in the presence of data updates without examining the entire dataset. We then show how to parallelize this incremental algorithm. We also propose a distributed asynchronous algorithm, which imposes minimal communication overhead for mining distributed dynamic datasets. Our distributed approach is capable of generating local models (in which each site has a summary of its own database) as well as the global model of frequent itemsets (in which all sites have a summary of the entire database). This ability permits our approach not only to generate frequent itemsets, but also to generate high-contrast frequent itemsets, which allows one to examine how the data is skewed over different sites.
425294
A Decomposition-Based Probabilistic Framework for Estimating the Selectivity of XML Twig Queries
In this paper we present a novel approach for estimating the selectivity of XML twig queries. Such a technique is useful for answering approximate queries as well as for determining an optimal query plan for complex queries based on said estimates. Our approach relies on a summary structure that contains the occurrence statistics of small twigs. We rely on a novel probabilistic approach for decomposing larger twig queries into smaller ones. We then show how it can be used to estimate the selectivity of the larger query in conjunction with the summary information. We present and evaluate different strategies for decomposition and compare this work against a state-of-the-art selectivity estimation approach on synthetic and real datasets. The experimental results show that our proposed approach is very effective in estimating the selectivity of XML twig queries.
9741529
Mining frequent itemsets in distributed and dynamic databases
Traditional methods for frequent itemset mining typically assume that data is centralized and static. Such methods impose excessive communication overhead when data is distributed, and they waste computational resources when data is dynamic. We present what we believe to be the first unified approach that overcomes these assumptions. Our approach makes use of parallel and incremental techniques to generate frequent itemsets in the presence of data updates without examining the entire database, and imposes minimal communication overhead when mining distributed databases. Further, our approach is able to generate both local and global frequent itemsets. This ability permits our approach to identify high-contrast frequent itemsets, which allows one to examine how the data is skewed over different sites.
46393116
Learning Approximate MRFs from Large Transaction Data
In this paper we consider the problem of learning approximate Markov Random Fields (MRFs) from large transaction data. We rely on frequent itemsets to learn MRFs on the data. Since learning exact large MRFs is generally intractable, we resort to learning approximate MRFs. Our proposed modeling approach first employs graph partitioning to cluster variables into balanced disjoint partitions, and then augments important interactions across partitions to capture interdependencies across them. A novel treewidth based augmentation scheme is proposed to boost performance. We learn an exact local MRF for each partition and then combine all the local MRFs together to derive a global model of the data. A greedy approximate inference scheme is developed on this global model. We demonstrate the use of the learned MRFs on the selectivity estimation problem. Empirical evaluation on real datasets demonstrates the advantage of our approach over extant solutions.
13965365
Distributed caching and adaptive search in multilayer P2P networks
To improve the scalability of Gnutella-like unstructured peer-to-peer (P2P) networks, a uniform index caching (UIC) mechanism was suggested in some earlier work. In UIC, query results are cached in all peers along the inverse query path such that the same query of other peers can be replied from their nearby-cached results. However, our experiments show that the UIC method causes a large amount of duplicated and unnecessary caching of items among neighboring peers. Aiming at improving the search efficiency, we propose a distributed caching mechanism, which distributes the cache results among neighboring peers. Furthermore, based on the distributed caching mechanism, an adaptive search approach is built which selectively forwards the query to the peers with a high probability of providing the desired cache results. All the enhancements above are defined in a protocol called distributed caching and adaptive search (DiCAS). In the DiCAS enhanced Gnutella network, all the peers are logically divided into multiple layers, with the character that all the peers in the same layer have the same group ID. The query flooding is restricted in one layer with the matched group ID. Our simulation study shows that, with the help of the index caching and search space division, the DiCAS protocol can significantly reduce the network search traffic in unstructured P2P systems without degrading query success rate.
1928330
A near-optimal broadcasting algorithm in all-port wormhole-routed hypercubes
For massively parallel computing mechanism, broadcasting is widely used in a variety of applications. When the computation is distributed among the powerful processors, communication overhead always limits the speedup. To reduce the influence of communication latency, a variety of routing methods have been discussed. In this paper, we propose an efficient routing method of broadcasting on all-port worrnhole-routed hybercubes. By exploiting the distance-insensitivity of wormhole switching, this efficient broadcasting algorithm reduce communication latency to l_rzl_logz(n+l )11steps in an n-dimensional hypercube, which is much better than previous results and very close to the theoretical optimum, [rz/log2(n+l )1.Further, by adopting the Hamming code, a well-known algebraic approach in coding theory, this proposed algorithm is contention-free in each broadcasting step.
36109723
Generation and Matching of Ontology Data for the Semantic Web in a Peer-to-Peer Framework
The abundance of ontology data is very crucial to the emerging semantic web. This paper proposes a framework that supports the generation of ontology data in a peer-to-peer environment. It not only enables users to convert existing structured data to ontology data aligned with given ontology schemas, but also allows them to publish new ontology data with ease. Besides ontology data generation, the common issue of data overlapping over the peers is addressed by the process of ontology data matching in the framework. This process helps turn the implicitly related data among the peers caused by overlapping into explicitly interlinked ontology data, which increases the overall quality of the ontology data. To improve matching accuracy, we explore ontology related features in the matching process. Experiments show that adding these features achieves better overall performance than using traditional features only.
6254649
Disjunctive Image Computation for Embedded Software Verification
Finite state models generated from software programs have unique characteristics that are not exploited by existing model checking algorithms. In this paper, we propose a novel disjunctive image computation algorithm and other simplifications based on these characteristics. Our algorithm divides an image computation into a disjunctive set of easier ones that can be performed in isolation. Hypergraph partitioning is used to minimize the number of live variables in each disjunctive component. We use the live variables to simplify transition relations and reachable state subsets. Our experiments on a set of real-world C programs show that the new algorithm achieves orders-of-magnitude performance improvement over the best known conjunctive image computation algorithm
18478446
On Parallel Complexity of Planar Triangulations
The greedy triangulation of a finite planar point set is obtained by repeatedly inserting a shortest diagonal that doesn't intersect those already in the plane. We show that the problem of constructing the greedy triangulation of a finite set of points with integer coordinates in the plane is P-complete. This is the first known geometric P-complete problem where the input is given as a set of points. On the other hand, we provide general NC-methods for testing whether a given triangulation of a set of points and/or line segments can be built by inserting the diagonals in a given partial order, and for constructing such triangulations for simple polygons. As corollaries, we obtain NC-algorithms for testing whether a triangulation is respectively the greedy triangulation or the so called sweep-line triangulation, and for constructing respectively the greedy triangulation or the sweep-line triangulation of a simple polygon. The latter result solves the open problem posed by Atallah et al.