33716862
Computation of Sparse Data Cubes with Constraints
For a data cube there are always constraints between dimensions or between attributes in a dimension, such as functional dependencies. We introduce the problem that when there are functional dependencies, how to use them to speed up the computation of sparse data cubes. A new algorithm CFD is presented to satisfy this demand. CFD determines the order of dimensions by considering their cardinalities and functional dependencies between them together. It makes dimensions with functional dependencies adjacent and their codes satisfy monotonic mapping, thus reduces the number of partitions for such dimensions. It also combines partitioning from bottom to up and aggregate computation from top to bottom to speed up the computation further. In addition CFD can efficiently compute a data cube with hierarchies from the smallest granularity to the coarsest one, and at most one attribute in a dimension takes part in the computation each time. The experiments have shown that the performance of CFD has a significant improvement.
45051194
2-PS Based Associative Text Classification
Recent studies reveal that associative classification can achieve higher accuracy than traditional approaches. The main drawback of this approach is that it generates a huge number of rules, which makes it difficult to select a subset of rules for accurate classification. In this study, we propose a novel association-based approach especially suitable for text classification. The approach first builds a classifier through a 2-PS (Two-Phase) method. The first phase aims for pruning rules locally, i.e., rules mined within every category are pruned by a sentence-level constraint, and this makes the rules more semantically correlated and less redundant. In the second phase, all the remaining rules are compared and selected with a global view, i.e., training examples from different categories are merged together to evaluate these rules. Moreover, when labeling a new document, the multiple sentence-level appearances of a rule are taken into account. Experimental results on the well-known text corpora show that our method can achieve higher accuracy than many well-known methods. In addition, the performance study shows that our method is quite efficient in comparison with other classification methods.
27133006
Binary partition based algorithms for mining association rules
Mining association rules is an important data mining problem. A fast binary partition-based algorithm (BPA) for mining association rules in large databases is presented in this paper. Basically, the framework of BPA is similar to that of the algorithm Apriori. In the first pass, all the frequent 1-item sets are divided into two disjoint parts. Accordingly, in each subsequent pass k, we partition the set of all the frequent k-item sets into three subsets. Any two different partitions are disjoint. If necessary, this partitioning procedure can be a recursive one. Therefore, we get a binary partition tree in the first pass and a corresponding ternary partition tree in each subsequent pass k. Due to such a partition, BPA can be very easily parallelized, assuming a shared-memory architecture.
11780341
Sentential Association Based Text Classification Systems
We recently proposed a novel sentential association based approach SAT-MOD for text classification, which views a sentence rather than a document as an association transaction, and uses a novel heuristic called MODFIT to select the most significant itemsets for constructing a category classifier. Based on SAT-MOD, we have developed a prototype system called SAT-Class. In this demo, we demonstrate the effectiveness of our text classification system, and also the readability and refinability of acquired classification rules.
30099215
Indexing and incremental updating condensed data cube
OLAP (online analytical processing) servers usually pre-compute data cubes to improve the response time of possible aggregate queries over cuboids with different grouping attributes. To reduce the huge size of a sparse data cube, the base single tuples (BSTs) are explored to condense cube tuples aggregated from the same set of source tuples into one tuple, whenever such condensing will not require further aggregate when the cube is used to answer queries. We propose the CuboidTree to index the BST condensed cube. Using both synthetic and real world data, we conducted experiments to demonstrate query processing and bulk incremental updating performance of the indexing scheme.
28810751
Counting Graph Matches with Adaptive Statistics Collection
High performance of query processing in large scale graph-structured data poses a pressing demand for high-quality statistics collection and selectivity estimation. Precise and succinct statistics collection about graph-structured data plays a crucial role for graph query selectivity estimation. In this paper, we propose the approach SMT, Succinct Markov Table, which achieves high precision in selectivity estimation with low memory space consumed. Four core notions of SMT are constructing, refining, compressing and estimating. The efficient algorithm SMTBuilder provides facility to build adaptive statistics model in the form of SMT. Versatile optimization rules, which investigate local bi-directional reachability, are introduced in SMT refining. During compressing, affective SMT grouping techniques are introduced. Statistical methods are used for selectivity estimations of various graph queries basing on SMT, especially for twig queries. By a thorough experimental study, we demonstrate SMT's advantages in accuracy and space by comparing with previously known alternative, as well as the preferred optimization rules and compressing technique that would favor different real-life data.
10484901
An ontology-based method for querying the Web data
Nowadays large collections of documents of various types of information are available on the Internet, and finding relevant information on the World Wide Web is becoming a critical issue. Available search engines mostly rely on information retrieval techniques in finding information. They lack the desired power in query formulation which is beyond the keywords search. We introduce a method for querying the Web data that integrates both search engine and database techniques to support more complex search requests and get more precise results.
15649874
A New Query Processing Scheme in a Web Data Engine
The explosion of information on the web turns the search for interested information from the web into a great challenge. In this paper, we present a system called Web Data Engine-SESQ (Search Extract Store Query) that is designed to solve this problem by integrating database techniques with search engine techniques. In contrast with traditional database systems and searching engines, SESQ is different in data model, query expression, data storage schema and the use of index.
28964163
Hidden Conditioned Homomorphism for XPath Fragment Containment
As a query language for navigating XML trees and selecting a set of element nodes, XPath is ubiquitous in XML applications. One important issue of XPath queries is checking containment. In particular, we investigate a frequently used fragment of XPath that consists of node tests, the child axis (/), the descendant axis (//), branches ([]) and label wildcards (*). For special classes of pattern trees, the homomorphism algorithm returns false negatives. In order to address this problem, we propose two containment techniques, conditioned homomorphism and hidden conditioned homomorphism, and then present sound algorithms to check containment. The analytical result is given with an experiment.
9428764
OLAP Query Processing Algorithm Based on Relational Storage
When the relational storage method is adopted in the data warehouse system, the most time-consuming operations in OLAP query are multitable join and group-by aggregation. Based on the characteristics of the data warehouse itself and the applications run on it, one new multi-table join algorithm -- Mjoin is given in this paper. And the performance of this new Mjoin algorithm is improved greatly, compared with other traditional multitable join algorithms. In this paper, a new sorting based group-by aggregation algorithm -- MuSA is also given, based on the Mjoin algorithm. The speed of sorting is remarkably improved in this new sorting based aggregation algorithm, as the keyword mapping technology is used to compress the sort keywords in the course of sorting.