18937658
Mining associations by pattern structure in large relational tables
Association rule mining aims at discovering patterns whose support is beyond a given threshold. Mining patterns composed of items described by an arbitrary subset of attributes in a large relational table represents a new challenge and has various practical applications, including the event management systems that motivated this work. The attribute combinations that define the items in a pattern provide the structural information of the pattern. Current association algorithms do not make full use of the structural information of the patterns: the information is either lost after it is encoded with attribute values, or is constrained by a given hierarchy or taxonomy. Pattern structures convey important knowledge about the patterns. We present an architecture that organizes the mining space based on pattern structures. By exploiting the interrelationships among pattern structures, execution times for mining can be reduced significantly. This advantage is demonstrated by our experiments using both synthetic and real-life datasets.
2230972
Pruning and dynamic scheduling of cost-sensitive ensembles
Previous research has shown that averaging ensemble can scale up learning over very large cost-sensitive datasets with linear speedup independent of the learning algorithms. At the same time, it achieves the same or even better accuracy than a single model computed from the entire dataset. However, one major drawback is its inefficiency in prediction since every base model in the ensemble has to be consulted in order to produce a final prediction. In this paper, we propose several approaches to reduce the number of base classifiers. Among various methods explored, our empirical studies have shown that the benefit-based greedy approach can safely remove more than 90% of the base models while maintaining or even exceeding the prediction accuracy of the original ensemble. Assuming that each base classifier consumes one unit of prediction time, the removal of 90% of base classifiers translates to a prediction speedup of 10 times. On top of pruning, we propose a novel dynamic scheduling approach to further reduce the "expected" number of classifiers employed in prediction. It measures the confidence of a prediction by a subset of classifiers in the pruned ensemble. This confidence is used to decide if more classifiers are needed in order to produce a prediction that is the same as the original unpruned ensemble. This approach reduces the "expected" number of classifiers by another 25% to 75% without loss of accuracy.
1498767
Clustering by pattern similarity in large data sets
Clustering is the process of grouping a set of objects into classes of similar objects. Although definitions of similarity vary from one clustering model to another, in most of these models the concept of similarity is based on distances, e.g., Euclidean distance or cosine distance. In other words, similar objects are required to have close values on at least a set of dimensions. In this paper, we explore a more general type of similarity. Under the pCluster model we proposed, two objects are similar if they exhibit a coherent pattern on a subset of dimensions. For instance, in DNA microarray analysis, the expression levels of two genes may rise and fall synchronously in response to a set of environmental stimuli. Although the magnitude of their expression levels may not be close, the patterns they exhibit can be very much alike. Discovery of such clusters of genes is essential in revealing significant connections in gene regulatory networks. E-commerce applications, such as collaborative filtering, can also benefit from the new model, which captures not only the closeness of values of certain leading indicators but also the closeness of (purchasing, browsing, etc.) patterns exhibited by the customers. Our paper introduces an effective algorithm to detect such clusters, and we perform tests on several real and synthetic data sets to show its effectiveness.
5103963
Adaptive Load Diffusion for Multiway Windowed Stream Joins
In this paper, we present an adaptive load diffusion operator to enable scalable processing of multiway windowed stream joins (MWSJs) using a cluster system. The load diffusion is achieved by a set of novel semantics-pre serving tuple routing algorithms. Different from previous work, the load diffusion operator can (1) preserve the MWSJ semantics while spreading tuples to different hosts for parallel join processing; (2) achieve fine-grained load balancing among distributed hosts; and (3) perform semantics-preserving online adaptations to maintain optimal performance in dynamic stream environments. We have implemented a prototype of the distributed MWSJ framework on top of the System S distributed stream processing system. Our experiment results based on both real data streams and synthetic workloads show that the load diffusion algorithms can efficiently scale-up the performance of MWSJ processing with low overhead.
2129781
ATLaS: A Native Extension of SQL for Data Mining
A lack of power and extensibility in their query languages has seriously limited the generality of DBMSs and hampered their ability to support new application domains. Considerable efforts by database researchers and commercial DBMS vendors have led to major extensions; yet there remain important applications—particularly data mining—that are not supported well in SQL-3. Thus, there is a pressing need for more general mechanisms for extending SQL and dealing with new application areas, particularly database-centric data mining. To satisfy this need, we allow database users to add new table functions and stream-oriented aggregate functions by defining them in SQL—rather than in external procedural languages as O-R DBMSs currently do. This simple extension turns SQL into a powerful database language, which can express a wide range of applications, including recursive queries, ROLAP aggregates, time-series queries, stream-oriented processing, and data mining functions. In addition to adding great power and flexibility to SQL, these extensions are conducive to performance and data independence. The paper also discusses the system we have developed to support these SQL extensions, and the architecture and techniques used in its realization.
5368193
Progressive modeling
Presently, inductive learning is still performed in a frustrating batch process. The user has little interaction with the system and no control over the final accuracy and training time. If the accuracy of the produced model is too low, all the computing resources are misspent. In this paper we propose a progressive modeling framework. In progressive modeling, the learning algorithm estimates online both the accuracy of the final model and remaining training time. If the estimated accuracy is far below expectation, the user can terminate training prior to completion without wasting further resources. If the user chooses to complete the learning process, progressive modeling will compute a model with expected accuracy in expected time. We describe one implementation of progressive modeling using ensemble of classifiers.
41722021
Correction of bias for motion estimation algorithms
Ego motion can be estimated by epipolar equations from point correspondences using statistically unbiased method. Such technique involves a positive weight being added to the fraction of residuals. The side effect of introduction of positive weights is that additional computation is required. We propose three algorithms: Two are derived from the statistically unbiased method, the third one has been derived based on minimising the error defined as distance between the image point and the corresponding epipolar line. It is proven that the errors are Gaussian distributed and comparable performance is achieved without the positive weights. Verified with experiments using real and synthetic data, the findings agree with the theory and the new algorithms outperform analytical solutions.
7336686
Online aggregation
Aggregation in traditional database systems is performed in batch mode: a query is submitted, the system processes a large volume of data over a long period of time, and, eventually, the final answer is returned. This archaic approach is frustrating to users and has been abandoned in most other areas of computing. In this paper we propose a new online aggregation interface that permits users to both observe the progress of their aggregation queries and control execution on the fly. After outlining usability and performance requirements for a system supporting online aggregation, we present a suite of techniques that extend a database system to meet these requirements. These include methods for returning the output in random order, for providing control over the relative rate at which different aggregates are computed, and for computing running confidence intervals. Finally, we report on an initial implementation of online aggregation in POSTGRES.
38802324
Experiments with improved approximate mean value analysis algorithms
Abstract Approximate mean value analysis (MVA) is a popular technique for analyzing queueing networks because of the efficiency and accuracy that it affords. In this paper, we present a new software package, called the improved approximate mean value analysis library (IAMVAL) , which can be easily integrated into existing commercial and research queueing network analysis packages. The IAMVAL packages include two new approximate MVA algorithms, the queue line (QL) algorithm and the fraction line (FL) algorithm, for analyzing multiple class separable queueing networks. The QL algorithm is always more accurate than, and yet has approximately the same computational efficiency as, the Bard–Schweitzer proportional estimation (PE) algorithm, which is currently the most widely used approximate MVA algorithm. The FL algorithm has the same computational cost and, in noncongested separable queueing networks where queue lengths are quite small, yields more accurate solutions than both the QL and PE algorithms.
2846401
Subgraph Join: Efficient Processing Subgraph Queries on Graph-Structured XML Document
The information in many applications can be naturally represented as graph-structured XML document. Structural query on graph structured XML document matches the subgraph of graph structured XML document on some given schema. The query processing of graph-structured XML document brings new challenges. 
 
In this paper, for the processing of subgraph query, we design a subgraph join algorithm based on reachability coding. Using efficient data structure, subgraph join algorithm can process subgraph query with various structures efficiently.
783838
Practical Indexing XML Document for Twig Query
Answering structural queries of XML with index is an important approach of efficient XML query processing. Among existing structural indexes for XML data, F&B index is the smallest index that can answer all branching queries. However, an F&B index for less regular XML data often contains a large number of index nodes, and hence a large amount of main memory. If the F&B index cannot be accommodated in the available memory, its performance will degrade significantly. This issue has practically limited wider application of the F&B index. 
 
In this paper, we propose a disk organization method for the F&B index which shift part of the leave nodes in the F&B index to the disk and organize them judiciously on the disk. Our method is based on the observation that the majority of the nodes in a F&B index is often the leaf nodes, yet their access frequencies are not high. 
 
We select some leaves to output to disk. With the support of reasonable storage structure in main memory and in disk, we design efficient query processing method). We further optimize the design of the F&B index based on the query workload. Experimental results verified the effectiveness of our proposed approach.