2370634
Pattern-based similarity search for microarray data
One fundamental task in near-neighbor search as well as other similarity matching efforts is to find a distance function that can efficiently quantify the similarity between two objects in a meaningful way. In DNA microarray analysis, the expression levels of two closely related genes may rise and fall synchronously in response to a set of experimental stimuli. Although the magnitude of their expression levels may not be close, the patterns they exhibit can be very similar. Unfortunately, none of the conventional distance metrics such as the Lp norm can model this similarity effectively. In this paper, we study the near-neighbor search problem based on this new type of similarity. We propose to measure the distance between two genes by subspace pattern similarity, i.e., whether they exhibit a synchronous pattern of rise and fall on a subset of dimensions. We then present an efficient algorithm for subspace near-neighbor search based on pattern similarity distance, and we perform tests on various data sets to show its effectiveness.
5103963
Adaptive Load Diffusion for Multiway Windowed Stream Joins
In this paper, we present an adaptive load diffusion operator to enable scalable processing of multiway windowed stream joins (MWSJs) using a cluster system. The load diffusion is achieved by a set of novel semantics-pre serving tuple routing algorithms. Different from previous work, the load diffusion operator can (1) preserve the MWSJ semantics while spreading tuples to different hosts for parallel join processing; (2) achieve fine-grained load balancing among distributed hosts; and (3) perform semantics-preserving online adaptations to maintain optimal performance in dynamic stream environments. We have implemented a prototype of the distributed MWSJ framework on top of the System S distributed stream processing system. Our experiment results based on both real data streams and synthetic workloads show that the load diffusion algorithms can efficiently scale-up the performance of MWSJ processing with low overhead.
14876394
User Defined Aggregates for Logical Data Languages
A new wave of data-intensive and knowledge-based applications|such as data mining and decision support|require the introduction of complex application-speciic aggregate functions. In this paper, we propose extensions for deductive database systems to support these new applications. We develop constructs, formal semantics, and implementation techniques for user-deened aggregates, and describe their realization in an extended LDL++ system recently built at UCLA. With these extensions, the system can support on-line aggregation, roll-ups for data cubing, temporal aggregates for time-series, iceberg queries, and other recently proposed operators used in decision support and data mining procedures. We then discuss the application of this technology to other DBMSs, and in particular to the SQL3 speciications that support the notion of user-deened aggregates. We show that SQL3 suuers from limitations that severely restrict its use in new applications; thus we propose simple extensions similar to those used for LDL++ to overcome such limitations.
1498767
Clustering by pattern similarity in large data sets
Clustering is the process of grouping a set of objects into classes of similar objects. Although definitions of similarity vary from one clustering model to another, in most of these models the concept of similarity is based on distances, e.g., Euclidean distance or cosine distance. In other words, similar objects are required to have close values on at least a set of dimensions. In this paper, we explore a more general type of similarity. Under the pCluster model we proposed, two objects are similar if they exhibit a coherent pattern on a subset of dimensions. For instance, in DNA microarray analysis, the expression levels of two genes may rise and fall synchronously in response to a set of environmental stimuli. Although the magnitude of their expression levels may not be close, the patterns they exhibit can be very much alike. Discovery of such clusters of genes is essential in revealing significant connections in gene regulatory networks. E-commerce applications, such as collaborative filtering, can also benefit from the new model, which captures not only the closeness of values of certain leading indicators but also the closeness of (purchasing, browsing, etc.) patterns exhibited by the customers. Our paper introduces an effective algorithm to detect such clusters, and we perform tests on several real and synthetic data sets to show its effectiveness.
2129781
ATLaS: A Native Extension of SQL for Data Mining
A lack of power and extensibility in their query languages has seriously limited the generality of DBMSs and hampered their ability to support new application domains. Considerable efforts by database researchers and commercial DBMS vendors have led to major extensions; yet there remain important applications—particularly data mining—that are not supported well in SQL-3. Thus, there is a pressing need for more general mechanisms for extending SQL and dealing with new application areas, particularly database-centric data mining. To satisfy this need, we allow database users to add new table functions and stream-oriented aggregate functions by defining them in SQL—rather than in external procedural languages as O-R DBMSs currently do. This simple extension turns SQL into a powerful database language, which can express a wide range of applications, including recursive queries, ROLAP aggregates, time-series queries, stream-oriented processing, and data mining functions. In addition to adding great power and flexibility to SQL, these extensions are conducive to performance and data independence. The paper also discusses the system we have developed to support these SQL extensions, and the architecture and techniques used in its realization.
5368193
Progressive modeling
Presently, inductive learning is still performed in a frustrating batch process. The user has little interaction with the system and no control over the final accuracy and training time. If the accuracy of the produced model is too low, all the computing resources are misspent. In this paper we propose a progressive modeling framework. In progressive modeling, the learning algorithm estimates online both the accuracy of the final model and remaining training time. If the estimated accuracy is far below expectation, the user can terminate training prior to completion without wasting further resources. If the user chooses to complete the learning process, progressive modeling will compute a model with expected accuracy in expected time. We describe one implementation of progressive modeling using ensemble of classifiers.
4332805
Stable egress route selection for interdomain traffic engineering: model and analysis
We present a general model of interdomain route selection to study interdomain traffic engineering. In this model, the routing of multiple destinations can be coordinated. Thus the model can capture general traffic engineering behaviors such as load balancing and link capacity constraints. We first identify potential routing instability and inefficiency of interdomain traffic engineering. We then derive a sufficient condition to guarantee convergence. We also show that the constraints on local policies imposed by business considerations in the Internet can guarantee stability without global coordination. Using realistic Internet topology, we evaluate the extent to which routing instability of interdomain traffic engineering can happen when the constraints are violated
1302015
An evaluation of scalable application-level multicast built using peer-to-peer overlays
Structured peer-to-peer overlay networks such as CAN, Chord, Pastry, and Tapestry can be used to implement Internet-scale application-level multicast. There are two general approaches to accomplishing this: tree building and flooding. This paper evaluates these two approaches using two different types of structured overlay: 1) overlays which use a form of generalized hypercube routing, e.g., Chord, Pastry and Tapestry, and 2) overlays which use a numerical distance metric to route through a Cartesian hyperspace, e.g., CAN. Pastry and CAN are chosen as the representatives of each type of overlay. To the best of our knowledge, this paper reports the first head-to-head comparison of CAN-style versus Pastry-style overlay networks, using multicast communication workloads running on an identical simulation infrastructure. The two approaches to multicast are independent of overlay network choice, and we provide a comparison of flooding versus tree-based multicast on both overlays. Results show that the tree-based approach consistently outperforms the flooding approach. Finally, for tree-based multicast, we show that Pastry provides better performance than CAN.
2060937
Efficient prediction algorithm of integer DCT coefficients for H.264/AVC optimization
This paper presents a novel efficient prediction algorithm to reduce redundant discrete cosine transform (DCT) and quantization computations for H.264 encoding optimization. A theoretical analysis is performed to study the sufficient condition for DCT coefficients to be quantized to zeros. As a result, three sufficient conditions corresponding to three types of transform and quantization methods in H.264 are proposed. Compared with other algorithms in the literature, the proposed algorithm derives more precise and efficient conditions to predict zero quantized DCT coefficients. Both the theoretical analysis and experimental results demonstrate that the proposed algorithm is superior to other algorithms in terms of the computational complexity reduction, encoded video quality, false acceptance rate, and false rejection rate.
15731582
Effective Appearance Model and Similarity Measure for Particle Filtering and Visual Tracking
In this paper, we adaptively model the appearance of objects based on Mixture of Gaussians in a joint spatial-color space (the approach is called SMOG). We propose a new SMOG-based similarity measure. SMOG captures richer information than the general color histogram because it incorporates spatial layout in addition to color. This appearance model and the similarity measure are used in a framework of Bayesian probability for tracking natural objects. In the second part of the paper, we propose an Integral Gaussian Mixture (IGM) technique, as a fast way to extract the parameters of SMOG for target candidate. With IGM, the parameters of SMOG can be computed efficiently by using only simple arithmetic operations (addition, subtraction, division) and thus the computation is reduced to linear complexity. Experiments show that our method can successfully track objects despite changes in foreground appearance, clutter, occlusion, etc.; and that it outperforms several color-histogram based methods.
15210668
A Music Retrieval System Based on Query-by-Singing for Karaoke Jukebox
This paper investigates the problem of retrieving Karaoke music by singing. The Karaoke music encompasses two audio channels in each track: one is a mix of vocal and background accompaniment, and the other is composed of accompaniment only. The accompaniments in the two channels often resemble each other, but are not identical. This characteristic is exploited to infer the vocal’s background music from the accompaniment-only channel, so that the main melody underlying the vocal signals can be extracted more effectively. To enable an efficient and accurate search for a large music database, we propose a phrase onset detection method based on Bayesian Information Criterion (BIC) for predicting the most likely beginning of a sung query, and adopt a multiple-level multiple-pass Dynamic Time Warping (DTW) for melody similarity comparison. The experiments conducted on a Karaoke database consisting of 1,071 popular songs show the promising results of query-by-singing retrieval for Karaoke music.