14038708
Efficiently computing frequent tree-like topology patterns in a Web environment
Data mining has recently emerged in a number of real life applications, such as banking, supermarketing, and interneting. The authors investigate a novel data mining problem derived from the applications of WWW. They present an efficient algorithm for solving the problem. The initial implementation suggests that the algorithm is very efficient and scalable in practice.
2460253
Approximate processing of massive continuous quantile queries over high-speed data streams
Quantile computation has many applications including data mining and financial data analysis. It has been shown that an /spl epsi/-approximate summary can be maintained so that, given a quantile query (/spl phi/,/spl epsi/), the data item at rank /spl lceil//spl phi/N/spl rceil/ may be approximately obtained within the rank error precision /spl epsi/N over all N data items in a data stream or in a sliding window. However, scalable online processing of massive continuous quantile queries with different /spl phi/ and /spl epsi/ poses a new challenge because the summary is continuously updated with new arrivals of data items. In this paper, first we aim to dramatically reduce the number of distinct query results by grouping a set of different queries into a cluster so that they can be processed virtually as a single query while the precision requirements from users can be retained. Second, we aim to minimize the total query processing costs. Efficient algorithms are developed to minimize the total number of times for reprocessing clusters and to produce the minimum number of clusters, respectively. The techniques are extended to maintain near-optimal clustering when queries are registered and removed in an arbitrary fashion against whole data streams or sliding windows. In addition to theoretical analysis, our performance study indicates that the proposed techniques are indeed scalable with respect to the number of input queries as well as the number of items and the item arrival rate in a data stream.
28636420
Efficient computation of a proximity matching in spatial databases
Abstract Spatial data mining recently emerges from a number of real applications, such as real-estate marketing, urban planning, weather forecasting, medical image analysis, road traffic accident analysis, etc. It demands for efficient solutions for many new, expensive, and complicated problems. In this paper, we investigate a proximity matching problem among clusters and features. The investigation involves proximity relationship measurement between clusters and features. We measure proximity in an average fashion to address possible non-uniform data distribution in a cluster. An efficient algorithm is proposed and evaluated to solve the problem. The algorithm applies a standard multi-step paradigm in combining with novel lower and upper proximity bounds. The algorithm is implemented in several different modes. Our experiment results not only give a comparison among them but also illustrate the efficiency of the algorithm.
1993887
Locating Motifs in Time-Series Data
Finding motifs in time-series is proposed to make clustering of time-series subsequences meaningful, because most existing algorithms of clustering time-series subsequences are reported meaningless in recent studies. The existing motif finding algorithms emphasize the efficiency at the expense of quality, in terms of the number of time-series subsequences in a motif and the total number of motifs found. In this paper, we formalize the problem as a continuous top-k motif balls problem in an m-dimensional space, and propose heuristic approaches that can significantly improve the quality of motifs with reasonable overhead, as shown in our experimental studies.
4018890
Clustering Moving Objects for Spatio-temporal Selectivity Estimation
Many spatio-temporal applications involve managing and querying moving objects. In such an environment, predictive spatio-temporal queries become an important query class to be processed to capture the nature of moving objects. In this paper, we investigated the problem of selectivity estimation for predictive spatio-temporal queries. We propose a novel histogram technique based on a clustering paradigm. To avoid expensive computation costs, we developed linear time heuristics to construct such a histogram. Our performance study indicated that the new techniques improve the accuracy of the existing techniques by one order of magnitude.
17501980
Term Graph Model for Text Classification
Most existing text classification methods (and text mining methods at large) are based on representing the documents using the traditional vector space model. We argue that important information, such as the relationship among words, is lost. We propose a term graph model to represent not only the content of a document but also the relationship among the keywords. We demonstrate that the new model enables us to define new similarity functions, such as considering rank correlation based on PageRank-style algorithms, for the classification purpose. Our preliminary results show promising results of our new model.
8765334
Privacy preserving regression modelling via distributed computation
Reluctance of data owners to share their possibly confidential or proprietary data with others who own related databases is a serious impediment to conducting a mutually beneficial data mining analysis. We address the case of vertically partitioned data -- multiple data owners/agencies each possess a few attributes of every data record. We focus on the case of the agencies wanting to conduct a linear regression analysis with complete records without disclosing values of their own attributes. This paper describes an algorithm that enables such agencies to compute the exact regression coefficients of the global regression equation and also perform some basic goodness-of-fit diagnostics while protecting the confidentiality of their data. In more general settings beyond the privacy scenario, this algorithm can also be viewed as method for the distributed computation for regression analyses.
8724659
Estimating device availability in pervasive peer-to-peer environment
In pervasive computing environments, devices often communicate in a peer-to-peer manner, either for passing messages or collaboratively running applications. For any particular device, information about the availability of other devices can lead to more efficient communication and better execution of distributed tasks. For example, a device can distribute computation tasks to reliable devices and temporarily avoid unreliable ones. Since there usually is not a central server to coordinate or monitor the communication in such environment, a key technical challenge is for each individual device to predict the availability of other devices. In this paper, we describe some methods for predicting the availability of other devices using historic availability data of those devices obtained in routine usage. In this scheme, each device separately maintains data about the past communication with the other devices, and predicts current and future availability using the statistics of those data. These methods do not require any centralized monitoring or extra probing, and thus have very low computation cost. These characteristics make them suitable for small devices in a peer-to-peer environment.
27372511
A comprehensive image processing suite for book re-mastering
Converting paper books into electronic form provides benefits for archiving, distribution and content reuse. However, directly scanned images are usually undesirable for electronic books, and automated content re-mastering is required. In this paper, we describe a comprehensive image processing suite consisting of three major components: 1) image enhancement with deskew, cropping, color correction, contrast enhancement and text sharpening, 2) compound document image compression, and 3) extraction of TOC (table of content) and linking. We built a processing pipeline that automatically converts a set of scanned page images into a high quality and highly compressed e-book in the popular PDF format.
13382977
Quality assurance in high volume document digitization: a survey
Quality assurance (QA) plays a critical role in high volume document digitization projects by making sure that the specified quality standard is reached under cost and time constraints. This paper takes a systematic view on this issue by summarizing and abstracting related existing work: quality bottlenecks and technical solutions throughout the whole processing pipeline, including cataloging, capture, image analysis and recognition, and error cascading; various strategies to conduct cost-effective QA, such as combination of auto-QA and manual QA, batch QA, special QA user interface, and open source QA
26474180
Adaptive confidence transform based classifier combination for Chinese character recognition
Abstract Classifier combination is an effective way to improve recognition performance. However, in Chinese character recognition the extremely large number of categories results in several difficulties for the combination. In order to overcome these difficulties a novel combination method is presented in this paper. It consists of three main components: adaptive confidence transform (ACT), consensus theoretic combination and reliability based speedup scheme. ACT, which can estimate a posteriori probabilities from raw measurement values, is the focus of this paper. Experimental results show a significant reduction of error rates in both printed (PCCR) and handwritten Chinese character recognition (HCCR).