28636420
Efficient computation of a proximity matching in spatial databases
Abstract Spatial data mining recently emerges from a number of real applications, such as real-estate marketing, urban planning, weather forecasting, medical image analysis, road traffic accident analysis, etc. It demands for efficient solutions for many new, expensive, and complicated problems. In this paper, we investigate a proximity matching problem among clusters and features. The investigation involves proximity relationship measurement between clusters and features. We measure proximity in an average fashion to address possible non-uniform data distribution in a cluster. An efficient algorithm is proposed and evaluated to solve the problem. The algorithm applies a standard multi-step paradigm in combining with novel lower and upper proximity bounds. The algorithm is implemented in several different modes. Our experiment results not only give a comparison among them but also illustrate the efficiency of the algorithm.
7595419
Efficient Computation of k-Medians over Data Streams Under Memory Constraints
In this paper, we study the problem of efficiently computing k-medians over high-dimensional and high speed data streams. The focus of this paper is on the issue of minimizing CPU time to handle high speed data streams on top of the requirements of high accuracy and small memory. Our work is motivated by the following observation: the existing algorithms have similar approximation behaviors in practice, even though they make noticeably different worst case theoretical guarantees. The underlying reason is that in order to achieve high approximation level with the smallest possible memory, they need rather complex techniques to maintain a sketch, along time dimension, by using some existing off-line clustering algorithms. Those clustering algorithms cannot guarantee the optimal clustering result over data segments in a data stream but accumulate errors over segments, which makes most algorithms behave the same in terms of approximation level, in practice. We propose a new grid-based approach which divides the entire data set into cells (not along time dimension). We can achieve high approximation level based on a novel concept called (1−∊)-dominant. We further extend the method to the data stream context, by leveraging a density-based heuristic and frequent item mining techniques over data streams. We only need to apply an existing clustering once to computing k-medians, on demand, which reduces CPU time significantly. We conducted extensive experimental studies, and show that our approaches outperform other well-known approaches.
14356230
Delay Optimizations in Quorum Consensus
The management of replicated data in distributed database systems is a classic problem with great practical importance. Quorum consensus is one of the popular methods, combining with eager replication, for managing replicated data. In this paper, we will investigate the problems of delay-optimal quorum consensus. Firstly, we will show that the problem of minimizing the total-delay (or mean-delay) is NP-hard. However, we can show that the problem restricted to some specific network topology, such as trees, rings, and meshes, can be solved in polynomial time. We also developed an approximate algorithm for a general case. The algorithm gives an approximate ratio less than 2. Secondly, we will present an efficient algorithm, based on the dynamic programming technique, to solve the problem of minimizing the maximal-delay.
6221918
Efficiently Matching Proximity Relationships in Spatial Databases
Spatial data mining recently emerges from a number of real applications, such as real-estate marketing, urban planning, weather forecasting, medical image analysis, road traffic accident analysis, etc. It demands for efficient solutions for many new, expensive, and complicated problems. In this paper, we investigate a proximity matching problem among clusters and features. The investigation involves proximity relationship measurement between clusters and features. We measure proximity in an average fashion to address possible nonuniform data distribution in a cluster. An efficient algorithm, for solving the problem, is proposed and evaluated. The algorithm applies a standard multistep paradigm in combining with novel lower and upper proximity bounds. The algorithm is implemented in several different modes. Our experiment results do not only give a comparison among them but also illustrate the efficiency of the algorithm.
1993887
Locating Motifs in Time-Series Data
Finding motifs in time-series is proposed to make clustering of time-series subsequences meaningful, because most existing algorithms of clustering time-series subsequences are reported meaningless in recent studies. The existing motif finding algorithms emphasize the efficiency at the expense of quality, in terms of the number of time-series subsequences in a motif and the total number of motifs found. In this paper, we formalize the problem as a continuous top-k motif balls problem in an m-dimensional space, and propose heuristic approaches that can significantly improve the quality of motifs with reasonable overhead, as shown in our experimental studies.
2460253
Approximate processing of massive continuous quantile queries over high-speed data streams
Quantile computation has many applications including data mining and financial data analysis. It has been shown that an /spl epsi/-approximate summary can be maintained so that, given a quantile query (/spl phi/,/spl epsi/), the data item at rank /spl lceil//spl phi/N/spl rceil/ may be approximately obtained within the rank error precision /spl epsi/N over all N data items in a data stream or in a sliding window. However, scalable online processing of massive continuous quantile queries with different /spl phi/ and /spl epsi/ poses a new challenge because the summary is continuously updated with new arrivals of data items. In this paper, first we aim to dramatically reduce the number of distinct query results by grouping a set of different queries into a cluster so that they can be processed virtually as a single query while the precision requirements from users can be retained. Second, we aim to minimize the total query processing costs. Efficient algorithms are developed to minimize the total number of times for reprocessing clusters and to produce the minimum number of clusters, respectively. The techniques are extended to maintain near-optimal clustering when queries are registered and removed in an arbitrary fashion against whole data streams or sliding windows. In addition to theoretical analysis, our performance study indicates that the proposed techniques are indeed scalable with respect to the number of input queries as well as the number of items and the item arrival rate in a data stream.
1998178
A Hierarchical Approach to Internet Distance Prediction
Internet distance prediction gives pair-wise latency information with limited measurements. Recent studies have revealed that the quality of existing prediction mechanisms from the application perspective is short of satisfactory. In this paper, we explore the root causes and remedies for this problem. Our experience with different landmark selection schemes shows that although selecting nearby landmarks can increase the prediction accuracy for short distances, it can cause the prediction accuracy for longer distances to degrade. Such uneven prediction quality significantly impacts application performance. Instead of trying to select the landmark nodes in some "intelligent" fashion, we propose a hierarchical prediction approach with straightforward landmark selection. Hierarchical prediction utilizes multiple coordinate sets at multiple distance scales, with the "right" scale being chosen for prediction each time. Experiments with Internet measurement datasets show that this hierarchical approach is extremely promising for increasing the accuracy of network distance prediction.
17019459
Semantic video classification by integrating flexible mixture model with adaptive EM algorithm
Digital video now plays an important role in medical education and healthcare, but our ability to automatic video indexing at semantic level is currently primitive. In this paper, we propose a novel framework to enable more effective semantic video classification and indexing in a specific surgery education video domain. Specifically, this framework includes: (a) A novel semantic-sensitive video content characterization and representation framework by using principal video shots and their perceptual multimodal features. (b) A novel semantic medical concept interpretation technique by using flexible mixture model. (c) A semantic video classifier by using an adaptive Expectation-Maximization (EM) algorithm for automatic parameter estimation and model selection (i.e., selecting the optimal number of mixture Gaussian components). Since more effective video content characterization framework has been integrated with an adaptive EM algorithm for video classification, our semantic video classifier has improved the classification accuracy significantly. For skin classification, its accuracy is close to 95.5%. For semantic surgical video classification, it achieves overall ≈ 84.6% accuracy.
12662254
Creating digital libraries: content generation and re-mastering
This article has two main goals: to describe the automatic creation of a digital library and to provide an overview of the metaalgorithmic patterns that can be applied to increase the accuracy of its creation. Automating the creation of useful digital libraries - that is, digital libraries affording searchable text and reusable ("repurposable") output - is a complicated process, whether the original library is paper-based or already available in electronic form. We outline the steps involved in the creation of a deployable digital library (>1.2 /spl times/ 10/sup 6/ pages) for MIT Press, as well as its implications to other aspects of digital library creation, management, use and repurposing. Input, transformation, information extraction, and output processes are considered in light of their utility in creating layers of content. Interestingly, in some aspects, scanning directly from paper offers extra opportunities for error-checking through feedback-feedforward combination. Strategies for quality assurance (QA) at the document, chapter and book level are also discussed. We emphasize the use of metaalgorithmic design patterns for application towards improving the content generation, extraction and remastering. This approach also increases the ease with which modules and algorithms are added to and deprecated from the system.
13382977
Quality assurance in high volume document digitization: a survey
Quality assurance (QA) plays a critical role in high volume document digitization projects by making sure that the specified quality standard is reached under cost and time constraints. This paper takes a systematic view on this issue by summarizing and abstracting related existing work: quality bottlenecks and technical solutions throughout the whole processing pipeline, including cataloging, capture, image analysis and recognition, and error cascading; various strategies to conduct cost-effective QA, such as combination of auto-QA and manual QA, batch QA, special QA user interface, and open source QA
2250003
ICDAR 2003 robust reading competitions: entries, results, and future directions
Abstract.This paper describes the robust reading competitions for ICDAR 2003. With the rapid growth in research over the last few years on recognizing text in natural scenes, there is an urgent need to establish some common benchmark datasets and gain a clear understanding of the current state of the art. We use the term ‘robust reading’ to refer to text images that are beyond the capabilities of current commercial OCR packages. We chose to break down the robust reading problem into three subproblems and run competitions for each stage, and also a competition for the best overall system. The subproblems we chose were text locating, character recognition and word recognition. By breaking down the problem in this way, we hoped to gain a better understanding of the state of the art in each of the subproblems. Furthermore, our methodology involved storing detailed results of applying each algorithm to each image in the datasets, allowing researchers to study in depth the strengths and weaknesses of each algorithm. The text-locating contest was the only one to have any entries. We give a brief description of each entry and present the results of this contest, showing cases where the leading entries succeed and fail. We also describe an algorithm for combining the outputs of the individual text locators and show how the combination scheme improves on any of the individual systems.