37278978
PC-Filter: A Robust Filtering Technique for Duplicate Record Detection in Large Databases
In this paper, we will propose PC-Filter (PC stands for Partitio n Comparison), a robust data filter for approximately duplicate record detection in large databases. PC-Filter distinguishes itself from all of existing methods by using the notion of partition in duplicate detection. It first sorts the whole database and splits the sorted database into a number of record partitions. The Partition Comparison Graph (PCG) is then constructed by performing fast partition pruning. Finally, duplicate records are effectively detected by using internal and external partition comparison based on PCG. Four properties, used as heuristics, have been devised to achieve a remarkable efficiency of the filter based on triangle inequity of record similarity. PC-Filter is insensitive to the key used to sort the database, and can achieve a very good recall level that is comparable to that of the pair-wise record comparison method but only with a complexity of O(N 4/3). Equipping existing detection methods with PC-Filter, we are able to well solve the ”Key Selection” problem, the ”Scope Specification” problem and the ”Low Recall” problem that the existing methods suffer from.
15427261
Anomaly detection in high-dimensional network data streams: A case study
In this paper, we study the problem of anomaly detection in high-dimensional network streams. We have developed a new technique, called Stream Projected Outlier deTector (SPOT), to deal with the problem of anomaly detection from high-dimensional data streams. We conduct a case study of SPOT in this paper by deploying it on 1999 KDD Intrusion Detection application. Innovative approaches for training data generation, anomaly classification and false positive reduction are proposed in this paper as well. Experimental results demonstrate that SPOT is effective in detecting anomalies from network data streams and outperforms existing anomaly detection methods.
11328445
Gene Specific Co-regulation Discovery: An Improved Approach
Discovering gene co-regulatory relationships is a new but important research problem in DNA microarray data analysis. The problem of gene specific co-regulation discovery is to, for a particular gene of interest, called the target gene , identify its strongly co-regulated genes and the condition subsets where such strong gene co-regulations are observed. The study on this problem can contribute to a better understanding and characterization of the target gene. The existing method, using the genetic algorithm (GA), is slow due to its expensive fitness evaluation and long individual representation. In this paper, we propose an improved method for finding gene specific co-regulations. Compared with the current method, our method features a notably improved efficiency. We employ k NN Search Table to substantially speed up fitness evaluation in the GA. We also propose a more compact representation scheme for encoding individuals in the GA, which contributes to faster crossover and mutation operations. Experimental results with a real-life gene microarray data set demonstrate the improved efficiency of our technique compared with the current method.
4732400
XAR-miner: efficient association rules mining for XML data
In this paper, we propose a framework, called XAR-Miner, for mining ARs from XML documents efficiently. In XAR-Miner, raw data in the XML document are first preprocessed to transform to either an Indexed Content Tree (IX-tree) or Multi-relational databases (Multi-DB), depending on the size of XML document and memory constraint of the system, for efficient data selection and AR mining. Task-relevant concepts are generalized to produce generalized meta-patterns, based on which the large ARs that meet the support and confidence levels are generated.
16517186
A Color Indexing Scheme Using Two-level Clustering Processing for Effective and Efficient Image Retrieval
[Abstract]: In this paper, we present a clustering-based color indexing scheme for effective amd efficient image retrieval, which is essentially an exploration on the application of clustering technique to image retrieval. In our approach, the color features are clustered automatically using a color clustering algorithm twice (called two-level clustering processing) and two color feature summarizations are obtained, e.g. Local Color Centriods (LCCs) and Global Color Centroids (GCCs). Based upon LCCs and GCCs, a three-level R-tree is bulit for indexing database images and performing effecitve and efficient image retrieval. The experiments show that this indexing scheme is effective and efficient in performing image retrieval.
34332782
Generalized Semi-Infinite Optimization and its Application in Robotics' Path Planning Problem
46405724
A Novel Approach Using PCA and SVM for Face Detection
Nowadays, face detection and recognition have gained importance in security and information access. In this paper, an efficient method of face detection based on principal components analysis (PCA) and support vector machine (SVM) is proposed. It firsly filter the face potential area using statistical feature which is generated by analyzing local histogram distribution. And then, SVM classifier is used to detect face feature in the test image, SVM has great performance in classification task. PCA is used to reduce dimension of sample data. After PCA transform, the feature vectors, which are used for training SVM classifier, are generated. Our tests in this paper are based on CMU face database. The experimental results demonstrate that the proposed method is encouraging with a successful detection rate.
47115701
An Adaptive Selection of Motion for Online Hand-Eye Calibration
As the robot makes unplanned movement, online hand-eye calibration determines the relative pose between the robot gripper/end-effector and the sensors mounted on it. With noisy measurements, hand-eye calibration is sensitive to small rotations in real applications. Moreover, degenerate cases such as pure translations have no effect in hand-eye calibration. This paper proposes an adaptive motion selection algorithm for online hand-eye calibration, which can adaptively set the thresholds of motion selection according to the characteristics of the unplanned motion sequence. It is achieved by using polynomial-regression to predict the relationship between RMS of calibration error and thresholds. Thus, this procedure leads to an adaptive method of motion selection. It can adapt itself to online hand-eye calibration in various applications. Experiments using simulated data are conducted and present good results. Experiments using real scenes also show that the method is promising.
5904664
A Novel Approach of Face Detection based on Skin Color Segmentation and PCA
Nowadays, face detection and recognition have gained importance in security and information access. In this paper, an efficient method of face detection based on skin color segmentation and principal components analysis(PCA) is proposed. Firstly, segmenting image using color model to filter candidate faces roughly; And then Eye-analogue segments at a given scale are discovered by finding regions which are darker than their neighborhoods to filter candidate faces farther; at the end, PCA method is used to extract the relevant information in human faces.
9382236
A Task-Based Approach to User Interface Design for a Web-Based Hydrologic Information Systems
Hydroinformatics is a new and rapidly developing field that integrates knowledge and understanding of water resources with the latest developments in information technology to improve decision-making in many critical applications. It encompasses methods for data capture, storage, processing, analysis and visualization, and the use of advanced modeling, simulation, optimization and knowledge-based tools and systems infrastructure. Three types of hydrological data are most commonly used: flow rate in major rivers and streams, height of water in wells, and precipitation. To get a complete view of the state of water at a given point in space and time, one must analyze many different types of hydrological data together to derive information using an online GIS tool. To help use these disparate data sources more effectively and efficiently, we have built an online interface called the IJEDI WebCenter for Hydroinformatics using a task-based approach. In this design, we first identify the tasks that users perform to study water-related issues, then organize data for each task, and build task-specific tools to present and analyze data and information. In a study involving both novices and experts in hydrology, we found that the both groups performed water-related studies more effectively and efficiently than they would have without the WebCenter.
17409111
Real-time Competition Simulation System
In tennis competition, there are some reasons leading to the competition results inaccurately. So I put forward a real-time competition simulation system to solve the tennis problem. It can overcome the limitations and the blind spots that occur in human observation. The system includes four parts: image collection, moving object detection and tracking, static scene simulation, competition simulation. This paper presents an algorithm for the detection and tracking of moving objects in sequence images that can be applied in simulations of tennis competition. The approach differs from most other methods that solve the problems of object occlusion and object deformation. The experimental results proved that our method is feasibility and usefulness.