60677377
Towards outlier detection for high-dimensional data streams using projected outlier analysis strategy
Outlier detection is an important research problem in data mining that aims to discover useful abnormal and irregular patterns hidden in large data sets. Most existing outlier detection methods only deal with static data with relatively low dimensionality. Recently, outlier detection for high-dimensional stream data became a new emerging research problem. A key observation that motivates this research is that outliers in high-dimensional data are projected outliers, i.e., they are embedded in lower-dimensional subspaces. Detecting projected outliers from high-dimensional stream data is a very challenging task for several reasons. First, detecting projected outliers is difficult even for high-dimensional static data. The exhaustive search for the outlying subspaces where projected outliers are embedded is a NP problem. Second, the algorithms for handling data streams are constrained to take only one pass to process the streaming data with the conditions of space limitation and time criticality. The currently existing methods for outlier detection are found to be ineffective for detecting projected outliers in high-dimensional data streams. 
In this thesis, we present a new technique, called the Stream Project Outlier deTector (SPOT), which attempts to detect projected outliers in high-dimensional data streams. SPOT employs an innovative window-based time model in capturing dynamic statistics from stream data, and a novel data structure containing a set of top sparse subspaces to detect projected outliers effectively. SPOT also employs a multi-objective genetic algorithm as an effective search method for finding the outlying subspaces where most projected outliers are embedded. The experimental results demonstrate that SPOT is efficient and effective in detecting projected outliers for high-dimensional data streams. The main contribution of this thesis is that it provides a backbone in tackling the challenging problem of outlier detection for high-dimensional data streams. SPOT can facilitate the discovery of useful abnormal patterns and can be potentially applied to a variety of high demand applications, such as for sensor network data monitoring, online transaction protection, etc.
16517186
A Color Indexing Scheme Using Two-level Clustering Processing for Effective and Efficient Image Retrieval
[Abstract]: In this paper, we present a clustering-based color indexing scheme for effective amd efficient image retrieval, which is essentially an exploration on the application of clustering technique to image retrieval. In our approach, the color features are clustered automatically using a color clustering algorithm twice (called two-level clustering processing) and two color feature summarizations are obtained, e.g. Local Color Centriods (LCCs) and Global Color Centroids (GCCs). Based upon LCCs and GCCs, a three-level R-tree is bulit for indexing database images and performing effecitve and efficient image retrieval. The experiments show that this indexing scheme is effective and efficient in performing image retrieval.
11328445
Gene Specific Co-regulation Discovery: An Improved Approach
Discovering gene co-regulatory relationships is a new but important research problem in DNA microarray data analysis. The problem of gene specific co-regulation discovery is to, for a particular gene of interest, called the target gene , identify its strongly co-regulated genes and the condition subsets where such strong gene co-regulations are observed. The study on this problem can contribute to a better understanding and characterization of the target gene. The existing method, using the genetic algorithm (GA), is slow due to its expensive fitness evaluation and long individual representation. In this paper, we propose an improved method for finding gene specific co-regulations. Compared with the current method, our method features a notably improved efficiency. We employ k NN Search Table to substantially speed up fitness evaluation in the GA. We also propose a more compact representation scheme for encoding individuals in the GA, which contributes to faster crossover and mutation operations. Experimental results with a real-life gene microarray data set demonstrate the improved efficiency of our technique compared with the current method.
15427261
Anomaly detection in high-dimensional network data streams: A case study
In this paper, we study the problem of anomaly detection in high-dimensional network streams. We have developed a new technique, called Stream Projected Outlier deTector (SPOT), to deal with the problem of anomaly detection from high-dimensional data streams. We conduct a case study of SPOT in this paper by deploying it on 1999 KDD Intrusion Detection application. Innovative approaches for training data generation, anomaly classification and false positive reduction are proposed in this paper as well. Experimental results demonstrate that SPOT is effective in detecting anomalies from network data streams and outperforms existing anomaly detection methods.
4732400
XAR-miner: efficient association rules mining for XML data
In this paper, we propose a framework, called XAR-Miner, for mining ARs from XML documents efficiently. In XAR-Miner, raw data in the XML document are first preprocessed to transform to either an Indexed Content Tree (IX-tree) or Multi-relational databases (Multi-DB), depending on the size of XML document and memory constraint of the system, for efficient data selection and AR mining. Task-relevant concepts are generalized to produce generalized meta-patterns, based on which the large ARs that meet the support and confidence levels are generated.
37278978
PC-Filter: A Robust Filtering Technique for Duplicate Record Detection in Large Databases
In this paper, we will propose PC-Filter (PC stands for Partitio n Comparison), a robust data filter for approximately duplicate record detection in large databases. PC-Filter distinguishes itself from all of existing methods by using the notion of partition in duplicate detection. It first sorts the whole database and splits the sorted database into a number of record partitions. The Partition Comparison Graph (PCG) is then constructed by performing fast partition pruning. Finally, duplicate records are effectively detected by using internal and external partition comparison based on PCG. Four properties, used as heuristics, have been devised to achieve a remarkable efficiency of the filter based on triangle inequity of record similarity. PC-Filter is insensitive to the key used to sort the database, and can achieve a very good recall level that is comparable to that of the pair-wise record comparison method but only with a complexity of O(N 4/3). Equipping existing detection methods with PC-Filter, we are able to well solve the ”Key Selection” problem, the ”Scope Specification” problem and the ”Low Recall” problem that the existing methods suffer from.
2408711
A Mixture Model for Expert Finding
This paper addresses the issue of identifying persons with expertise knowledge on a given topic. Traditional methods usually estimate the relevance between the query and the support documents of candidate experts using, for example, a language model. However, the language model lacks the ability of identifying semantic knowledge, thus results in some right experts cannot be found due to not occurrence of the query terms in the support documents. In this paper, we propose a mixture model based on Probabilistic Latent Semantic Analysis (PLSA) to estimate a hidden semantic theme layer between the terms and the support documents. The hidden themes are used to capture the semantic relevance between the query and the experts. We evaluate our mixture model in a real-world system, ArnetMiner. Experimental results indicate that the proposed model outperforms the language models.
1821709
A Hand-Eye Robotic Model for Total Knee Replacement Surgery
This paper presents a hand-eye robotic model for total knee replacement (TKR) surgery. Unlike existent robot assisted TKR surgery, the proposed model is a surgical robot that combines with a movable hand-eye navigation system, which would use the full potential of both computer-assisted systems. Without using CT images and landmark pins in the patient's bones, it can directly measure the mechanical axis with high precision. This system provides a new approach of the minimally invasive surgery. Experiment results show that the proposed model is promising in the future application.
45190241
Factor analysis based anomaly detection
We propose a novel anomaly detection algorithm based on factor analysis and Mahalanobis distance. Factor analysis is used to uncover the latent structure (dimensions) of a set of variables. It reduces attribute space from a larger number of variables to a smaller number of factors. The Mahalanobis distance is used to determine the "similarity" of a set of values from an "unknown" sample to a set of values measured from a collection of "known" samples. Combined with factor analysis, Mahalanobis distance is extended to examine whether a given vector is an outlier from a model identified by "factors" based on factor analysis. We present a factor analysis-based network anomaly detection algorithm and apply it to DARPA intrusion detection evaluation data. The experimental results show that the proposed algorithm is able to detect network intrusions with relatively low false alarms.
3348552
ArnetMiner: extraction and mining of academic social networks
This paper addresses several key issues in the ArnetMiner system, which aims at extracting and mining academic social networks. Specifically, the system focuses on: 1) Extracting researcher profiles automatically from the Web; 2) Integrating the publication data into the network from existing digital libraries; 3) Modeling the entire academic network; and 4) Providing search services for the academic network. So far, 448,470 researcher profiles have been extracted using a unified tagging approach. We integrate publications from online Web databases and propose a probabilistic framework to deal with the name ambiguity problem. Furthermore, we propose a unified modeling approach to simultaneously model topical aspects of papers, authors, and publication venues. Search services such as expertise search and people association search have been provided based on the modeling results. In this paper, we describe the architecture and main features of the system. We also present the empirical evaluation of the proposed methods.
13312738
Autonomous Objects and Bottom-Up Composition in ZOO Applied to a Case Study of Biological Reactivity
As part of our work on the formal analysis of object-oriented models, we turn to systems where many autonomous individuals interact to give rise to complex collective behaviour. We adapt our ZOO [1,2] structuring and apply it to a case study based on a published model of part of the immune system [3]. The formalisation calls for a bottom-up solution with no central control over individual units, and includes an approach to represent feedback channelsenabling broadcast communication between individuals and across levels.