8576217
Statistical Issues with Labeled Sample Size Analysis for Semi-Supervised Linear Discriminant Analysis
37278978
PC-Filter: A Robust Filtering Technique for Duplicate Record Detection in Large Databases
In this paper, we will propose PC-Filter (PC stands for Partitio n Comparison), a robust data filter for approximately duplicate record detection in large databases. PC-Filter distinguishes itself from all of existing methods by using the notion of partition in duplicate detection. It first sorts the whole database and splits the sorted database into a number of record partitions. The Partition Comparison Graph (PCG) is then constructed by performing fast partition pruning. Finally, duplicate records are effectively detected by using internal and external partition comparison based on PCG. Four properties, used as heuristics, have been devised to achieve a remarkable efficiency of the filter based on triangle inequity of record similarity. PC-Filter is insensitive to the key used to sort the database, and can achieve a very good recall level that is comparable to that of the pair-wise record comparison method but only with a complexity of O(N 4/3). Equipping existing detection methods with PC-Filter, we are able to well solve the ”Key Selection” problem, the ”Scope Specification” problem and the ”Low Recall” problem that the existing methods suffer from.
15427261
Anomaly detection in high-dimensional network data streams: A case study
In this paper, we study the problem of anomaly detection in high-dimensional network streams. We have developed a new technique, called Stream Projected Outlier deTector (SPOT), to deal with the problem of anomaly detection from high-dimensional data streams. We conduct a case study of SPOT in this paper by deploying it on 1999 KDD Intrusion Detection application. Innovative approaches for training data generation, anomaly classification and false positive reduction are proposed in this paper as well. Experimental results demonstrate that SPOT is effective in detecting anomalies from network data streams and outperforms existing anomaly detection methods.
16517186
A Color Indexing Scheme Using Two-level Clustering Processing for Effective and Efficient Image Retrieval
[Abstract]: In this paper, we present a clustering-based color indexing scheme for effective amd efficient image retrieval, which is essentially an exploration on the application of clustering technique to image retrieval. In our approach, the color features are clustered automatically using a color clustering algorithm twice (called two-level clustering processing) and two color feature summarizations are obtained, e.g. Local Color Centriods (LCCs) and Global Color Centroids (GCCs). Based upon LCCs and GCCs, a three-level R-tree is bulit for indexing database images and performing effecitve and efficient image retrieval. The experiments show that this indexing scheme is effective and efficient in performing image retrieval.
4732400
XAR-miner: efficient association rules mining for XML data
In this paper, we propose a framework, called XAR-Miner, for mining ARs from XML documents efficiently. In XAR-Miner, raw data in the XML document are first preprocessed to transform to either an Indexed Content Tree (IX-tree) or Multi-relational databases (Multi-DB), depending on the size of XML document and memory constraint of the system, for efficient data selection and AR mining. Task-relevant concepts are generalized to produce generalized meta-patterns, based on which the large ARs that meet the support and confidence levels are generated.
11328445
Gene Specific Co-regulation Discovery: An Improved Approach
Discovering gene co-regulatory relationships is a new but important research problem in DNA microarray data analysis. The problem of gene specific co-regulation discovery is to, for a particular gene of interest, called the target gene , identify its strongly co-regulated genes and the condition subsets where such strong gene co-regulations are observed. The study on this problem can contribute to a better understanding and characterization of the target gene. The existing method, using the genetic algorithm (GA), is slow due to its expensive fitness evaluation and long individual representation. In this paper, we propose an improved method for finding gene specific co-regulations. Compared with the current method, our method features a notably improved efficiency. We employ k NN Search Table to substantially speed up fitness evaluation in the GA. We also propose a more compact representation scheme for encoding individuals in the GA, which contributes to faster crossover and mutation operations. Experimental results with a real-life gene microarray data set demonstrate the improved efficiency of our technique compared with the current method.
10449221
C-SAW and genAWeave: a two-level aspect weaving toolsuite
This demonstration will feature overviews of the C-SAW and GenAWeave projects. The first half of the presentation will introduce the concept of two-level aspect weaving, which unites a <i>model transformation</i> tool with a <i>program transformation</i> engine. From models representing an avionics application, it will be shown how changes to model properties trigger corresponding adaptations to the related source code. The second half of the demonstration is focused on using a program transformation engine to perform the task of aspect weaving. In particular, several crosscutting concerns will be woven into an Object Pascal application.
16882534
Systematic benchmarking of microarray data feature extraction and classification
A combination of microarrays with classification methods is a promising approach to supporting clinical management decisions in oncology. The aim of this paper is to systematically benchmark the role of classification models. Each classification model is a combination of one feature extraction method and one classification method. We consider four feature extraction methods and five classification methods, from which 20 classification models can be derived. The feature extraction methods are t-statistics, non-parametric Wilcoxon statistics, ad hoc signal-to-noise statistics, and principal component analysis (PCA), and the classification methods are Fisher linear discriminant analysis (FLDA), the support vector machine (SVM), the k nearest-neighbour classifier (kNN), diagonal linear discriminant analysis (DLDA), and diagonal quadratic discriminant analysis (DQDA). Twenty randomizations of each of three binary cancer classification problems derived from publicly available datasets are examined. PCA plus FLDA is found to be the optimal classification model.
15782545
Rapid and Accurate Protein Side Chain Prediction with Local Backbone Information
High-accuracy protein structure modeling demands accurate and very fast side chain prediction since such a procedure must be repeatedly called at each step of structure refinement. Many known side chain prediction programs, such as SCWRL and TreePack, depend on the philosophy that global information and pairwise energy function must be used to achieve high accuracy. These programs are too slow to be used in the case when side chain packing has to be used thousands of times, such as protein structure refinement and protein design. 
 
We present an unexpected study that local backbone information can determine side chain conformations accurately. LocalPack, our side chain packing program which is based on only local information, achieves equal accuracy as SCWRL and TreePack, yet runs 4-14 times faster, hence providing a key missing piece in our efforts to high-accuracy protein structure modeling.
259035
Factor-analysis based anomaly detection and clustering
This paper presents a novel anomaly detection and clustering algorithm for the network intrusion detection based on factor analysis and Mahalanobis distance. Factor analysis is used to uncover the latent structure of a set of variables. The Mahalanobis distance is used to determine the "similarity" of a set of values from an "unknown" sample to a set of values measured from a collection of "known" samples. By utilizing factor analysis and Mahalanobis distance, we developed an algorithm 1) to identify outliers based on a trained model, and 2) to cluster attacks by abnormal features.
11446949
Composite Evaluation of Replacing Small Generation Units with Large Ones in the Electricity Sector in China
This paper applies two methods to evaluate the economic effects of the policy of "replacing small units with large ones (RSWL)," one is Discounted Cash Flow (DCF) method which performs a technology economic analysis both from the financial and the economic perspectives, the other one is to use the Real Option theory to study the lagging investment problem of the big unit. Through an illustrative example, three alternatives of replacing 200 MW units with 600 MW units are compared. The sensitivity analysis is made to evaluate the impact of uncertain factors on the financial and economic indicators of the execution of RSWL. The environmental costs are also taken into account in the comparison and analysis. The proposed methods can aid the decision-making in the implementation of the policy of RSWL from a wider perspective and it can be used to evaluate the economic benefits and costs of the RSWL projects.