4726034
D-GridMST: Clustering Large Distributed Spatial Databases
In this paper, we will propose a distributable clustering algorithm, called Distributed-GridMST (D-GridMST), which deals with large distributed spatial databases. D-GridMST employs the notions of multi-dimensional cube to partition the data space involved and uses density criteria to extract representative points from spatial databases, based on which a global MST of representatives is constructed. Such a MST is partitioned according to users’ clustering specification and used to label data points in the respective distributed spatial database thereafter. Since only the compact information of the distributed spatial databases is transferred via network, D-GridMST is characterized by small network transferring overhead. Experimental results show that D-GridMST is effective since it is able to produce exactly the same clustering result as that produced in centralized paradigm, making D-GridMST a promising tool for clustering large distributed spatial databases.
37278978
PC-Filter: A Robust Filtering Technique for Duplicate Record Detection in Large Databases
In this paper, we will propose PC-Filter (PC stands for Partitio n Comparison), a robust data filter for approximately duplicate record detection in large databases. PC-Filter distinguishes itself from all of existing methods by using the notion of partition in duplicate detection. It first sorts the whole database and splits the sorted database into a number of record partitions. The Partition Comparison Graph (PCG) is then constructed by performing fast partition pruning. Finally, duplicate records are effectively detected by using internal and external partition comparison based on PCG. Four properties, used as heuristics, have been devised to achieve a remarkable efficiency of the filter based on triangle inequity of record similarity. PC-Filter is insensitive to the key used to sort the database, and can achieve a very good recall level that is comparable to that of the pair-wise record comparison method but only with a complexity of O(N 4/3). Equipping existing detection methods with PC-Filter, we are able to well solve the ”Key Selection” problem, the ”Scope Specification” problem and the ”Low Recall” problem that the existing methods suffer from.
15427261
Anomaly detection in high-dimensional network data streams: A case study
In this paper, we study the problem of anomaly detection in high-dimensional network streams. We have developed a new technique, called Stream Projected Outlier deTector (SPOT), to deal with the problem of anomaly detection from high-dimensional data streams. We conduct a case study of SPOT in this paper by deploying it on 1999 KDD Intrusion Detection application. Innovative approaches for training data generation, anomaly classification and false positive reduction are proposed in this paper as well. Experimental results demonstrate that SPOT is effective in detecting anomalies from network data streams and outperforms existing anomaly detection methods.
4732400
XAR-miner: efficient association rules mining for XML data
In this paper, we propose a framework, called XAR-Miner, for mining ARs from XML documents efficiently. In XAR-Miner, raw data in the XML document are first preprocessed to transform to either an Indexed Content Tree (IX-tree) or Multi-relational databases (Multi-DB), depending on the size of XML document and memory constraint of the system, for efficient data selection and AR mining. Task-relevant concepts are generalized to produce generalized meta-patterns, based on which the large ARs that meet the support and confidence levels are generated.
16517186
A Color Indexing Scheme Using Two-level Clustering Processing for Effective and Efficient Image Retrieval
[Abstract]: In this paper, we present a clustering-based color indexing scheme for effective amd efficient image retrieval, which is essentially an exploration on the application of clustering technique to image retrieval. In our approach, the color features are clustered automatically using a color clustering algorithm twice (called two-level clustering processing) and two color feature summarizations are obtained, e.g. Local Color Centriods (LCCs) and Global Color Centroids (GCCs). Based upon LCCs and GCCs, a three-level R-tree is bulit for indexing database images and performing effecitve and efficient image retrieval. The experiments show that this indexing scheme is effective and efficient in performing image retrieval.
11328445
Gene Specific Co-regulation Discovery: An Improved Approach
Discovering gene co-regulatory relationships is a new but important research problem in DNA microarray data analysis. The problem of gene specific co-regulation discovery is to, for a particular gene of interest, called the target gene , identify its strongly co-regulated genes and the condition subsets where such strong gene co-regulations are observed. The study on this problem can contribute to a better understanding and characterization of the target gene. The existing method, using the genetic algorithm (GA), is slow due to its expensive fitness evaluation and long individual representation. In this paper, we propose an improved method for finding gene specific co-regulations. Compared with the current method, our method features a notably improved efficiency. We employ k NN Search Table to substantially speed up fitness evaluation in the GA. We also propose a more compact representation scheme for encoding individuals in the GA, which contributes to faster crossover and mutation operations. Experimental results with a real-life gene microarray data set demonstrate the improved efficiency of our technique compared with the current method.
17362173
A New Heuristic Reduct Algorithm Base on Rough Sets Theory
Real world data sets usually have many features, which increases the complexity of data mining task. Feature selection, as a preprocessing step to the data mining, has been shown very effective in reducing dimensionality, removing irrelevant data, increasing learning accuracy, and improving comprehensibility. To find the optimal feature subsets is the aim of feature selection. Rough sets theory provides a mathematical approach to find optimal feature subset, but this approach is time consuming. In this paper, we propose a novel heuristic algorithm based on rough sets theory to find out the feature subset. This algorithm employs appearing frequency of attribute as heuristic information. Experiment results show in most times our algorithm can find out optimal feature subset quickly and efficiently.
26954040
Multi-User Joint Doppler Spread Estimation in Generalized Distributed Antenna System
Generalized distributed antenna is one of the most promising architectures likely to be used in next generation mobile communication systems. In this paper, based on the architecture, multi-user co-channel interference in Doppler spread estimation is analyzed and several useful conclusions are obtained. Moreover, in order to combat this interference, a novel ACF-based multi-user joint Doppler spread estimation (M- JDSE) algorithm is proposed, taking advantage of centralized signal processing characteristic in generalized distributed antenna systems. Both theoretical analysis and simulation results demonstrate the significant performance improvement of proposed M-JDSE algorithm in multi-user environments.
33955645
A Dynamic Scalable Video Conference System Based on SIP
The Session Initiation Protocol (SIP) provides powerful and flexible signaling capabilities for building video conferencing services. Traditionally, for SIP-based centralized video conference systems, the conferencing scale is mainly limited by both the capability of conference server and the availability of bandwidth. In this paper, our design focuses on how to provide dynamic scalability for the SIP-based video conferencing system when the number of conference users increases continually. Based on the study of the SIP protocol and the existing video conferencing models, we propose a dynamic scalable service model that can support to dynamically increase the number of conference servers without negative influence on the stability of system. This enables the extra service requests to be transferred and served in the cooperated conference servers. The paper also addresses the SIP-enabled conferencing flows based on the model in detail. We developed a prototype of video conference system based on the proposed model. Experimental results demonstrate the validity of this service model.
9511290
The Cluster Distribution of Regulatory Motifs of Transcription in Yeast Introns
A comparative analysis of olignucleotide frequencies in two sets of introns of genes highly-transcribed and lowly-transcribed respectively has suggested that the existence of potential positive regulatory motifs of transcription in yeast introns. To further reveal the distribution feature of these motifs, we detected significant clusters of these motifs (mainly pentanucleotides) in introns by r-scan analysis. The results showed that there are more clusters of regulatory motifs in the introns of ribosomal protein genes (highly-transcribed genes) than in lowly-transcribed introns. Experimental studies show that the transcription factors function cooperatively in transcriptional activation and the corresponding binding sites for factors generally cluster in DNA. Accordingly, we speculated that the cluster distribution of regulatory motifs of transcription in yeast introns could becorrelated with the cooperative action of transcription factors and the transcriptional rates of genes could be improved by the cooperativity.
14759570
Simplifying the Reconstruction of 3D Models using Parameter Elimination
Reconstructing large models from images is a significant challenge for computer vision, computer graphics, and related fields. In this paper, we present an approach for simplifying the reconstruction process by mathematically eliminating external camera parameters. This results in less parameters to estimate and in an overall significantly more robust and accurate reconstruction. We reformulate the problem in such a manner as to be able to identify invariants, eliminate superfluous parameters, and measure the performance of our formulation under various conditions. We compare a two-step camera orientation-free method, where the majority of the points are reconstructed using a linear equation set, and a camera position-and- orientation free method, using a degree-two equation set. Both approaches use a full perspective camera and are applied to synthetic and real-world datasets.