28734493
AGRID: An Efficient Algorithm for Clustering Large High-Dimensional Datasets
The clustering algorithm GDILC relies on density-based clustering with grid and is designed to discover clusters of arbitrary shapes and eliminate noises. However, it is not scalable to large high-dimensional datasets. In this paper, we improved this algorithm in five important directions. Through these improvements, AGRID is of high scalability and can process large high-dimensional datasets. It can discover clusters of various shapes and eliminate noises effectively. Besides, it is insensitive to the order of input and is a non-parametric algorithm. The high speed and accuracy of the AGRID clustering algorithm was shown in our experiments.
5595688
Discovering Interesting Association Rules by Clustering
There are a great many metrics available for measuring the interestingness of rules In this paper, we design a distinct approach for identifying association rules that maximizes the interestingness in an applied context More specifically, the interestingness of association rules is defined as the dissimilarity between corresponding clusters In addition, the interestingness assists in filtering out those rules that may be uninteresting in applications Experiments show the effectiveness of our algorithm.
38902380
Construction Algorithm of Principal Curves in the Sense of Limit
Principal curves have been defined as self-consistent, smooth, one-dimensional curves which pass through the middle of a multidimensional data set. They are nonlinear generalization of the first Principal Component. In this paper, we take a new approach by defining principal curves as continuous curves based on the local tangent space in the sense of limit. It is proved that this new principal curves not only satisfy the self-consistent property, but also are the unique existence for any given open covering. Based on the new definition, a new practical algorithm for constructing principal curves is given. And the convergence properties of this algorithm are analyzed. The new construction algorithm of principal curves is illustrated on some simulated data sets.
11721850
Estimating LOC for information systems from their conceptual data models
Effort and cost estimation is crucial in software management. Estimation of software size plays a key role in the estimation. Line of Code (LOC) is still a commonly used software size measure. Despite the fact that software sizing is well recognized as an important problem for more than two decades, there is still much problem in existing methods. Conceptual data model is widely used in the requirements analysis for information systems. It is also not difficult to construct conceptual data models in the early stage of developing information systems. Much characteristic of an information system is actually reflected from its conceptual data model. We explore into the use of conceptual data model for estimating LOC. This paper proposes a novel method for estimating LOC for an information system from its conceptual data model through the use of multiple linear regression model. We have validated the method through collecting samples from both the industry and open-source systems.
1706033
Hierarchical Clustering Algorithms for Document Datasets
Fast and high-quality document clustering algorithms play an important role in providing intuitive navigation and browsing mechanisms by organizing large amounts of information into a small number of meaningful clusters. In particular, clustering algorithms that build meaningful hierarchies out of large document collections are ideal tools for their interactive visualization and exploration as they provide data-views that are consistent, predictable, and at different levels of granularity. This paper focuses on document clustering algorithms that build such hierarchical solutions and (i) presents a comprehensive study of partitional and agglomerative algorithms that use different criterion functions and merging schemes, and (ii) presents a new class of clustering algorithms called constrained agglomerative algorithms, which combine features from both partitional and agglomerative approaches that allows them to reduce the early-stage errors made by agglomerative methods and hence improve the quality of clustering solutions. The experimental evaluation shows that, contrary to the common belief, partitional algorithms always lead to better solutions than agglomerative algorithms; making them ideal for clustering large document collections due to not only their relatively low computational requirements, but also higher clustering quality. Furthermore, the constrained agglomerative methods consistently lead to better solutions than agglomerative methods alone and for many cases they outperform partitional methods, as well.
10814374
The DNF exception problem
Given a disjunctive normal form (DNF) expression ϕ and a set A of vectors satisfying the expression, called the set of exceptions, we would like to update ϕ to get a new DNF which is false on A, and otherwise is equivalent to ϕ. Is there an algorithm with running time polynomial in the number of variables, the size of the original formula and the number of exceptions, which produces an updated formula of size bounded by a certain type of function of the same parameters?We give an efficient updating algorithm, which shows that the previously known best upper bound for the size of the updated expression is not optimal in order of magnitude. We then present a lower bound for the size of the updated formula in terms of the parameters, which is the first known lower bound for this problem. We also consider the special case (studied previously in the complexity theory of disjunctive normal forms) where the initial formula is identically true, and give efficient updating algorithms, providing new upper bounds for the size of the updated expression.
1680683
Subscription Propagation in Highly-Available Publish/Subscribe Middleware
Achieving availability and scalability while providing service guarantees such as in-order, gapless delivery is essential for deploying publish/subscribe messaging middleware in wide area networks. Scalability often requires a publish/subscribe system to propagate subscription information and perform content matching across the network. Existing subscription propagation algorithms do not support in-order, gapless delivery in a redundant overlay network.This paper presents a novel approach that utilizes virtual time (VT) vectors to convey temporal consistency in propagating incremental and consolidated subscription information. The VT vectors provide a means of testing sufficiency of filtering information, by comparing a broker's VT vector with that of a message. When the test fails, indicating insufficient broker subscription information, safety may be preserved by "flooding" the message to all neighbors on a routing tree. This approach does not require subscription state agreement across redundant paths and hence is highly available. We present a detailed evaluation of the approach.
8245310
Packet classification using independent sets
This paper describes a new algorithm for packet classification using the concept of independent sets. The algorithm has very small memory requirements. The search speed is neither sensitive to the rule table nor to the percentage of wildcards in the fields. It also scales well from two dimensional classifiers to high dimensional ones. In particular, the algorithm is inherently parallel. Hardware tailored to this algorithm can achieve very fast search speed.