17533834
Efficiently Computing the Top N Averages in Iceberg Cubes
Data cubes are an enabling approach for efficient on-line analytical processing (OLAP) systems. Iceberg cubes are special cubes consisting of the aggregates of multi-dimensional groups that satisfy user-specified thresholds. When there are a relatively large number of dimensions, the number of groups in an iceberg cube is huge. End users cannot fully understand the aggregate results or directly use them to make a decision. Approaches for the efficient computation of the top n groups have been proposed and have been shown to work well on iceberg cubes with simple aggregate functions such as COUNT and SUM. In this paper, we study the efficient computation of the top n groups with the complex aggregate function AVERAGE. As the average of a sub-group does not increase/decrease monotonically with its super-group, AVERAGE constraints cannot be used directly for pruning. We propose a new technique upper-bounding average which is anti-monotonic and can be used for low-cost effective pruning. Based on a tree structure representing groups, search and pruning techniques are developed, and an algorithm is proposed to compute the top n averages efficiently.
30931074
A Distributed Object Database's Basic Design Issues
1166438
Building Behaviour Knowledge Space to Make Classification Decision
CAEP, namely Classification by Aggregating Emerging Patterns, builds classifiers from Emerging Patterns (EPs). EPs mined from the training data of a class are distinguishing features of the class. To classify a test instance t, the scores by aggregating EPs in t measures the weight we put on each class; direct comparison of scores decides t's class. However the skewed distribution of EPs among classes and intricate relationship between EPs sometimes make the decision by directly comparing scores unreliable. In this paper, we propose to build Score Behaviour Knowledge Space (SBKS) to record the behaviour of training data on scores; classification decision is drawn from SBKS from a statistical point of view. Extensive experiments on real-world datasets show that SBKS frequently improves CAEP classifiers, especially on datasets where they have relatively poor performance. The improved CAEP classifiers outperform the start-of-the-art decision tree classifier C5.0.
8448018
Efficient Frequent Pattern Mining on Web Logs
Mining frequent patterns from Web logs is an important data mining task. Candidate-generation-and-test and pattern-growth are two representative frequent pattern mining approaches. We have conducted extensive experiments on real world Web log data to analyse the characteristics of Web logs and the behaviours of these two approaches on Web logs. To improve the performance of current algorithms on mining Web logs, we propose a new algorithm – Combined Frequent Pattern Mining (CFPM) to cater for Web log data specifically. We use heuristics to prune search space and reduce costs in mining so that better efficiency is achieved. Experimental results show that CFPM significantly improves the performance of the pattern-growth approach by 1.2–7.8 times on mining frequent patterns from Web logs.
717712
Efficient Computation of Iceberg Cubes by Bounding Aggregate Functions
The iceberg cubing problem is to compute the multidimensional group-by partitions that satisfy given aggregation constraints. Pruning unproductive computation for iceberg cubing when nonantimonotone constraints are present is a great challenge because the aggregate functions do not increase or decrease monotonically along the subset relationship between partitions. In this paper, we propose a novel bound prune cubing (BP-Cubing) approach for iceberg cubing with nonantimonotone aggregation constraints. Given a cube over n dimensions, an aggregate for any group-by partition can be computed from aggregates for the most specific n--dimensional partitions (MSPs). The largest and smallest aggregate values computed this way become the bounds for all partitions in the cube. We provide efficient methods to compute tight bounds for base aggregate functions and, more interestingly, arithmetic expressions thereof, from bounds of aggregates over the MSPs. Our methods produce tighter bounds than those obtained by previous approaches. We present iceberg cubing algorithms that combine bounding with efficient aggregation strategies. Our experiments on real-world and artificial benchmark data sets demonstrate that BP-Cubing algorithms achieve more effective pruning and are several times faster than state-of-the-art iceberg cubing algorithms and that BP-Cubing achieves the best performance with the top-down cubing approach.
35361263
Reducing Overfitting in Predicting Intrinsically Unstructured Proteins
Intrinsically unstructured or disordered proteins are proteins that lack fixed 3-D structure globally or contain long disordered regions. Predicting disordered regions has attracted significant research recently. In developing a decision tree based disordered region predictor, we note that many previous predictors applying 20 amino acid compositions as training parameter tend to overfit the data. In this paper we propose to alleviate overfitting in prediction of intrinsically unstructured proteins by reducing input parameters. We also compare this approach with the random forest model, which is inherently tolerant to overfitting. Our experiments suggest that reducing 20 amino acid compositions into 4 groups according to amino acid property can reduce the overfitting in decision tree model. Alternatively, ensemble-learning techniques like random forest is inherently more tolerant to this kind of overfitting and can be a promising candidate in disordered region prediction.
12113641
Neighbor number, valley seeking and clustering
This paper proposes a novel nonparametric clustering algorithm capable of identifying shape-free clusters. This algorithm is based on a nonparametric estimation of the normalized density derivative (NDD) and the local convexity of the density distribution function, both of which are represented in a very concise form in terms of neighbor numbers. We use NDD to measure the dissimilarity between each pair of observations in a local neighborhood and to build a connectivity graph. Combined with the local convexity, this similarity measure can detect observations in local minima (valleys) of the density function, which separate observations in different major clusters. We demonstrate that this algorithm has a close relationship with the single-linkage hierarchical clustering and can be viewed as its extension. The performance of the algorithm is tested with both synthetic and real datasets. An example of color image segmentation is also given. Comparisons with several representative existing algorithms show that the proposed method can robustly identify major clusters even when there are complex configurations and/or large overlaps.
5632028
On the Laplacian Spectra of Graphs
We first establish the relationship between the largest eigenvalue of the Laplacian matrix of a graph and its bipartite density. Then we present lower and upper bounds for the largest Laplacian eigenvalue of a graph in terms of its largest degree and diameter.
8353152
An Efficient Algorithm for Mining Coherent Patterns from Heterogeneous Microarrays
DNA microarray techniques present a novel way for geneticists to monitor interactions among tens of thousands of genes simultaneously, and have become standard lab routines in gene discovery, disease diagnosis, and drug design. There has been extensive research on coherent subspace clustering of gene expressions measured under consistent experimental settings. This implies that all experiments are run using the same batch of microarray chips with similar characteristics of noise. Algorithms developed under this assumption may not be applicable for analyzing data collected from heterogeneous settings, where the set of genes being monitored may be different and expression levels may be not directly comparable even for the same gene. In this paper, we propose a model, F-cluster, for mining subspace coherent patterns from heterogeneous gene expression data, which is shown effective for revealing truthful patterns and reducing spurious ones. We also develop an efficient and scalable hybrid approach that combines gene-pair based and sample-pair based pruning to generate F-clusters from multiple gene expression matrices simultaneously. The experimental results demonstrate that our model can discover significant clusters that may not be identified by previous models.
1109125
An on-demand Bluetooth scatternet formation and routing protocol for wireless sensor networks
Bluetooth is a promising short-range wireless communication technology with the characteristics of interference resilience and power efficiency which are required by wireless sensor networks. As an enhanced sensor node platform, the Intel Mote uses Bluetoth as its radio scheme and MAC protocol. However, most of the existing scatternet formation protocols intend to connect all the nodes within the networks regardless of traffic pattern. For wireless sensor network applications with traffic of low duty cycle, on-demand scatternet formation and routing achieve significant power saving by avoiding to maintain the entire network connectivity. We propose in this paper an on-demand scatternet and route formation protocol for Bluetooth-based wireless sensor networks. It is able to cope with multiple sources initiating traffic simultaneously in densely deployed wireless sensor networks. We also introduce a modified inquiry scheme using extended ID packet for power efficient propagation of route request messages. Furthermore, we propose a mechanism employing POLL packets in Page processes to transfer scatternet formation and route reply information without extra overhead. Simulation results show that our on-demand scatternet formation and routing protocol can provide multihop channels with reasonable delay for Bluetooth-base wireless sensor networks.
14269691
Multiprocessor scalability predictions through detailed program execution analysis
Scalability measures the ability of a parallel system to improve performance as the size of an application problem and the number of processors involved increase. There are some limits to existing scalability studies. First, the problem size in a computation is not well-defined. Second, the methods used to differentiate algorithmic and architectural scalabilities are not effective enough. Thh-dly, most approaches to scalability study are either highly time-consuming or restricted to simple problem/architecture structures. A major effort of this work is to address these limits. We have extended the latency metric [11] for more complex scaling of problems, and to possibly isolate the scalability of an algorithm from a parallel system. The scalability y prediction is based on a semi-empirical approach [10] that significantly reduces the time and cost of measurements and simulation. Our prediction results were validated on the KSR1 and on the CM-5.