1166438
Building Behaviour Knowledge Space to Make Classification Decision
CAEP, namely Classification by Aggregating Emerging Patterns, builds classifiers from Emerging Patterns (EPs). EPs mined from the training data of a class are distinguishing features of the class. To classify a test instance t, the scores by aggregating EPs in t measures the weight we put on each class; direct comparison of scores decides t's class. However the skewed distribution of EPs among classes and intricate relationship between EPs sometimes make the decision by directly comparing scores unreliable. In this paper, we propose to build Score Behaviour Knowledge Space (SBKS) to record the behaviour of training data on scores; classification decision is drawn from SBKS from a statistical point of view. Extensive experiments on real-world datasets show that SBKS frequently improves CAEP classifiers, especially on datasets where they have relatively poor performance. The improved CAEP classifiers outperform the start-of-the-art decision tree classifier C5.0.
35361263
Reducing Overfitting in Predicting Intrinsically Unstructured Proteins
Intrinsically unstructured or disordered proteins are proteins that lack fixed 3-D structure globally or contain long disordered regions. Predicting disordered regions has attracted significant research recently. In developing a decision tree based disordered region predictor, we note that many previous predictors applying 20 amino acid compositions as training parameter tend to overfit the data. In this paper we propose to alleviate overfitting in prediction of intrinsically unstructured proteins by reducing input parameters. We also compare this approach with the random forest model, which is inherently tolerant to overfitting. Our experiments suggest that reducing 20 amino acid compositions into 4 groups according to amino acid property can reduce the overfitting in decision tree model. Alternatively, ensemble-learning techniques like random forest is inherently more tolerant to this kind of overfitting and can be a promising candidate in disordered region prediction.
750246
Noise Tolerance of EP-Based Classifiers
Emerging Pattern (EP)-based classifiers are a type of new classifiers based on itemsets whose occurrence in one dataset varies significantly from that of another. These classifiers are very promising and have shown to perform comparably with some popular classifiers. In this paper, we conduct two experiments to study the noise tolerance of EP-based classifiers. A primary concern is to ascertain if overfitting occurs in them. Our results highlight the fact that the aggregating approach in constructing EP-based classifiers prevents them from overfitting. We further conclude that perfect training accuracy does not necessarily lead to overfitting of a classifier as long as there exists a suitable mechanism, such as an aggregating approach, to counterbalance any propensity to overfit.
11467593
Generating Multidimensional Schemata from Relational Aggregation Queries
Queries on operational databases can be viewed as a form of business rules on data warehouse schema design. We propose to use such queries to automatically generate measures, dimensions and dimension hierarchies and their representation in a star schema. The schema produced with our approach has good properties such as non-redundant dimensional attributes and orthogonality among dimensions and can answer many more queries than just those it was generated from.
8448018
Efficient Frequent Pattern Mining on Web Logs
Mining frequent patterns from Web logs is an important data mining task. Candidate-generation-and-test and pattern-growth are two representative frequent pattern mining approaches. We have conducted extensive experiments on real world Web log data to analyse the characteristics of Web logs and the behaviours of these two approaches on Web logs. To improve the performance of current algorithms on mining Web logs, we propose a new algorithm – Combined Frequent Pattern Mining (CFPM) to cater for Web log data specifically. We use heuristics to prune search space and reduce costs in mining so that better efficiency is achieved. Experimental results show that CFPM significantly improves the performance of the pattern-growth approach by 1.2–7.8 times on mining frequent patterns from Web logs.
30931074
A Distributed Object Database's Basic Design Issues
2396831
Mining coherent patterns from heterogeneous microarray data
Microarray technology is a powerful tool for geneticists to monitor interactions among tens of thousands of genes simultaneously. There has been extensive research on coherent subspace clustering of gene expressions measured under consistent experimental settings. However, these methods assume that all experiments are run using the same batch of microarray chips with similar characteristics of noise. Algorithms developed under this assumption may not be applicable for analyzing data collected from heterogeneous settings, where the set of genes being monitored may be different and expression levels may be not directly comparable even for the same gene. In this paper, we propose a model, F-cluster, for mining subspace coherent patterns from heterogeneous gene expression data. We compare our model with previously proposed models. We analyze the search space of the problem and give a naïve solution for it.
6142922
Improving memory performance of sorting algorithms
Memory hierarchy considerations during sorting algorithm design and implementation play an important role in significantly improving execution performance. Existing algorithms mainly attempt to reduce capacity misses on direct-mapped caches. To reduce other types of cache misses that occur in the more common set-associative caches and the TLB, we restructure the mergesort and quicksort algorithms further by integrating tiling, padding, and buffering techniques and by repartitioning the data set. Our study shows that substantial performance improvements can be obtained using our new methods.
16051355
DPPC-RE: TCAM-based distributed parallel packet classification with range encoding
Packet classification has been a critical data path function for many emerging networking applications. An interesting approach is the use of ternary content addressable memory (TCAM) to achieve deterministic, high-speed packet classification performance. However, apart from high cost and power consumption, due to slow growing clock rate for memory technology, in general, the traditional single TCAM-based solution has difficulty to keep up with fast growing line rates. Moreover, the TCAM storage efficiency is largely affected by the need to support rules with ranges or range matching. In this paper, a distributed TCAM scheme that exploits chip-level-parallelism is proposed to greatly improve the throughput performance. This scheme seamlessly integrates with a range encoding scheme which not only solves the range matching problem, but also ensures a balanced high throughput performance. A thorough theoretical worst-case analysis of throughput, processing delay, and power consumption, as well as the experimental results show that the proposed solution can achieve scalable throughput performance matching up to OC768 line rate or higher. The added TCAM storage overhead is found to be reasonably small for the five real-world classifiers studied
47233627
Integrative negotiation in complex organizational agent systems
This paper introduces an integrative negotiation mechanism, which enables agents to choose any attitude from the extremes of self-interested and fully cooperative to those that are partially self-interested and partially cooperative. Experimental work verifies this mechanism and explores the question whether it always improves the social welfare to have an agent be completely cooperative. It is found that it is good for the organization to have agents being partially cooperative in their local negotiation with other agents rather than being fully cooperative, in order to deal more effectively with the uncertainty of not having a more informed view of the state of the entire agent organization.
3575990
Pruning dynamic slices with confidence
Given an incorrect value produced during a failed program run (e.g., a wrong output value or a value that causes the program to crash), the backward dynamic slice of the value very frequently captures the faulty code responsible for producing the incorrect value. Although the dynamic slice often contains only a small percentage of the statements executed during the failed program run, the dynamic slice can still be large and thus considerable effort may be required by the programmer to locate the faulty code.In this paper we develop a strategy for pruning the dynamic slice to identify a subset of statements in the dynamic slice that are likely responsible for producing the incorrect value. We observe that some of the statements used in computing the incorrect value may also have been involved in computing correct values (e.g., a value produced by a statement in the dynamic slice of the incorrect value may also have been used in computing a correct output value prior to the incorrect value). For each such executed statement in the dynamic slice, using the value profiles of the executed statements, we compute a confidence value ranging from 0 to 1 - a higher confidence value corresponds to greater likelihood that the execution of the statement produced a correct value. Given a failed run involving execution of a single error, we demonstrate that the pruning of a dynamic slice by excluding only the statements with the confidence value of 1 is highly effective in reducing the size of the dynamic slice while retaining the faulty code in the slice. Our experiments show that the number of distinct statements in a pruned dynamic slice are 1.79 to 190.57 times less than the full dynamic slice. Confidence values also prioritize the statements in the dynamic slice according to the likelihood of them being faulty. We show that examining the statements in the order of increasing confidence values is an effective strategy for reducing the effort of fault location.