26868159
A compression-boosting transform for 2D data
In this paper, we present an invertible transform for 2D data which has the objective of reordering the matrix to improve its (lossless) compression at later stages. Given a binary matrix, the transform involves first searching for the largest uniform submatrix, that is, a submatrix solely composed by the same symbol (either 0 or 1) induced by a subset of rows and columns (which are not necessarily contiguous). Then, the rows and the columns are reordered such that the uniform submatrix is moved to the left-upper corner of the matrix. The transform is recursively applied on the rest of the matrix. The recursion is stopped when the partition produces a matrix which is smaller than a predetermined threshold. The inverse transform (decompression) is fast and can be implemented in linear time in the size of the matrix. The effects of the transform on the compressibility of 2D data is studied empirically by comparing the performance of gzip and bzip2 before and after the application of the transform on several inputs. The preliminary results show that the transform boosts compression.
5241796
Finding Biclusters by Random Projections
Given a matrix X composed of symbols, a bicluster is a submatrix of X obtained by removing some of the rows and some of the columns of X in such a way that each row of what is left reads the same string. In this paper, we are concerned with the problem of finding the bicluster with the largest area in a large matrix X. The problem is first proved to be NP-complete. We present a fast and efficient randomized algorithm that discovers the largest bicluster by random projections. A detailed probabilistic analysis of the algorithm and an asymptotic study of the statistical significance of the solutions are given. We report results of extensive simulations on synthetic data.
1859019
A parallel algorithm for clustering protein-protein interaction networks
The increasing availability of interaction graphs requires new resource-efficient tools capable of extracting valuable biological knowledge from these networks. In this paper we report on a novel parallel implementation of Girvan and Newman's clustering algorithm that is capable of running on clusters of computers. Our parallel implementation achieves almost linear speed-up up to 32 processors and allows us to run this computationally intensive algorithm on large protein-protein interaction networks. Preliminary experiments show that the algorithm has very high accuracy in identifying functional related protein modules.
32009068
Finding biclusters by random projections
Given a matrix X composed of symbols, a bicluster is a submatrix of X obtained by removing some of the rows and some of the columns of X in such a way that each row of what is left reads the same string. In this paper, we are concerned with the problem of finding the bicluster with the largest area in a large matrix X. The problem is first proved to be NP-complete. We present a fast and efficient randomized algorithm that discovers the largest bicluster by random projections. A detailed probabilistic analysis of the algorithm and an asymptotic study of the statistical significance of the solutions are given. We report results of extensive simulations on synthetic data.
18229979
A Compression-Boosting Transform for Two-Dimensional Data
We introduce a novel invertible transform for two- dimensional data which has the objective of reordering the matrix so it will improve its (lossless) compression at later stages. The transform requires to solve a computationally hard problem for which a randomized algorithm is used. The inverse transform is fast and can be implemented in linear time in the size of the matrix. Preliminary experimental results show that the reordering improves the compressibility of digital images.
8789759
Mining web logs for prediction models in WWW caching and prefetching
Web caching and prefetching are well known strategies for improving the performance of Internet systems. When combined with web log mining, these strategies can decide to cache and prefetch web documents with higher accuracy. In this paper, we present an application of web log mining to obtain web-document access patterns and use these patterns to extend the well-known GDSF caching policies and prefetching policies. Using real web logs, we show that this application of data mining can achieve dramatic improvement to web-access performance.
61940695
Extracting Actionable Knowledge from Decision Trees
Most data mining algorithms and tools stop at discovered customer models, producing distribution information on customer profiles. Such techniques, when applied to industrial problems such as customer relationship management (CRM), are useful in pointing out customers who are likely attritors and customers who are loyal, but they require human experts to postprocess the discovered knowledge manually. Most of the postprocessing techniques have been limited to producing visualization results and interestingness ranking, but they do not directly suggest actions that would lead to an increase in the objective function such as profit. In this paper, we present novel algorithms that suggest actions to change customers from an undesired status (such as attritors) to a desired one (such as loyal) while maximizing an objective function: the expected net profit. These algorithms can discover cost-effective actions to transform customers from undesirable classes to desirable ones. The approach we take integrates data mining and decision making tightly by formulating the decision making problems directly on top of the data mining results in a postprocessing step. To improve the effectiveness of the approach, we also present an ensemble of decision trees which is shown to be more robust when the training data changes. Empirical tests are conducted on both a realistic insurance application domain and UCI benchmark data
30032661
Data preparation for data mining
Data preparation is a fundamental stage of data analysis. While a lot of low-quality information is available in various data sources and on the Web, many organizations or companies are interested in how to transform the data into cleaned forms which can be used for high-profit purposes. This goal generates an urgent need for data analysis aimed at cleaning the raw data. In this paper, we first show the importance of data preparation in data analysis, then introduce some research achievements in the area of data preparation. Finally, we suggest some future directions of research and development.
5698182
Measurement, analysis and performance improvement of the Apache Web server
Performance of Web servers is critical to the success of many corporations and organizations. However, very few results have been published that quantitatively study the server behavior and identify the performance bottlenecks. In this paper we measured and analyzed the behavior of the popular Apache Web server on a uniprocessor system and a 4-CPU SMP (Symmetric Multi-Processor) system running the IBM AIX operating system. Using the AIX built-in tracing facility, we obtained detailed information on kernel events and system activities while running Apache driven by the SPECweb96 and the WebStone benchmarks. After quantitatively identifying the performance bottlenecks, we proposed and implemented 6 techniques that improve the throughput of Apache by 61%. These techniques are general purpose and can be applied to other Web servers as well.
31114007
Cooperative Target Localization Method for Heterogeneous Sensor Networks
Based on the binary sensor model, a novel method for target localization in heterogeneous sensor networks is presented. With the binary information reported by nodes, target's position is locked into the intersection area of sensing areas of all nodes which detect the same target, and then the estimated position is computed by geometric means. The proposed method adapts to heterogeneous sensor networks, moreover, it can integrate with other target localization approaches easily. Simulation results demonstrate that, in sensor networks composed of the same type of sensors, our method lead to a decrease in average localization errors compared with the traditional method; in heterogeneous sensor networks, the method renders more accurate estimate of the target's location.