30032661
Data preparation for data mining
Data preparation is a fundamental stage of data analysis. While a lot of low-quality information is available in various data sources and on the Web, many organizations or companies are interested in how to transform the data into cleaned forms which can be used for high-profit purposes. This goal generates an urgent need for data analysis aimed at cleaning the raw data. In this paper, we first show the importance of data preparation in data analysis, then introduce some research achievements in the area of data preparation. Finally, we suggest some future directions of research and development.
15099164
A scalable supervised algorithm for dimensionality reduction on streaming data
Algorithms on streaming data have attracted increasing attention in the past decade. Among them, dimensionality reduction algorithms are greatly interesting due to the desirability of real tasks. Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) are two of the most widely used dimensionality reduction approaches. However, PCA is not optimal for general classification problems because it is unsupervised and ignores valuable label information for classification. On the other hand, the performance of LDA is degraded when encountering limited available low-dimensional spaces and singularity problem. Recently, Maximum Margin Criterion (MMC) was proposed to overcome the shortcomings of PCA and LDA. Nevertheless, the original MMC algorithm could not satisfy the streaming data model to handle large-scale high-dimensional data set. Thus an effective, efficient and scalable approach is needed. In this paper, we propose a supervised incremental dimensionality reduction algorithm and its extension to infer adaptive low-dimensional spaces by optimizing the maximum margin criterion. Experimental results on a synthetic dataset and real datasets demonstrate the superior performance of our proposed algorithm on streaming data.
5296157
A Novel Scalable Algorithm for Supervised Subspace Learning
Subspace learning approaches aim to discover important statistical distribution on lower dimensions for high dimensional data. Methods such as principal component analysis (PCA) do not make use of the class information, and linear discriminant analysis (LDA) could not be performed efficiently in a scalable way. In this paper, we propose a novel highly scalable supervised subspace learning algorithm called as supervised Kampong measure (SKM). It assigns data points as close as possible to their corresponding class mean, simultaneously assigns data points to be as far as possible from the other class means in the transformed lower dimensional subspace. Theoretical derivation shows that our algorithm is not limited by the number of classes or the singularity problem faced by LDA. Furthermore, our algorithm can be executed in an incremental manner in which learning is done in an online fashion as data streams are received. Experimental results on several datasets, including a very large text data set RCV1, show the outstanding performance of our proposed algorithm on classification problems as compared to PCA, LDA and a popular feature selection approach, information gain (IG).
11767978
Real-Time Scheduling for Multi-agent Call Center Automation
In a call center, service agents with different capabilities are available for solving incoming customer problems at any time. To supply quick response and better problem solution to customers, it is necessary to schedule customer problems to appropriate service agents efficiently. We developed SANet, a service agent network for call center, which integrates multiple service agents including both software agents and human agents, and employs a broker to schedule customer problems to service agents for better solutions according to their changing capabilities and availability. This paper describes the real-time scheduling method in SANet as well as its architecture. There are two phases in our scheduling method. One is problem-type learning. The broker is trained to learn the problem types and hence can decide the type of incoming problems automatically. The other is the scheduling algorithm based on problem types, capabilities and availability of service agents. We highlight an application in which we apply SANet to a call center problem for a cable-TV company. Finally, we support our claims via experimental results and discuss related works.
9645876
Handling feature interactions in process-planning
Abstract The development of a successful approach to the integration of computer-aided design and computer-aided process-planning will require solutions to several fundamental problems in planning and geometric reasoning. In this article, we consider two such problems, both of which arise from interactions among machinable features. The first problem is that interactions can lead to several alternative interpretations of what the machinable features are. We address this problem by developing an algebra of features. The second problem is that interactions among features can cause problems in optimizing the process plans. For this problem, we have developed a branch-and-bound optimization algorithm with good heuristics to guide the search. This article presents the theory and mathematical framework for both problems and also the details of the implementation and experimental results.
13595177
Multiple-Goal Recognition from Low-Level Signals
Researchers and practitioners from both the artificial intelligence and pervasive computing communities have been paying increasing attention to the task of inferring users' high-level goals from low-level sensor readings. A common assumption made by most approaches is that a user either has a single goal in mind, or achieves several goals sequentially. However, in real-world environments, a user often has multiple goals that are concurrently carried out, and a single action can serve as a common step towards multiple goals. In this paper, we formulate the multiple-goal recognition problem and exemplify it in an indoor environment where an RF-based wireless network is available. We propose a goal-recognition algorithm based on a dynamic model set and show how goal models evolve over time based on pre-defined states. Experiments with real data demonstrate that our method can accurately and efficiently recognize multiple interleaving goals in a user's trace.
8767636
The Design and Implementation of a DCD Device Driver for Unix
Recent research results [1, 2] using simulation have demonstrated that Disk Caching Disk (DCD), a new disk I/O architecture, has the potential for drastically improving disk write performance besides its higher reliability than traditional disk systems. To validate whether DCD can live up to its promise in the real world environment, we have designed and implemented a DCD device driver for the Sun's Solaris operating system. Measured performance results are very promising. For metadata intensive benchmarks, our DCD driver outperforms the traditional system by a factor of 2-6 in terms of program execution speeds. The driver also guarantees file system integrity in the events of system crashes or failures. Moreover, unlike other approaches such as Log-structured File Systems or Soft Updates, the DCD driver is completely transparent to the OS. It does not require any changes to the OS or the on-disk data layout. As a result, it can be used as a "drop-in" replacement for the traditional disk device driver in an existing system to obtain immediate performance improvement. Our multi-layered device-driver approach significantly reduces the implementation overhead and improves portability.
18229979
A Compression-Boosting Transform for Two-Dimensional Data
We introduce a novel invertible transform for two- dimensional data which has the objective of reordering the matrix so it will improve its (lossless) compression at later stages. The transform requires to solve a computationally hard problem for which a randomized algorithm is used. The inverse transform is fast and can be implemented in linear time in the size of the matrix. Preliminary experimental results show that the reordering improves the compressibility of digital images.
854297
Performance of Cache Memories for Vector Computers
Abstract Recent studies [5] have shown that the memory system is the major bottleneck of vector computers. Performance measurements on CRAY-2 computers indicate that as much as a factor of 4.4 performance degradation results from memory latency. In this paper, we study the performance of incorporating cache memories into vector computers. Two cache organizations that we proposed [22] are considered: direct-mapped cache and prime-mapped cache. We analyze the caching behavior of three typical blocked algorithms for numerical applications: matrix multiplication, Gaussian elimination, and FFT. By analyzing the algorithm structures in conjunction with system architectures, we develop analytical models based on real applications rather than on statistical estimates. Our performance models give the expected value of execution time of an algorithm averaged over a wide range of problem sizes. Performance measurements of the algorithms on a real machine are carried out to validate our analysis. Numerical results show that the prime-mapped cache minimizes the cache miss ratio caused by line interferences that are critical for numerical applications.
5241796
Finding Biclusters by Random Projections
Given a matrix X composed of symbols, a bicluster is a submatrix of X obtained by removing some of the rows and some of the columns of X in such a way that each row of what is left reads the same string. In this paper, we are concerned with the problem of finding the bicluster with the largest area in a large matrix X. The problem is first proved to be NP-complete. We present a fast and efficient randomized algorithm that discovers the largest bicluster by random projections. A detailed probabilistic analysis of the algorithm and an asymptotic study of the statistical significance of the solutions are given. We report results of extensive simulations on synthetic data.
13194241
BUCS - a bottom-up cache structure for networked storage servers
This paper introduces a new caching structure to improve server performance by minimizing data traffic over the system bus. The idea is to form a bottom-up caching hierarchy in a networked storage server. The bottom level cache is located on an embedded controller that is a combination of a network interface card (NIC) and a storage host bus adapter (HBA). Storage data coming from or going to a network are cached at this bottom level cache and meta-data related to these data are passed to the host for processing. When cached data exceed the capacity of the bottom level cache, some data are moved to the host RAM that is usually larger than the bottom level cache. This new cache hierarchy is referred to as bottom-up cache structure (BUGS) in contrast to a traditional CPU-centric top-down cache where the top-level cache is the smallest and fastest, and the lower in the hierarchy the larger and slower the cache. Such data caching at the controller level dramatically reduces bus traffic and leads to great performance improvement for networked storages. We have implemented a proof-of-concept prototype using Intel's IQ80310 reference board and Linux network block device. Through performance measurements on the prototype implementation, we observed up to 3 times performance improvement of BUCS over traditional systems in terms of response time and system throughput.