40952064
LDA/QR: an efficient and effective dimension reduction algorithm and its theoretical foundation
LDA/QR, a linear discriminant analysis (LDA) based dimension reduction algorithm is presented. It achieves the efficiency by introducing a QR decomposition on a small-size matrix, while keeping competitive classification accuracy. Its theoretical foundation is also presented.
7169369
A two-stage linear discriminant analysis via QR-decomposition
Linear discriminant analysis (LDA) is a well-known method for feature extraction and dimension reduction. It has been used widely in many applications involving high-dimensional data, such as image and text classification. An intrinsic limitation of classical LDA is the so-called singularity problems; that is, it fails when all scatter matrices are singular. Many LDA extensions were proposed in the past to overcome the singularity problems. Among these extensions, PCA+LDA, a two-stage method, received relatively more attention. In PCA+LDA, the LDA stage is preceded by an intermediate dimension reduction stage using principal component analysis (PCA). Most previous LDA extensions are computationally expensive, and not scalable, due to the use of singular value decomposition or generalized singular value decomposition. In this paper, we propose a two-stage LDA method, namely LDA/QR, which aims to overcome the singularity problems of classical LDA, while achieving efficiency and scalability simultaneously. The key difference between LDA/QR and PCA+LDA lies in the first stage, where LDA/QR applies QR decomposition to a small matrix involving the class centroids, while PCA+LDA applies PCA to the total scatter matrix involving all training data points. We further justify the proposed algorithm by showing the relationship among LDA/QR and previous LDA methods. Extensive experiments on face images and text documents are presented to show the effectiveness of the proposed algorithm.
37613840
Improving medical/biological data classification performance by wavelet preprocessing
Many real-world datasets contain noise which could degrade the performances of learning algorithms. Motivated from the success of wavelet denoising techniques in image data, we explore a general solution to alleviate the effect of noisy data by wavelet preprocessing for medical/biological data classification. Our experiments are divided into two categories: one is of different classification algorithms on a specific database, and the other is of a specific classification algorithm (decision tree) on different databases. The experiment results show that the wavelet denoising of noisy data is able to improve the accuracies of those classification methods, if the localities of the attributes are strong enough.
1550828
Two-Dimensional Linear Discriminant Analysis
Linear Discriminant Analysis (LDA) is a well-known scheme for feature extraction and dimension reduction. It has been used widely in many applications involving high-dimensional data, such as face recognition and image retrieval. An intrinsic limitation of classical LDA is the so-called singularity problem, that is, it fails when all scatter matrices are singular. A well-known approach to deal with the singularity problem is to apply an intermediate dimension reduction stage using Principal Component Analysis (PCA) before LDA. The algorithm, called PCA+LDA, is used widely in face recognition. However, PCA+LDA has high costs in time and space, due to the need for an eigen-decomposition involving the scatter matrices. 
 
In this paper, we propose a novel LDA algorithm, namely 2DLDA, which stands for 2-Dimensional Linear Discriminant Analysis. 2DLDA overcomes the singularity problem implicitly, while achieving efficiency. The key difference between 2DLDA and classical LDA lies in the model for data representation. Classical LDA works with vectorized representations of data, while the 2DLDA algorithm works with data in matrix representation. To further reduce the dimension by 2DLDA, the combination of 2DLDA and classical LDA, namely 2DLDA+LDA, is studied, where LDA is preceded by 2DLDA. The proposed algorithms are applied on face recognition and compared with PCA+LDA. Experiments show that 2DLDA and 2DLDA+LDA achieve competitive recognition accuracy, while being much more efficient.
663178
Efficient Kernel Discriminant Analysis via QR Decomposition
Linear Discriminant Analysis (LDA) is a well-known method for feature extraction and dimension reduction. It has been used widely in many applications such as face recognition. Recently, a novel LDA algorithm based on QR Decomposition, namely LDA/QR, has been proposed, which is competitive in terms of classification accuracy with other LDA algorithms, but it has much lower costs in time and space. However, LDA/QR is based on linear projection, which may not be suitable for data with nonlinear structure. This paper first proposes an algorithm called KDA/QR, which extends the LDA/QR algorithm to deal with nonlinear data by using the kernel operator. Then an efficient approximation of KDA/QR called AKDA/QR is proposed. Experiments on face image data show that the classification accuracy of both KDA/QR and AKDA/QR are competitive with Generalized Discriminant Analysis (GDA), a general kernel discriminant analysis algorithm, while AKDA/QR has much lower time and space costs.
3013042
Feature extraction via generalized uncorrelated linear discriminant analysis
Feature extraction is important in many applications, such as text and image retrieval, because of high dimensionality. Uncorrelated Linear Discriminant Analysis (ULDA) was recently proposed for feature extraction. The extracted features via ULDA were shown to be statistically uncorrelated, which is desirable for many applications. In this paper, we will first propose the ULDA/QR algorithm to simplify the previous implementation of ULDA. Then we propose the ULDA/GSVD algorithm, based on a novel optimization criterion, to address the singularity problem. It is applicable for undersampled problem, where the data dimension is much larger than the data size, such as text and image retrieval. The novel criterion used in ULDA/GSVD is the perturbed version of the one from ULDA/QR, while surprisingly, the solution to ULDA/GSVD is shown to be independent of the amount of perturbation applied. We did extensive experiments on text and face image data to show the effectiveness of ULDA/GSVD and compare with other popular feature extraction algorithms.
3084819
Distributed energy-conserving routing protocols
This paper discusses several distributed power-aware routing protocols in wireless ad-hoc networks (especially sensor networks). We seek to optimize the lifetime of the network. We have developed three distributed power-aware algorithms and analyzed their efficiency in terms of the number of message broadcasts and the overall network lifetime modelled as the time to the first message that can not be sent. These are: (1) a distributed min power algorithm (modelled on a distributed version of Dijkstra's algorithm), (2) a distributed max-min algorithm, and (3) the distributed version of the centralized online max-min zP/sub min/ algorithm presented by Qun Li et al. (2001). The first two algorithms are used to define the third, although they are very interesting and useful on their own for applications where the optimization criterion is the minimum power, respectively the maximum residual power. The distributed max-min zP/sub min/ algorithm optimizes the overall lifetime of the network by avoiding nodes of low power, while not using too much total power.
10818971
Efficient Implementation of Public Key Cryptosystems on Mote Sensors (Short Paper)
We report our implementation of the RSA and ECC public-key cryptosystem on Berkeley Motes. We detail the implementation of 1024-bit RSA and 160-bit ECC cryptosystems on MICA mote sensors. We have achieved the performance of 0.79s for RSA public key operation and 21.5s for private operation, and 1.3s for ECC signature generation and 2.8s for verification. For comparison, we also show our new ECC implementation on TelosB motes with a signature time 1.60s and a verification time 3.30s. For the detailed description of the implementation, we refer to our technical report [13].
29490339
From intra-transaction to generalized inter-transaction: Landscaping multidimensional contexts in association rule mining
The problem of mining multidimensional inter-transactional association rules was recently introduced in [ACM Trans. Inform. Syst. 18(4) (2000) 423; Proc. of the ACM SIGMOD Workshop on Research Issues on Data Mining and Knowledge Discovery, Seattle, Washington, June 1998, p. 12:1]. It extends the scope of mining association rules from traditional single-dimensional intra-transactional associations to multidimensional inter-transactional associations. Inter-transactional association rules can represent not only the associations of items happening within transactions as traditional intra-transactional association rules do, but also the associations of items among different transactions under a multidimensional context. "After McDonald and Burger King open branches, KFC will open a branch two months later and one mile away" is an example of such rules. In this paper, we extend the previous problem definition based on context expansion, and present a more general form of association rules, named generalized multidimensional inter-transactional association rules. An algorithm for mining such generalized inter-transactional association rules is presented by extension of a priori. We report our experiments on applying the algorithm to both real-life and synthetic data sets. Empirical evaluation shows that with the generalized inter-transactional association rules, more comprehensive and interesting association relationships can be detected from data sets.
16882491
A MAP/G/1 Queue with Negative Customers
In this paper, we consider a MAP/G/1 queue with MAP arrivals of negative customers, where there are two types of service times and two classes of removal rules: the RCA and RCH, as introduced in section 2. We provide an approach for analyzing the system. This approach is based on the classical supplementary variable method, combined with the matrix-analytic method and the censoring technique. By using this approach, we are able to relate the boundary conditions of the system of differential equations to a Markov chain of GI/G/1 type or a Markov renewal process of GI/G/1 type. This leads to a solution of the boundary equations, which is crucial for solving the system of differential equations. We also provide expressions for the distributions of stationary queue length and virtual sojourn time, and the Laplace transform of the busy period. Moreover, we provide an analysis for the asymptotics of the stationary queue length of the MAP/G/1 queues with and without negative customers.
14645772
Three power-aware routing algorithms for sensor networks
Summary This paper discusses online power-aware routing in large wireless ad hoc networks (especially sensor networks) for applications in which the message sequence is not known. We seek to optimize the lifetime of the network. We show that online power-aware routing does not have a constant competitive ratio to the off-line optimal algorithm. We develop an approximation algorithm called max –min zPmin that has a good empirical competitive ratio. To ensure scalability, we introduce a second online algorithm for power-aware routing. This hierarchical algorithm is called zone-based routing. Our experiments show that its performance is quite good. Finally, we describe a distributed version of this algorithm that does not depend on any centralization. Copyright  2003 John Wiley & Sons, Ltd.