24987149
LMRL: a multi-agent reinforcement learning model and algorithm
Multi-agent reinforcement learning technologies are mainly investigated from two perspectives: one is from the concurrence, and the other from the game theory. The former chiefly applies to cooperative multi-agent systems, while the latter usually applies to coordinated multi-agent systems. However, There exist some problems such as that of the credit assignment and the multiple Nash equilibriums for agents with them. In this paper, we propose a new multi-agent reinforcement learning model and algorithm LMRL from a layer perspective. LMRL model is composed of an offline training layer that employs a single agent reinforcement learning technology to acquire stationary strategy knowledge and an online interaction layer that employs a multi-agent reinforcement learning technology and the strategy knowledge that can be revised dynamically to interact with environment. An agent with LMRL can improve its generalization, adaptability and coordination ability. Experiments show that the performance of LMRL can be better than that of single agent reinforcement learning and Nash-Q.
28841129
Believability Based Iterated Belief Revision
Classical iterated belief revision methods rarely take into account the impact of the uncertain information. In this paper, an approach of believability based iterated belief revision(BIBR) is presented. BIBR relates the belief revision in the multi-agent system to the believability of information, which plays an important role in the revision process. Based on the Dempster-Shafer theory of evidence and believability function formalism, the believability of information can be obtained. The revised belief set by BIBR is dependent on the history of revision, namely, on the information received prior to the current belief set. It is proved that the BIBR operation meets the AGM postulates for belief revision and the Darwiche and Pearl postulates for iterated belief revision.
10779366
An efficient adaptive focused crawler based on ontology learning
The enormous growth of the World Wide Web has made it important to perform resource discovery efficiently. Consequently, several new ideas have been proposed; among them a key technique is focused crawling which is able to crawl particular topical portions of the World Wide Web quickly without having to explore all Web pages. In this paper, we present an intelligent focused crawler algorithm in which we embed ontology to evaluate the page's relevance to the topic. Compared with other algorithms using domain knowledge, our algorithm can evolve the ontology automatically during crawl process. Considering the instinct characteristics of the ontology, propagation has also been imported to accelerate the evolution of the ontology. We applied our approaches in several tasks and provided an empirical evaluation which has shown promising results.
23206649
Learning classifier system ensemble for data mining
This paper proposes LCSE, a learning classifier system ensemble, which is an extension of the classical learning classifier system(LCS). The classical LCS includes two major modules, a genetic algorithm module used to facilitate rule discovery, and a reinforcement learning module used to adjust the strength of the corresponding rules while it receives the rewards from the environment. In LCSE we build a two-level ensemble architecture to enhance the generalization of LCS. In the first-level, new instances are first bootstrapped and sent to several LCSs for classification. Then, in the second-level, a plurality-vote method is used to combine the classification results of individual LCSs into a final decision. Experiments on some benchmark data sets from the UCI repository have shown that LCSE has better generalization ability than the single LCS and other supervised learning methods.
15420304
Incremental gradient descent imputation method for missing data in learning classifier systems
Learning with incomplete or missing data has been a major challenge in learning classifier system. One method for covering missing data is imputing missing values based on the statistic of known values. Another is marking them matching arbitrary case. A new approach using incremental gradient descent imputation model is proposed in this paper, which use the relationship among variables to estimate the missing value. And, some experiments are conducted in order to compare the performance of new approach and other classical covering methods.
6963840
Applying Neural Network to Reinforcement Learning in Continuous Spaces
This paper is concerned with the problem of Reinforcement Learning (RL) in large or continuous spaces. Function approximation is the main method to solve such kind of problem. We propose using neural networks as function approximators in this paper. Then we experiment with three kind of neural networks in Mountain-Car task and illustrate comparisons among them. The result shows that CMAC and Fuzzy ARTMAP perform better than BP in Reinforcement Learning with Function Approximation (RLFA).
17160551
Pattern discovery on character sets and real-valued data: linear bound on irredundant motifs and an efficient polynomial time algorithm
Given an input sequence of data, a motif is a repeating pattern, possibly interspersed with "dont care" characters. The data could be a sequence of characters or sets of characters or even real values. In the first two cases, the number of motifs could potentially be exponential in the size of the input sequence and in the third case there could be infinite number of motifs (assuming two real numbers are equal if they are within some specified 5 > 0 of each other). However, it has been shown in some motif based applications, such as multiple sequence aligz~nent [Par98, PFR99], that a small subset of these motifs capture all the relevant information. Also, from an information-theoretic viewpoint, the special motifs represent the sets with low co-relation, hence of higher informational value. In this paper, we show, for any sequence with n characters, there exists only a linear (or no more than 3n) number of these special motifs and every other motif can be constructed from this set. We term these special motifs as irredundant motifs. We also present an efficient polynomial time algorithm to generate these motifs. This bound on the number of useful motifs ~ves validation to motif-based approaches, since the total number of irredundant motifs does not explode. This result is potentially of significance to most applications that use pattern discovery as the basic engine such as data mining, clustering, matching and others.
14861370
On transforming statistical models for non-frontal face verification
We address the pose mismatch problem which can occur in face verification systems that have only a single (frontal) face image available for training. In the framework of a Bayesian classifier based on mixtures of gaussians, the problem is tackled through extending each frontal face model with artificially synthesized models for non-frontal views. The synthesis methods are based on several implementations of maximum likelihood linear regression (MLLR), as well as standard multi-variate linear regression (LinReg). All synthesis techniques rely on prior information and learn how face models for the frontal view are related to face models for non-frontal views. The synthesis and extension approach is evaluated by applying it to two face verification systems: a holistic system (based on PCA-derived features) and a local feature system (based on DCT-derived features). Experiments on the FERET database suggest that for the holistic system, the LinReg-based technique is more suited than the MLLR-based techniques; for the local feature system, the results show that synthesis via a new MLLR implementation obtains better performance than synthesis based on traditional MLLR. The results further suggest that extending frontal models considerably reduces errors. It is also shown that the local feature system is less affected by view changes than the holistic system; this can be attributed to the parts based representation of the face, and, due to the classifier based on mixtures of gaussians, the lack of constraints on spatial relations between the face parts, allowing for deformations and movements of face areas.
18106801
Mining Protein Sequences for Motifs
We use methods from Data Mining and Knowledge Discovery to design an algorithm for detecting motifs in protein sequences. The algorithm assumes that a motif is constituted by the presence of a "good" combination of residues in appropriate locations of the motif. The algorithm attempts to compile such good combinations into a "pattern dictionary" by processing an aligned training set of protein sequences. The dictionary is subsequently used to detect motifs in new protein sequences. Statistical significance of the detection results are ensured by statistically determining the various parameters of the algorithm. Based on this approach, we have implemented a program called GYM. The Helix-Turn-Helix motif was used as a model system on which to test our program. The program was also extended to detect Homeodomain motifs. The detection results for the two motifs compare favorably with existing programs. In addition, the GYM program provides a lot of useful information about a given protein sequence.
16604799
Face Recognition Using Line Edge Map
The automatic recognition of human faces presents a significant challenge to the pattern recognition research community. Typically, human faces are very similar in structure with minor differences from person to person. They are actually within one class of "human face". Furthermore, lighting conditions change, while facial expressions and pose variations further complicate the face recognition task as one of the difficult problems in pattern analysis. This paper proposes a novel concept: namely, that faces can be recognized using a line edge map (LEM). The LEM, a compact face feature, is generated for face coding and recognition. A thorough investigation of the proposed concept is conducted which covers all aspects of human face recognition, i.e. face recognition under (1) controlled/ideal conditions and size variations, (2) varying lighting conditions, (3) varying facial expressions, and (4) varying pose. The system performance is also compared with the eigenface method, one of the best face recognition techniques, and with reported experimental results of other methods. A face pre-filtering technique is proposed to speed up the search process. It is a very encouraging to find that the proposed face recognition technique has performed better than the eigenface method in most of the comparison experiments. This research demonstrates that the LEM, together with the proposed generic line-segment Hausdorff distance measure, provides a new method for face coding and recognition.
16464835
Automated Face Pose Estimation Using Elastic Energy Models
Face pose estimation forms an important part in a face recognition system. However, fully automated and accurate pose determination still remains an unsolved problem in the research community. In this paper, we propose a novel elastic energy model to automatically estimate face poses. Our method employs statistical energy contributions of a set of feature points, which can avoid over-trusting selected anchor points. It provides a robust solution to the feature localisation inaccuracy problem, which is inevitable in practical applications with cluttered backgrounds. As a general configuration, our model can be easily implemented and extended to other non-rigid objects. Its effectiveness and robustness are revealed in our experiments