6963840
Applying Neural Network to Reinforcement Learning in Continuous Spaces
This paper is concerned with the problem of Reinforcement Learning (RL) in large or continuous spaces. Function approximation is the main method to solve such kind of problem. We propose using neural networks as function approximators in this paper. Then we experiment with three kind of neural networks in Mountain-Car task and illustrate comparisons among them. The result shows that CMAC and Fuzzy ARTMAP perform better than BP in Reinforcement Learning with Function Approximation (RLFA).
40250467
How to Choose the Optimal Policy in Multi-agent Belief Revision?
In multi-agent system, it is not enough to maintain the coherence of single agent's belief set only. By communication, one agent removing or adding a belief sentence may influence the certainty of other agents' belief sets. In this paper, we investigate carefully some features of belief revision process in different multi-agent systems and bring forward a uniform framework - Bereframe. In Bereframe, we think there are other objects rather than maintaining the coherence or maximize certainty of single belief revision process, and agent must choose the optimal revision policy to realize these. The credibility measure is brought forward, which is used to compute the max credibility degree of the knowledge background. In cooperative multi-agent system, agents always choose the combinative policy to maximize the certainty of whole system's belief set according to the welfare principle. And in semi-competitive multi-agent system, agents choose the revision policy according to the Pareto efficiency principle. In competitive multi-agent system, the Nash equilibrium criterion is applied into agent's belief revision process.
28841129
Believability Based Iterated Belief Revision
Classical iterated belief revision methods rarely take into account the impact of the uncertain information. In this paper, an approach of believability based iterated belief revision(BIBR) is presented. BIBR relates the belief revision in the multi-agent system to the believability of information, which plays an important role in the revision process. Based on the Dempster-Shafer theory of evidence and believability function formalism, the believability of information can be obtained. The revised belief set by BIBR is dependent on the history of revision, namely, on the information received prior to the current belief set. It is proved that the BIBR operation meets the AGM postulates for belief revision and the Darwiche and Pearl postulates for iterated belief revision.
10779366
An efficient adaptive focused crawler based on ontology learning
The enormous growth of the World Wide Web has made it important to perform resource discovery efficiently. Consequently, several new ideas have been proposed; among them a key technique is focused crawling which is able to crawl particular topical portions of the World Wide Web quickly without having to explore all Web pages. In this paper, we present an intelligent focused crawler algorithm in which we embed ontology to evaluate the page's relevance to the topic. Compared with other algorithms using domain knowledge, our algorithm can evolve the ontology automatically during crawl process. Considering the instinct characteristics of the ontology, propagation has also been imported to accelerate the evolution of the ontology. We applied our approaches in several tasks and provided an empirical evaluation which has shown promising results.
23206649
Learning classifier system ensemble for data mining
This paper proposes LCSE, a learning classifier system ensemble, which is an extension of the classical learning classifier system(LCS). The classical LCS includes two major modules, a genetic algorithm module used to facilitate rule discovery, and a reinforcement learning module used to adjust the strength of the corresponding rules while it receives the rewards from the environment. In LCSE we build a two-level ensemble architecture to enhance the generalization of LCS. In the first-level, new instances are first bootstrapped and sent to several LCSs for classification. Then, in the second-level, a plurality-vote method is used to combine the classification results of individual LCSs into a final decision. Experiments on some benchmark data sets from the UCI repository have shown that LCSE has better generalization ability than the single LCS and other supervised learning methods.
20003562
Mapping Web Usage Patterns to MDP Model and Mining with Reinforcement Learning
For many web usage mining applications, it is crucial to compare navigation paths of different users. This paper presents a reinforcement learning based method for mining the sequential usage patterns of user behaviors. In detail, the temporal data set about every user is constructed from the web log file, and then the navigation paths of the users are modelled using the extended Markov decision process. The proposed method could learn the dynamical sequential usage patterns on-line.
16464835
Automated Face Pose Estimation Using Elastic Energy Models
Face pose estimation forms an important part in a face recognition system. However, fully automated and accurate pose determination still remains an unsolved problem in the research community. In this paper, we propose a novel elastic energy model to automatically estimate face poses. Our method employs statistical energy contributions of a set of feature points, which can avoid over-trusting selected anchor points. It provides a robust solution to the feature localisation inaccuracy problem, which is inevitable in practical applications with cluttered backgrounds. As a general configuration, our model can be easily implemented and extended to other non-rigid objects. Its effectiveness and robustness are revealed in our experiments
5548574
On local features for GMM based face verification
It has been recently shown that local feature approaches to face verification are considerably more robust than holistic approaches, in terms of translations (caused by automatic face localization) and pose variations. In this paper, we first investigate whether features based on local principal component analysis (LPCA) are more discriminative than features based on the 2D discrete cosine transform (2D DCT). We also investigate several methods for modifying the two feature extraction techniques in order to counteract the effects of linear and non-linear illumination changes, without losing discriminative information. Results on the XM2VTS database show that when using a Bayesian classifier based on Gaussian mixture models (GMMs), the performances of 2D DCT and LPCA techniques are quite similar, suggesting that the 2D DCT technique is preferable due to its lower computational complexity. When using 8/spl times/8 blocks, modifying the 2D DCT and LPCA techniques by removing the first coefficient, which is the most affected by illumination changes, enhances robustness with little change in discrimination ability; removing further coefficients causes a noticeable reduction in performance on clean images and provides little gain in robustness. When using the 2D DCT with 16/spl times/16 blocks, the first three coefficients need to be removed in order to achieve good robustness. It is further shown that contrary to previously published results, the use of deltas of low-order coefficients (to alleviate performance losses caused by removing coefficients) can adversely affect robustness.
8115984
Facial expression recognition from line-based caricatures
The automatic recognition of facial expression presents a significant challenge to the pattern analysis and man-machine interaction research community. Recognition from a single static image is particularly a difficult task. In this paper, we present a methodology for facial expression recognition from a single static image using line-based caricatures. The recognition process is completely automatic. It also addresses the computational expensive problem and is thus suitable for real-time applications. The proposed approach uses structural and geometrical features of a user sketched expression model to match the line edge map (LEM) descriptor of an input face image. A disparity measure that is robust to expression variations is defined. The effectiveness of the proposed technique has been evaluated and promising results are obtained. This work has proven the proposed idea that facial expressions can be characterized and recognized by caricatures.
41738297
Human face profile recognition using attributed string
A new attributed string matching method for human face profile recognition is proposed in this work. It is a novel idea to apply structural and syntactic technique on face profile matching. The approach works on a chain of profile line segments and highlights the favor of curve matching by suppressing the operations of insert and delete. The technique relies mainly on the merge and change operations of string to tackle the inconsistency problem of feature point detection. A quadratic penalty function is proposed to prohibit large angle changes and overmerging. The method produces very encouraging results and is found to be suitable for similar shape classification.
17124128
FCAN: A Structured P2P System Based on Content Query
The existing DHT-based file storage systems are suffering problems of being restricted to exact match and lacking semantic-based query. To solve these problems, a DHT-based file storage system named FCAN is proposed. Mapping a document vector from semantic space to P2P space by FastMap, FCAN can effectively reduce the dimensions of it to implement semantic-based query without affecting the semantic integrality of this document. By clustering the nearest neighbors based on centroid, FCAN is further improved to reduce the redundant messages and search time. Simulations show that FCAN has not only higher recall and precision, but also fewer search time and redundant messages