10844784
Meta-game Equilibrium for Multi-agent Reinforcement Learning
This paper proposes a multi-agent Q-learning algorithm called meta-game-Q learning that is developed from the meta-game equilibrium concept Different from Nash equilibrium, meta-game equilibrium can achieve the optimal joint action game through deliberating its preference and predicting others' policies in the general-sum game A distributed negotiation algorithm is used to solve the meta-game equilibrium problem instead of using centralized linear programming algorithms We use the repeated prisoner's dilemma example to empirically demonstrate that the algorithm converges to meta-game equilibrium.
10779366
An efficient adaptive focused crawler based on ontology learning
The enormous growth of the World Wide Web has made it important to perform resource discovery efficiently. Consequently, several new ideas have been proposed; among them a key technique is focused crawling which is able to crawl particular topical portions of the World Wide Web quickly without having to explore all Web pages. In this paper, we present an intelligent focused crawler algorithm in which we embed ontology to evaluate the page's relevance to the topic. Compared with other algorithms using domain knowledge, our algorithm can evolve the ontology automatically during crawl process. Considering the instinct characteristics of the ontology, propagation has also been imported to accelerate the evolution of the ontology. We applied our approaches in several tasks and provided an empirical evaluation which has shown promising results.
40250467
How to Choose the Optimal Policy in Multi-agent Belief Revision?
In multi-agent system, it is not enough to maintain the coherence of single agent's belief set only. By communication, one agent removing or adding a belief sentence may influence the certainty of other agents' belief sets. In this paper, we investigate carefully some features of belief revision process in different multi-agent systems and bring forward a uniform framework - Bereframe. In Bereframe, we think there are other objects rather than maintaining the coherence or maximize certainty of single belief revision process, and agent must choose the optimal revision policy to realize these. The credibility measure is brought forward, which is used to compute the max credibility degree of the knowledge background. In cooperative multi-agent system, agents always choose the combinative policy to maximize the certainty of whole system's belief set according to the welfare principle. And in semi-competitive multi-agent system, agents choose the revision policy according to the Pareto efficiency principle. In competitive multi-agent system, the Nash equilibrium criterion is applied into agent's belief revision process.
28841129
Believability Based Iterated Belief Revision
Classical iterated belief revision methods rarely take into account the impact of the uncertain information. In this paper, an approach of believability based iterated belief revision(BIBR) is presented. BIBR relates the belief revision in the multi-agent system to the believability of information, which plays an important role in the revision process. Based on the Dempster-Shafer theory of evidence and believability function formalism, the believability of information can be obtained. The revised belief set by BIBR is dependent on the history of revision, namely, on the information received prior to the current belief set. It is proved that the BIBR operation meets the AGM postulates for belief revision and the Darwiche and Pearl postulates for iterated belief revision.
24987149
LMRL: a multi-agent reinforcement learning model and algorithm
Multi-agent reinforcement learning technologies are mainly investigated from two perspectives: one is from the concurrence, and the other from the game theory. The former chiefly applies to cooperative multi-agent systems, while the latter usually applies to coordinated multi-agent systems. However, There exist some problems such as that of the credit assignment and the multiple Nash equilibriums for agents with them. In this paper, we propose a new multi-agent reinforcement learning model and algorithm LMRL from a layer perspective. LMRL model is composed of an offline training layer that employs a single agent reinforcement learning technology to acquire stationary strategy knowledge and an online interaction layer that employs a multi-agent reinforcement learning technology and the strategy knowledge that can be revised dynamically to interact with environment. An agent with LMRL can improve its generalization, adaptability and coordination ability. Experiments show that the performance of LMRL can be better than that of single agent reinforcement learning and Nash-Q.
20003562
Mapping Web Usage Patterns to MDP Model and Mining with Reinforcement Learning
For many web usage mining applications, it is crucial to compare navigation paths of different users. This paper presents a reinforcement learning based method for mining the sequential usage patterns of user behaviors. In detail, the temporal data set about every user is constructed from the web log file, and then the navigation paths of the users are modelled using the extended Markov decision process. The proposed method could learn the dynamical sequential usage patterns on-line.
16609037
DELPHI: A pattern-based method for detecting sequence similarity
We describe DELPHI, a new computational tool for identifying sequence similarity between a query sequence and a database of proteins. Use is made of a set of patterns obtained from the underlying database through a one-time computation. The patterns are subsequently matched against every query sequence presented to the system. A pattern matched by a region of the query pinpoints a potential local similarity between that region and all of the database sequences also matching that pattern. In a final step, all such local similarities are examined more closely by aligning and scoring the corresponding query and database regions. By prudently choosing a set of patterns, the method can be used to discover weak but biologically important similarities. We provide a number of examples using both classified and unclassified proteins that corroborate this claim.
17124128
FCAN: A Structured P2P System Based on Content Query
The existing DHT-based file storage systems are suffering problems of being restricted to exact match and lacking semantic-based query. To solve these problems, a DHT-based file storage system named FCAN is proposed. Mapping a document vector from semantic space to P2P space by FastMap, FCAN can effectively reduce the dimensions of it to implement semantic-based query without affecting the semantic integrality of this document. By clustering the nearest neighbors based on centroid, FCAN is further improved to reduce the redundant messages and search time. Simulations show that FCAN has not only higher recall and precision, but also fewer search time and redundant messages
6765906
On the Threshold of Having a Linear Treewidth in Random Graphs
The concept of tree-width and tree-decomposition of a graph plays an important role in algorithms and graph theory. Many NP-hard problems have been shown to be polynomially sovable when restricted to the class of instances with a bounded tree-width. In this paper, we establish an improved lower bound on the threshold for a random graph to have a linear treewidth, which improves the previous result by Kloks [1].
8326762
A Space Efficient Algorithm for Sequence Alignment with Inversions
A dynamic programming algorithm to find an optimal alignment for a pair of DNA sequences has been described by Schoniger and Waterman. The alignments use not only substitutions, insertions, and deletions of single nucleotides, but also inversions, which are the reversed complements, of substrings of the sequences. With the restriction that the inversions are pairwise non-intersecting, their proposed algorithm runs in O(n2m2) time and consumes O(n2m2) space, where n and m are the lengths of the input sequences respectively. We develop a space efficient algorithm to compute such an optimal alignment which consumes only O(nm) space within the same amount of time. Our algorithm enables the computation for a pair of DNA sequences of length up to 10,000 to be carried out on an ordinary desktop computer. Simulation study is conducted to verify some biological facts about gene shuffling across species.
45061344
Space Complexity of Estimation of Distribution Algorithms
In this paper, we investigate the space complexity of the Estimation of Distribution Algorithms (EDAs), a class of sampling-based variants of the genetic algorithm. By analyzing the nature of EDAs, we identify criteria that characterize the space complexity of two typical implementation schemes of EDAs, the factorized distribution algorithm and Bayesian network-based algorithms. Using random additive functions as the prototype, we prove that the space complexity of the factorized distribution algorithm and Bayesian network-based algorithms is exponential in the problem size even if the optimization problem has a very sparse interaction structure.