12769414
Robot learning with GA-based fuzzy reinforcement learning agents
Abstract How to learn from both expert knowledge and measurement-based information for a robot to acquire perception and motor skills is a challenging research topic in the field of autonomous robotic systems. For this reason, a general GA (genetic algorithm)-based fuzzy reinforcement learning (GAFRL) agent is proposed in this paper. We first characterize the robot learning problem and point out some major issues that need to be addressed in conjunction with reinforcement learning. Based on a neural fuzzy network architecture of the GAFRL agent, we then discuss how different kinds of expert knowledge and measurement-based information can be incorporated in the GAFRL agent so as to accelerate its learning. By making use of the global optimization capability of GAs, the GAFRL can solve the local minima problem in traditional actor-critic reinforcement learning. On the other hand, with the prediction capability of the critic network, GAs can evaluate the candidate solutions regularly even during the periods without external feedback from the environment. This can guide GAs to perform a more effective global search. Finally, different types of GAFRL agents are constructed and verified using the simulation model of a physical biped robot.
9275481
RoboCup 2004 Competitions and Symposium: A Small Kick for Robots, a Giant Score for Science
RoboCup is an international initiative with the main goals of fostering research and education in artificial intelligence and robotics, as well as of promoting science and technology to world citizens. The idea behind RoboCup is to provide a standard problem for which a wide range of technologies can be integrated and examined, as well as being used for project-oriented education, and to organize annual events open to the general public, at which different solutions to the problem are compared. The eighth annual RoboCup -- RoboCup 2004 -- was held in Lisbon, Portugal, from 27 June to 5 July. In this article, a general description of RoboCup 2004 is presented, including summaries concerning teams, participants, distribution into leagues, main research advances, as well as detailed descriptions for each league.
7697882
Incorporating Perception-Based Information in Reinforcement Learning Using Computing with Words
In this paper, a general fuzzy reinforcement learning (FRL) agent that can utilise not only measurement-based information but also perception-based information by means of computing with words (CW) is proposed. By introducing fuzzy numbers and their arithmetic operations and fuzzy Lyapunov synthesis in the domain of CW, a set of stable fuzzy control rules can be derived from perception-based information. Moreover, based on a neurofuzzy network architecture, the fuzzy rules can be incorporated in the FRL agent to initialise its action network, critic network and evaluation feedback module so as to improve the learning. The performance and applicability of the proposed approach are illustrated through the practical implementation of learning control of an autonomous pole-balancing mobile robot.
14695460
Evaluation of a Decentralized Architecture for Large Scale Collaborative Intrusion Detection
An important problem in network intrusion detection is how to detect large scale coordinated attacks such as scans, worms and denial-of-service attacks. These coordinated attacks can be difficult to detect at an early stage, since the evidence of the attack may be widely distributed across different subnetworks in the Internet. A critical issue for research is how to detect these large scale attacks by correlating information from multiple intrusion detection systems in an efficient manner. Several collaborative detection systems have been proposed in the literature. However, these proposals have lacked large scale testing in real networks, and the practicalities of how to optimize the trade-off between detection accuracy and reaction time of these systems has not been demonstrated. To address these challenges, we propose LarSID, a scalable decentralized large scale intrusion detection framework. LarSID provides a service for defending against attacks by sharing potential evidence of intrusions between participant intrusion detection systems via a distributed hash table (DHT) architecture. In particular, we investigate how to optimize the trade-off between detection accuracy and reaction time of LarSID based on an analysis of a large, real-world intrusion detection dataset (DShield Dataset), which has been collected from over 1600 firewall administrators across the world. LarSID has been deployed and tested on the PlanetLab testbed, and is built on top of OpenDHT - a public DHT service. Our experimental results show significant reductions in detection latency compared to a centralized detection architecture. Currently, LarSID has been deployed on 128 PlanetLab nodes as a large scale intrusion detection service.
1882154
Providing Quality of Service in IEEE 802.11 WLAN
This paper aims to provide quality of service (QoS) for IEEE 802.11 wireless local area network (WLAN). Specifically, we develop a new scheduling algorithm in link layer to support multimedia services with guaranteed QoS in WLAN. The proposed scheduling algorithm makes use of the idle system time to reduce average packet loss of real-time (RT) variable-bit-rate (VBR) services, while satisfying required transmission delay and bandwidth requirements of all RT services. Detailed analysis is given by taking into account several key statistical factors, such as service traffic model, the ratio between the polling period and the contention period (CP), burst generation probabilities. Performance is evaluated through OPNET simulation. Simulation results demonstrate that our proposed joint scheme is effective in terms of both system resource utilization and QoS provisioning
2612705
Credit Scoring Model Based on Neural Network with Particle Swarm Optimization
Credit scoring has gained more and more attentions both in academic world and the business community today. Many modeling techniques have been developed to tackle the credit scoring tasks. This paper presents a Structure-tuning Particle Swarm Optimization (SPSO) approach for training feed-forward neural networks (NNs). The algorithm is successfully applied to a real credit problem. By simultaneously tuning the structure and connection weights of NNs, the proposed algorithm generates optimized NNs with problem-matched information processing capacity and it also eliminates some ill effects introduced by redundant input features and the corresponding redundant structure. Compared with BP and GA, SPSO can improve the pattern classification accuracy of NNs while speeding up the convergence of training process.
1105765
Service discovery and measurement based on DAML-QoS ontology
As more and more Web services are deployed, Web service's discovery mechanisms become essential. Similar services can have quite different QoS behaviors. For service selection and management purpose, it is necessary to clearly specify QoS constraints and metrics definitions for Web services. We investigate on the semantic QoS specification and introduce our design principles on it. Based on the specification refinement and conformance, we introduce the QoS matchmaking algorithm with multiple matching degrees. The matchmaking prototype is designed to prove the feasibility. Well-defined Metrics can be further utilized by measurement organizations to monitor and evaluate the promised service level objectives.
17546380
Web Services Discovery with DAML-QoS Ontology
As more and more Web services are deployed, Web serviceâ€™s discovery mechanisms become essential. Similar services from different sources often exhibit quite different Quality of Service (QoS) levels. For service selection and management purpose, it is necessary to explicitly, precisely, and flexibly specify various constraints and QoS metrics for Web services descriptions. This paper provides a novel ontology as a complement to DARPA Agent Markup Languageâ€”Service (DAML-S) Ontology to provide a better QoS metrics model. Three layers are defined, together with clear role descriptions for development. Cardinality constraints are utilized to describe the QoS property constraints. A matchmaking algorithm with multiple matching degrees is provided accordingly. When incorporated with DAML-S, the multiple service level objectives can be described by assigning multiple QoS profiles to one service profile. A prototype system was developed, which demonstrates the feasibility for small to medium-sized service advertisement repository.