2025191
Cobot in LambdaMOO: A Social Statistics Agent
We describe our development of Cobot, a software agent who lives in LambdaMOO, a popular virtual world frequented by hundreds of users. We present a detailed discussion of the functionality that has made him one of the objects most frequently interacted with in LambdaMOO, human or artificial.
7241207
Intrinsically Motivated Reinforcement Learning
Psychologists call behavior intrinsically motivated when it is engaged in for its own sake rather than as a step toward solving a specific problem of clear practical value. But what we learn during intrinsically motivated behavior is essential for our development as competent autonomous entities able to efficiently solve a wide range of practical problems as they arise. In this paper we present initial results from a computational study of intrinsically motivated reinforcement learning aimed at allowing artificial agents to construct and extend hierarchies of reusable skills that are needed for competent autonomy.
2624439
Learning Predictive State Representations
We introduce the first algorithm for learning predictive state representations (PSRs), which are a way of representing the state of a controlled dynamical system. The state representation in a PSR is a vector of predictions of tests, where tests are sequences of actions and observations said to be true if and only if all the observations occur given that all the actions are taken. The problem of finding a good PSR. one that is a sufficient statistic for the dynamical system--can be divided into two parts: 1) discovery of a good set of tests, and 2) learning to make accurate predictions for those tests. In this paper, we present detailed empirical results using a gradient-based algorithm for addressing the second problem. Our results demonstrate several sample systems in which the algorithm learns to make correct predictions and several situations in which the algorithm is less successful. Our analysis reveals challenges that will need to be addressed in future PSR learning algorithms.
2695116
Near-Optimal Reinforcement Learning in Polynomial Time
We present new algorithms for reinforcement learning and prove that they have polynomial bounds on the resources required to achieve near-optimal return in general Markov decision processes. After observing that the number of actions required to approach the optimal return is lower bounded by the mixing time T of the optimal policy (in the undiscounted case) or by the horizon time T (in the discounted case), we then give algorithms requiring a number of actions and total computation time that are only polynomial in T and the number of states and actions, for both the undiscounted and discounted cases. An interesting aspect of our algorithms is their explicit handling of the Exploration-Exploitation trade-off.
2661414
Graphical Models for Game Theory
We introduce a compact graph-theoretic representation for multi-party game theory. Our main result is a provably correct and efficient algorithm for computing approximate Nash equilibria in one-stage games represented by trees or sparse graphs.
5053575
Bias-Variance Error Bounds for Temporal Difference Updates
Temporal difference (td) algorithms are used in reinforcement learning to compute estimates of the value of a given policy in an unknown Markov decision process (policy evaluation). We give rigorous upper bounds on the error of the closely related phased td algorithms (which differ from the standard updates in their treatment of the learning rate) as a function of the amount of experience. These upper bounds prove exponentially fast convergence, with both the rate of convergence and the asymptote strongly dependent on the length of the backups k or the parameter . Our bounds give formal verification to the well-known intuition that td methods are subject to a bias-variance tradeoff, and they lead to schedules for k and that are predicted to be better than any fixed values for these parameters. We give preliminary experimental confirmation of our theory for a version of the random walk problem.
9012983
First results in the autonomous retrieval of buried objects
We have developed an autonomous system for the retrieval of buried objects. It is designed to detect, locate and retrieve buried objects. The system is equipped with a hydraulic robot, laser range finder and a subsurface sensor. First, subsurface sensing is used to detect and locate buried objects. If an object can be reached with one dig, the excavator retrieves it directly. Otherwise a layer of soil above the object is removed and another subsurface scan is made to get a more accurate estimate of the object location. This loop is repeated until the object is retrieved. The paper presents some recent results in sensing and excavation from experiments conducted on our testbed. There are numerous potential applications for this system-hazardous waste removal, maintenance of subsurface structure, construction, and removal of unexploded buried ordnance.<<ETX>>
33975335
UCD of financial services at the smart internet technology centre
In this paper I describe the experience of contributing sociological and anthropological perspectives to the user-centered design of financial services in the Smart Internet Technology Cooperative Research Centre in Australia.
62491573
A Robotic Excavator for Autonomous Truck Loading
Excavators are used for the rapid removal of soil and other materials in mines, quarries, and construction sites. The automation of these machines offers promise for increasing productivity and improving safety. To date, most research in this area has focussed on selected parts of the problem. In this paper, we present a system that completely automates the truck loading task. The excavator uses two scanning laser rangefinders to recognize and localize the truck, measure the soil face, and detect obstacles. The excavator's software decides where to dig in the soil, where to dump in the truck, and how to quickly move between these points while detecting and stopping for obstacles. The system was fully implemented and was demonstrated to load trucks as fast as human operators.
1844435
Neural network analysis of MINERVA scene analysis benchmark
Scene analysis is an important area of research with the aim of identifying objects and their relationships in natural scenes. The MINERVA benchmark has recently been introduced in this area for testing different image processing and classification schemes. We present results on the classification of eight natural objects in the complete set of 448 natural images using neural networks. An exhaustive set of experiments with this benchmark has been conducted using four different segmentation methods and five texture-based feature extraction methods. The results in this paper show the performance of a neural network classifier on a ten fold cross-validation task. On the basis of the results produced, we are able to rank how well different image segmentation algorithms are suited to the task of region of interest identification in these images, and we also see how well texture extraction algorithms rank on the basis of classification results.
2924964
Exploiting heterogeneity in sensor networks
The presence of heterogeneous nodes (i.e., nodes with an enhanced energy capacity or communication capability) in a sensor network is known to increase network reliability and lifetime. However, questions of where how many, and what types of heterogeneous resources to deploy remain largely unexplored. We focus on energy and link heterogeneity in ad hoc sensor networks and consider resource-aware MAC and routing protocols to utilize those resources. Using analysis, simulation, and real testbed measurements, we evaluate the impact of number and placement of heterogeneous resources on performance in networks of different sizes and densities. While we prove that optimal deployment is very hard in general, we also show that only a modest number of reliable, long-range backhaul links and line-powered nodes are required to have a significant impact. Properly deployed, heterogeneity can triple the average delivery rate and provide a 5-fold increase in the lifetime (respectively) of a large batten-powered network of simple sensors.