14570975
DRMs, fair use and users' experience of sharing music
There is a mismatch between the law relating to fair use, personal use and copying; the central thrust of Digital Rights Management (DRM) and users' behavior relating to the listening and sharing of music. This paper reports on the different copyright regimes in the United States and Australia. It describes some of the current DRM systems. Against this background, the paper draws on a qualitative study to explore Australian users' experience of listening to and sharing music. A design for a good DRM has to take into account the schism between the copyright regimes and users' sharing behavior.
206599449
Downloading vs Purchase: Music Industry vs Consumers
The music industry argues that unauthorised downloading of music is detrimentally affecting the industry; a breach of copyright that needs to be stopped. The industry has taken many actions to stamp out unauthorised music downloading, including prosecution of peer-to-peer software users for breach of copyright and against peer-to-peer software suppliers for contributory copyright infringement. Industry commentators have questioned this line of reasoning particularly as there is now significant revenue from legal music downloads in the United States and Europe. This paper draws on a qualitative study of music consumers in Australia to show there is not a clear dichotomy between down- loading and purchase. It is more of a continuum. From the users' perspective, downloading is transformed to the activity of accessing and exploring music. The industry could more fruitfully focus on satisfying this basic aspect of the listening experience in DRM systems in order to work together with their customers.
8689480
Cobot: A Social Reinforcement Learning Agent
We report on the use of reinforcement learning with Cobot, a software agent residing in the well-known online community LambdaMOO. Our initial work on Cobot (Isbell et al.2000) provided him with the ability to collect social statistics and report them to users. Here we describe an application of RL allowing Cobot to take proactive actions in this complex social environment, and adapt behavior from multiple sources of human reward. After 5 months of training, and 3171 reward and punishment events from 254 different LambdaMOO users, Cobot learned nontrivial preferences for a number of users, modifing his behavior based on his current state. Here we describe LambdaMOO and the state and action spaces of Cobot, and report the statistical results of the learning experiment.
6042780
On the Convergence of Stochastic Iterative Dynamic Programming Algorithms
Recent developments in the area of reinforcement learning have yielded a number of new algorithms for the prediction and control of Markovian environments. These algorithms, including the TD() algorithm of Sutton (1988) and the Q-learning algorithm of Watkins (1989), can be motivated heuristically as approximations to dynamic programming (DP). In this paper we provide a rigorous proof of convergence of these DP-based learning algorithms by relating them to the powerful techniques of stochastic approximation theory via a new convergence theorem. The theorem establishes a general class of convergent algorithms to which both TD() and Q-learning belong.
11487617
Attaining situational awareness for sliding autonomy
We are interested in the problems of a human operator who is responsible for rapidly and accurately responding to requests for help from an autonomous robotic construction team. A difficult aspect of this problem is gaining an awareness of the requesting robot's situation quickly enough to avoid slowing the whole team down. One approach to speeding the initial acquisition of situational awareness is to maintain a buffer of data, and play it back for the human when their help is needed. We report here on an experiment to determine how the composition and length of this buffer affect the human's speed and accuracy in our multi-robot construction domain. The experiments show that, for our scenario, 5 - 10 seconds of one raw video feed led to the fastest operator attainment of situational awareness, while accuracy was maximized by viewing 10 seconds of three video feeds. These results are necessarily specific to our scenario, but we feel that they indicate general trends which may be of use in other situations. We discuss the interacting effects of buffer composition and length on operator speed and accuracy, and draw several conclusions from this experiment which may generalize to other scenarios.
18021791
Energy Consumption of TCP in Ad Hoc Networks
In this paper we study the energy cost (protocol processing and communication cost) and goodput of different flavors of TCP (Transmission Control Protocol) in ad hoc networks. We implemented a testbed and measured the actual energy cost as well as goodput of running TCP Reno, Newreno, SACK (Selective ACKnowledgement) and a version that combines Explicit Link Failure Notification (ELFN) [7] and Explicit Congestion Notification (ECN) [5] in Newreno. We see that the use of ECN & ELFN does yield higher goodput in most cases with a corresponding lower total energy cost. We see an energy savings of between 20% and 500% depending on the network conditions.
7241207
Intrinsically Motivated Reinforcement Learning
Psychologists call behavior intrinsically motivated when it is engaged in for its own sake rather than as a step toward solving a specific problem of clear practical value. But what we learn during intrinsically motivated behavior is essential for our development as competent autonomous entities able to efficiently solve a wide range of practical problems as they arise. In this paper we present initial results from a computational study of intrinsically motivated reinforcement learning aimed at allowing artificial agents to construct and extend hierarchies of reusable skills that are needed for competent autonomy.