415554
Disk scrubbing in large archival storage systems
Large archival storage systems experience long periods of idleness broken up by rare data accesses. In such systems, disks may remain powered off for long periods of time. These systems can lose data for a variety of reasons, including failures at both the device level and the block level. To deal with these failures, we must detect them early enough to be able to use the redundancy built into the storage system. We propose a process called "disk scrubbing" in a system in which drives are periodically accessed to detect drive failure. By scrubbing all of the data stored on all of the disks, we can detect block failures and compensate for them by rebuilding the affected blocks. Our research shows how the scheduling of disk scrubbing affects overall system reliability, and that "opportunistic" scrubbing, in which the system scrubs disks only when they are powered on for other reasons, performs very well without the need to power on disks solely to check them.
15027712
Performing file prediction with a program-based successor model
Increases in CPU performance have surpassed those in hard drives. As a result, disk operations have become more expensive in terms of the number of CPU cycles spent waiting for them to complete. File prediction can mitigate this problem by prefetching files into cache before they are accessed. Identifying relationships between individual files plays a key role in successfully performing file prefetching. It is well-known that previous patterns of file references can be used to predict future references. Nevertheless, knowledge about the programs producing the relationships between individual files has rarely been investigated. We present a program-based successor (PBS) model that identifies relationships between files through the names of the programs accessing them. We develop a program-based last successor (PLS) model derived from PBS to do file prediction. Our simulation results show that PLS makes 21% fewer incorrect predictions and roughly the same number of correct predictions as the last-successor (LS) model. We also examine the cache hit ratio achieved by applying PLS to the least recently used (LRU) caching algorithm and show that a cache using PLS and LRU together can perform better than a cache up to 40 times larger using LRU alone. Finally, we argue that because program-based successors are more likely to be used soon, incorrectly prefetched program-based successors are more likely to be used and thus less incorrect than incorrectly prefetched files from non-program-based models.
8965925
Power conservation strategies for MEMS-based storage devices
Power dissipation by storage systems in mobile computers accounts for a large percentage of the power consumed by the entire system. Reducing the power used by the storage device is crucial for reducing overall power consumption. A new class of secondary storage devices based on microelectromechanical systems (MEMS) promises to consume an order of magnitude less power with 10-20 times shorter latency and 10 times greater storage densities. We describe three strategies to reduce power consumption: aggressive spin-down; sequential request merging; sub-sector accesses. We show that aggressive spin-down can save up to 50% of the total energy consumed by the device at the cost of increased response time. Merging of sequential requests can save up to 18% of the servicing energy and reduce response time by about 20%. Transferring less data for small requests such as those for metadata can save 40% of the servicing energy. Finally, we show that by applying all three power management strategies simultaneously the total power consumption of MEMS-based storage devices can be reduced by about 54% with no impact on I/O performance.
5159428
Ceph: a scalable, high-performance distributed file system
We have developed Ceph, a distributed file system that provides excellent performance, reliability, and scalability. Ceph maximizes the separation between data and metadata management by replacing allocation tables with a pseudo-random data distribution function (CRUSH) designed for heterogeneous and dynamic clusters of unreliable object storage devices (OSDs). We leverage device intelligence by distributing data replication, failure detection and recovery to semi-autonomous OSDs running a specialized local object file system. A dynamic distributed metadata cluster provides extremely efficient metadata management and seamlessly adapts to a wide range of general purpose and scientific computing file system workloads. Performance measurements under a variety of workloads show that Ceph has excellent I/O performance and scalable metadata management, supporting more than 250,000 metadata operations per second.
17941466
Predicting when not to predict
File prefetching based on previous file access patterns has been shown to be an effective means of reducing file system latency by implicitly loading caches with files that are likely to be needed in the near future. Mistaken prefetching requests can be very costly in terms of added performance overheads, including increased latency and bandwidth consumption. Such costs of mispredictions are easily overlooked when considering access prediction algorithms only in terms of their accuracy; we describe a novel algorithm that uses machine learning not only to improve overall prediction accuracy, but also as a means to avoid those costly mispredictions. Our algorithm is fully adaptive to changing workloads, and is fully automated in its ability to refrain from offering predictions when they are likely to be mistaken. Our trace-based simulations show that our algorithm produces prediction accuracies of up to 98%. While this appears to be at the expense of a very slight reduction in cache hit ratios, application of this algorithm actually results in substantial reductions in unnecessary (and costly) I/O operations.
5148304
HeRMES: high-performance reliable MRAM-enabled storage
Magnetic RAM (MRAM) is a new memory technology with access and cost characteristics comparable to those of conventional dynamic RAM (DRAM) and the non-volatility of magnetic media such as disk. Simply replacing DRAM with MRAM will make main memory non-volatile, but it will not improve file system performance. However, effective use of MRAM in a file system has the potential to significantly improve performance over existing file systems. The HeRMES file system will use MRAM to dramatically improve file system performance by using it as a permanent store for both file system data and metadata. In particular, metadata operations, which make up over 50% of all file system requests [14], are nearly free in HeRMES because they do not require any disk accesses. Data requests will also be faster, both because of increased metadata request speed and because using MRAM as a non-volatile cache will allow HeRMES to better optimize data placement on disk. Though MRAM capacity is too small to replace disk entirely, HeRMES will use MRAM to provide high-speed access to relatively small units of data and metadata, leaving most file data stored on disk.
3125175
A dynamic heuristic broadcasting protocol for video-on-demand
Most existing distribution protocols for video-on-demand are tailored for a specific range of video access rates and perform poorly beyond that range. We present a dynamic heuristic broadcasting protocol that performs as well as stream tapping with unlimited extra tapping at low video access rates and has the same average bandwidth requirements as the best existing broadcasting protocols at high video access rates. We also show how our protocol can handle compressed video and adapt itself to the individual bandwidth requirements of each video.
5757820
Persuasion as a Form of Inter-Agent Negotiation
Agents in a multi-agent environment must often cooperate to achieve their objectives. In this paper an agent, B, cooperates with another agent, A, if B adopts a goal that furthers A's objectives in the environment. If agents are independent and motivated by their own interests, cooperation cannot be relied upon and it may be necessary for A to persuade B to adopt a cooperative goal. This paper is concerned with the organisation and construction of persuasive argument, and examines how a rational agent comes to hold a belief, and thus, how new beliefs might be engendered and existing beliefs altered, through the process of argumentation. Argument represents an opportunity for an agent to convince a possibly sceptical or resistant audience of the veracity of its own beliefs. This ability is a vital component of rich communication, facilitating explanation, instruction, cooperation and conflict resolution. An architecture is described in which a hierarchical planner is used to develop discourse plans which can be realised in natural language using the LOLITA system. Planning is concerned with the intentional, contextual and pragmatic aspects of discourse structure as well as with the logical form of the argument and its stylistic organisation. In this paper attention is restricted to the planning of persuasive discourse, or monologue.