8299246
HyLog: A High Performance Approach to Managing Disk Layout
Our objective is to improve disk I/O performance in multi-disk systems supporting multiple concurrent users, such as file servers, database servers, and email servers. In such systems, many disk reads are absorbed by large in-memory buffers, and so disk writes comprise a large portion of the disk I/O traffic. LFS (Log-structured File System) has the potential to achieve superior write performance by accumulating small writes into large blocks and writing them to new places, rather than overwriting on top of their old copies (called Overwrite). Although it is commonly believed that the high segment cleaning overhead of LFS makes it a poor choice for workloads with random updates, in this paper we find that because of the fast improvement of disk technologies, LFS significantly outperforms Overwrite in a wide range of system configurations and workloads (including the random update workload) under modern and future disks. 
 
LFS performs worse than Overwrite, however, when the disk space utilization is very high due to the high cleaning cost. In this paper, we propose a new approach, the Hybrid Log-structured (Hy-Log) disk layout, to overcome this problem. Hy-Log uses a log-structured approach for hot pages to achieve high write performance, and Overwrite for cold pages to reduce the cleaning cost. We compare the performance of HyLog to that of Overwrite, LFS and WOLF (the latest improvement on LFS) under various system configurations and workloads. Our results show that, in most cases, Hylog performs comparably to the best of the other three approaches.
7784369
Network bandwidth requirements for scalable on-demand streaming
Recently proposed streaming protocols are able to deliver multimedia files on-demand with the required server bandwidth growing only logarithmically with the file request rate. The same efficiencies are achieved for network bandwidth if delivery is over a true broadcast channel. This paper considers the required network bandwidth for on-demand streaming over multicast delivery trees. We consider both simple canonical delivery trees, and more complex cases in which delivery trees are constructed using both existing and new algorithms for various randomly generated network topologies and client site locations. Our results quantify the potential savings from the use of multicast trees that are configured to minimize network bandwidth rather than the latency to the content server. Further, we show that it is possible to achieve reasonably close to the minimum possible bandwidth usage for both network and server simultaneously, with a practical on-demand streaming protocol.
8188664
Scalable On-Demand Streaming of Nonlinear Media
A conventional video file contains a single temporally-ordered sequence of video frames. Clients requesting on-demand streaming of such a file receive (all or intervals of) the same content. For popular files that receive many requests during a file playback time, scalable streaming protocols based on multicast or broadcast have been devised. Such protocols require server and network bandwidth that grow much slower than linearly with the file request rate. This paper considers ldquononlinearrdquo video content in which there are parallel sequences of frames. Clients dynamically select which branch of the video they wish to follow, sufficiently ahead of each branch point so as to allow the video to be delivered without jitter. An example might be ldquochoose-your-own-endingrdquo movies. With traditional scalable delivery architectures such as movie theaters or TV broadcasting, such personalization of the delivered video content is very difficult or impossible. It becomes feasible, in principle at least, when the video is streamed to individual clients over a network. For on-demand streaming of nonlinear media, this paper analyzes the minimal server bandwidth requirements, and proposes and evaluates practical scalable delivery protocols.
8373854
Scalably supporting durable subscriptions in a publish/subscribe system
We describe algorithms to scalably support durable subscriptions in a publish-subscribe system. Durable subscriptions are guaranteed exactly-once message delivery, despite periods of disconnection from the system. Our algorithms persistently log each message only once in the system, and can support administratively specified ’early-release’ policies that reclaim persistent storage in the presence of misbehaving durable subscribers. To efficiently recover messages missed by a disconnected durable subscriber, without refiltering messages published while the subscriber was disconnected, we persistently log filtering information in a manner optimized for the read/write pattern of durable subscriptions. Consolidation of data-structures across all subscribers that are done with catching up (after a disconnection), helps the system support a larger number of subscribers. We experimentally demonstrate the low-latency and scalability properties of our implementation, both in the presence and absence of failures.
13369698
Make-to-stock systems with backorders: IPA gradients
We consider single-stage, single-product make-to-stock systems with random demand and random service (production) rate, where demand shortages at the inventory facility are backordered. The make-to-stock system is modeled as a stochastic fluid model (SFM), in which the traditional discrete arrival, service and departure stochastic processes are replaced by corresponding stochastic fluid-flow rate processes. IPA (infinitesimal perturbation analysis) gradients of various performance metrics are derived with respect to parameters of interest (here, base-stock level and production rate), and are showed to be unbiased and easy to compute. The (random) IPA gradients are obtained via sample path analysis under very mild assumptions, and are inherently nonparametric in the sense that no specific probability law need be postulated. The formulas derived can be used in simulation, as well as in real-life systems.
15790608
Clustering High-Dimensional Data with Low-Order Neighbors
Density-based and grid-based clustering are two main clustering approaches. The former is famous for its capability of discovering clusters of various shapes and eliminating noises, while the latter is well known for its high speed. Combination of the two approaches seems to provide better clustering results. To the best of our knowledge, however, all existing algorithms that combine density-based clustering and grid-based clustering take cells as atomic units, in the sense that either all objects in a cell belong to a cluster or no object in the cell belong to any cluster. This requires the cells to be small enough to ensure the fine resolution of results. In high-dimensional spaces, however, the number of cells can be very large when cells are small, which would make the clustering process extremely costly. On the other hand, the number of neighbors of a cell grows exponentially with the dimensionality of datasets, which makes the complexity increase further. In this paper, we present a new approach that takes objects (or points) as the atomic units, so that the restriction of cell size can be relaxed without degrading the resolution of clustering results. In addition, a concept of ith-order neighbors is introduced to avoid considering the exponential number of neighboring cells. By considering only low-order neighbors, our algorithm is very efficient while losing only a little bit of accuracy. Experiments on synthetic and public data show that our algorithm can cluster high-dimensional data effectively and efficiently.
42162120
Digitizing Old Photographs for the Web
In the last few years, both small family photo collections and huge library and museum collections have been digitized and made available on the Web. In this article, we discuss reasons for digitizing. We describe an ongoing project in which children and adults in a technology-oriented after-school program in rural Michigan are digitizing photographs for a local historical society.
1706033
Hierarchical Clustering Algorithms for Document Datasets
Fast and high-quality document clustering algorithms play an important role in providing intuitive navigation and browsing mechanisms by organizing large amounts of information into a small number of meaningful clusters. In particular, clustering algorithms that build meaningful hierarchies out of large document collections are ideal tools for their interactive visualization and exploration as they provide data-views that are consistent, predictable, and at different levels of granularity. This paper focuses on document clustering algorithms that build such hierarchical solutions and (i) presents a comprehensive study of partitional and agglomerative algorithms that use different criterion functions and merging schemes, and (ii) presents a new class of clustering algorithms called constrained agglomerative algorithms, which combine features from both partitional and agglomerative approaches that allows them to reduce the early-stage errors made by agglomerative methods and hence improve the quality of clustering solutions. The experimental evaluation shows that, contrary to the common belief, partitional algorithms always lead to better solutions than agglomerative algorithms; making them ideal for clustering large document collections due to not only their relatively low computational requirements, but also higher clustering quality. Furthermore, the constrained agglomerative methods consistently lead to better solutions than agglomerative methods alone and for many cases they outperform partitional methods, as well.