8942941
Improving the WWW: Caching or Multicast?
Abstract We consider two schemes for the distribution of popular Web documents. In the first scheme the sender repeatedly transmits the Web document into a multicast address, and receivers asynchronously join the corresponding multicast tree to receive a copy. In the second scheme, the document is distributed to the receivers through a hierarchy of Web caches. We develop analytical models for both schemes, and use the models to compare the two schemes in terms of latency and bandwidth usage. We find that except for documents that change very frequently, hierarchical caching gives lower latency and uses less bandwidth than multicast. For rapidly changing documents, multicast distribution reduces latency, saves network bandwidth, and reduces the load on the origin server. Furthermore, if a document is updated randomly rather than periodically, the relative performance of CMP improves. Therefore, the best overall performance is achieved when the Internet implements both solutions, hierarchical caching and multicast.
31310473
Packet multiplexers with adversarial regulated traffic
We consider a finite-buffer packet multiplexer to which traffic arrives from several independent sources. The traffic from each of the sources is regulated, i.e., the amount of traffic that can enter the multiplexer is constrained by known regulator constraints. The regulator constraints depend on, the source and are more general than those resulting from cascaded leaky buckets. We assume that the traffic is adversarial to the extent permitted by the regulators. For lossless multiplexing, we show that if the original multiplexer is lossless it is possible to allocate bandwidth and buffer to the sources so that the resulting segregated systems are lossless. For lossy multiplexing, we use our results for lossless multiplexing to estimate the loss probability of the multiplexer. Our estimate involves transforming the original system into two independent resource systems, and using adversarial sources for the two independent resources to obtain a bound on the loss probabilities for the transformed system. We show that the adversarial sources are not extremal on-off sources, even when the regulator consists of a peak rate controller in series with a leaky bucket. We explicitly characterize the form of the adversarial source for the transformed problem. We also provide numerical results for the case of the simple regulator.
15937667
A dynamic programming methodology for managing prerecorded VBR sources in packet-switched networks
Broadband networks will increasingly carry prerecorded traffic, such as high-fidelity audio, short multimedia clips, and full-length movies. We study how to manage the transmission and transport of traffic from prerecorded VBR sources so that network resources are efficiently utilized and end users receive satisfactory service. Specifically, we study Piecewise Constant-Rate Transmission and Transport (PCRTT), whereby the server transmits and the network transports each connection's packets at different constant rates over a small number of intervals. We show how Dynamic Programming (DP) can be applied to find optimal PCRTT rates and intervals for a wide variety of optimization criteria, including criteria which explicitly account for delays to user interaction (pause, fast forward, and rewind). We also introduce two admission policies for PCRTT: peak-rate admission and packing admission. For peak-rate admission, for each video the network allocates an end-to-end CBR connection; the rate of the CBR connection is equal to the maximum PCRTT rate. For packing admission, for each video the network reserves a series of CBR connections of different rates over specific time intervals. Using public domain MPEG traces, we present several numerical examples which illustrate the traffic management schemes and the DP methodology.
32291503
On Nonblocking Multirate Interconnection Networks
In a recent paper, Melen and Turner [SIAM J. Comput., 18 (1989), pp. 301–313] determined sufficient conditions for multirate interconnection networks to be strictly nonblocking and rearrangeable. They considered the continuous bandwidth case, which permits the bandwidth of a connection to take an arbitrary value in a given closed interval. In this paper, simple necessary and sufficient conditions for multirate interconnection networks to be strictly nonblocking for both discrete and continuous bandwidth cases are determined. New results for rearrangeable multirate networks with discrete bandwidth requirements are also given.
6975415
Distributing Layered Encoded Video through Caches
The efficient distribution of stored information has become a major concern in the Internet which has increasingly become a vehicle for the transport of stored video. Because of the highly heterogeneous access to the Internet, researchers and engineers have argued for layered encoded video. We investigate delivering layered encoded video using caches. Based on the stochastic knapsack theory, we develop a model for the layered video caching problem. We propose heuristics to determine which videos and which layers in the videos should be cached in order to maximize the revenue from the streaming service. We evaluate the performance of our heuristics through extensive numerical experiments. We find that, for typical scenarios, the revenue increases nearly logarithmically with the cache size and linearly with the link bandwidth that connects the cache to the origin servers. We also consider service models with request queuing and negotiations about the delivered stream quality and find that both extensions provide only small revenue increases.
369703
Object replication strategies in content distribution networks
Recently, the Internet has witnessed the emergence of content distribution networks (CDNs). In this paper, we study the problem of optimally replicating objects in CDN servers. In our model, each Internet autonomous system (AS) is a node with finite storage capacity for replicating objects. The optimization problem is to replicate objects so that when clients fetch objects from the nearest CDN server with the requested object, the average number of ASs traversed is minimized. We formulate this problem as a combinatorial optimization problem. We show that this optimization problem is NP complete. We develop four natural heuristics and compare them numerically using real Internet topology data. We find that the best results are obtained with heuristics that have all the CDN servers cooperating in making the replication decisions. We also develop a model for studying the benefits of cooperation between nodes, which provides insight into peer-to-peer content distribution.
7701041
Optimizing selections over datacubes
Datacube queries compute aggregates over database relations at a variety of granularities. Often one wants only datacube output tuples whose aggregate value satisfies a certain condition, such as exceeding a given threshold. We develop algorithms for processing a datacube query using the selection condition internally during the computation. Thus, we can safely prune parts of the computation and end up with a more efficient computation of the answer Our first technique, called "specialization", uses the fact that a tuple in the datacube does not meet the given threshold to infer that all finer level aggregates cannot meet the threshold. Our second technique is called "generalization", and applies in the case where the actual value of the aggregate is not needed in the output, but used just to compare with the threshold. We demonstrate the efficiency of these techniques by implementing them within the sparse datacube algorithm of Ross and Srivastava. We present a performance study using synthetic and real-world data sets. Our results indicate substantial performance improvements for queries with selective conditions.
11741717
Cost-based unbalanced R-trees
Cost-based unbalanced R-trees (CUR-trees) are a cost-function-based data structure for spatial data. CUR-trees are constructed specifically to improve the evaluation of intersection queries, the most basic selection query in an R-tree. A CUR-tree is built taking into account a given query distribution for the queries and a cost model for their execution. Depending on the expected frequency of access, objects or subtrees are stored higher up in the tree. After each insertion in the tree, local reorganizations of a node and its children have their expected query cost evaluated, and a reorganization is performed if this is beneficial. No strict balancing of the trees applies, allowing the tree to unfold solely based on the result of the cost evaluation. We present our cost-based approach and describe the evaluation and reorganization operations based on the cost function. We present a cost model for in-memory access costs and we present three different query models. In our experiments, we compare the performance of the CUR-tree to the R-tree and the R*-tree. The CUR-tree is able to significantly improve intersection query performance, without unacceptably increasing the cost of building the tree. The use of R-trees for in-memory data reflects the high (and growing) cost of bringing data from RAM into the CPU cache relative to the cost of other computations.
206043136
Faster Joins, Self-Joins and Multi-Way Joins Using Join Indices
We propose a new algorithm, called Stripe-join, for performing a join given a join index. Stripe-join is inspired by an algorithm called \Jive-join" developed by Li and Ross. Stripe-join makes a single sequential pass through each input relation, in addition to one pass through the join index and two passes through a set of temporary les that contain tuple identiiers but no input tuples. Stripe-join performs this eeciently even when the input relations are much larger than main memory, as long as the number of blocks in main memory is of the order of the square root of the number of blocks in the participating relations. Stripe-join is particularly eecient for self-joins. To our knowledge, Stripe-join is the rst algorithm that, given a join index and a relation signiicantly larger than main memory, can perform a self-join with just a single pass over the input relation and without storing input tuples in intermediate les. Almost all the I/O is sequential, thus minimizing the impact of seek and rotational latency. The algorithm is resistant to data skew. It can also join multiple relations while still making only a single pass over each input relation. Using a detailed cost model, Stripe-join is analyzed and compared with competing algorithms. For large input relations, Stripe-join performs signiicantly better than Valduriez's algorithm and hash join algorithms. We demonstrate circumstances under which Stripe-join performs signiicantly better than Jive-join. Unlike Jive-join, Stripe-join makes no assumptions about the order of the join index.
19447750
Local search scheduling algorithms for maximal throughput in packet switches
We consider the (generalized) packet switch scheduling problem, where the switch service configuration has to be dynamically chosen based on observed queue backlogs, so as to maximize the throughput. A class of recently developed 'projective' scheduling algorithms, which substantially generalize the well-known maximum weight matching (MWM) algorithms for crossbar switches, are explored from the perspective of complexity. The typically huge number of possible switch configurations that the scheduler has to consider in each timeslot has been previously observed to lead to an impractical computational requirement. We introduce a new class of projective schedules based on 'local search' concepts. In particular, rather than searching the entire (typically huge) set of available service configurations to find the best one, the new scalable scheduling algorithms search 'locally' over a small neighborhood of service configurations to find a 'better' one in each time slot. We show that local projective scheduling algorithms can provide dramatic reduction in complexity without causing any loss of throughput (although they may observe higher delay). We explore the nature and structure of such schedules, which show a much higher promise for practical implementation than their global versions.
2564021
Database research at Columbia University
Columbia University has a number of projects that touch on database systems issues. In this report, we describe the Columbia Fast Query Project (Section 2), the JAM project (Section 3), the CARDGIS project (Section 4), the Columbia Internet Information Searching Project (Section 5), the Columbia Content-Based Visual Query project (Section 6), and projects associated with Columbia’s Programming Systems Laboratory (Section 7).