2443306
Long range dependence in network traffic and the closed loop behaviour of buffers under adaptive window control
We consider an Internet link carrying http-like traffic, i.e., transfers of finite volume files arriving at random time instants. These file transfers are controlled by an adaptive window protocol (AWP); an example of such a protocol is TCP. We provide analysis for the auto-covariance function of the AWP-controlled traffic into the link's buffer; this traffic, in general, cannot be represented by an on-off process. The analysis establishes that, for TCP-controlled transfer of Pareto-distributed file sizes with infinite second moment, the traffic into the link buffer is long range-dependent (LRD). We also develop an analysis for obtaining the stationary distribution of the link buffer occupancy under an AWP-controlled transfer of files sampled from some distribution. For any AWP, the analysis provides us with the Laplace-Stieltjes transform (LST) of the distribution of the link buffer occupancy process in terms of the functions defining the AWP and the file size distribution. The analysis also provides a necessary and a sufficient condition for the finiteness of the mean link buffer content; these conditions again have explicit dependence on the AWP used and the file size distribution. This establishes the sensitivity of the buffer occupancy process to the file size distribution. Combining the results from the above analyses, we provide various examples in which the closed loop control of an AWP results in finite mean link buffer occupancy even though the file sizes are Pareto-distributed (with infinite second moment), and the traffic into the link buffer is long range-dependent (with Hurst parameters which would suggest an infinite mean queue occupancy under open loop analysis). We also study the effect of window reductions due to active queue management and find that window reductions lead to further lightening of the tail of buffer occupancy distribution. The significance of this work is three-fold: (i) by looking at the window evolution as a function of the amount of data served and not as a function of time, this work provides a new framework for analysing various processes related to the link buffer under AWP-controlled transfer of files with a general file size distribution; (ii) it indicates that the buffer behaviour in the Internet may not be as poor as predicted from an open loop analysis of a queue fed with LRD traffic; and (iii) it shows that the buffer behaviour (and hence the throughput performance for finite buffers) is sensitive to the distribution of file sizes.
43715944
Performance Analysis and Scheduling of Stochastic Fork-Join Jobs in a Multicomputer System
The authors model a parallel processing system comprising several homogeneous computers interconnected by a communication network. Jobs arriving to this system have a linear fork-join structure. Each fork of the job gives rise to a random number of tasks that can be processed independently on any of the computers. Since exact analysis of fork-join models is known to be intractable, the authors resort to obtaining analytical bounds to the mean job response time of the fork-join job. For jobs with a single fork-join and, probabilistic allocation of tasks of the job to the N processors, they obtain upper and lower bounds to the mean job response time. Upper bounds are obtained using the concept of associated random variables and are found to be a good approximation to the mean job response time. A simple lower bound is obtained by neglecting queueing delays. They also find two lower bounds that include queueing delays. For multiple fork-join jobs, they study an approximation based on associated random variables. Finally, two versions of the join-the-shortest-queue (JSQ) allocation policy (i.e., JSQ by batch and JSQ by task) are studied and compared, via simulations and diffusion limits. >
15724188
Instability and performance limits of distributed simulators of feedforward queueing networks
In this article we study the performance of distributed simulation of open feedforward queueing networks, by analyzing queueing models of message flows in distributed discrete event simulators. We view each logical process in a distributed simulation as comprising a message sequencer with associated message queues, followed aby an event processor. We introduce the idealized, but analytically useful, comcept of maximum lookahead. We show that, with quite general stochatstic assumptions for message arrival and time-stamp processes, the meassage queues are unstable for conservative sequencing, and for conservative sequencing with maximum lookahead and hence for optimistic resequencing, and for any resequenceing algorithm that does not employ interprocessor flow control. Finally, we provide formulas for the throughput of distributed simulators of feedforward queueing networks.
5368108
A stochastic approximation approach for max-min fair adaptive rate control of ABR sessions with MCRs
The available bit rate (ABR) sessions in an ATM network share the bandwidth left over after guaranteeing service to constant bit rate (CBR) and variable bit rate (VBR) traffic. Hence the bandwidth available to ABR sessions is randomly varying. This bandwidth must be shared by the sessions in a max-min fair fashion. Our point of departure in this paper is to formulate the problem of determining the max-min fair session rates as the problem of finding the root of a certain nonlinear vector equation; the same formulation also arises with our notion of max-min fairness with positive minimum cell rates (MCRs). This formulation allows us to use a stochastic approximation algorithm for online distributed computation of the max-min fair rates. We use the well known ordinary differential equation technique to prove convergence of the algorithm in the synchronous update case. We provide simulation results using the NIST simulator to show that the algorithm is able to track the max-min fair rates for slowly varying random available link bandwidths.
15446858
A new approach for asynchronous distributed rate control of elastic sessions in integrated packet networks
We develop a new class of asynchronous distributed algorithms for the explicit rate control of elastic sessions in an integrated packet network. Sessions can request for minimum guaranteed rate allocations (e.g., minimum cell rates in the ATM context), and, under this constraint, we seek to allocate the max-min fair rates to the sessions. We capture the integrated network context by permitting the link bandwidths available to elastic sessions to be stochastically time varying. The available capacity of each link is viewed as some statistic of this stochastic process [e.g., a fraction of the mean, or a large deviations-based equivalent service capacity (ESC)]. The ESC is obtained so as to satisfy an overflow probability constraint on the buffer length. For fixed available capacity at each link, we show that the vector of max-min fair rates can be computed from the root of a certain vector equation. A distributed asynchronous stochastic approximation technique is then used to develop a provably convergent distributed algorithm for obtaining the root of the equation, even when the link flows and the available capacities are obtained from on-line measurements. The switch algorithm does not require per connection monitoring, nor does it require per connection marking of control packets. A virtual buffer based approach for on-line estimation of the ESC is utilized. We also propose techniques for handling large variations in the available capacity owing to the arrivals or departures of CBR/VBR sessions. Finally, simulation results are provided to demonstrate the performance of this class of algorithms in the local and wide area network context.
17761375
Comparative performance analysis of versions of TCP in a local network with a lossy link
We use a stochastic model to study the throughput performance of various transport control protocol (TCP) versions (Tahoe (including its older version that we call OldTahoe), Reno, and NewReno) in the presence of random losses on a wireless link in a local network. We model the cyclic evolution of TCP, each cycle starting at the epoch at which recovery starts from the losses in the previous cycle. TCP throughput is computed as the reward rate in a certain Markov renewal-reward process. Our model allows us to study the performance implications of various protocol features, such as fast retransmit and fast recovery. We show the impact of coarse timeouts. In the local network environment the key issue is to avoid a coarse timeout after a loss occurs. We show the effect of reducing the number of duplicate acknowledgements (ACKs) for triggering a fast retransmit. A large coarse timeout granularity seriously affects the performance of TCP, and the various protocol versions differ in their ability to avoid a coarse timeout when random loss occurs; we quantify these differences. We show that, for large packet-loss probabilities, TCP-Reno performs no better, or worse, than TCP-Tahoe. TCP-NewReno is a considerable improvement over TCP-Tahoe, and reducing the fast-retransmit threshold from three to one yields a large gain in throughput; this is similar to one of the modifications in the TCP-Vegas proposal. We explain some of these observations in terms of the variation of fast-recovery probabilities with packet-loss probability. The results of our analysis compare well with a simulation that uses actual TCP code.
7308298
Data streaming algorithms for accurate and efficient measurement of traffic and flow matrices
The traffic volume between origin/destination (OD) pairs in a network, known as traffic matrix, is essential for efficient network provisioning and traffic engineering. Existing approaches of estimating the traffic matrix, based on statistical inference and/or packet sampling, usually cannot achieve very high estimation accuracy. In this work, we take a brand new approach in attacking this problem. We propose a novel data streaming algorithm that can process traffic stream at very high speed (e.g., 40 Gbps) and produce traffic digests that are orders of magnitude smaller than the traffic stream. By correlating the digests collected at any OD pair using Bayesian statistics, the volume of traffic flowing between the OD pair can be accurately determined. We also establish principles and techniques for optimally combining this streaming method with sampling, when sampling is necessary due to stringent resource constraints. In addition, we propose another data streaming algorithm that estimates flow matrix, a finer-grained characterization than traffic matrix. Flow matrix is concerned with not only the total traffic between an OD pair (traffic matrix), but also how it splits into flows of various sizes. Through rigorous theoretical analysis and extensive synthetic experiments on real Internet traffic, we demonstrate that these two algorithms can produce very accurate estimation of traffic matrix and flow matrix respectively.
28468806
Space-code bloom filter for efficient per-flow traffic measurement
Per-flow traffic measurement is critical for usage accounting, traffic engineering, and anomaly detection. Previous methodologies are either based on random sampling (e.g., Cisco's NetFlow), which is inaccurate, or only account for the "elephants". We introduce a novel technique for measuring per-flow traffic approximately, for all flows regardless of their sizes, at very high-speed (say, OC768). The core of this technique is a novel data structure called space code bloom filter (SCBF). A SCBF is an approximate representation of a multiset; each element in this multiset is a traffic flow and its multiplicity is the number of packets in the flow. The multiplicity of an element in the multiset represented by SCBF can be estimated through either of two mechanisms-maximum likelihood estimation (MLE) or mean value estimation (MVE). Through parameter tuning, SCBF allows for graceful tradeoff between measurement accuracy and computational and storage complexity. SCBF also contributes to the foundation of data streaming by introducing a new paradigm called blind streaming. We evaluate the performance of SCBF through mathematical analysis and through experiments on packet traces gathered from a tier-1 ISP backbone. Our results demonstrate that SCBF achieves reasonable measurement accuracy with very low storage and computational complexity
2987975
Data streaming algorithms for efficient and accurate estimation of flow size distribution
Knowing the distribution of the sizes of traffic flows passing through a network link helps a network operator to characterize network resource usage, infer traffic demands, detect traffic anomalies, and accommodate new traffic demands through better traffic engineering. Previous work on estimating the flow size distribution has been focused on making inferences from sampled network traffic. Its accuracy is limited by the (typically) low sampling rate required to make the sampling operation affordable. In this paper we present a novel data streaming algorithm to provide much more accurate estimates of flow distribution, using a "lossy data structure" which consists of an array of counters fitted well into SRAM. For each incoming packet, our algorithm only needs to increment one underlying counter, making the algorithm fast enough even for 40 Gbps (OC-768) links. The data structure is lossy in the sense that sizes of multiple flows may collide into the same counter. Our algorithm uses Bayesian statistical methods such as Expectation Maximization to infer the most likely flow size distribution that results in the observed counter values after collision. Evaluations of this algorithm on large Internet traces obtained from several sources (including a tier-1 ISP) demonstrate that it has very high measurement accuracy (within 2%). Our algorithm not only dramatically improves the accuracy of flow distribution measurement, but also contributes to the field of data streaming by formalizing an existing methodology and applying it to the context of estimating the flow-distribution.
38260906
An Efficient SuperGrid Protocol for High Availability and Load Balancing
Recently there has been a resurgence of interest in quorum-based methods for maintaining consistency of a group of replicated servers as a way to achieve load balancing in addition to high availability. We describe a SuperGrid protocol that performs well for small number of items (less than 100) and evaluate its performance analytically and through simulation experiments. Although it has been shown that it is possible to construct quorum systems where the load is O(1//spl radic/N) and the availability is exp(-/spl Omega/(/spl radic/N)), these asymptotic results do not perform very well when the number of items is relatively small. In such cases, our results show that the Supergrid protocol outperforms the recent Crumbling Walls protocol and the Paths system. It is also simple and easy to implement.
10184291
Efficient and scalable query routing for unstructured peer-to-peer networks
Searching for content in peer-to-peer networks is an interesting and challenging problem. Queries in Gnutella-like unstructured systems that use flooding or random walk to search must visit O(n) nodes in a network of size n, thus consuming significant amounts of bandwidth. In this paper, we propose a query routing protocol that allows low bandwidth consumption during query forwarding using a low cost mechanism to create and maintain information about nearby objects. To achieve this, our protocol maintains a lightweight probabilistic routing table at each node that suggests the location of each object in the network. Following the corresponding routing table entries, a query can reach the destination in a small number of hops with high probability. However, maintaining routing tables in a large and highly dynamic network requires non-traditional mechanisms. We design a novel data structure called an exponentially decaying bloom filter (EDBF) that encodes such probabilistic routing tables in a highly compressed manner, and allows for efficient aggregation and propagation. The search primitives provided by our system can be used to search for single keys or multiple keywords with equal ease. Analytical modeling of our design predicts significant improvements in search efficiency, verified through extensive simulations in which we observed an order of magnitude reduction in query path length over previous proposals.