722371
Design and Analysis of Deadline and Budget Constrained Autoscaling (DBCA) Algorithm for 5G Mobile Networks
In cloud computing paradigm, virtual resource autoscaling approaches have been intensively studied recent years. Those approaches dynamically scale in/out virtual resources to adjust system performance for saving operation cost. However, designing the autoscaling algorithm for desired performance with limited budget, while considering the existing capacity of legacy network equipment, is not a trivial task. In this paper, we propose a Deadline and Budget Constrained Autoscaling (DBCA) algorithm for addressing the budget-performance tradeoff. We develop an analytical model to quantify the tradeoff and cross-validate the model by extensive simulations. The results show that the DBCA can significantly improve system performance given the budget upper-bound. In addition, the model provides a quick way to evaluate the budget-performance tradeoff and system design without wide deployment, saving on cost and time.
114034803
Dynamic Resource Allocation using Virtual Machines for Cloud Computing Environment
Cloud computing allows business customers to scale up and down their resource usage based on needs. Many of the touted gains in the cloud model come from resource multiplexing through virtualization technology. In this paper, we present a system that uses virtualization technology to alloc ate data center resources dynamically based on application demands and support green computing by optimizing the number of servers in use. We introduce the concept of "skewness" to measure the unevenness in the multi-dimensional resource utilization of a ser ver. By minimizing skewness, we can combine different types of workloads nicely and improve the overall utilization of server resources. We develop a set of heuristics that prev ent overload in the system effectively while saving energy used. Tra ce driven simulation and experiment results demonstrate that our algorithm achieves good performance
5042811
Dynamic Auto Scaling Algorithm (DASA) for 5G Mobile Networks
Network Function Virtualization (NFV) enables mobile operators to virtualize their network entities as Virtualized Network Functions (VNFs), offering fine-grained on-demand network capabilities. VNFs can be dynamically scale-in/out to meet the performance desire and other dynamic behaviors. However, designing the auto-scaling algorithm for desired characteristics with low operation cost and low latency, while considering the existing capacity of legacy network equipment, is not a trivial task. In this paper, we propose a VNF Dynamic Auto Scaling Algorithm (DASA) considering the tradeoff between performance and operation cost. We develop an analytical model to quantify the tradeoff and validate the analysis through extensive simulations. The results show that the DASA can significantly reduce operation cost given the latency upper-bound. Moreover, the models provide a quick way to evaluate the cost- performance tradeoff and system design without wide deployment, which can save cost and time.
28557693
Service center trade-offs between customer impatience and power consumption
We consider the problem of managing a service center where it is desirable to keep power consumption low. Customers defect (i.e. leave the system) if they have to wait too long before starting service. A block of servers is designated as 'reserve'. The reserves are powered up when the number of jobs in the system is sufficiently high, and are powered down when that number is sufficiently low. Powering up takes an interval of time during which the reserves consume power but do not serve jobs. The objective function to be minimized takes into account both the number of servers that are powered up and the number of jobs that are lost through defections. The question of how to choose the number of reserves, and the up and down thresholds, is answered by analyzing a suitable queueing model. Heuristics and numerical results are also presented.
29676581
Multiserver Queues with Finite Capacity and Setup Time
Multiserver queues with setup time have been extensively studied because they have application in modelling of power-saving data centers. Although the infinite buffer models are extensively investigated, less attention has been paid to finite buffer models. This paper considers an M/M/\(c\)/\(K\) queue with setup time for which we suggest a simple and numerically stable recursion for the stationary distribution of the system state. Numerical experiments show various insights into the performance of the system such as performance-energy tradeoff as well as the effect of the capacity on the blocking probability and the mean queue length.
62642759
Early observations on the performance of Windows Azure
A significant open issue in cloud computing is the real performance of the infrastructure. Few, if any, cloud providers or technologies offer quantitative performance guarantees. Regardless of the potential advantages of the cloud in comparison to enterprise-deployed applications, cloud infrastructures may ultimately fail if deployed applications cannot predictably meet behavioral requirements. In this paper, we present the results of comprehensive performance experiments we conducted on Windows Azure from October 2009 to February 2010. In general, we have observed good performance of the Windows Azure mechanisms, although the average 10 min VM startup time must be accounted for in application design. We also present performance and reliability observations and analysis from our deployment of a large-scale scientific application hosted on Azure, called ModisAzure, that show unusual and sporadic VM execution slowdown of over 4× in some cases and affected up to 16% of task executions at times. In addition to a detailed performance evaluation of Windows Azure, we provide recommendations for potential users of Windows Azure based on these early observations. Although the discussion and analysis is tailored to scientific applications, the results are broadly applicable to the range of existing and future applications running in Windows Azure.
16135116
Prediction-Based Dynamic Resource Allocation for Video Transcoding in Cloud Computing
This paper presents prediction-based dynamic resource allocation algorithms to scale video transcoding service on a given Infrastructure as a Service cloud. The proposed algorithms provide mechanisms for allocation and deallocation of virtual machines (VMs) to a cluster of video transcoding servers in a horizontal fashion. We use a two-step load prediction method, which allows proactive resource allocation with high prediction accuracy under real-time constraints. For cost-efficiency, our work supports transcoding of multiple on-demand video streams concurrently on a single VM, resulting in a reduced number of required VMs. We use video segmentation at group of pictures level, which splits video streams into smaller segments that can be transcoded independently of one another. The approach is demonstrated in a discrete-event simulation and an experimental evaluation involving two different load patterns.
5995177
Efficient Autoscaling in the Cloud Using Predictive Models for Workload Forecasting
Large-scale component-based enterprise applications that leverage Cloud resources expect Quality of Service(QoS) guarantees in accordance with service level agreements between the customer and service providers. In the context of Cloud computing, auto scaling mechanisms hold the promise of assuring QoS properties to the applications while simultaneously making efficient use of resources and keeping operational costs low for the service providers. Despite the perceived advantages of auto scaling, realizing the full potential of auto scaling is hard due to multiple challenges stemming from the need to precisely estimate resource usage in the face of significant variability in client workload patterns. This paper makes three contributions to overcome the general lack of effective techniques for workload forecasting and optimal resource allocation. First, it discusses the challenges involved in auto scaling in the cloud. Second, it develops a model-predictive algorithm for workload forecasting that is used for resource auto scaling. Finally, empirical results are provided that demonstrate that resources can be allocated and deal located by our algorithm in a way that satisfies both the application QoS while keeping operational costs low.
15668504
Predictive Data Grouping and Placement for Cloud-Based Elastic Server Infrastructures
Workload variations on Internet platforms such as YouTube, Flickr, LastFM require novel approaches to dynamic resource provisioning in order to meet QoS requirements, while reducing the Total Cost of Ownership (TCO) of the infrastructures. The economy of scale promise of cloud computing is a great opportunity to approach this problem, by developing elastic large scale server infrastructures. However, a proactive approach to dynamic resource provisioning requires prediction models forecasting future load patterns. On the other hand, unexpected volume and data spikes require reactive provisioning for serving unexpected surges in workloads. When workload can not be predicted, adequate data grouping and placement algorithms may facilitate agile scaling up and down of an infrastructure. In this paper, we analyze a dynamic workload of an on-line music portal and present an elastic Web infrastructure that adapts to workload variations by dynamically scaling up and down servers. The workload is predicted by an autoregressive model capturing trends and seasonal patterns. Further, for enhancing data locality, we propose a predictive data grouping based on the history of content access of a user community. Finally, in order to facilitate agile elasticity, we present a data placement based on workload and access pattern prediction. The experimental results demonstrate that our forecasting model predicts workload with a high precision. Further, the predictive data grouping and placement methods provide high locality, load balance and high utilization of resources, allowing a server infrastructure to scale up and down depending on workload.
2523313
Quality-assured cloud bandwidth auto-scaling for video-on-demand applications
There has been a recent trend that video-on-demand (VoD) providers such as Netflix are leveraging resources from cloud services for multimedia streaming. In this paper, we consider the scenario that a VoD provider can make reservations for bandwidth guarantees from cloud service providers to guarantee the streaming performance in each video channel. We propose a predictive resource auto-scaling system that dynamically books the minimum bandwidth resources from multiple data centers for the VoD provider to match its short-term demand projections. We exploit the anti-correlation between the demands of video channels for statistical multiplexing and for hedging the risk of under-provision. The optimal load direction from channels to data centers is derived with provable performance. We further provide suboptimal solutions that balance bandwidth and storage costs. The system is backed up by a demand predictor that forecasts the demand expectation, volatility and correlations based on learning. Extensive simulations are conducted driven by the workload traces from a commercial VoD system.
8682512
Workload Prediction Using ARIMA Model and Its Impact on Cloud Applications’ QoS
As companies shift from desktop applications to cloud-based software as a service (SaaS) applications deployed on public clouds, the competition for end-users by cloud providers offering similar services grows. In order to survive in such a competitive market, cloud-based companies must achieve good quality of service (QoS) for their users, or risk losing their customers to competitors. However, meeting the QoS with a cost-effective amount of resources is challenging because workloads experience variation overtime. This problem can be solved with proactive dynamic provisioning of resources, which can estimate the future need of applications in terms of resources and allocate them in advance, releasing them once they are not required. In this paper, we present the realization of a cloud workload prediction module for SaaS providers based on the autoregressive integrated moving average (ARIMA) model. We introduce the prediction based on the ARIMA model and evaluate its accuracy of future workload prediction using real traces of requests to Web servers. We also evaluate the impact of the achieved accuracy in terms of efficiency in resource utilization and QoS. Simulation results show that our model is able to achieve an average accuracy of up to 91 percent, which leads to efficiency in resource utilization with minimal impact on the QoS.
14983111
Empirical prediction models for adaptive resource provisioning in the cloud
Cloud computing allows dynamic resource scaling for enterprise online transaction systems, one of the key characteristics that differentiates the cloud from the traditional computing paradigm. However, initializing a new virtual instance in a cloud is not instantaneous; cloud hosting platforms introduce several minutes delay in the hardware resource allocation. In this paper, we develop prediction-based resource measurement and provisioning strategies using Neural Network and Linear Regression to satisfy upcoming resource demands. Experimental results demonstrate that the proposed technique offers more adaptive resource management for applications hosted in the cloud environment, an important mechanism to achieve on-demand resource allocation in the cloud.
17875157
Cloud Client Prediction Models for Cloud Resource Provisioning in a Multitier Web Application Environment
In order to meet Service Level Agreement (SLA) requirements, efficient scaling of Virtual Machine (VM) resources must be provisioned few minutes ahead due to the VM boot-up time. One way to proactively provision resources is by predicting future resource demands. In this research, we have developed and evaluated cloud client prediction models for TPC-W benchmark web application using three machine learning techniques: Support Vector Machine (SVM), Neural Networks (NN) and Linear Regression (LR). We included the SLA metrics for Response Time and Throughput to the prediction model with the aim of providing the client with a more robust scaling decision choice. Our results show that Support Vector Machine provides the best prediction model.
216963
CloudScale: elastic resource scaling for multi-tenant cloud systems
Elastic resource scaling lets cloud systems meet application service level objectives (SLOs) with minimum resource provisioning costs. In this paper, we present CloudScale, a system that automates fine-grained elastic resource scaling for multi-tenant cloud computing infrastructures. CloudScale employs online resource demand prediction and prediction error handling to achieve adaptive resource allocation without assuming any prior knowledge about the applications running inside the cloud. CloudScale can resolve scaling conflicts between applications using migration, and integrates dynamic CPU voltage/frequency scaling to achieve energy savings with minimal effect on application SLOs. We have implemented CloudScale on top of Xen and conducted extensive experiments using a set of CPU and memory intensive applications (RUBiS, Hadoop, IBM System S). The results show that CloudScale can achieve significantly higher SLO conformance than other alternatives with low resource and energy cost. CloudScale is non-intrusive and light-weight, and imposes negligible overhead (< 2% CPU in Domain 0) to the virtualized computing cluster.
7032175
Workload characterization and prediction in the cloud: A multiple time series approach
Cloud computing promises high scalability, flexibility and cost-effectiveness to satisfy emerging computing requirements. To efficiently provision computing resources in the cloud, system administrators need the capabilities of characterizing and predicting workload on the Virtual Machines (VMs). In this paper, we use data traces obtained from a real data center to develop such capabilities. First, we search for repeatable workload patterns by exploring cross-VM workload correlations resulted from the dependencies among applications running on different VMs. Treating workload data samples as time series, we develop a co-clustering technique to identify groups of VMs that frequently exhibit correlated workload patterns, and also the time periods in which these VM groups are active. Then, we introduce a method based on Hidden Markov Modeling (HMM) to characterize the temporal correlations in the discovered VM clusters and to predict variations of workload patterns. The experimental results show that our method can not only help better understand group-level workload characteristics, but also make more accurate predictions on workload changes in a cloud.
31809147
Exact analysis of the M/M/k/setup class of Markov chains via recursive renewal reward
The M/M/k/setup model, where there is a penalty for turning servers on, is common in data centers, call centers, and manufacturing systems. Setup costs take the form of a time delay, and sometimes there is additionally a power penalty, as in the case of data centers. While the M/M/1/setup was exactly analyzed in 1964, no exact analysis exists to date for the M/M/k/setup with $$k>1$$k>1. In this paper, we provide the first exact, closed-form analysis for the M/M/k/setup and some of its important variants including systems in which idle servers delay for a period of time before turning off or can be put to sleep. Our analysis is made possible by a new way of combining renewal reward theory and recursive techniques to solve Markov chains with a repeating structure. Our renewal-based approach uses ideas from renewal reward theory and busy period analysis to obtain closed-form expressions for metrics of interest such as the transform of time in system and the transform of power consumed by the system. The simplicity, intuitiveness, and versatility of our renewal-based approach makes it useful for analyzing Markov chains far beyond the M/M/k/setup. In general, our renewal-based approach should be used to reduce the analysis of any 2-dimensional Markov chain which is infinite in at most one dimension and repeating to the problem of solving a system of polynomial equations. In the case where all transitions in the repeating portion of the Markov chain are skip-free and all up/down arrows are unidirectional, the resulting system of equations will yield a closed-form solution.
12276102
Managing performance and power consumption in a server farm
We examine the problem of managing a server farm in a way that attempts to satisfy the conflicting objectives of high performance and low power consumption. A subset of servers is designated as ‘reserve’. The reserves are powered up when the number of jobs in the system is sufficiently high, and are powered down when that number is sufficiently low. Powering up takes an interval of time during which the reserves consume power but do not serve jobs. The question of how to choose the number of reserves, and the up and down thresholds, is answered by analyzing a suitable queueing model and minimizing an appropriate cost function. Heuristics and numerical results are also presented.
109828225
Power consumption analysis for data centers with independent setup times and threshold controls
Data center consumes a large amount of energy since it has a huge number of servers. Therefore, a reduction of a few percent of the power consumption means saving a large amount of money and the environment. In the current technology, an idle server still consumes about 65% of its peak. Thus, an efficient way to save energy is the shutdown of server when the workload is low. Mitrani [3] proposes a threshold control policy where a block of servers is turned on and off according to two thresholds for the number of jobs present in the system. Relaxing the assumption in Mitrani [1] that after a setup time, all the servers in the block are active concurrently, we consider a more realistic model where each server has an independent setup time. We formulate the system by a two dimensional Markov chain and derive performance measures using a generating function approach. The results allow us to investigate the performance and energy consumption trade-off.
32157345
Trading Power Consumption against Performance by Reserving Blocks of Servers
We consider the problem of managing a service center where it is desirable to keep power consumption low. Customers may be patient or they may defect (i.e. leave the system) if waiting times are too long. Several blocks of servers are designated as 'reserves'. They are powered up and down when the queue size increases above or falls below certain thresholds. Objective functions to be minimized take into account the number of servers that are powered up, the number of occasions when blocks are powered up and a performance measure involving either waiting time or number of jobs lost through defections. The question of how to choose the blocks of reserves is answered by analyzing a suitable queueing model. Heuristic policies are proposed and numerical results are presented.
11574687
NFV: state of the art, challenges, and implementation in next generation mobile networks (vEPC)
As mobile network users look forward to the connectivity speeds of 5G networks, service providers are facing challenges in complying with connectivity demands without substantial financial investments. Network function virtualization (NFV) is introduced as a new methodology that offers a way out of this bottleneck. NFV is poised to change the core structure of telecommunications infrastructure to be more cost-efficient. In this article, we introduce an NFV framework, and discuss the challenges and requirements of its use in mobile networks. In particular, an NFV framework in the virtual environment is proposed. Moreover, in order to reduce signaling traffic and achieve better performance, this article proposes a criterion to bundle multiple functions of a virtualized evolved packet core in a single physical device or a group of adjacent devices. The analysis shows that the proposed grouping can reduce the network control traffic by 70 percent.
12407443
Inside the Social Network's (Datacenter) Network
Large cloud service providers have invested in increasingly larger datacenters to house the computing infrastructure required to support their services. Accordingly, researchers and industry practitioners alike have focused a great deal of effort designing network fabrics to efficiently interconnect and manage the traffic within these datacenters in performant yet efficient fashions. Unfortunately, datacenter operators are generally reticent to share the actual requirements of their applications, making it challenging to evaluate the practicality of any particular design. Moreover, the limited large-scale workload information available in the literature has, for better or worse, heretofore largely been provided by a single datacenter operator whose use cases may not be widespread. In this work, we report upon the network traffic observed in some of Facebook's datacenters. While Facebook operates a number of traditional datacenter services like Hadoop, its core Web service and supporting cache infrastructure exhibit a number of behaviors that contrast with those reported in the literature. We report on the contrasting locality, stability, and predictability of network traffic in Facebook's datacenters, and comment on their implications for network architecture, traffic engineering, and switch design.
14175988
Application and network performance of Amazon elastic compute cloud instances
The computing and networking infrastructure in public clouds is shared between multiple users and can create abnormal variations in the performance of applications running in cloud. In this paper we compare the performance of compute-intensive applications on CPUs and GPUs offered by Amazon's cloud and show that not all applications exhibit speedups when executed on the GPU. Even for applications that exhibit speedup on the GPU, the overall application performance may be bottlenecked by network delay. For such cases, the high cost of GPU instances is inefficient since it does not improve application performance. We also show that the network performance of different instances can vary significantly overtime and public clouds may throttle network bandwidth for applications that generate significant network traffic.
1285357
A Performance Study on the VM Startup Time in the Cloud
One of many advantages of the cloud is the elasticity, the ability to dynamically acquire or release computing resources in response to demand. However, this elasticity is only meaningful to the cloud users when the acquired Virtual Machines (VMs) can be provisioned in time and be ready to use within the user expectation. The long unexpected VM startup time could result in resource under-provisioning, which will inevitably hurt the application performance. A better understanding of the VM startup time is therefore needed to help cloud users to plan ahead and make in-time resource provisioning decisions. In this paper, we study the startup time of cloud VMs across three real-world cloud providers -- Amazon EC2, Windows Azure and Rackspace. We analyze the relationship between the VM startup time and different factors, such as time of the day, OS image size, instance type, data center location and the number of instances acquired at the same time. We also study the VM startup time of spot instances in EC2, which show a longer waiting time and greater variance compared to on-demand instances.
11468926
Gram Charlier Expansion for Time Varying Multiserver Queues with Abandonment
In this paper, we introduce a new approximation for estimating the dynamics of multiserver queues with abandonment. The approximation involves a four-dimensional dynamical system that uses the skewness and kurtosis of the queueing distribution via the Gram Charlier expansion. We show that the additional information captured in the skewness and kurtosis allows us to estimate the dynamics of the mean and variance much better than fluid and diffusion limit theorems or other methods that use only mean and variance behavior. Lastly, our approach also yields accurate approximations for the probability of delay, which is an important metric for quality of service.