# Infosys Whitepaper 
Title: Performance testing Internet of Things (IoT) 
Author: Infosys Limited 
Format: PDF 1.7 

---

  Page: 1 / 4  

---

 VIEW POINT PERFORMANCE TESTING INTERNET  OF THINGS (IOT) - Yakub Reddy Gurijala     Senior Technology Architect 

---

  Page: 2 / 4  

---

 External Document © 2018 Infosys Limited Internet of things (IoT) is a network of  systems, devices, and sensors which are  connected and this connectivity enables  these objects to share the data. It is a  platform which allows to manage the data  and controls the devices remotely based on  the requirement.<|endoftext|>The IoT has gained momentum in the recent  years due to Internet availability, cloud  evaluation, and micro services evaluation.  According to Gartner, IoT connected devices  are growing at 30 percent year-on-year and  there will be 20 billion devices connected  by 2020 (more than human population).  IoT business is growing at 22 percent year- on-year and will reach US$3010 billion.  People, government, and business will be  hugely affected by IoT in the coming years  resulting in smart cities, smart homes, smart  hospitals, and so on.  IoT devices produce data continuously. This  data needs to be saved and analyzed for  future decisions and these decisions may be  immediate or may be taken later by using  business intelligence (BI) analytics. IoT helps  in improving the operational performance  and cost optimization. So achieve this, IoT  systems must be built for high performance  and scalalability. To measure these two  key attributes of an IoT application, it is  important to understand the business value  for which it is built. In addition, to measure  performance, it is necessary to simulate  real-world workload models, which can  be created using business requirements,  historic data and future growth require- ments, type of devices, network conditions,  usage patterns, and geographic spread.  Application usage patterns are arrived  by analyzing the IoT application logs for  peak hours and normal hours. Using these  data points, different workload conditions  (real-world load test / simulation) can be  created for peak usage, normal usage, future  growth, and daylong / multiday simulations.<|endoftext|>IoT performance testing (PT) is little different  from traditional performance testing.  Following table illustrates differences  between traditional PT vs IoT PT.<|endoftext|>Because of these differences, IoT PT poses a lot of challenges to performance engineers. Below sections will describe different challenges  posed by IoT applications and Infosys solution elements for each of the challenges.<|endoftext|>Some of key differences between traditional PT and IoT PT: Key diferences Simulation Simulation of users Simulation of devices / sensors Scale Few hundred users to few thousand users Few thousand devices to few million devices Amount of data Sends and receives large amount of data per request Sends and receives minimal data per request but data is shared continuously with time interval Protocols Uses standard protocols to communicate Uses non-standard and new protocols to communicate Requests / responses In most of the cases, users create the requests and receive the response Generally IoT devices create the requests and receive response as well as request, and provide response BI Only few applications have BI as part of testing BI will be a part of IoT; needs to measure performance by applying loads on IoT app Traditional PT IoT PT 

---

  Page: 3 / 4  

---

 External Document © 2018 Infosys Limited Performance testing  challenges Protocols and performance testing tool  IoT does not have standard protocol set  to establish the connectivity between IoT  application and devices. IoT protocols used  range from HTTP, AllJoyn, IoTivity, MQTT,  CoAP, AMQP, and more. These protocols  are still in the early phases of development  and different IoT solution vendors come  up with specific protocol standards (sets).  These protocols are continuously evolving  with IoT applications. Since these are new  technologies / protocols, and current  performance testing tools may or may not  support them.  Geographical spread and network  conditions IoT devices / sensors are spread across  the world and use different networks to  connect to the IoT servers to send and  receive the data. As part of performance  testing, there is need to simulate devices  from different locations (to simulate  latency) with required network technolo- gies like 2G, 3G, 4G, Bluetooth, etc.<|endoftext|>Load conditions It is necessary to load test the applications  by simulating real-world conditions. These  patterns are complex in nature and it will  be extremely difficult to collect and predict  the data. To recreate real-world load  conditions, we may land up simulating  millions of devices.<|endoftext|>Real-time decision making Some IoT implementations may require  the data from a device that needs to be  processed at runtime and based on the  data received, the corresponding decision  is taken. These decisions are generally no- tifications / requests to different devices /  sensors or different systems which perform  particular action. As part of testing,  these notifications / requests need to be  monitored for performance (time taken to  generate the notification / request from  the data received by IoT application).  IoT application monitoring and BI  processing Monitoring is essential for any application.  It helps understand the system behavior  under real-world conditions. For IoT ap- plications, both the application and the  backend BI systems need to be monitored.  This will help understand data processing,  both in terms of the volume and accuracy.  Infosys IoT PT solution Infosys created a comprehensive  framework using JMeter to support all the  needs of IoT PT.  Protocols and performance testing tool  Infosys selected JMeter as performance  test tool to conduct PT. JMeter already  has support to most of the IoT protocols  like HTTP, CoAP, AMQP, MQTT and Kafka.  As IoT is an emerging area, new protocols  are being developed over the time. To  on-board new protocols Infosys has come  up with a protocol framework using  protocol SDK and extending the JMeter.  Using these JMeter extensions, scripts  can be prepared to simulate new protocol  requests and devices.<|endoftext|>Geographical spread and network  conditions To simulate geographical spread, JMeter  is integrated with cloud solutions like  Amazon web services (AWS) to setup  the load generators across different  geographies. Using AWS integration,  JMeter is able to generate the traffic from  different locations of the world to IoT  application to mimic the geographical  spread and network latency.  Infosys has in-house IP-based solution,  Infosys Network Simulation tool (iNITS),  to simulate different network conditions  required for any requests which use  transmission control protocol (TCP). We  have integrated iNITS solution with JMeter  to simulate different network conditions  required by IoT PT.<|endoftext|>Load conditions To collect the accurate real-world scenarios,  Infosys developed different tools /  frameworks like non-functional require- ments (NFR) questionnaire, workload  modeling tools, and others. These tools  / frameworks reduce the requirement  gathering and collect the information more  accurately.  To simulate millions of devices, JMeter  integrated with cloud using automated  scripts. These scripts will create required  number of load generators in cloud, setup  the JMeter, copy the scripts, test data,  execute the results, collect the results,  shutdown the LG’s which are created, and  process the results.<|endoftext|>Real-time decision making Notifications, which are sent to other  devices / sensors / systems, need to be  monitored using stubs / service virtualiza- tion technologies. IoT application logs  are collected and analyzed for processing  time and response time of the real-time  processing and decision making scenarios  under different load conditions.<|endoftext|>IoT application monitoring and BI  processing Infosys created predefined process /  performance metrics collection to monitor  the systems (Web / app / database layers)  deployed in cloud and data center. These  metrics are analyzed to uncover possible  performance bottlenecks. If BI systems  were built using batch jobs, then enough  test data needs to be created using  performance test scripts and the batch  jobs executed to monitor the BI system.  If real-time BI systems were implemented  using hot channels then, BI systems need  to be monitored as a part of different  performance tests by generating different  amount of data per second / minute /  hour. Using this approach, IoT applica- tions are comprehensively monitored and  performance results are benchmarked  against different load conditions.<|endoftext|>IoT PT resources Infosys presently has 1200+ performance  testing resources having experience in  testing different types of applications,  technologies, and tools. And more than  500 employees have working experience  on JMeter.  Infosys has dedicated resources who  are trained on IoT performance test  frameworks (JMeter, new protocols,  network simulation, and IoT monitoring).  These resources continuously explore the 