# Infosys Whitepaper 
Title: Today, the most popular form of evaluating an individual capabilities and competency level in any organization is done by Appr 
Author: Infosys Technologies 
Format: PDF 1.4 

---

  Page: 1 / 12  

---

   Presented at the 3rd International Software Testing Conference, India  The When & How of Test Automation  Vivek Motwani  Programmer Analyst  Product Competency Center  Infosys Technologies Limited, Bangalore    Abstract  Test automation has always been looked upon as a magic formula to improve the quality  processes of products/applications right from the day when first commercial product/application  was made. But when one actually starts automating the testing, the ground realities are realised.  More often than not, the teething troubles of deciding the right time to go for automation,  defining the scope of automation and selection of right tool for automation are over-whelming in  the first place. And even if these teething troubles are overcome, the automation tool developed is  usually inefficient as lots of important considerations are over-looked. This paper aims to suggest  the solution of these issues, and suggest best practices to be followed while doing the automation  so as to maximise the efficiency of the automation tool developed.  Introduction  Automation is the only long-term solution for reduced costs in software testing and better quality  products. But these aims are achieved only when certain best practices are followed before and  while developing the automation suite.  Howard Fear has aptly stated, "Take care with test automation. When done well, it can bring  many benefits. When not, it can be a very expensive exercise, resulting in frustration”. [3]  More often that not, after automating the testing of a product, the automation team finds the  automation tool more of a headache because of the unplanned and thoughtless approach adopted  while developing the tool. Generally, lots of effort is spent in developing the tool, only to  discover that the tool is limited in scope, lacks user-friendliness and requires frequent re-work  every now and then.  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.  

---

  Page: 2 / 12  

---

   Presented at the 3rd International Software Testing Conference, India  And if sufficient care is exercised and proper practices are followed before and while automating  the same product/application, the resulting automation tool not only saves time and effort, but is  also a sheer beauty in itself because of the amount of user-friendliness, flexibility, reusability and  extensibility it ensures.  Let us, therefore, discuss what all needs to be taken care of before going for test automation and  also while actually doing the automation.  Automation when?  (The Desiderata)  Lots of effort has to be spent even before you actually start with automation. It needs to be  ensured that following things have been taken care of: -  Stability of the product/application is ensured: The first thing that needs to be ensured is that  the product/application is fairly stable in terms of functionality. Even if it is slated to incorporate  new features, the new features should not disturb the existing functionality. There is no sense in  automation the testing of a product that is supposed to change functionality-wise.  Besides, the error messages generated by the product/application should remain consistent across  different releases. If the testing is GUI-based, then it needs to be ascertained that the future  releases of the product would not be undergoing GUI changes which might prove critical for the  automation suite.  Interface to be tested has been identified: Three different interfaces a product might have are  command line interfaces (CLIs), application-programming interfaces (APIs), and graphical user  interfaces (GUIs). Some may have all three, but many will have only one or two. These are the  interfaces that are available to you for your testing. By their nature, APIs and command line  interfaces are easier to automate than GUIs. Find out if your product has either one; sometimes  these are hidden or meant for internal use only. After this, you need to decide which interface’s  testing has to be automated. Some relevant points are: -  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.  

---

  Page: 3 / 12  

---

   Presented at the 3rd International Software Testing Conference, India  � GUI test automation is more difficult than test automation of the other two interfaces.  This is because firstly, GUI test automation will invariably require some manual script  writing. Secondly, there will always be some amount of technical challenge of getting the  tool to work with your product. Thirdly, GUI test automation involves keeping up with  design changes made to a GUI. GUIs are notorious for being modified and redesigned  throughout the development process.  � Despite the reasons for not depending on GUI test automation as the basis for testing  your product functionality, the GUI still needs to be tested, of course, and you may  choose to automate these tests. But you should have additional tests you can depend on to  test core product functionality that will not break when the GUI is redesigned. These tests  will need to work through a different interface: a command line or API.  � In order to simplify the testing of an API, you may want to bind it to an interpreter, such  as TCL or Perl or even Python. This enables interactive testing and should also speed up  the development cycle for your automated tests. Working with API’s may also allow you  to automate unit tests for individual product components.  Scope of automation has been defined: Before setting out to automate the testing of your  application/product, it is essential to define the scope/intended coverage of the automation tool.  The scope may encompass functionality testing, regression testing or simply acceptance testing.  You can even select to automate the testing of certain particular features or certain selective  testcases of different features.   Individual testcases to be automated have been identified: Automation suite should be looked  upon as a baseline test suite to be used in conjunction with manual testing, rather than as a  replacement for it. It should aim at reducing the manual testing effort gradually, but not doing  away with manual testing altogether. It needs to be understood that automation can aid manual  testing effort but cannot replace manual testing. What machines are good at and humans are slow  at should be chosen for automation. Setting realistic goals in early stages of test automation is  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.  

---

  Page: 4 / 12  

---

   Presented at the 3rd International Software Testing Conference, India  important for achieving long-term success. So, even after defining the scope of the automation  tool in terms of acceptance/regression testing, etc, it needs be made sure that following kinds of  testcases are eliminated from the scope of automation: -  � Testcases that are long and complicated and require manual inspection/intervention in  between.  � Testcases that take tremendous amount of time in automation and it is difficult to ensure  re-usability even if they are automated.  � Testcases pertaining to usability testing. Usability testing means testing in a true end-user  environment in order to check whether the system is able to operate properly in  accordance with the exact set of processes and steps applied by the end-user, including  user's interface and system convenience estimation.  It’s very important to include the right testcases in the suite. If the selection of testcases for the  automation suite is not meticulous, you might end up discovering nothing really important about  the software you are testing even if you develop a highly robust and reliable test suite.  Testcases have been fine-tuned: The testcases need to be fine-tuned for automation. The  expectation level from the testcases for automating is widely different from the expectation from  manual testing point-of-view. The salient features that need to be taken care of include: -  � Manual regression tests are usually documented so that each test picks up after the  preceding test, taking advantage of any objects or records that may already have been  created. Manual testers can usually figure out what is going on. A common mistake is to  use the same approach with automated tests. But because of this approach, a failure in  one test will topple successive tests. Moreover, these tests also cannot be run  individually. This makes it difficult to use the automated test to help analyze legitimate  failures. So, it is advised to revamp the testcases so as to make them independent. Each  testcase should setup its test environment.  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.  

---

  Page: 5 / 12  

---

   Presented at the 3rd International Software Testing Conference, India  � The testcases need to be equipped with proper test-data. E.g. –