Continue # Infosys Whitepaper 
 suite might have changed some configuration  files or properties files before it received the signal. So, if the changes are not reverted back  before the termination of the suite, then things will go for a toss.  Self-sufficiency in individual testcases: Testcases should not be dependent on preceding  testcases for execution. If there is dependency on testcases occurring before in the sequence, then  the subsequent testcases will fail without any reason. If at all such dependence is unavoidable, the  error message in the log file, when such testcases fail because of the failing of preceding testcase,  should be explanatory enough.  Equipped with test data: The automation suite should be equipped with all the test data required  by the different testcases. The test data may consist of simple data input as required by the  testcases to supply parameters to the functions for testing different conditions like numeric input,  alpha-numeric input, non-alpha-numeric input, etc. It may as well consist of specific files to be  supplied to the testcases to test particular functionality of the application/product. The automation  suite has to be accompanied with such test data and this test data has to be prepared for the suite  with precision. Example: - A particular feature of the application/product may have to be tested  with files of different sizes, say 0 bytes, 64 KB, 1MB, 30 MB, etc. So, the suite requires to have  the files precisely of these sizes only. All such files may be kept in a particular folder from where  the suite picks them up. The regular input data, which is required by the functions as parameters  can be supplied through the input data files. The individual testcases may parse the parameters to  be supplied to the testcases while reading them from the input data files. The tools available in the  market support different types of file to be used as data input files. Example: - WinRunner uses  excel sheets for reading data, while SilkTest uses.dat files.  Dynamic generation of names for temporary files and input data: Sometimes, the automation  suite would require to create certain temporary files. If the suite does not delete the temporary  files created by itself, then they will get over-written in the next run of the suite. Besides, if a file  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.  

---

  Page: 10 / 12  

---

   Presented at the 3rd International Software Testing Conference, India  by the same name exists even before the first run of the suite, then that file may get over-written  in the first run itself. The consequence will be even worse if the write permission is not there on  the already existing file. The script will fail to over-write also in such a case and the testcase  might eventually bombard. Similar problems are faced when the suite contains a positive  testcases like creating a folder with a given name. If the suite does delete this folder created as a  part of the clean-up process, the testcases fails unnecessarily when the suite is run again with the  testcase trying to create a folder with the same name.   A solution to all such problems is to dynamically generate the names for temporary files  and all such input data at the run time. This way the names will not conflict with those of the  existing files and fresh data. Such dynamic generation of names can be accomplished by several  ways. One typical way of generation can be stripping the microsecond part of the current time  and appending it to a name. This way there will be an extremely rare probability (10 to the power  of –6, to be precise) that a conflict in the names will take place.  Cleaning-up: It has to be ensured that the automation suite brings the application/product back to  the original state it was in before the suite executed. If any configuration or properties file was  changed for the execution of some testcase, then the changes must be reverted back. If the suite  generates some temporary files, they should be deleted by the suite towards the end.  Incorporating user-friendliness: The automation suite should be as user-friendly as possible.  Some basic points for ensuring user-friendliness are: -  � The user should have freedom to put the test data files anywhere on the m/c.  � The suite can be run from anywhere on the m/c.  � It can be installed anywhere on the m/c.  � Once it is run, the suite should not require any manual intervention till completion. The  user should be able to run the suite unattended.  For incorporating such user-friendliness, the suite needs to be designed in a proper way. A  separate configuration file can be created that contains all the variables that the user might want  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.  

---

  Page: 11 / 12  

---

   Presented at the 3rd International Software Testing Conference, India  to change. E.g. – The user might want the log files to be generated on the desktop instead of a  hard-coded path. The user might as well want the suite to pick-up the test data/files from a  directory of his choice. All such entities can be placed in the configuration file in the form of  variables that the user can change easily. The suite can read these variables from the  configuration files every time it is run. If such a design is used, all that the user would need to do  before running the script is to change the configuration file as per his needs. Thus, the user will  get a tremendous amount of flexibility and independence.  Developing an efficient error recovery routine: Error Recovery routine enables the test suite to  run continuously unattended. The function of this routine is to anticipate errors, decide on  corrective action, log the error and proceed further with next test, if possible. E.g. - If unexpected  termination of application under test happens, the routine should recognize the interruption and  restart the application. This prevents cascading effect or reporting wrong defects after a test suite  execution. In a nutshell, this will ensure that failures in test execution are effectively trapped,  interpreted and suite continues to run without failures. Without such an error recovery system,  automated test suite runs will never take off. Manual presence will become a necessity during test  suite execution.  Test scripts for test data setup and cleaning up: If the automation suite does not take care of  test data setup, it will have to be done manually by the user, which reduces the fun of test  automation. This becomes all the more important when test data setup requirements are huge and  as a result, the whole exercise become highly time consuming. Hence, an ideal automation suite  should incorporate dedicated scripts for test data set-up. These scripts are executed before any  other functionality test can be executed on the product. E.g. – When the application/product in  focus is an ERP suite or a banking software, the test data setup part itself may take 3-4 man-days  of effort. With the automation in place for this setup, the effort is reduced drastically.    Similarly, scripts for cleaning-up should also be incorporated in the automation suite.  Such scripts will aim at bringing the application to the ground state it was in before the  © QAI India Ltd, 3rd Annual International Software Testing Conference, 2001. No use for profit permitted.  

---

  Page: 12 / 12  

---

   Presented at the 3rd International Software Testing Conference, India  automation suite was run, i.e., they will undo all the changes that any testcases in the suite  brought about while executing. E.g. – If there is a testcases for creating a folder, then the clean-up  action will delete this folder.  Testing the test scripts: Test scripts should be tested before they are used in a test suite. Testing  of all test scripts should be planned in test automation activity. Adequate tests need to be  performed for each test script. When test error simulation and rectification is difficult and time- consuming process, reporting false errors can cost more and defeat the objective. The goal for the  automation team has to be that a test program should never report a false problem. All scripts  should satisfy the following criteria: -  � When given a valid input, they produce the correct output.  � When given an invalid input, they correctly and gracefully reject the input.  � Do not hang or crash, given either valid or invalid input.    Conclusion  Test automation is a great idea. But it is not a magic wand. Proper time and effort has to be spent  for the development of the test automation suite. And the key is to follow the right processes. In  eagerness to achieve fast results, the desirable processes are compromised. And that is the reason  why, more often than not, it only promises and raises hopes, but simply fails to deliver.   Acknowledgement  I wish to express my sincere gratitude towards Sridhar Kumanduri and KiranK