Continue # Infosys Whitepaper 
 framework is complex. Typically, it follows many of the steps used during functional testing.  Data source and conditioning testing • Verify the quality of data from various systems – data correctness, completeness and appropriateness along with format checks, data  lineage checks and pattern analysis • Verify transformation rules and logic applied on raw data to get the desired output format. The testing methodology/automation  framework should function irrespective of the nature of data – tables, flat files or big data • Verify that the output queries or programs provide the intended data output • Test for positive and negative scenarios Algorithm testing • Split input data for learning and for the algorithm • If the algorithm uses ambiguous datasets, i.e., the output for a single input is not known, the software should be tested by feeding a set  of inputs and checking if the output is related. Such relationships must be soundly established to ensure that algorithms do not have  defects • Check the cumulative accuracy of hits (TPs and TNs) over misses (FPs and FNs) API integration • Verify input request and response from each application programming interface (API) • Verify request response pairs • Test communication between components – input and response returned as well as response format and correctness  • Conduct integration testing of API and algorithms and verify reconciliation/visualization of output System/regression testing • Conduct end-to-end implementation testing for specific use cases, i.e., provide an input, verify data ingestion and quality, test the  algorithms, verify communication through the API layer, and reconcile the final output on the data visualization platform with expected  output • Check for system security, i.e., static and dynamic security testing • Conduct user interface and regression testing of the systems 

---

  Page: 5 / 8  

---

 External Document © 2018 Infosys Limited Use case 3: Testing ML-based analytical models Organizations build analytical models for three main purposes as shown in Fig 2 Fig 2: Types and purposes of analytical models  The validation strategy used while testing the analytical model involves the following three steps: •  Split the historical data into ‘test’ and ‘train’ datasets  •  Train and test the model based on generated datasets •  Report the accuracy of model for the various generated scenarios Fig 2: Types and purposes of analytical models  External Document © 2018 Infosys Limited 

---

  Page: 6 / 8  

---

 External Document © 2018 Infosys Limited Fig 3: Testing analytical models  Organizations build analytical models for three main purposes as shown in Fig 2 Fig 2: Types and purposes of analytical models  Use case 4: Testing of AI-powered solutions Chatbot testing framework • Test the chatbot framework using semantically equivalent sentences and create an automated library  for this purpose • Maintain configurations of basic and advanced semantically equivalent sentences with formal and  informal tones and complex words • Automate end-to-end scenario (requesting chatbot, getting a response and validating the response  action with accepted output) • Generate automated scripts in Python for execution  RPA testing framework • Use open source automation or functional testing tools (Selenium, Sikuli, Robot Class, AutoIT) for mul- tiple applications • Use flexible test scripts with the ability to switch between machine language programming (where  required as an input to the robot) and high-level language for functional automation • Use a combination of pattern, text, voice, image, and optical character recognition testing techniques  with functional automation for true end-to-end testing of applications While testing a model, it is critical to do the following to ensure success:  • Devise the right strategy to split and subset historical dataset using deep knowledge of development model and code to understand how it  works on data • Model the end-to-end evaluation strategy to train and recreate model in test environments with associated components • Customize test automation to optimize testing throughput and predictability by leveraging customized solutions to split the dataset, evaluate  the model and enable reporting  

---

  Page: 7 / 8  

---

 External Document © 2018 Infosys Limited Conclusion AI frameworks typically follow 5 stages –  learning from various data sources, input  data conditioning, machine learning and  analytics, visualization, and feedback. Each  stage has specific failure points that can  be identified using several techniques.  Thus, when testing the AI systems, QA  departments must clearly define the  test strategy by considering the various  challenges and failure points across all  stages. Some of the important testing  use cases to be considered are testing  standalone cognitive features, AI platforms,  ML-based analytical models, and AI- powered solutions. Such a comprehensive  testing strategy will help organizations  streamline their AI frameworks and  minimize failures, thereby improving output  quality and accuracy.  External Document © 2018 Infosys Limited 

---

  Page: 8 / 8  

---

 © 2018 Infosys Limited, Bengaluru, India. All Rights Reserved. Infosys believes the information in this document is accurate as of its publication date; such information is subject to change without notice. Infosys  acknowledges the proprietary rights of other companies to the trademarks, product names and such other intellectual property rights mentioned in this document. Except as expressly permitted, neither this  documentation nor any part of it may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, printing, photocopying, recording or otherwise, without the  prior permission of Infosys Limited and/ or any named intellectual property rights holders under this document.  For more information, contact askus@infosys.com Infosys.com | NYSE: INFY Stay Connected   


***


  