# Infosys Whitepaper 
Title: Big data testing for a leading global brewer 
Author: Infosys Limited 
Format: PDF 1.7 

---

  Page: 1 / 8  

---

 WHITE PAPER MOVING FRAGMENTED TEST  DATA MANAGEMENT TOWARDS A  CENTRALIZED APPROACH  Abstract Test Data Management (TDM) ensures managing test data requests  in an automated way to ensure a high degree of test coverage  by providing the right data, in the right quantity, and at the right  time, in non-production environments. Automated TDM service  facilitates test data management across test environments through  a structured approach of data subsetting, cleansing, gold copy  creation, data refresh, and sensitive data masking.  Typically, a centralized TDM system with well-defined processes  is more effectual than the traditional manual or decentralized  approach, but in some cases, a decentralized approach is adopted.  This paper takes a deeper dive into the considerations for the  centralization of TDM processes within enterprise ITs.<|endoftext|>

---

  Page: 2 / 8  

---

 Introduction In most organizations where TDM is at  its infancy, test data-related activities  are done by the individual project teams  themselves. There will not be a dedicated  team identified or process defined to  handle test data requests. Such projects  with a primitive TDM approach possess  several drawbacks: •  Lack of a defined ownership for the test  environment and test data setup:  results  in unintentionally losing the test data  setup or data overstepping •  Unavailability of data setup for testing  end-to-end scenarios: Lack of data setup  between inter-dependent and third- party applications •  Lack of referential integrity defined  in the databases: Absence of primary,  foreign, relationships defined in the  database makes it difficult to identify  related tables and generate the correct  test data set •  Insufficient data available for  performance load testing: Manually  generating bulk data is a tedious task  and less feasible •  Increased number of defects due to  incorrect test data: Leads to re-work  and losing time unnecessarily analyzing  issues caused due to incorrect test data  used for testing •  Outdated test data in QA database:  Periodic refresh of test data does not  happen from production •  Inability to provision data since data  is unavailable: Lack the mechanism  required for generating synthetic data •  Risk of exposing sensitive data to testing  teams: Sensitive fields need to be  masked before provisioning for testing •  Multiple copies of data: Storage costs  can be reduced by maintaining required  gold copies and refreshing and reusing  gold copies after major releases Having a well-defined practice for handling  all the test, data-related, requirements  across all non-production environments  in an organization is the essence of TDM.  Aimed to address all the above stated  issues, it will bring in more control and  make TDM more effective.<|endoftext|>Based on the TDM requirement type,  organizations can opt for either a de- centralized or a centralized approach.  This paper gives a detailed view of both  approaches and highlights how the  centralized approach is more efficient and  beneficial.<|endoftext|>External Document © 2018 Infosys Limited 

---

  Page: 3 / 8  

---

 Centralized TDM deals with consolidating  the test data provisioning for all non- production environments across the  organization. It provides a systematic  approach to analyze and provision test data.<|endoftext|>Pros •  Well-established TDM team with  a workflow-based mechanism for  managing test data requests •  Reduced latency in provisioning test  data with quick turnaround time •  Automated end-to-end approach with  tools and processes •  Reduced infrastructure cost by storing  only the required data for provisioning   in gold copy database  •  Reduced risk of incorrect test data,  resulting in lesser defects  •  Resolution of data overstepping issues  by the TDM team •  Periodic refresh of the gold copy makes  the latest data available for testing by  the QA team •  Reusable masking configurations and  test data generation scripts provides  quick turnaround time •  Easy handling of complex end-to-end  test scenarios that require data setup  across heterogeneous data sources  having federated relationships through   a centralized test data managment  •  Creation of bulk data which is  relationally intact for non-functional  testing requirements is achieved using  automated solutions •  Varied techniques available for creating  synthetic data in scenarios where source  data is not available for provisioning Cons •  Considerable time and effort is required  to consolidate the TDM across various  portfolios •  High knowledge acquisition effort  required to understand the different  application data models •  Sporadic bottlenecks and dependency  on the TDM team in case of high  workload from all LOBs Centralized TDM External Document © 2018 Infosys Limited 

---

  Page: 4 / 8  

---

 Decentralized TDM It is not necessary that all applications in  different portfolios in the organization’s  landscape have to be covered under the  consolidated TDM umbrella. There are  instances where some applications can  follow the de-centralized TDM approach.  This is mostly determined by the level  of integration between the applications,  technologies supported, data sensitivity,  environment constraints, etc. For example,  data in HR, infrastructure applications, etc.,  may be independent and not related to  marketing, sales, inventory, or corporate  data. These systems, hence, can adopt a de- centralized TDM approach and need to be  handled outside the centralized umbrella.<|endoftext|>Pros •   Minimal effort required to set up TDM  for individual applications •   Good understanding of the respective  application data models, which makes  the team capable to address the test  data requests quickly Cons •   Multiple copies of data without owner- ship because individual teams store  separate copies of production data.  •   Unmasked sensitive data in non- production environments can lead to a  security breach •   Less uniformity in standards and pro- cesses •   Increase in data overstepping issues •   Minimal automation may be present  with lack of coordinated processes  •   Limitations in setting up data across  multiple data sources due to decentral- ized systems. Data set up in one ap- plication may not be in sync with other  inter-dependent applications Sales Corporate HR Infrastructure Inventory Marketing Independent TDM Consolidated TDM Centralized TDM implementa- tion approaches Primarily, there are two approaches for  implementing centralized test data man- agement within an organization: •   Big Bang approach: In this approach,  all major applications under the TDM  scope in the organization are identi- fied, test data requirements across  applications are analyzed, and gold  copies for these applications are cre- ated at one go. A TDM team is set up  to address test data needs for all the  applications.     This approach will take considerable  time for the initial setup and knowl- edge of the application stack across  the organization’s portfolio is a must.  Another key challenge with this ap- proach is keeping up with the database  (DB) changes happening in production  during the initial setup •   Incremental approach: In this ap- proach, based on the business  requirements, TDM is established  for an application or a prioritized set  of applications. Separate test data  management implementations will be  carried out which can be progressively  integrated. The TDM team will address  the test data needs as soon as the gold  copies for the applications are set up.<|endoftext|>   In this approach, TDM is more manage- able, and can reap early benefits. TDM  set up for smaller set of applications  takes lesser time compared to the Big  Bang approach.<|endoftext|>A phased approach for TDM  implementation Centralized and automated test data man- agement implementations can follow a  phased approach. Each stage has a defined  set of activities to achieve the goals and  these stages are: •   Analysis  •   Design  •   Implementation  •   Steady State From the initial assessment phase, it moves  to a stabilized stage, expanding TDM  services to other domains and portfolios in  the organization, and working for continu- ous improvements on the way.<|endoftext|>The timelines proposed in the diagram are  highly indicative. Time duration for each  phase will depend on factors like: •   TDM requirement complexity •   Number of portfolios or applications  involved •   Knowledge or understanding about  the application landscape or database  models Figure 1: TDM Landscape External Document © 2018 Infosys Limited 

---

  Page: 5 / 8  

---

 Figure 2: Phased TDM Approach When to go for centralized  TDM? •   Applications or technologies used are  mostly compatible with tools in the  market •   Scope of TDM for applications across  various portfolios continue throughout  the life cycle of the application •   Incoming T