In many business environments, it may be necessary to execute bulk data transfers between a variety of persistent data stores associated with applications or other systems both internal and external to an enterprise. With previous techniques, to handle such bulk data movement from a source data store to a target data store in connection with operation of an application in a business workflow, in addition to developing the code for the application itself an application developer was typically required to: (1) custom develop a first piece of code, specific to the application and the source data store, for extracting data from the source data store and placing the extracted data in an intermediate storage location (such as a flat file for example) in an intermediate format; (2) custom develop a second piece of code (such as a Perl script for example), specific to the application and the intermediate format, for transforming the stored data into a format suitable for the target data store; and (3) custom develop a third piece of code, specific to the application and the target data store, for loading the transformed data into the target data store. Such custom-developed code is seldom reusable, is typically difficult to maintain, and typically makes application integration difficult as additional applications and data stores are added into the integration environment. Available Extract-Transform Load (ETL) tools can handle extraction of data from particular source data stores, aggregation or other straightforward transformations of extracted data, and loading of transformed data into particular target data stores. Although such ETL tools may be adequate for certain simple integration scenarios involving linking an existing application or other system to a database, such tools are limited in their capabilities and do not relieve an application developer from the burdens discussed above in designing and developing a new application. Accordingly, supporting bulk data integration between persistent data stores remains a pressing need.
USPTO Backgrounds
USPTO Backgrounds