Metadata management for federated databases
A federated database consists of several loosely integrated databases, where each database may contain hundreds of tables and thousands of columns,interrelated by complex foreign key relationships. In general, there exists a lot of semistructured data elements outside the database represented by documents (files), created and updated by multiple users and programs. Documents have references to multiple databases and subsets of their tables and columns. Manually tracking which specific tables and columns are referred to by a document, accessed by a specific program or user is a daunting task. With such a goal in mind, we present a system that builds metadata models for a federated database using a relational database as a central object type and metadata repository. Metadata includes table and columns coming from logical data models corresponding to each database, as well as documents representing external semistructured data sources. SQL statements assemble metadata and data types to create objects and relationships as relational tables for easy querying. We discuss potential applications in federated scientific databases.