Earth Science Markup Language: A Solution to Address Data Format Heterogeneity Problems in Atmospheric Sciences
| cientists are often confounded by various data types, formats, and systems used in Earth science. For example, regional atmospheric air pollution models have to deal with over 20 datasets in different formats, where each format has a large user manual describing the structure and format of the data. It is almost impossible for individual scientists or even scientific groups to have expertise at their disposal in each of these data formats. Often, important new data are not incorporated into models because they are in a format that is new to the research group. Dealing with heterogeneous formats can become a major bottleneck in the process of data analysis. Since most scientists are not programmers, much of their time and effort are required to understand and decode data in unfamiliar formats. 1 Scientists have to either write a data decoder or reader module for a new data format or translate the data into a format that their analysis tool can handle. One solution to this problem would be to have all data-producing and data-consuming communities agree upon a single self-describing format that includes the metadata defined in a community-accepted convention. The metadata annotations in this " standard " data format would allow software developers to write analysis tools that can decode any data in the scientific domain. Although a single, " standard " self-describing data format would greatly simplify the use, integration, or fusion of disparate data types, the science community has found this solution to be impractical for the following reasons: First, there is a considerable volume of legacy datasets. Converting all of these datasets to a standard format would be exceedingly time-consuming and expensive. Furthermore, it is difficult to design a standard format that can match the capabilities of specialized formats designed for specific uses. For example, some formats were designed for efficient storage and exchange across a network, whereas other formats were designed for efficient data retrieval. The Earth Science Markup Language (ESML) provides a more flexible solution to this data usability problem, supporting multiple data formats with metadata-based interchange technologies. ESML is a specialized markup language for Earth science metadata based on XML (eXtensible Markup Language), allowing analysis tools to seamlessly utilize datasets in heterogeneous formats. ESML description files contain metadata with content and structural information for the corresponding data file format. These descriptions can be generated by either the data producer or the data consumer …