scientific data
8755162
The FAIR Guiding Principles for scientific data management and stewardship
There is an urgent need to improve the infrastructure supporting the reuse of scholarly data. A diverse set of stakeholders representing academia, industry, funding agencies, and scholarly publishers have come together to design and jointly endorse a concise and measureable set of principles that we refer to as the FAIR Data Principles. The intent is that these may act as a guideline for those wishing to enhance the reusability of their data holdings. Distinct from peer initiatives that focus on the human scholar, the FAIR Principles put specific emphasis on enhancing the ability of machines to automatically find and use the data, in addition to supporting its reuse by individuals. This Comment is the first formal publication of the FAIR Principles, and includes the rationale behind them, and some exemplar implementations in the community.
Scientific data
Mark D Wilkinson Michel Dumontier IJsbrand Jan Aalbersberg Gabrielle Appleton Myles Axton Arie Baak Niklas Blomberg Jan-Willem Boiten Luiz Olavo Bonino da Silva Santos Philip E Bourne Jildau Bouwman Anthony J Brookes Tim Clark Merce Crosas Ingrid Dillo Olivier Dumon Scott C Edmunds Chris T A Evelo Richard Finkers Alejandra N Gonzalez-Beltran Alasdair J G Gray Paul T Groth Carole A Goble Jeffrey S Grethe Jaap Heringa Peter A C 't Hoen Rob W W Hooft Tobias Kuhn Ruben G Kok Joost N Kok Scott J Lusher Maryann E Martone Albert Mons Abel Laerte Packer Bengt Persson Philippe Rocca-Serra Marco Roos Rene C van Schaik Susanna-Assunta Sansone Erik Schultes Thierry Sengstag Ted Slater George O Strawn Morris A Swertz Mark Thompson Johan van der Lei Erik M van Mulligen Jan Velterop Andra Waagmeester Peter Wittenburg Katy Wolstencroft Jun Zhao Barend Mons
174809523
Make scientific data FAIR
All disciplines should follow the geosciences and demand best practice for publishing and sharing data, argue Shelley Stall and colleagues.All disciplines should follow the geosciences and demand best practice for publishing and sharing data, argue Shelley Stall and colleagues.
Nature
Shelley Stall Lynn Yarmey Joel Cutcher-Gershenfeld Brooks Hanson Kerstin A Lehnert Brian A Nosek Mark A Parsons Erin M Robinson Lesley Wyborn
51924604
Understanding and Modeling Lossy Compression Schemes on HPC Scientific Data
Scientific simulations generate large amounts of floating point data, which are often not very compressible using the traditional reduction schemes, such as deduplication or lossless compression. The emergence of lossy floating point compression holds promise to satisfy the data reduction demand from HPC applications; however, lossy compression has not been widely adopted in science production. We believe a fundamental reason is that there is a lack of understanding of the benefits, pitfalls, and performance of lossy compression on scientific data. In this paper, we conduct a comprehensive study on state of the art lossy compression, including ZFP, SZ, and ISABELA, using real and representative HPC datasets. Our evaluation reveals the complex interplay between compressor design, data features and compression performance. The impact of reduced accuracy on data analytics is also examined through a case study of fusion blob detection, offering domain scientists with the insights of what to expect from fidelity loss. Furthermore, the trial and error approach to understanding compression performance involves substantial compute and storage overhead. To this end, we propose a sampling based estimation method that extrapolates the reduction ratio from data samples, to guide domain scientists to make more informed data reduction decisions.
2018 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
Tao Lu Qing Liu Xubin He Huizhang Luo Eric Suchyta J Choi Norbert Podhorszki Scott Klasky Matthew Wolf Tong Liu Zhenbo Qiao
713219
Journey to the centre of the cell: Virtual reality immersion into scientific data
Visualization of scientific data is crucial not only for scientific discovery but also to communicate science and medicine to both experts and a general audience. Until recently, we have been limited to visualizing the three dimensional (3D) world of biology in 2 dimensions. Renderings of 3D cells are still traditionally displayed using two dimensional (2D) media, such as on a computer screen or paper. However, the advent of consumer grade virtual reality (VR) headsets such as Oculus Rift and HTC Vive means it is now possible to visualize and interact with scientific data in a 3D virtual world. In addition, new microscopic methods provide an unprecedented opportunity to obtain new 3D data sets. In this perspective article, we highlight how we have used cutting edge imaging techniques to build a 3D virtual model of a cell from serial block face scanning electron microscope (SBEM) imaging data. This model allows scientists, students and members of the public to explore and interact with a "real" cell. Early testing of this immersive environment indicates a significant improvement in students' understanding of cellular processes and points to a new future of learning and public engagement. In addition, we speculate that VR can become a new tool for researchers studying cellular architecture and processes by populating VR models with molecular data.
Traffic
Angus P R Johnston James Rae Nicholas Ariotti Benjamin J Bailey Andrew R Lilja Robyn Webb Charles Ferguson Sheryl Maher Thomas P Davis Richard Webb John B McGhee Robert G Parton
4399600
Scientific data interpolation with low dimensional manifold model
Abstract We propose to apply a low dimensional manifold model to scientific data interpolation from regular and irregular samplings with a significant amount of missing information. The low dimensionality of the patch manifold for general scientific data sets has been used as a regularizer in a variational formulation. The problem is solved via alternating minimization with respect to the manifold and the data set, and the Laplace Beltrami operator in the Euler Lagrange equation is discretized using the weighted graph Laplacian. Various scientific data sets from different fields of study are used to illustrate the performance of the proposed algorithm on data compression and interpolation from both regular and irregular samplings.
J. Comput. Phys.
Wei Zhu Bao Wang Richard C Barnard Cory D Hauck Frank Jenko S Osher
2178023
Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error Controlled Quantization
Today's HPC applications are producing extremely large amounts of data, such that data storage and analysis are becoming more challenging for scientific research. In this work, we design a new error controlled lossy compression algorithm for large scale scientific data. Our key contribution is significantly improving the prediction hitting rate (or prediction accuracy) for each data point based on its nearby data values along multiple dimensions. We derive a series of multilayer prediction formulas and their unified formula in the context of data compression. One serious challenge is that the data prediction has to be performed based on the preceding decompressed values during the compression in order to guarantee the error bounds, which may degrade the prediction accuracy in turn. We explore the best layer for the prediction by considering the impact of compression errors on the prediction accuracy. Moreover, we propose an adaptive error controlled quantization encoder, which can further improve the prediction hitting rate considerably. The data size can be reduced significantly after performing the variable length encoding because of the uneven distribution produced by our quantization encoder. We evaluate the new compressor on production scientific data sets and compare it with many other state of the art compressors: GZIP, FPZIP, ZFP, SZ 1.1, and ISABELA. Experiments show that our compressor is the best in class, especially with regard to compression factors (or bit rates) and compression errors (including RMSE, NRMSE, and PSNR) Our solution is better than the second best solution by more than a 2x increase in the compression factor and 3.8x reduction in the normalized root mean squared error on average, with reasonable error bounds and user desired bit rates.
2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
Dingwen Tao Sheng Di Zizhong Chen Franck Cappello
199082903
Papers' Information in Scientific Data and China Scientific Data
Data Description:papers' information in Scientific Data and China Scientific Data Time range: 2018 01 11 to 2018 02 25; Data volume: 487KB; Data Format: .xlsx.
Wen Liangming Wang Xiaodong Guo Lei Wu Zhangsheng Wang Weijun
63050084
On the Reuse of Scientific Data
While science policy promotes data sharing and open data, these are not ends in themselves. Arguments for data sharing are to reproduce research, to make public assets available to the public, to leverage investments in research, and to advance research and innovation. To achieve these expected benefits of data sharing, data must actually be reused by others. Data sharing practices, especially motivations and incentives, have received far more study than has data reuse, perhaps because of the array of contested concepts on which reuse rests and the disparate contexts in which it occurs. Here we explicate concepts of data, sharing, and open data as a means to examine data reuse. We explore distinctions between use and reuse of data. Lastly we propose six research questions on data reuse worthy of pursuit by the community: How can uses of data be distinguished from reuses? When is reproducibility an essential goal? When is data integration an essential goal? What are the tradeoffs between collecting new data and reusing existing data? How do motivations for data collection influence the ability to reuse data? How do standards and formats for data release influence reuse opportunities? We conclude by summarizing the implications of these questions for science policy and for investments in data reuse.
Data Sci. J.
Irene V Pasquetto Bernadette M Randles Christine L Borgman
14224621
Parallel Tensor Compression for Large Scale Scientific Data
As parallel computing trends towards the exascale, scientific data produced by high fidelity simulations are growing increasingly massive. For instance, a simulation on a three dimensional spatial grid with 512 points per dimension that tracks 64 variables per grid point for 128 time steps yields 8 TB of data, assuming double precision. By viewing the data as a dense five way tensor, we can compute a Tucker decomposition to find inherent low dimensional multilinear structure, achieving compression ratios of up to 5000 on real world data sets with negligible loss in accuracy. So that we can operate on such massive data, we present the first ever distributed memory parallel implementation for the Tucker decomposition, whose key computations correspond to parallel linear algebra operations, albeit with nonstandard data layouts. Our approach specifies a data distribution for tensors that avoids any tensor data redistribution, either locally or in parallel. We provide accompanying analysis of the computation and communication costs of the algorithms. To demonstrate the compression and accuracy of the method, we apply our approach to real world data sets from combustion science simulations. We also provide detailed performance results, including parallel performance in both weak and strong scaling experiments.
2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
Woody Austin Grey Ballard Tamara G Kolda
53557203
Are Scientific Data Repositories Coping with Research Data Publishing?
Research data publishing is intended as the release of research data to make it possible for practitioners to (re)use them according to "open science" dynamics. There are three main actors called to deal with research data publishing practices: researchers, publishers, and data repositories. This study analyses the solutions offered by generalist scientific data repositories, i.e. repositories supporting the deposition of any type of research data. These repositories cannot make any assumption on the application domain. They are actually called to face with the almost open ended typologies of data used in science. The current practices promoted by such repositories are analysed with respect to eight key aspects of data publishing, i.e. dataset formatting, documentation, licensing, publication costs, validation, availability, discovery and access, and citation. From this analysis it emerges that these repositories implement well consolidated practices and pragmatic solutions for literature repositories. These practices and solutions can not totally meet the needs of management and use of datasets resources, especially in a context where rapid technological changes continuously open new exploitation prospects.
Data Sci. J.
Massimiliano Assante Leonardo Candela Donatella Castelli Alice Tani