-
DESC DC2 Data Release Note
Authors:
LSST Dark Energy Science Collaboration,
Bela Abolfathi,
Robert Armstrong,
Humna Awan,
Yadu N. Babuji,
Franz Erik Bauer,
George Beckett,
Rahul Biswas,
Joanne R. Bogart,
Dominique Boutigny,
Kyle Chard,
James Chiang,
Johann Cohen-Tanugi,
Andrew J. Connolly,
Scott F. Daniel,
Seth W. Digel,
Alex Drlica-Wagner,
Richard Dubois,
Eric Gawiser,
Thomas Glanzman,
Salman Habib,
Andrew P. Hearin,
Katrin Heitmann,
Fabio Hernandez,
Renée Hložek
, et al. (32 additional authors not shown)
Abstract:
In preparation for cosmological analyses of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), the LSST Dark Energy Science Collaboration (LSST DESC) has created a 300 deg$^2$ simulated survey as part of an effort called Data Challenge 2 (DC2). The DC2 simulated sky survey, in six optical bands with observations following a reference LSST observing cadence, was processed with th…
▽ More
In preparation for cosmological analyses of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), the LSST Dark Energy Science Collaboration (LSST DESC) has created a 300 deg$^2$ simulated survey as part of an effort called Data Challenge 2 (DC2). The DC2 simulated sky survey, in six optical bands with observations following a reference LSST observing cadence, was processed with the LSST Science Pipelines (19.0.0). In this Note, we describe the public data release of the resulting object catalogs for the coadded images of five years of simulated observations along with associated truth catalogs. We include a brief description of the major features of the available data sets. To enable convenient access to the data products, we have developed a web portal connected to Globus data services. We describe how to access the data and provide example Jupyter Notebooks in Python to aid first interactions with the data. We welcome feedback and questions about the data release via a GitHub repository.
△ Less
Submitted 13 June, 2022; v1 submitted 12 January, 2021;
originally announced January 2021.
-
Toward an Automated HPC Pipeline for Processing Large Scale Electron Microscopy Data
Authors:
Rafael Vescovi,
Hanyu Li,
Jeffery Kinnison,
Murat Keceli,
Misha Salim,
Narayanan Kasthuri,
Thomas D. Uram,
Nicola Ferrier
Abstract:
We present a fully modular and scalable software pipeline for processing electron microscope (EM) images of brain slices into 3D visualization of individual neurons and demonstrate an end-to-end segmentation of a large EM volume using a supercomputer. Our pipeline scales multiple packages used by the EM community with minimal changes to the original source codes. We tested each step of the pipelin…
▽ More
We present a fully modular and scalable software pipeline for processing electron microscope (EM) images of brain slices into 3D visualization of individual neurons and demonstrate an end-to-end segmentation of a large EM volume using a supercomputer. Our pipeline scales multiple packages used by the EM community with minimal changes to the original source codes. We tested each step of the pipeline individually, on a workstation, a cluster, and a supercomputer. Furthermore, we can compose workflows from these operations using a Balsam database that can be triggered during the data acquisition or with the use of different front ends and control the granularity of the pipeline execution. We describe the implementation of our pipeline and modifications required to integrate and scale up existing codes. The modular nature of our environment enables diverse research groups to contribute to the pipeline without disrupting the workflow, i.e. new individual codes can be easily integrated for each step on the pipeline.
△ Less
Submitted 6 November, 2020;
originally announced November 2020.
-
The LSST DESC DC2 Simulated Sky Survey
Authors:
LSST Dark Energy Science Collaboration,
Bela Abolfathi,
David Alonso,
Robert Armstrong,
Éric Aubourg,
Humna Awan,
Yadu N. Babuji,
Franz Erik Bauer,
Rachel Bean,
George Beckett,
Rahul Biswas,
Joanne R. Bogart,
Dominique Boutigny,
Kyle Chard,
James Chiang,
Chuck F. Claver,
Johann Cohen-Tanugi,
Céline Combet,
Andrew J. Connolly,
Scott F. Daniel,
Seth W. Digel,
Alex Drlica-Wagner,
Richard Dubois,
Emmanuel Gangler,
Eric Gawiser
, et al. (55 additional authors not shown)
Abstract:
We describe the simulated sky survey underlying the second data challenge (DC2) carried out in preparation for analysis of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) by the LSST Dark Energy Science Collaboration (LSST DESC). Significant connections across multiple science domains will be a hallmark of LSST; the DC2 program represents a unique modeling effort that stresses…
▽ More
We describe the simulated sky survey underlying the second data challenge (DC2) carried out in preparation for analysis of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) by the LSST Dark Energy Science Collaboration (LSST DESC). Significant connections across multiple science domains will be a hallmark of LSST; the DC2 program represents a unique modeling effort that stresses this interconnectivity in a way that has not been attempted before. This effort encompasses a full end-to-end approach: starting from a large N-body simulation, through setting up LSST-like observations including realistic cadences, through image simulations, and finally processing with Rubin's LSST Science Pipelines. This last step ensures that we generate data products resembling those to be delivered by the Rubin Observatory as closely as is currently possible. The simulated DC2 sky survey covers six optical bands in a wide-fast-deep (WFD) area of approximately 300 deg^2 as well as a deep drilling field (DDF) of approximately 1 deg^2. We simulate 5 years of the planned 10-year survey. The DC2 sky survey has multiple purposes. First, the LSST DESC working groups can use the dataset to develop a range of DESC analysis pipelines to prepare for the advent of actual data. Second, it serves as a realistic testbed for the image processing software under development for LSST by the Rubin Observatory. In particular, simulated data provide a controlled way to investigate certain image-level systematic effects. Finally, the DC2 sky survey enables the exploration of new scientific ideas in both static and time-domain cosmology.
△ Less
Submitted 26 January, 2021; v1 submitted 12 October, 2020;
originally announced October 2020.
-
Balsam: Automated Scheduling and Execution of Dynamic, Data-Intensive HPC Workflows
Authors:
Michael A. Salim,
Thomas D. Uram,
J. Taylor Childers,
Prasanna Balaprakash,
Venkatram Vishwanath,
Michael E. Papka
Abstract:
We introduce the Balsam service to manage high-throughput task scheduling and execution on supercomputing systems. Balsam allows users to populate a task database with a variety of tasks ranging from simple independent tasks to dynamic multi-task workflows. With abstractions for the local resource scheduler and MPI environment, Balsam dynamically packages tasks into ensemble jobs and manages their…
▽ More
We introduce the Balsam service to manage high-throughput task scheduling and execution on supercomputing systems. Balsam allows users to populate a task database with a variety of tasks ranging from simple independent tasks to dynamic multi-task workflows. With abstractions for the local resource scheduler and MPI environment, Balsam dynamically packages tasks into ensemble jobs and manages their scheduling lifecycle. The ensembles execute in a pilot "launcher" which (i) ensures concurrent, load-balanced execution of arbitrary serial and parallel programs with heterogeneous processor requirements, (ii) requires no modification of user applications, (iii) is tolerant of task-level faults and provides several options for error recovery, (iv) stores provenance data (e.g task history, error logs) in the database, (v) supports dynamic workflows, in which tasks are created or killed at runtime. Here, we present the design and Python implementation of the Balsam service and launcher. The efficacy of this system is illustrated using two case studies: hyperparameter optimization of deep neural networks, and high-throughput single-point quantum chemistry calculations. We find that the unique combination of flexible job-packing and automated scheduling with dynamic (pilot-managed) execution facilitates excellent resource utilization. The scripting overheads typically needed to manage resources and launch workflows on supercomputers are substantially reduced, accelerating workflow development and execution.
△ Less
Submitted 18 September, 2019;
originally announced September 2019.
-
HACC Cosmological Simulations: First Data Release
Authors:
Katrin Heitmann,
Thomas D. Uram,
Hal Finkel,
Nicholas Frontiere,
Salman Habib,
Adrian Pope,
Esteban Rangel,
Joseph Hollowed,
Danila Korytov,
Patricia Larsen,
Benjamin S. Allen,
Kyle Chard,
Ian Foster
Abstract:
We describe the first major public data release from cosmological simulations carried out with Argonne's HACC code. This initial release covers a range of datasets from large gravity-only simulations. The data products include halo information for multiple redshifts, down-sampled particles, and lightcone outputs. We provide data from two very large LCDM simulations as well as beyond-LCDM simulatio…
▽ More
We describe the first major public data release from cosmological simulations carried out with Argonne's HACC code. This initial release covers a range of datasets from large gravity-only simulations. The data products include halo information for multiple redshifts, down-sampled particles, and lightcone outputs. We provide data from two very large LCDM simulations as well as beyond-LCDM simulations spanning eleven w0-wa cosmologies. Our release platform uses Petrel, a research data service, located at the Argonne Leadership Computing Facility. Petrel offers fast data transfer mechanisms and authentication via Globus, enabling simple and efficient access to stored datasets. Easy browsing of the available data products is provided via a web portal that allows the user to navigate simulation products efficiently. The data hub will be extended by adding more types of data products and by enabling computational capabilities to allow direct interactions with simulation results.
△ Less
Submitted 3 October, 2019; v1 submitted 26 April, 2019;
originally announced April 2019.
-
DESCQA: An Automated Validation Framework for Synthetic Sky Catalogs
Authors:
Yao-Yuan Mao,
Eve Kovacs,
Katrin Heitmann,
Thomas D. Uram,
Andrew J. Benson,
Duncan Campbell,
Sofía A. Cora,
Joseph DeRose,
Tiziana Di Matteo,
Salman Habib,
Andrew P. Hearin,
J. Bryce Kalmbach,
K. Simon Krughoff,
François Lanusse,
Zarija Lukić,
Rachel Mandelbaum,
Jeffrey A. Newman,
Nelson Padilla,
Enrique Paillas,
Adrian Pope,
Paul M. Ricker,
Andrés N. Ruiz,
Ananth Tenneti,
Cristian Vega-Martínez,
Risa H. Wechsler
, et al. (2 additional authors not shown)
Abstract:
The use of high-quality simulated sky catalogs is essential for the success of cosmological surveys. The catalogs have diverse applications, such as investigating signatures of fundamental physics in cosmological observables, understanding the effect of systematic uncertainties on measured signals and testing mitigation strategies for reducing these uncertainties, aiding analysis pipeline developm…
▽ More
The use of high-quality simulated sky catalogs is essential for the success of cosmological surveys. The catalogs have diverse applications, such as investigating signatures of fundamental physics in cosmological observables, understanding the effect of systematic uncertainties on measured signals and testing mitigation strategies for reducing these uncertainties, aiding analysis pipeline development and testing, and survey strategy optimization. The list of applications is growing with improvements in the quality of the catalogs and the details that they can provide. Given the importance of simulated catalogs, it is critical to provide rigorous validation protocols that enable both catalog providers and users to assess the quality of the catalogs in a straightforward and comprehensive way. For this purpose, we have developed the DESCQA framework for the Large Synoptic Survey Telescope Dark Energy Science Collaboration as well as for the broader community. The goal of DESCQA is to enable the inspection, validation, and comparison of an inhomogeneous set of synthetic catalogs via the provision of a common interface within an automated framework. In this paper, we present the design concept and first implementation of DESCQA. In order to establish and demonstrate its full functionality we use a set of interim catalogs and validation tests. We highlight several important aspects, both technical and scientific, that require thoughtful consideration when designing a validation framework, including validation metrics and how these metrics impose requirements on the synthetic sky catalogs.
△ Less
Submitted 8 February, 2018; v1 submitted 27 September, 2017;
originally announced September 2017.
-
Adapting the serial Alpgen event generator to simulate LHC collisions on millions of parallel threads
Authors:
J. T. Childers,
T. D. Uram,
T. J. LeCompte,
M. E. Papka,
D. P. Benjamin
Abstract:
As the LHC moves to higher energies and luminosity, the demand for computing resources increases accordingly and will soon outpace the growth of the Worldwide LHC Computing Grid. To meet this greater demand, event generation Monte Carlo was targeted for adaptation to run on Mira, the supercomputer at the Argonne Leadership Computing Facility. Alpgen is a Monte Carlo event generation application th…
▽ More
As the LHC moves to higher energies and luminosity, the demand for computing resources increases accordingly and will soon outpace the growth of the Worldwide LHC Computing Grid. To meet this greater demand, event generation Monte Carlo was targeted for adaptation to run on Mira, the supercomputer at the Argonne Leadership Computing Facility. Alpgen is a Monte Carlo event generation application that is used by LHC experiments in the simulation of collisions that take place in the Large Hadron Collider. This paper details the process by which Alpgen was adapted from a single-processor serial-application to a large-scale parallel-application and the performance that was achieved.
△ Less
Submitted 23 November, 2015;
originally announced November 2015.
