-
Completeness of the NASA/IPAC Extragalactic Database (NED) -- Local Volume Sample
Authors:
D. O. Cook,
J. M. Mazzarella,
G. Helou,
A. Alcala,
T. X. Chen,
R. Ebert,
C. Frayer,
J. Kim,
T. Lo,
B. F. Madore,
P. M. Ogle,
M. Schmitz,
L. P. Singer,
S. Terek,
J. Valladon,
X. Wu
Abstract:
We introduce the NASA/IPAC Extragalactic Database (NED) Local Volume Sample (NED-LVS), a subset of $\sim$1.9 million objects with distances out to 1000~Mpc. We use UV and IR fluxes available in NED from all-sky surveys to derive physical properties, and estimate the completeness relative to the expected local luminosity density. The completeness relative to NIR luminosities (which traces a galaxy'…
▽ More
We introduce the NASA/IPAC Extragalactic Database (NED) Local Volume Sample (NED-LVS), a subset of $\sim$1.9 million objects with distances out to 1000~Mpc. We use UV and IR fluxes available in NED from all-sky surveys to derive physical properties, and estimate the completeness relative to the expected local luminosity density. The completeness relative to NIR luminosities (which traces a galaxy's stellar mass) is roughly 100% at $D<$30~Mpc and remains moderate (70%) out to 300~Mpc. For brighter galaxies ($\gtrsim L^{*}$), NED-LVS is $\sim$100% complete out to $\sim$400~Mpc. When compared to other local Universe samples (GLADE and HECATE), all three are $\sim$100% complete below 30~Mpc. At distances beyond $\sim$80~Mpc, NED-LVS is more complete than both GLADE and HECATE by $\sim$10-20%. NED-LVS is the underlying sample for the NED gravitational wave follow-up (NED-GWF) service, which provides prioritized lists of host candidates for GW events within minutes of alerts issued by the LIGO-Virgo-KAGRA collaboration. We test the prioritization of galaxies in the volume of GW170817 by 3 physical properties, where we find that both stellar mass and inverse specific star formation rate place the correct host galaxy in the top ten. In addition, NED-LVS can be used for a wide variety of other astrophysical studies: galaxy evolution, star formation, large-scale structure, galaxy environments, and more. The data in NED are updated regularly, and NED-LVS will be updated concurrently. Consequently, NED-LVS will continue to provide an increasingly complete sample of galaxies for a multitude of astrophysical research areas for years to come.
△ Less
Submitted 9 June, 2023;
originally announced June 2023.
-
Classification of Astrophysics Journal Articles with Machine Learning to Identify Data for NED
Authors:
Tracy X. Chen,
Rick Ebert,
Joseph M. Mazzarella,
Cren Frayer,
Scott Terek,
Ben H. P. Chan,
David Cook,
Tak Lo,
Marion Schmitz,
Xiuqin Wu
Abstract:
The NASA/IPAC Extragalactic Database (NED) is a comprehensive online service that combines fundamental multi-wavelength information for known objects beyond the Milky Way and provides value-added, derived quantities and tools to search and access the data. The contents and relationships between measurements in the database are continuously augmented and revised to stay current with astrophysics li…
▽ More
The NASA/IPAC Extragalactic Database (NED) is a comprehensive online service that combines fundamental multi-wavelength information for known objects beyond the Milky Way and provides value-added, derived quantities and tools to search and access the data. The contents and relationships between measurements in the database are continuously augmented and revised to stay current with astrophysics literature and new sky surveys. The conventional process of distilling and extracting data from the literature involves human experts to review the journal articles and determine if an article is of extragalactic nature, and if so, what types of data it contains. This is both labor intensive and unsustainable, especially given the ever-increasing number of publications each year. We present here a machine learning (ML) approach developed and integrated into the NED production pipeline to help automate the classification of journal article topics and their data content for inclusion into NED. We show that this ML application can successfully reproduce the classifications of a human expert to an accuracy of over 90% in a fraction of the time it takes a human, allowing us to focus human expertise on tasks that are more difficult to automate.
△ Less
Submitted 10 January, 2022;
originally announced January 2022.
-
Best Practices for Data Publication in the Astronomical Literature
Authors:
Tracy X. Chen,
Marion Schmitz,
Joseph M. Mazzarella,
Xiuqin Wu,
Julian C. van Eyken,
Alberto Accomazzi,
Rachel L. Akeson,
Mark Allen,
Rachael Beaton,
G. Bruce Berriman,
Andrew W. Boyle,
Marianne Brouty,
Ben Chan,
Jessie L. Christiansen,
David R. Ciardi,
David Cook,
Raffaele D'Abrusco,
Rick Ebert,
Cren Frayer,
Benjamin J. Fulton,
Christopher Gelino,
George Helou,
Calen B. Henderson,
Justin Howell,
Joyce Kim
, et al. (20 additional authors not shown)
Abstract:
We present an overview of best practices for publishing data in astronomy and astrophysics journals. These recommendations are intended as a reference for authors to help prepare and publish data in a way that will better represent and support science results, enable better data sharing, improve reproducibility, and enhance the reusability of data. Observance of these guidelines will also help to…
▽ More
We present an overview of best practices for publishing data in astronomy and astrophysics journals. These recommendations are intended as a reference for authors to help prepare and publish data in a way that will better represent and support science results, enable better data sharing, improve reproducibility, and enhance the reusability of data. Observance of these guidelines will also help to streamline the extraction, preservation, integration and cross-linking of valuable data from astrophysics literature into major astronomical databases, and consequently facilitate new modes of science discovery that will better exploit the vast quantities of panchromatic and multi-dimensional data associated with the literature. We encourage authors, journal editors, referees, and publishers to implement the best practices reviewed here, as well as related recommendations from international astronomical organizations such as the International Astronomical Union (IAU) for publication of nomenclature, data, and metadata. A convenient Checklist of Recommendations for Publishing Data in the Literature is included for authors to consult before the submission of the final version of their journal articles and associated data files. We recommend that publishers of journals in astronomy and astrophysics incorporate a link to this document in their Instructions to Authors.
△ Less
Submitted 16 April, 2022; v1 submitted 2 June, 2021;
originally announced June 2021.
-
The Zwicky Transient Facility: Science Objectives
Authors:
Matthew J. Graham,
S. R. Kulkarni,
Eric C. Bellm,
Scott M. Adams,
Cristina Barbarino,
Nadejda Blagorodnova,
Dennis Bodewits,
Bryce Bolin,
Patrick R. Brady,
S. Bradley Cenko,
Chan-Kao Chang,
Michael W. Coughlin,
Kishalay De,
Gwendolyn Eadie,
Tony L. Farnham,
Ulrich Feindt,
Anna Franckowiak,
Christoffer Fremling,
Avishay Gal-yam,
Suvi Gezari,
Shaon Ghosh,
Daniel A. Goldstein,
V. Zach Golkhou,
Ariel Goobar,
Anna Y. Q. Ho
, et al. (92 additional authors not shown)
Abstract:
The Zwicky Transient Facility (ZTF), a public-private enterprise, is a new time domain survey employing a dedicated camera on the Palomar 48-inch Schmidt telescope with a 47 deg$^2$ field of view and 8 second readout time. It is well positioned in the development of time domain astronomy, offering operations at 10% of the scale and style of the Large Synoptic Survey Telescope (LSST) with a single…
▽ More
The Zwicky Transient Facility (ZTF), a public-private enterprise, is a new time domain survey employing a dedicated camera on the Palomar 48-inch Schmidt telescope with a 47 deg$^2$ field of view and 8 second readout time. It is well positioned in the development of time domain astronomy, offering operations at 10% of the scale and style of the Large Synoptic Survey Telescope (LSST) with a single 1-m class survey telescope. The public surveys will cover the observable northern sky every three nights in g and r filters and the visible Galactic plane every night in g and r. Alerts generated by these surveys are sent in real time to brokers. A consortium of universities which provided funding ("partnership") are undertaking several boutique surveys. The combination of these surveys producing one million alerts per night allows for exploration of transient and variable astrophysical phenomena brighter than r $\sim$ 20.5 on timescales of minutes to years. We describe the primary science objectives driving ZTF including the physics of supernovae and relativistic explosions, multi-messenger astrophysics, supernova cosmology, active galactic nuclei and tidal disruption events, stellar variability, and Solar System objects.
△ Less
Submitted 5 February, 2019;
originally announced February 2019.
-
The Zwicky Transient Facility: System Overview, Performance, and First Results
Authors:
Eric C. Bellm,
Shrinivas R. Kulkarni,
Matthew J. Graham,
Richard Dekany,
Roger M. Smith,
Reed Riddle,
Frank J. Masci,
George Helou,
Thomas A. Prince,
Scott M. Adams,
C. Barbarino,
Tom Barlow,
James Bauer,
Ron Beck,
Justin Belicki,
Rahul Biswas,
Nadejda Blagorodnova,
Dennis Bodewits,
Bryce Bolin,
Valery Brinnel,
Tim Brooke,
Brian Bue,
Mattia Bulla,
Rick Burruss,
S. Bradley Cenko
, et al. (91 additional authors not shown)
Abstract:
The Zwicky Transient Facility (ZTF) is a new optical time-domain survey that uses the Palomar 48-inch Schmidt telescope. A custom-built wide-field camera provides a 47 deg$^2$ field of view and 8 second readout time, yielding more than an order of magnitude improvement in survey speed relative to its predecessor survey, the Palomar Transient Factory (PTF). We describe the design and implementation…
▽ More
The Zwicky Transient Facility (ZTF) is a new optical time-domain survey that uses the Palomar 48-inch Schmidt telescope. A custom-built wide-field camera provides a 47 deg$^2$ field of view and 8 second readout time, yielding more than an order of magnitude improvement in survey speed relative to its predecessor survey, the Palomar Transient Factory (PTF). We describe the design and implementation of the camera and observing system. The ZTF data system at the Infrared Processing and Analysis Center provides near-real-time reduction to identify moving and varying objects. We outline the analysis pipelines, data products, and associated archive. Finally, we present on-sky performance analysis and first scientific results from commissioning and the early survey. ZTF's public alert stream will serve as a useful precursor for that of the Large Synoptic Survey Telescope.
△ Less
Submitted 5 February, 2019;
originally announced February 2019.
-
The Zwicky Transient Facility: Data Processing, Products, and Archive
Authors:
Frank J. Masci,
Russ R. Laher,
Ben Rusholme,
David L. Shupe,
Steven Groom,
Jason Surace,
Edward Jackson,
Serge Monkewitz,
Ron Beck,
David Flynn,
Scott Terek,
Walter Landry,
Eugean Hacopians,
Vandana Desai,
Justin Howell,
Tim Brooke,
David Imel,
Stefanie Wachter,
Quan-Zhi Ye,
Hsing-Wen Lin,
S. Bradley Cenko,
Virginia Cunningham,
Umaa Rebbapragada,
Brian Bue,
Adam A. Miller
, et al. (24 additional authors not shown)
Abstract:
The Zwicky Transient Facility (ZTF) is a new robotic time-domain survey currently in progress using the Palomar 48-inch Schmidt Telescope. ZTF uses a 47 square degree field with a 600 megapixel camera to scan the entire northern visible sky at rates of ~3760 square degrees/hour to median depths of g ~ 20.8 and r ~ 20.6 mag (AB, 5sigma in 30 sec). We describe the Science Data System that is housed…
▽ More
The Zwicky Transient Facility (ZTF) is a new robotic time-domain survey currently in progress using the Palomar 48-inch Schmidt Telescope. ZTF uses a 47 square degree field with a 600 megapixel camera to scan the entire northern visible sky at rates of ~3760 square degrees/hour to median depths of g ~ 20.8 and r ~ 20.6 mag (AB, 5sigma in 30 sec). We describe the Science Data System that is housed at IPAC, Caltech. This comprises the data-processing pipelines, alert production system, data archive, and user interfaces for accessing and analyzing the products. The realtime pipeline employs a novel image-differencing algorithm, optimized for the detection of point source transient events. These events are vetted for reliability using a machine-learned classifier and combined with contextual information to generate data-rich alert packets. The packets become available for distribution typically within 13 minutes (95th percentile) of observation. Detected events are also linked to generate candidate moving-object tracks using a novel algorithm. Objects that move fast enough to streak in the individual exposures are also extracted and vetted. The reconstructed astrometric accuracy per science image with respect to Gaia is typically 45 to 85 milliarcsec. This is the RMS per axis on the sky for sources extracted with photometric S/N >= 10. The derived photometric precision (repeatability) at bright unsaturated fluxes varies between 8 and 25 millimag. Photometric calibration accuracy with respect to Pan-STARRS1 is generally better than 2%. The products support a broad range of scientific applications: fast and young supernovae, rare flux transients, variable stars, eclipsing binaries, variability from active galactic nuclei, counterparts to gravitational wave sources, a more complete census of Type Ia supernovae, and Solar System objects.
△ Less
Submitted 5 February, 2019;
originally announced February 2019.
-
Processing Images from the Zwicky Transient Facility
Authors:
Russ R. Laher,
Frank J. Masci,
Steve Groom,
Benjamin Rusholme,
David L. Shupe,
Ed Jackson,
Jason Surace,
Dave Flynn,
Walter Landry,
Scott Terek,
George Helou,
Ron Beck,
Eugean Hacopians,
Umaa Rebbapragada,
Brian Bue,
Roger M. Smith,
Richard G. Dekany,
Adam A. Miller,
S. B. Cenko,
Eric Bellm,
Maria Patterson,
Thomas Kupfer,
Lin Yan,
Tom Barlow,
Matthew Graham
, et al. (3 additional authors not shown)
Abstract:
The Zwicky Transient Facility is a new robotic-observing program, in which a newly engineered 600-MP digital camera with a pioneeringly large field of view, 47~square degrees, will be installed into the 48-inch Samuel Oschin Telescope at the Palomar Observatory. The camera will generate $\sim 1$~petabyte of raw image data over three years of operations. In parallel related work, new hardware and s…
▽ More
The Zwicky Transient Facility is a new robotic-observing program, in which a newly engineered 600-MP digital camera with a pioneeringly large field of view, 47~square degrees, will be installed into the 48-inch Samuel Oschin Telescope at the Palomar Observatory. The camera will generate $\sim 1$~petabyte of raw image data over three years of operations. In parallel related work, new hardware and software systems are being developed to process these data in real time and build a long-term archive for the processed products. The first public release of archived products is planned for early 2019, which will include processed images and astronomical-source catalogs of the northern sky in the $g$ and $r$ bands. Source catalogs based on two different methods will be generated for the archive: aperture photometry and point-spread-function fitting.
△ Less
Submitted 16 October, 2017; v1 submitted 4 August, 2017;
originally announced August 2017.
-
Redshift-Independent Distances in the NASA/IPAC Extragalactic Database: Methodology, Content and Use of NED-D
Authors:
Ian Steer,
Barry F. Madore,
Joseph M. Mazzarella,
Marion Schmitz,
Harold G. Corwin, Jr.,
Ben H. P. Chan,
Rick Ebert,
George Helou,
Kay Baker,
Xi Chen,
Cren Frayer,
Jeff Jacobson,
Tak Lo,
Patrick Ogle,
Olga Pevunova,
Scott Terek
Abstract:
Estimates of galaxy distances based on indicators that are independent of cosmological redshift are fundamental to astrophysics. Researchers use them to establish the extragalactic distance scale, to underpin estimates of the Hubble constant, and to study peculiar velocities induced by gravitational attractions that perturb the motions of galaxies with respect to the Hubble flow of universal expan…
▽ More
Estimates of galaxy distances based on indicators that are independent of cosmological redshift are fundamental to astrophysics. Researchers use them to establish the extragalactic distance scale, to underpin estimates of the Hubble constant, and to study peculiar velocities induced by gravitational attractions that perturb the motions of galaxies with respect to the Hubble flow of universal expansion. In 2006 the NASA/IPAC Extragalactic Database (NED) began making available a comprehensive compilation of redshift-independent extragalactic distance estimates. A decade later, this compendium of distances (NED-D) now contains more than 100,000 individual estimates based on primary and secondary indicators, available for more than 28,000 galaxies, and compiled from over 2,000 references in the refereed astronomical literature. This article describes the methodology, content, and use of NED-D, and addresses challenges to be overcome in compiling such distances. Currently, 75 different distance indicators are in use. We include a figure that facilitates comparison of the indicators with significant numbers of estimates in terms of the minimum, 25th percentile, median, 75th percentile, and maximum distances spanned. Brief descriptions of the indicators, including examples of their use in the database, are given in an Appendix.
△ Less
Submitted 29 December, 2016;
originally announced December 2016.
-
Rule-based Cross-matching of Very Large Catalogs in NED
Authors:
Patrick M. Ogle,
Joe Mazzarella,
Rick Ebert,
Dario Fadda,
Tak Lo,
Scott Terek,
Marion Schmitz
Abstract:
The NASA/IPAC Extragalactic Database (NED) has deployed a new rule-based cross-matching algorithm called Match Expert (MatchEx), capable of cross-matching very large catalogs (VLCs) with >10 million objects. MatchEx goes beyond traditional position-based cross-matching algorithms by using other available data together with expert logic to determine which candidate match is the best. Furthermore, t…
▽ More
The NASA/IPAC Extragalactic Database (NED) has deployed a new rule-based cross-matching algorithm called Match Expert (MatchEx), capable of cross-matching very large catalogs (VLCs) with >10 million objects. MatchEx goes beyond traditional position-based cross-matching algorithms by using other available data together with expert logic to determine which candidate match is the best. Furthermore, the local background density of sources is used to determine and minimize the false-positive match rate and to estimate match completeness. The logical outcome and statistical probability of each match decision is stored in the database, and may be used to tune the algorithm and adjust match parameter thresholds. For our first production run, we cross-matched the GALEX All Sky Survey Catalog (GASC), containing nearly 40 million NUV-detected sources, against a directory of 180 million objects in NED. Candidate matches were identified for each GASC source within a 7.5 arcsecond radius. These candidates were filtered on position-based matching probability, and on other criteria including object type and object name. We estimate a match completeness of 97.6% and a match accuracy of 99.75%. MatchEx is being used to cross-match over 2 billion catalog sources to NED, including the Spitzer Source List, the 2MASS Point-Source Catalog, AllWISE, and SDSS DR 10. It will also speed up routine cross-matching of sources as part of the NED literature pipeline.
△ Less
Submitted 3 March, 2015;
originally announced March 2015.
