-
Mining archival data from wide-field astronomical surveys in search of near-Earth objects
Authors:
Teymoor Saifollahi,
Gijs Verdoes Kleijn,
Rees Williams,
Marco Micheli,
Toni Santana-Ros,
Ewout Helmich,
Detlef Koschny,
Luca Conversi
Abstract:
Increasing our knowledge of the orbits and compositions of Near-Earth Objects (NEOs) is important for a better understanding of the evolution of the Solar System and of life. The detection of serendipitous NEO appearances among the millions of archived exposures from large astronomical imaging surveys can provide a contribution which is complementary to NEO surveys. Using the AstroWISE information…
▽ More
Increasing our knowledge of the orbits and compositions of Near-Earth Objects (NEOs) is important for a better understanding of the evolution of the Solar System and of life. The detection of serendipitous NEO appearances among the millions of archived exposures from large astronomical imaging surveys can provide a contribution which is complementary to NEO surveys. Using the AstroWISE information system, this work aims to assess the detectability rate, the achieved recovery rate and the quality of astrometry when data mining the ESO archive for the OmegaCAM wide-field imager at the VST. We developed an automatic pipeline that searches for the NEO appearances inside the AstroWISE environment. Throughout the recovery process, the pipeline uses several public web-tools to identify possible images that overlap with the position of NEOs, and acquires information on the NEOs predicted position and other properties (e.g., magnitude, rate and direction of motion) at the time of observations. We have recovered 196 appearances of NEOs from a set of 968 appearances predicted to be recoverable. It includes appearances for three NEOs which were on the impact risk list at that point. These appearances were well before their discovery. The subsequent risk assessment using the extracted astrometry removes these NEOs from the risk list. We estimate a detectability rate of 0.05 per NEO at an SNR>3 for NEOs in the OmegaCAM archive. Our automatic recovery rates are 40% and 20% for NEOs on the risk list and the full list, respectively. The achieved astrometric and photometric accuracy is on average 0.12 arcsec and 0.1 mag. These results show the high potential of the archival imaging data of the ground-based wide-field surveys as useful instruments for the search, (p)recovery and characterization of NEOs. Highly automated approaches, as possible using AstroWISE, make this undertaking feasible.
△ Less
Submitted 6 March, 2023;
originally announced March 2023.
-
The third data release of the Kilo-Degree Survey and associated data products
Authors:
J. T. A. de Jong,
G. A. Verdoes Kleijn,
T. Erben,
H. Hildebrandt,
K. Kuijken,
G. Sikkema,
M. Brescia,
M. Bilicki,
N. R. Napolitano,
V. Amaro,
K. G. Begeman,
D. R. Boxhoorn,
H. Buddelmeijer,
S. Cavuoti,
F. Getman,
A. Grado,
E. Helmich,
Z. Huang,
N. Irisarri,
F. La Barbera,
G. Longo,
J. P. McFarland,
R. Nakajima,
M. Paolillo,
E. Puddu
, et al. (18 additional authors not shown)
Abstract:
The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky W…
▽ More
The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products, adding further area, homogenized photometric calibration, photometric redshifts and weak lensing shear measurements to the first two releases. A dedicated pipeline embedded in the Astro-WISE information system is used for the production of the main release. Modifications with respect to earlier releases are described in detail. Photometric redshifts have been derived using both Bayesian template fitting, and machine-learning techniques. For the weak lensing measurements, optimized procedures based on the THELI data reduction and lensfit shear measurement packages are used. In DR3 stacked ugri images, weight maps, masks, and source lists for 292 new survey tiles (~300 sq.deg) are made available. The multi-band catalogue, including homogenized photometry and photometric redshifts, covers the combined DR1, DR2 and DR3 footprint of 440 survey tiles (447 sq.deg). Limiting magnitudes are typically 24.3, 25.1, 24.9, 23.8 (5 sigma in a 2 arcsec aperture) in ugri, respectively, and the typical r-band PSF size is less than 0.7 arcsec. The photometric homogenization scheme ensures accurate colors and an absolute calibration stable to ~2% for gri and ~3% in u. Separately released are a weak lensing shear catalogue and photometric redshifts based on two different machine-learning techniques.
△ Less
Submitted 21 May, 2017; v1 submitted 8 March, 2017;
originally announced March 2017.
-
The SAMI Galaxy Survey: The cluster redshift survey, target selection and cluster properties
Authors:
M. S. Owers,
J. T. Allen,
I. Baldry,
J. J. Bryant,
G. N. Cecil,
L. Cortese,
S. M. Croom,
S. P. Driver,
L. M. R. Fogarty,
A. W. Green,
E. Helmich,
J. T. A. de Jong,
K. Kuijken,
S. Mahajan,
J. McFarland,
M. B. Pracy,
A. G. S. Robotham,
G. Sikkema,
S. Sweet,
E. N. Taylor,
G. Verdoes Kleijn,
A. E. Bauer,
J. Bland-Hawthorn,
S. Brough,
M. Colless
, et al. (18 additional authors not shown)
Abstract:
We describe the selection of galaxies targeted in eight low redshift clusters (APMCC0917, A168, A4038, EDCC442, A3880, A2399, A119 and A85; $0.029 < z < 0.058$) as part of the Sydney-AAO Multi-Object integral field Spectrograph Galaxy Survey (SAMI-GS). We have conducted a redshift survey of these clusters using the AAOmega multi-object spectrograph on the 3.9m Anglo-Australian Telescope. The redsh…
▽ More
We describe the selection of galaxies targeted in eight low redshift clusters (APMCC0917, A168, A4038, EDCC442, A3880, A2399, A119 and A85; $0.029 < z < 0.058$) as part of the Sydney-AAO Multi-Object integral field Spectrograph Galaxy Survey (SAMI-GS). We have conducted a redshift survey of these clusters using the AAOmega multi-object spectrograph on the 3.9m Anglo-Australian Telescope. The redshift survey is used to determine cluster membership and to characterise the dynamical properties of the clusters. In combination with existing data, the survey resulted in 21,257 reliable redshift measurements and 2899 confirmed cluster member galaxies. Our redshift catalogue has a high spectroscopic completeness ($\sim 94\%$) for $r_{\rm petro} \leq 19.4$ and clustercentric distances $R< 2\rm{R}_{200}$. We use the confirmed cluster member positions and redshifts to determine cluster velocity dispersion, $\rm{R}_{200}$, virial and caustic masses, as well as cluster structure. The clusters have virial masses $14.25 \leq {\rm log }({\rm M}_{200}/\rm{M}_{\odot}) \leq 15.19$. The cluster sample exhibits a range of dynamical states, from relatively relaxed-appearing systems, to clusters with strong indications of merger-related substructure. Aperture- and PSF-matched photometry are derived from SDSS and VST/ATLAS imaging and used to estimate stellar masses. These estimates, in combination with the redshifts, are used to define the input target catalogue for the cluster portion of the SAMI-GS. The primary SAMI-GS cluster targets have $R< \rm{R}_{200}$, velocities $|v_{\rm pec}| < 3.5σ_{200}$ and stellar masses $9.5 \leq {\rm log(M}^*_{approx}/\rm{M}_{\odot}) \leq 12$. Finally, we give an update on the SAMI-GS progress for the cluster regions.
△ Less
Submitted 2 March, 2017;
originally announced March 2017.
-
The first and second data releases of the Kilo-Degree Survey
Authors:
Jelte T. A. de Jong,
Gijs A. Verdoes Kleijn,
Danny R. Boxhoorn,
Hugo Buddelmeijer,
Massimo Capaccioli,
Fedor Getman,
Aniello Grado,
Ewout Helmich,
Zhuoyi Huang,
Nancy Irisarri,
Konrad Kuijken,
Francesco La Barbera,
John P. McFarland,
Nicola R. Napolitano,
Mario Radovich,
Gert Sikkema,
Edwin A. Valentijn,
Kor G. Begeman,
Massimo Brescia,
Stefano Cavuoti,
Ami Choi,
Oliver-Mark Cordes,
Giovanni Covone,
Massimo Dall'Ora,
Hendrik Hildebrandt
, et al. (24 additional authors not shown)
Abstract:
The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the su…
▽ More
The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the survey is mapping the large-scale matter distribution in the Universe back to a redshift of ~0.5. Secondary science cases are manifold, covering topics such as galaxy evolution, Milky Way structure, and the detection of high-redshift clusters and quasars.
KiDS is an ESO Public Survey and dedicated to serving the astronomical community with high-quality data products derived from the survey data, as well as with calibration data. Public data releases will be made on a yearly basis, the first two of which are presented here. For a total of 148 survey tiles (~160 sq.deg.) astrometrically and photometrically calibrated, coadded ugri images have been released, accompanied by weight maps, masks, source lists, and a multi-band source catalog.
A dedicated pipeline and data management system based on the Astro-WISE software system, combined with newly developed masking and source classification software, is used for the data production of the data products described here. The achieved data quality and early science projects based on the data products in the first two data releases are reviewed in order to validate the survey data. Early scientific results include the detection of nine high-z QSOs, fifteen candidate strong gravitational lenses, high-quality photometric redshifts and galaxy structural parameters for hundreds of thousands of galaxies. (Abridged)
△ Less
Submitted 19 August, 2015; v1 submitted 2 July, 2015;
originally announced July 2015.
-
Gravitational Lensing Analysis of the Kilo Degree Survey
Authors:
Konrad Kuijken,
Catherine Heymans,
Hendrik Hildebrandt,
Reiko Nakajima,
Thomas Erben,
Jelte T. A. de Jong,
Massimo Viola,
Ami Choi,
Henk Hoekstra,
Lance Miller,
Edo van Uitert,
Alexandra Amon,
Chris Blake,
Margot Brouwer,
Axel Buddendiek,
Ian Fenech Conti,
Martin Eriksen,
Aniello Grado,
Joachim Harnois-Déraps,
Ewout Helmich,
Ricardo Herbonnet,
Nancy Irisarri,
Thomas Kitching,
Dominik Klaes,
Francesco Labarbera
, et al. (10 additional authors not shown)
Abstract:
The Kilo-Degree Survey (KiDS) is a multi-band imaging survey designed for cosmological studies from weak lensing and photometric redshifts. It uses the ESO VLT Survey Telescope with its wide-field camera OmegaCAM. KiDS images are taken in four filters similar to the SDSS ugri bands. The best-seeing time is reserved for deep r-band observations that reach a median 5-sigma limiting AB magnitude of 2…
▽ More
The Kilo-Degree Survey (KiDS) is a multi-band imaging survey designed for cosmological studies from weak lensing and photometric redshifts. It uses the ESO VLT Survey Telescope with its wide-field camera OmegaCAM. KiDS images are taken in four filters similar to the SDSS ugri bands. The best-seeing time is reserved for deep r-band observations that reach a median 5-sigma limiting AB magnitude of 24.9 with a median seeing that is better than 0.7arcsec.
Initial KiDS observations have concentrated on the GAMA regions near the celestial equator, where extensive, highly complete redshift catalogues are available. A total of 109 survey tiles, one square degree each, form the basis of the first set of lensing analyses, which focus on measurements of halo properties of GAMA galaxies. 9 galaxies per square arcminute enter the lensing analysis, for an effective inverse shear variance of 69 per square arcminute. Accounting for the shape measurement weight, the median redshift of the sources is 0.53.
KiDS data processing follows two parallel tracks, one optimized for galaxy shape measurement (for weak lensing), and one for accurate matched-aperture photometry in four bands (for photometric redshifts). This technical paper describes how the lensing and photometric redshift catalogues have been produced (including an extensive description of the Gaussian Aperture and Photometry pipeline), summarizes the data quality, and presents extensive tests for systematic errors that might affect the lensing analyses. We also provide first demonstrations of the suitability of the data for cosmological measurements, and explain how the shear catalogues were blinded to prevent confirmation bias in the scientific analyses.
The KiDS shear and photometric redshift catalogues, presented in this paper, are released to the community through http://kids.strw.leidenuniv.nl .
△ Less
Submitted 16 September, 2015; v1 submitted 2 July, 2015;
originally announced July 2015.
-
Monitoring the photometric behavior of OmegaCAM with Astro-WISE
Authors:
Gijs A. Verdoes Kleijn,
Konrad H. Kuijken,
Edwin A. Valentijn,
Danny R. Boxhoorn,
Kor G. Begeman,
Erik R. Deul,
Ewout M. Helmich,
Roeland Rengelink
Abstract:
The OmegaCAM wide-field optical imager is the sole instrument on the VLT Survey Telescope at ESO's Paranal Observatory. The instrument, as well as the telescope, have been designed for surveys with very good, natural seeing-limited image quality over a 1 square degree field. OmegaCAM was commissioned in 2011 and has been observing three ESO Public Surveys in parallel since October 15, 2011. We use…
▽ More
The OmegaCAM wide-field optical imager is the sole instrument on the VLT Survey Telescope at ESO's Paranal Observatory. The instrument, as well as the telescope, have been designed for surveys with very good, natural seeing-limited image quality over a 1 square degree field. OmegaCAM was commissioned in 2011 and has been observing three ESO Public Surveys in parallel since October 15, 2011. We use the Astro-WISE information system to monitor the calibration of the observatory and to produce the Kilo Degree Survey (KiDS).
Here we describe the photometric monitoring procedures in Astro-WISE and give a first impression of OmegaCAM's photometric behavior as a function of time. The long-term monitoring of the observatory goes hand in hand with the KiDS survey production in Astro-WISE. KiDS is observed under partially non-photometric conditions. Based on the first year of OmegaCAM operations it is expected that a $\sim 1%-2%$ photometric homogeneity will be achieved for KiDS.
△ Less
Submitted 26 December, 2012;
originally announced December 2012.
-
The Astro-WISE approach to quality control for astronomical data
Authors:
J. P. McFarland,
E. M. Helmich,
E. A. Valentijn
Abstract:
We present a novel approach to quality control during the processing of astronomical data. Quality control in the Astro-WISE Information System is integral to all aspects of data handing and provides transparent access to quality estimators for all stages of data reduction from the raw image to the final catalog. The implementation of quality control mechanisms relies on the core features in this…
▽ More
We present a novel approach to quality control during the processing of astronomical data. Quality control in the Astro-WISE Information System is integral to all aspects of data handing and provides transparent access to quality estimators for all stages of data reduction from the raw image to the final catalog. The implementation of quality control mechanisms relies on the core features in this Astro-WISE Environment (AWE): an object-oriented framework, full data lineage, and both forward and backward chaining. Quality control information can be accessed via the command-line awe-prompt and the web-based Quality-WISE service. The quality control system is described and qualified using archive data from the 8-CCD Wide Field Imager (WFI) instrument (http://www.eso.org/lasilla/instruments/wfi/) on the 2.2-m MPG/ESO telescope at La Silla and (pre-)survey data from the 32-CCD OmegaCAM instrument (http://www.astro-wise.org/~omegacam/) on the VST telescope at Paranal.
△ Less
Submitted 26 March, 2012; v1 submitted 19 March, 2012;
originally announced March 2012.
-
The Astro-WISE Optical Image Pipeline: Development and Implementation
Authors:
J. P. McFarland,
G. Verdoes-Kleijn,
G. Sikkema,
E. M. Helmich,
D. R. Boxhoorn,
E. A. Valentijn
Abstract:
We have designed and implemented a novel way to process wide-field astronomical data within a distributed environment of hardware resources and humanpower. The system is characterized by integration of archiving, calibration, and post-calibration analysis of data from raw, through intermediate, to final data products. It is a true integration thanks to complete linking of data lineage from the fin…
▽ More
We have designed and implemented a novel way to process wide-field astronomical data within a distributed environment of hardware resources and humanpower. The system is characterized by integration of archiving, calibration, and post-calibration analysis of data from raw, through intermediate, to final data products. It is a true integration thanks to complete linking of data lineage from the final catalogs back to the raw data. This paper describes the pipeline processing of optical wide-field astronomical data from the WFI (http://www.eso.org/lasilla/instruments/wfi/) and OmegaCAM (http://www.astro-wise.org/~omegacam/) instruments using the Astro-WISE information system (the Astro-WISE Environment or simply AWE). This information system is an environment of hardware resources and humanpower distributed over Europe. AWE is characterized by integration of archiving, data calibration, post-calibration analysis, and archiving of raw, intermediate, and final data products. The true integration enables a complete data processing cycle from the raw data up to the publication of science-ready catalogs. The advantages of this system for very large datasets are in the areas of: survey operations management, quality control, calibration analyses, and massive processing.
△ Less
Submitted 14 October, 2011; v1 submitted 11 October, 2011;
originally announced October 2011.
-
Astro-WISE: Chaining to the Universe
Authors:
Edwin A. Valentijn,
John P. McFarland,
Jan Snigula,
Kor G. Begeman,
Danny R. Boxhoorn,
Roeland Rengelink,
Ewout Helmich,
Philippe Heraudeau,
Gijs Verdoes Kleijn,
Ronald Vermeij,
Willem-Jan Vriend,
Michiel J. Tempelaar,
Erik Deul,
Konrad Kuijken,
Massimo Capaccioli,
Roberto Silvotti,
Ralf Bender,
Mark Neeser,
Roberto Saglia,
Emmanuel Bertin,
Yannick Mellier
Abstract:
The recent explosion of recorded digital data and its processed derivatives threatens to overwhelm researchers when analysing their experimental data or when looking up data items in archives and file systems. While current hardware developments allow to acquire, process and store 100s of terabytes of data at the cost of a modern sports car, the software systems to handle these data are lagging…
▽ More
The recent explosion of recorded digital data and its processed derivatives threatens to overwhelm researchers when analysing their experimental data or when looking up data items in archives and file systems. While current hardware developments allow to acquire, process and store 100s of terabytes of data at the cost of a modern sports car, the software systems to handle these data are lagging behind. This general problem is recognized and addressed by various scientific communities, e.g., DATAGRID/EGEE federates compute and storage power over the high-energy physical community, while the astronomical community is building an Internet geared Virtual Observatory, connecting archival data. These large projects either focus on a specific distribution aspect or aim to connect many sub-communities and have a relatively long trajectory for setting standards and a common layer. Here, we report "first light" of a very different solution to the problem initiated by a smaller astronomical IT community. It provides the abstract "scientific information layer" which integrates distributed scientific analysis with distributed processing and federated archiving and publishing. By designing new abstractions and mixing in old ones, a Science Information System with fully scalable cornerstones has been achieved, transforming data systems into knowledge systems. This break-through is facilitated by the full end-to-end linking of all dependent data items, which allows full backward chaining from the observer/researcher to the experiment. Key is the notion that information is intrinsic in nature and thus is the data acquired by a scientific experiment. The new abstraction is that software systems guide the user to that intrinsic information by forcing full backward and forward chaining in the data modelling.
△ Less
Submitted 7 February, 2007;
originally announced February 2007.
