-
Euclid. I. Overview of the Euclid mission
Authors:
Euclid Collaboration,
Y. Mellier,
Abdurro'uf,
J. A. Acevedo Barroso,
A. Achúcarro,
J. Adamek,
R. Adam,
G. E. Addison,
N. Aghanim,
M. Aguena,
V. Ajani,
Y. Akrami,
A. Al-Bahlawan,
A. Alavi,
I. S. Albuquerque,
G. Alestas,
G. Alguero,
A. Allaoui,
S. W. Allen,
V. Allevato,
A. V. Alonso-Tetilla,
B. Altieri,
A. Alvarez-Candal,
A. Amara,
L. Amendola
, et al. (1086 additional authors not shown)
Abstract:
The current standard model of cosmology successfully describes a variety of measurements, but the nature of its main ingredients, dark matter and dark energy, remains unknown. Euclid is a medium-class mission in the Cosmic Vision 2015-2025 programme of the European Space Agency (ESA) that will provide high-resolution optical imaging, as well as near-infrared imaging and spectroscopy, over about 14…
▽ More
The current standard model of cosmology successfully describes a variety of measurements, but the nature of its main ingredients, dark matter and dark energy, remains unknown. Euclid is a medium-class mission in the Cosmic Vision 2015-2025 programme of the European Space Agency (ESA) that will provide high-resolution optical imaging, as well as near-infrared imaging and spectroscopy, over about 14,000 deg^2 of extragalactic sky. In addition to accurate weak lensing and clustering measurements that probe structure formation over half of the age of the Universe, its primary probes for cosmology, these exquisite data will enable a wide range of science. This paper provides a high-level overview of the mission, summarising the survey characteristics, the various data-processing steps, and data products. We also highlight the main science objectives and expected performance.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
ScopeSim: A flexible general purpose astronomical instrument data simulation framework in Python
Authors:
Kieran Leschinski,
Hugo Buddelmeijer,
Oliver Czoske,
Miguel Verdugo,
Gijs Verdoes-Kleijn,
Werner Zeilinger
Abstract:
ScopeSim is a flexible multipurpose instrument data simulation framework built in Python.
It enables both raw and reduced observation data to be simulated for a wide range of telescopes and instruments quickly and efficiently on a personal computer. The software is currently being used to generate simulated raw input data for developing the data reduction pipelines for the MICADO and METIS instr…
▽ More
ScopeSim is a flexible multipurpose instrument data simulation framework built in Python.
It enables both raw and reduced observation data to be simulated for a wide range of telescopes and instruments quickly and efficiently on a personal computer. The software is currently being used to generate simulated raw input data for developing the data reduction pipelines for the MICADO and METIS instruments at the ELT. The ScopeSim environment consists of three main packages which are responsible for providing on-sky target templates (ScopeSim_templates), the data to build the optical models of various telescopes and instruments (instrument reference database), and the simulation engine (ScopeSim). This strict division of responsibilities allows ScopeSim to be used to simulate observation data for many different instrument and telescope configurations for both imaging and spectroscopic instruments. ScopeSim has been built to avoid redundant calculations wherever possible. As such it is able to deliver simulated observations on time scales of seconds to minutes. All the code and data is open source and hosted on Github. The community is also most welcome, and indeed encouraged to contribute to code ideas, target templates, and instrument packages.
△ Less
Submitted 28 September, 2021;
originally announced September 2021.
-
Data model as agile basis for evolving calibration software
Authors:
Hugo Buddelmeijer,
Gijs A. Verdoes Kleijn,
Kieran Leschinski
Abstract:
We design the imaging data calibration and reduction software for MICADO, the First Light near-IR instrument on the Extremely Large Telescope. In this process we have hit the limit of what can be achieved with a detailed software design that is primarily captured in pdf/word documents.
Trade-offs between hardware and calibration software are required to meet stringent science requirements. To su…
▽ More
We design the imaging data calibration and reduction software for MICADO, the First Light near-IR instrument on the Extremely Large Telescope. In this process we have hit the limit of what can be achieved with a detailed software design that is primarily captured in pdf/word documents.
Trade-offs between hardware and calibration software are required to meet stringent science requirements. To support such trade-offs, more software needs to be developed in the early phases of the project: simulators, archives, prototype recipes and pipelines. This requires continuous and efficient exchange of evolving designs between the software and hardware groups, which is hard to achieve with manually maintained documents. This, and maintaining the consistency between the design documents and various software components is possible with a machine readable version of the design.
We construct a detailed design that is readable by both software and humans. From this the design documentation, prototype pipelines and data archives are generated automatically. We present the implementation of such an approach for the calibration software detailed design for the ELT MICADO imager which is based on expertise and lessons learned in earlier projects (e.g. OmegaCAM, MUSE, Euclid).
△ Less
Submitted 9 November, 2020;
originally announced November 2020.
-
KiDS+VIKING-450: A new combined optical & near-IR dataset for cosmology and astrophysics
Authors:
Angus H. Wright,
Hendrik Hildebrandt,
Konrad Kuijken,
Thomas Erben,
Robert Blake,
Hugo Buddelmeijer,
Ami Choi,
Nicholas Cross,
Jelte T. A. de Jong,
Alastair Edge,
Carlos Gonzalez-Fernandez,
Eduardo González Solares,
Aniello Grado,
Catherine Heymans,
Mike Irwin,
Aybuke Kupcu Yoldas,
James R. Lewis,
Robert G. Mann,
Nicola Napolitano,
Mario Radovich,
Peter Schneider,
Cristóbal Sifón,
William Sutherland,
Eckhard Sutorius,
Gijs A. Verdoes Kleijn
Abstract:
We present the curation and verification of a new combined optical and near infrared dataset for cosmology and astrophysics, derived from the combination of $ugri$-band imaging from the Kilo Degree Survey (KiDS) and $ZY\!J\!H\!K_{\rm s}$-band imaging from the VISTA Kilo degree Infrared Galaxy (VIKING) survey. This dataset is unrivaled in cosmological imaging surveys due to its combination of area…
▽ More
We present the curation and verification of a new combined optical and near infrared dataset for cosmology and astrophysics, derived from the combination of $ugri$-band imaging from the Kilo Degree Survey (KiDS) and $ZY\!J\!H\!K_{\rm s}$-band imaging from the VISTA Kilo degree Infrared Galaxy (VIKING) survey. This dataset is unrivaled in cosmological imaging surveys due to its combination of area ($458$ deg$^2$ before masking), depth ($r\le25$), and wavelength coverage ($ugriZY\!J\!H\!K_{\rm s}$). The combination of survey depth, area, and (most importantly) wavelength coverage allows significant reductions in systematic uncertainties (i.e. reductions of between 10 and 60\% in bias, outlier rate, and scatter) in photometric-to-spectroscopic redshift comparisons, compared to the optical-only case at photo-$z$ above $0.7$. The complementarity between our optical and NIR surveys means that over $80\%$ of our sources, across all photo-$z$, have significant detections (i.e. not upper limits) in our $8$ reddest bands. We derive photometry, photo-$z$, and stellar masses for all sources in the survey, and verify these data products against existing spectroscopic galaxy samples. We demonstrate the fidelity of our higher-level data products by constructing the survey stellar mass functions in 8 volume-complete redshift bins. We find that these photometrically derived mass functions provide excellent agreement with previous mass evolution studies derived using spectroscopic surveys. The primary data products presented in this paper are publicly available at http://kids.strw.leidenuniv.nl/.
△ Less
Submitted 2 September, 2019; v1 submitted 14 December, 2018;
originally announced December 2018.
-
Statistical analysis of probability density functions for photometric redshifts through the KiDS-ESO-DR3 galaxies
Authors:
Valeria Amaro,
Stefano Cavuoti,
Massimo Brescia,
Civita Vellucci,
Giuseppe Longo,
Maciej Bilicki,
Jelte T. A. de Jong,
Crescenzo Tortora,
Mario Radovich,
Nicola R. Napolitano,
Hugo Buddelmeijer
Abstract:
Despite the high accuracy of photometric redshifts (zphot) derived using Machine Learning (ML) methods, the quantification of errors through reliable and accurate Probability Density Functions (PDFs) is still an open problem. First, because it is difficult to accurately assess the contribution from different sources of errors, namely internal to the method itself and from the photometric features…
▽ More
Despite the high accuracy of photometric redshifts (zphot) derived using Machine Learning (ML) methods, the quantification of errors through reliable and accurate Probability Density Functions (PDFs) is still an open problem. First, because it is difficult to accurately assess the contribution from different sources of errors, namely internal to the method itself and from the photometric features defining the available parameter space. Second, because the problem of defining a robust statistical method, always able to quantify and qualify the PDF estimation validity, is still an open issue. We present a comparison among PDFs obtained using three different methods on the same data set: two ML techniques, METAPHOR (Machine-learning Estimation Tool for Accurate PHOtometric Redshifts) and ANNz2, plus the spectral energy distribution template fitting method, BPZ. The photometric data were extracted from the KiDS (Kilo Degree Survey) ESO Data Release 3, while the spectroscopy was obtained from the GAMA (Galaxy and Mass Assembly) Data Release 2. The statistical evaluation of both individual and stacked PDFs was done through quantitative and qualitative estimators, including a dummy PDF, useful to verify whether different statistical estimators can correctly assess PDF quality. We conclude that, in order to quantify the reliability and accuracy of any zphot PDF method, a combined set of statistical estimators is required.
△ Less
Submitted 23 October, 2018;
originally announced October 2018.
-
KiDS-SQuaD: The KiDS Strongly lensed Quasar Detection project
Authors:
C. Spiniello,
A. Agnello,
N. R. Napolitano,
A. V. Sergeyev,
F. I. Getman,
C. Tortora,
M. Spavone,
M. Bilicki,
H. Buddelmeijer,
L. V. E. Koopmans,
K. Kuijken,
G. Vernardos,
E. Bannikova,
M. Capaccioli
Abstract:
New methods have been recently developed to search for strong gravitational lenses, in particular lensed quasars, in wide-field imaging surveys. Here, we compare the performance of three different, morphology- and photometry- based methods to find lens candidates over the Kilo-Degree Survey (KiDS) DR3 footprint (440 deg$^2$). The three methods are: i) a multiplet detection in KiDS-DR3 and/or Gaia-…
▽ More
New methods have been recently developed to search for strong gravitational lenses, in particular lensed quasars, in wide-field imaging surveys. Here, we compare the performance of three different, morphology- and photometry- based methods to find lens candidates over the Kilo-Degree Survey (KiDS) DR3 footprint (440 deg$^2$). The three methods are: i) a multiplet detection in KiDS-DR3 and/or Gaia-DR1, ii) direct modeling of KiDS cutouts and iii) positional offsets between different surveys (KiDS-vs-Gaia, Gaia-vs-2MASS), with purpose-built astrometric recalibrations. The first benchmark for the methods has been set by the recovery of known lenses. We are able to recover seven out of ten known lenses and pairs of quasars observed in the KiDS DR3 footprint, or eight out of ten with improved selection criteria and looser colour pre-selection. This success rate reflects the combination of all methods together, which, taken individually, performed significantly worse (four lenses each). One movelty of our analysis is that the comparison of the performances of the different methods has revealed the pros and cons of the approaches and, most of all, the complementarities. We finally provide a list of high-grade candidates found by one or more methods, awaiting spectroscopic follow-up for confirmation. Of these, KiDS 1042+0023 is to our knowledge the first confirmed lensed quasar from KiDS, exhibiting two quasar spectra at the same source redshift at either sides of a red galaxy, with uniform flux-ratio $f\approx1.25$ over the wavelength range $0.45μ\mathrm{m}<λ<0.75μ\mathrm{m}.$
△ Less
Submitted 11 July, 2018; v1 submitted 31 May, 2018;
originally announced May 2018.
-
The Astropy Project: Building an inclusive, open-science project and status of the v2.0 core package
Authors:
The Astropy Collaboration,
A. M. Price-Whelan,
B. M. Sipőcz,
H. M. Günther,
P. L. Lim,
S. M. Crawford,
S. Conseil,
D. L. Shupe,
M. W. Craig,
N. Dencheva,
A. Ginsburg,
J. T. VanderPlas,
L. D. Bradley,
D. Pérez-Suárez,
M. de Val-Borro,
T. L. Aldcroft,
K. L. Cruz,
T. P. Robitaille,
E. J. Tollerud,
C. Ardelean,
T. Babej,
M. Bachetti,
A. V. Bakanov,
S. P. Bamford,
G. Barentsen
, et al. (112 additional authors not shown)
Abstract:
The Astropy project supports and fosters the development of open-source and openly-developed Python packages that provide commonly-needed functionality to the astronomical community. A key element of the Astropy project is the core package Astropy, which serves as the foundation for more specialized projects and packages. In this article, we provide an overview of the organization of the Astropy p…
▽ More
The Astropy project supports and fosters the development of open-source and openly-developed Python packages that provide commonly-needed functionality to the astronomical community. A key element of the Astropy project is the core package Astropy, which serves as the foundation for more specialized projects and packages. In this article, we provide an overview of the organization of the Astropy project and summarize key features in the core package as of the recent major release, version 2.0. We then describe the project infrastructure designed to facilitate and support development for a broader ecosystem of inter-operable packages. We conclude with a future outlook of planned new features and directions for the broader Astropy project.
△ Less
Submitted 16 January, 2018; v1 submitted 8 January, 2018;
originally announced January 2018.
-
The galaxy environment in GAMA G3C groups using the Kilo Degree Survey Data Release 3
Authors:
M. V. Costa-Duarte,
M. Viola,
A. Molino,
K. Kuijken,
L. Sodré Jr.,
M. Bilicki,
M. M. Brouwer,
H. Buddelmeijer,
F. Getman,
A. Grado,
J. T. A. de Jong,
G. V. Kleijn,
N. Napolitano,
E. Puddu,
M. Radovich,
M. Vakili
Abstract:
We aim to investigate the galaxy environment in GAMA Galaxy Groups Catalogue (G3C) using a volume-limited galaxy sample from the Kilo Degree Survey Data Release 3. The k-Nearest Neighbour technique is adapted to take into account the probability density functions (PDFs) of photometric redshifts in our calculations. This algorithm was tested on simulated KiDS tiles, showing its capability of recove…
▽ More
We aim to investigate the galaxy environment in GAMA Galaxy Groups Catalogue (G3C) using a volume-limited galaxy sample from the Kilo Degree Survey Data Release 3. The k-Nearest Neighbour technique is adapted to take into account the probability density functions (PDFs) of photometric redshifts in our calculations. This algorithm was tested on simulated KiDS tiles, showing its capability of recovering the relation between galaxy colour, luminosity and local environment. The characterization of the galaxy environment in G3C groups shows systematically steeper density contrasts for more massive groups. The red galaxy fraction gradients in these groups is evident for most of group mass bins. The density contrast of red galaxies is systematically higher at group centers when compared to blue galaxy ones. In addition, distinct group center definitions are used to show that our results are insensitive to center definitions. These results confirm the galaxy evolution scenario which environmental mechanisms are responsible for a slow quenching process as galaxies fall into groups and clusters, resulting in a smooth observed colour gradients in galaxy systems.
△ Less
Submitted 12 April, 2018; v1 submitted 20 December, 2017;
originally announced December 2017.
-
The third data release of the Kilo-Degree Survey and associated data products
Authors:
J. T. A. de Jong,
G. A. Verdoes Kleijn,
T. Erben,
H. Hildebrandt,
K. Kuijken,
G. Sikkema,
M. Brescia,
M. Bilicki,
N. R. Napolitano,
V. Amaro,
K. G. Begeman,
D. R. Boxhoorn,
H. Buddelmeijer,
S. Cavuoti,
F. Getman,
A. Grado,
E. Helmich,
Z. Huang,
N. Irisarri,
F. La Barbera,
G. Longo,
J. P. McFarland,
R. Nakajima,
M. Paolillo,
E. Puddu
, et al. (18 additional authors not shown)
Abstract:
The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky W…
▽ More
The Kilo-Degree Survey (KiDS) is an ongoing optical wide-field imaging survey with the OmegaCAM camera at the VLT Survey Telescope. It aims to image 1500 square degrees in four filters (ugri). The core science driver is mapping the large-scale matter distribution in the Universe, using weak lensing shear and photometric redshift measurements. Further science cases include galaxy evolution, Milky Way structure, detection of high-redshift clusters, and finding rare sources such as strong lenses and quasars. Here we present the third public data release (DR3) and several associated data products, adding further area, homogenized photometric calibration, photometric redshifts and weak lensing shear measurements to the first two releases. A dedicated pipeline embedded in the Astro-WISE information system is used for the production of the main release. Modifications with respect to earlier releases are described in detail. Photometric redshifts have been derived using both Bayesian template fitting, and machine-learning techniques. For the weak lensing measurements, optimized procedures based on the THELI data reduction and lensfit shear measurement packages are used. In DR3 stacked ugri images, weight maps, masks, and source lists for 292 new survey tiles (~300 sq.deg) are made available. The multi-band catalogue, including homogenized photometry and photometric redshifts, covers the combined DR1, DR2 and DR3 footprint of 440 survey tiles (447 sq.deg). Limiting magnitudes are typically 24.3, 25.1, 24.9, 23.8 (5 sigma in a 2 arcsec aperture) in ugri, respectively, and the typical r-band PSF size is less than 0.7 arcsec. The photometric homogenization scheme ensures accurate colors and an absolute calibration stable to ~2% for gri and ~3% in u. Separately released are a weak lensing shear catalogue and photometric redshifts based on two different machine-learning techniques.
△ Less
Submitted 21 May, 2017; v1 submitted 8 March, 2017;
originally announced March 2017.
-
First test of Verlinde's theory of Emergent Gravity using Weak Gravitational Lensing measurements
Authors:
Margot M. Brouwer,
Manus R. Visser,
Andrej Dvornik,
Henk Hoekstra,
Konrad Kuijken,
Edwin A. Valentijn,
Maciej Bilicki,
Chris Blake,
Sarah Brough,
Hugo Buddelmeijer,
Thomas Erben,
Catherine Heymans,
Hendrik Hildebrandt,
Benne W. Holwerda,
Andrew M. Hopkins,
Dominik Klaes,
Jochen Liske,
Jon Loveday,
John McFarland,
Reiko Nakajima,
Cristóbal Sifón,
Edward N. Taylor
Abstract:
Verlinde (2016) proposed that the observed excess gravity in galaxies and clusters is the consequence of Emergent Gravity (EG). In this theory the standard gravitational laws are modified on galactic and larger scales due to the displacement of dark energy by baryonic matter. EG gives an estimate of the excess gravity (described as an apparent dark matter density) in terms of the baryonic mass dis…
▽ More
Verlinde (2016) proposed that the observed excess gravity in galaxies and clusters is the consequence of Emergent Gravity (EG). In this theory the standard gravitational laws are modified on galactic and larger scales due to the displacement of dark energy by baryonic matter. EG gives an estimate of the excess gravity (described as an apparent dark matter density) in terms of the baryonic mass distribution and the Hubble parameter. In this work we present the first test of EG using weak gravitational lensing, within the regime of validity of the current model. Although there is no direct description of lensing and cosmology in EG yet, we can make a reasonable estimate of the expected lensing signal of low redshift galaxies by assuming a background LambdaCDM cosmology. We measure the (apparent) average surface mass density profiles of 33,613 isolated central galaxies, and compare them to those predicted by EG based on the galaxies' baryonic masses. To this end we employ the ~180 square degrees overlap of the Kilo-Degree Survey (KiDS) with the spectroscopic Galaxy And Mass Assembly (GAMA) survey. We find that the prediction from EG, despite requiring no free parameters, is in good agreement with the observed galaxy-galaxy lensing profiles in four different stellar mass bins. Although this performance is remarkable, this study is only a first step. Further advancements on both the theoretical framework and observational tests of EG are needed before it can be considered a fully developed and solidly tested theory.
△ Less
Submitted 19 December, 2016; v1 submitted 9 December, 2016;
originally announced December 2016.
-
The first and second data releases of the Kilo-Degree Survey
Authors:
Jelte T. A. de Jong,
Gijs A. Verdoes Kleijn,
Danny R. Boxhoorn,
Hugo Buddelmeijer,
Massimo Capaccioli,
Fedor Getman,
Aniello Grado,
Ewout Helmich,
Zhuoyi Huang,
Nancy Irisarri,
Konrad Kuijken,
Francesco La Barbera,
John P. McFarland,
Nicola R. Napolitano,
Mario Radovich,
Gert Sikkema,
Edwin A. Valentijn,
Kor G. Begeman,
Massimo Brescia,
Stefano Cavuoti,
Ami Choi,
Oliver-Mark Cordes,
Giovanni Covone,
Massimo Dall'Ora,
Hendrik Hildebrandt
, et al. (24 additional authors not shown)
Abstract:
The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the su…
▽ More
The Kilo-Degree Survey (KiDS) is an optical wide-field imaging survey carried out with the VLT Survey Telescope and the OmegaCAM camera. KiDS will image 1500 square degrees in four filters (ugri), and together with its near-infrared counterpart VIKING will produce deep photometry in nine bands. Designed for weak lensing shape and photometric redshift measurements, the core science driver of the survey is mapping the large-scale matter distribution in the Universe back to a redshift of ~0.5. Secondary science cases are manifold, covering topics such as galaxy evolution, Milky Way structure, and the detection of high-redshift clusters and quasars.
KiDS is an ESO Public Survey and dedicated to serving the astronomical community with high-quality data products derived from the survey data, as well as with calibration data. Public data releases will be made on a yearly basis, the first two of which are presented here. For a total of 148 survey tiles (~160 sq.deg.) astrometrically and photometrically calibrated, coadded ugri images have been released, accompanied by weight maps, masks, source lists, and a multi-band source catalog.
A dedicated pipeline and data management system based on the Astro-WISE software system, combined with newly developed masking and source classification software, is used for the data production of the data products described here. The achieved data quality and early science projects based on the data products in the first two data releases are reviewed in order to validate the survey data. Early scientific results include the detection of nine high-z QSOs, fifteen candidate strong gravitational lenses, high-quality photometric redshifts and galaxy structural parameters for hundreds of thousands of galaxies. (Abridged)
△ Less
Submitted 19 August, 2015; v1 submitted 2 July, 2015;
originally announced July 2015.
-
Leveraging Data Lineage to Infer Logical Relationships between Astronomical Catalogs
Authors:
Hugo Buddelmeijer,
Edwin A. Valentijn
Abstract:
A novel method to infer logical relationships between sets is presented. These sets can be any collection of elements, for example astronomical catalogs of celestial objects. The method does not require the contents of the sets to be known explicitly. It combines incomplete knowledge about the relationships between sets to infer a priori unknown relationships. Relationships between sets are repres…
▽ More
A novel method to infer logical relationships between sets is presented. These sets can be any collection of elements, for example astronomical catalogs of celestial objects. The method does not require the contents of the sets to be known explicitly. It combines incomplete knowledge about the relationships between sets to infer a priori unknown relationships. Relationships between sets are represented by sets of Boolean hypercubes. This leads to deductive reasoning by application of logical operators to these sets of hypercubes. A pseudocode for an efficient implementation is described.
The method is used in the Astro-WISE information system to infer relationships between catalogs of astronomical objects. These catalogs can be very large and, more importantly, their contents do not have to be available at all times. Science products are stored in Astro-WISE with references to other science products from which they are derived, or their dependencies. This creates full data lineage that links every science product all the way back to the raw data. Catalogs are created in a way that maximizes knowledge about their relationship with their dependencies. The presented algorithm is used to determine which objects a catalog represents by leveraging this information.
△ Less
Submitted 5 March, 2012;
originally announced March 2012.
-
Query Driven Visualization
Authors:
Hugo Buddelmeijer,
Edwin A. Valentijn
Abstract:
The request driven way of deriving data in Astro-WISE is extended to a query driven way of visualization. This allows scientists to focus on the science they want to perform, because all administration of their data is automated. This can be done over an abstraction layer that enhances control and flexibility for the scientist.
The request driven way of deriving data in Astro-WISE is extended to a query driven way of visualization. This allows scientists to focus on the science they want to perform, because all administration of their data is automated. This can be done over an abstraction layer that enhances control and flexibility for the scientist.
△ Less
Submitted 29 November, 2011;
originally announced November 2011.
-
Astro-WISE processing of wide-field images and other data
Authors:
Hugo Buddelmeijer,
O. Rees Williams,
John P. McFarland,
Andrey Belikov
Abstract:
Astro-WISE is the Astronomical Wide-field Imaging System for Europe. It is a scientific information system which consists of hardware and software federated over about a dozen institutes throughout Europe. It has been developed to exploit the ever increasing avalanche of data produced by astronomical surveys and data intensive scientific experiments in general.
The demo explains the architecture…
▽ More
Astro-WISE is the Astronomical Wide-field Imaging System for Europe. It is a scientific information system which consists of hardware and software federated over about a dozen institutes throughout Europe. It has been developed to exploit the ever increasing avalanche of data produced by astronomical surveys and data intensive scientific experiments in general.
The demo explains the architecture of the Astro-WISE information system and shows the use of Astro-WISE interfaces. Wide-field astronomical images are derived from the raw image to the final catalog according to the user's request. The demo is based on the standard Astro-WISE guided tour, which can be accessed from the Astro-WISE website.
The typical Astro-WISE data processing chain is shown, which can be used for data handling for a variety of different instruments, currently 14, including OmegaCAM, MegaCam, WFI, WFC, ACS/HST, etc.
△ Less
Submitted 29 November, 2011;
originally announced November 2011.
-
Automatic Optimized Discovery, Creation and Processing of Astronomical Catalogs
Authors:
Hugo Buddelmeijer,
Danny Boxhoorn,
Edwin A. Valentijn
Abstract:
We present the design of a novel way of handling astronomical catalogs in Astro-WISE in order to achieve the scalability required for the data produced by large scale surveys. A high level of automation and abstraction is achieved in order to facilitate interoperation with visualization software for interactive exploration. At the same time flexibility in processing is enhanced and data is shared…
▽ More
We present the design of a novel way of handling astronomical catalogs in Astro-WISE in order to achieve the scalability required for the data produced by large scale surveys. A high level of automation and abstraction is achieved in order to facilitate interoperation with visualization software for interactive exploration. At the same time flexibility in processing is enhanced and data is shared implicitly between scientists.
This is accomplished by using a data model that primarily stores how catalogs are derived; the contents of the catalogs are only created when necessary and stored only when beneficial for performance. Discovery of existing catalogs and creation of new catalogs is done through the same process by directly requesting the final set of sources (astronomical objects) and attributes (physical properties) that is required, for example from within visualization software.
New catalogs are automatically created to provide attributes of sources for which no suitable existing catalogs can be found. These catalogs are defined to contain the new attributes on the largest set of sources the calculation of the attributes is applicable to, facilitating reuse for future data requests. Subsequently, only those parts of the catalogs that are required for the requested end product are actually processed, ensuring scalability.
The presented mechanisms primarily determine which catalogs are created and what data has to be processed and stored: the actual processing and storage itself is left to existing functionality of the underlying information system.
△ Less
Submitted 16 November, 2011;
originally announced November 2011.
-
Query Driven Visualization of Astronomical Catalogs
Authors:
Hugo Buddelmeijer,
Edwin A. Valentijn
Abstract:
Interactive visualization of astronomical catalogs requires novel techniques due to the huge volumes and complex structure of the data produced by existing and upcoming astronomical surveys. The creation as well as the disclosure of the catalogs can be handled by data pulling mechanisms. These prevent unnecessary processing and facilitate data sharing by having users request the desired end produc…
▽ More
Interactive visualization of astronomical catalogs requires novel techniques due to the huge volumes and complex structure of the data produced by existing and upcoming astronomical surveys. The creation as well as the disclosure of the catalogs can be handled by data pulling mechanisms. These prevent unnecessary processing and facilitate data sharing by having users request the desired end products.
In this work we present query driven visualization as a logical continuation of data pulling. Scientists can request catalogs in a declarative way and set process parameters directly from within the visualization. This results in profound interoperation between software with a high level of abstraction.
New messages for the Simple Application Messaging Protocol are proposed to achieve this abstraction. Support for these messages are implemented in the Astro-WISE information system and in a set of demonstrational applications.
△ Less
Submitted 11 October, 2011;
originally announced October 2011.
