-
The Future of Astronomical Data Infrastructure: Meeting Report
Authors:
Michael R. Blanton,
Janet D. Evans,
Dara Norman,
William O'Mullane,
Adrian Price-Whelan,
Luca Rizzi,
Alberto Accomazzi,
Megan Ansdell,
Stephen Bailey,
Paul Barrett,
Steven Berukoff,
Adam Bolton,
Julian Borrill,
Kelle Cruz,
Julianne Dalcanton,
Vandana Desai,
Gregory P. Dubois-Felsmann,
Frossie Economou,
Henry Ferguson,
Bryan Field,
Dan Foreman-Mackey,
Jaime Forero-Romero,
Niall Gaffney,
Kim Gillies,
Matthew J. Graham
, et al. (47 additional authors not shown)
Abstract:
The astronomical community is grappling with the increasing volume and complexity of data produced by modern telescopes, due to difficulties in reducing, accessing, analyzing, and combining archives of data. To address this challenge, we propose the establishment of a coordinating body, an "entity," with the specific mission of enhancing the interoperability, archiving, distribution, and productio…
▽ More
The astronomical community is grappling with the increasing volume and complexity of data produced by modern telescopes, due to difficulties in reducing, accessing, analyzing, and combining archives of data. To address this challenge, we propose the establishment of a coordinating body, an "entity," with the specific mission of enhancing the interoperability, archiving, distribution, and production of both astronomical data and software. This report is the culmination of a workshop held in February 2023 on the Future of Astronomical Data Infrastructure. Attended by 70 scientists and software professionals from ground-based and space-based missions and archives spanning the entire spectrum of astronomical research, the group deliberated on the prevailing state of software and data infrastructure in astronomy, identified pressing issues, and explored potential solutions. In this report, we describe the ecosystem of astronomical data, its existing flaws, and the many gaps, duplication, inconsistencies, barriers to access, drags on productivity, missed opportunities, and risks to the long-term integrity of essential data sets. We also highlight the successes and failures in a set of deep dives into several different illustrative components of the ecosystem, included as an appendix.
△ Less
Submitted 7 November, 2023;
originally announced November 2023.
-
Anti-Black racism workshop during the Vera C. Rubin Observatory virtual 2021 Project and Community Workshop
Authors:
Andrés A. Plazas Malagón,
Federica Bianco,
Ranpal Gill,
Robert D. Blum,
Rosaria,
Bonito,
Wil O'Mullane,
Alsyha Shugart,
Rachel Street,
Aprajita Verma
Abstract:
Systemic racism is a ubiquitous theme in societies worldwide and plays a central role in shaping our economic, social, and academic institutions. The Vera C. Rubin Observatory is a major US ground-based facility based in Chile with international participation. The Observatory is an example of excellence and will deliver the largest survey of the sky ever attempted. Rubin's full scientific and soci…
▽ More
Systemic racism is a ubiquitous theme in societies worldwide and plays a central role in shaping our economic, social, and academic institutions. The Vera C. Rubin Observatory is a major US ground-based facility based in Chile with international participation. The Observatory is an example of excellence and will deliver the largest survey of the sky ever attempted. Rubin's full scientific and social potential can not be attained without addressing systemic racism and associated barriers to equity, diversity, and inclusion (EDI). During Rubin's 2021 virtual Project and Community Workshop (PCW), the annual Rubin community-based meeting, an anti-Black racism workshop took place, facilitated by 'The BIPOC Project' organization. About 60 members from different parts of the Rubin ecosystem participated. We describe the motivation, organization, challenges, outcomes, and near- and long-term goals of this workshop.
△ Less
Submitted 16 October, 2023;
originally announced October 2023.
-
Software Architecture and System Design of Rubin Observatory
Authors:
William O'Mullane,
Frossie Economou,
Kian-Tat Lim,
Fritz Mueller,
Tim Jenness,
Gregory P. Dubois-Felsmann,
Leanne P. Guy,
Ian S. Sullivan,
Yusra AlSayyad,
John D. Swinbank,
K. Simon Krughoff
Abstract:
Starting from a description of the Rubin Observatory Data Management System Architecture, and drawing on our experience with and involvement in a range of other projects including Gaia, SDSS, UKIRT, and JCMT, we derive a series of generic design patterns and lessons learned.
Starting from a description of the Rubin Observatory Data Management System Architecture, and drawing on our experience with and involvement in a range of other projects including Gaia, SDSS, UKIRT, and JCMT, we derive a series of generic design patterns and lessons learned.
△ Less
Submitted 24 November, 2022;
originally announced November 2022.
-
From Data to Software to Science with the Rubin Observatory LSST
Authors:
Katelyn Breivik,
Andrew J. Connolly,
K. E. Saavik Ford,
Mario Jurić,
Rachel Mandelbaum,
Adam A. Miller,
Dara Norman,
Knut Olsen,
William O'Mullane,
Adrian Price-Whelan,
Timothy Sacco,
J. L. Sokoloski,
Ashley Villar,
Viviana Acquaviva,
Tomas Ahumada,
Yusra AlSayyad,
Catarina S. Alves,
Igor Andreoni,
Timo Anguita,
Henry J. Best,
Federica B. Bianco,
Rosaria Bonito,
Andrew Bradshaw,
Colin J. Burke,
Andresa Rodrigues de Campos
, et al. (75 additional authors not shown)
Abstract:
The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) dataset will dramatically alter our understanding of the Universe, from the origins of the Solar System to the nature of dark matter and dark energy. Much of this research will depend on the existence of robust, tested, and scalable algorithms, software, and services. Identifying and developing such tools ahead of time has the po…
▽ More
The Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST) dataset will dramatically alter our understanding of the Universe, from the origins of the Solar System to the nature of dark matter and dark energy. Much of this research will depend on the existence of robust, tested, and scalable algorithms, software, and services. Identifying and developing such tools ahead of time has the potential to significantly accelerate the delivery of early science from LSST. Developing these collaboratively, and making them broadly available, can enable more inclusive and equitable collaboration on LSST science.
To facilitate such opportunities, a community workshop entitled "From Data to Software to Science with the Rubin Observatory LSST" was organized by the LSST Interdisciplinary Network for Collaboration and Computing (LINCC) and partners, and held at the Flatiron Institute in New York, March 28-30th 2022. The workshop included over 50 in-person attendees invited from over 300 applications. It identified seven key software areas of need: (i) scalable cross-matching and distributed joining of catalogs, (ii) robust photometric redshift determination, (iii) software for determination of selection functions, (iv) frameworks for scalable time-series analyses, (v) services for image access and reprocessing at scale, (vi) object image access (cutouts) and analysis at scale, and (vii) scalable job execution systems.
This white paper summarizes the discussions of this workshop. It considers the motivating science use cases, identified cross-cutting algorithms, software, and services, their high-level technical specifications, and the principles of inclusive collaborations needed to develop them. We provide it as a useful roadmap of needs, as well as to spur action and collaboration between groups and individuals looking to develop reusable software for early LSST science.
△ Less
Submitted 4 August, 2022;
originally announced August 2022.
-
Rubin Science Platform on Google: the story so far
Authors:
William O'Mullane,
Frossie Economou,
Flora Huang,
Dan Speck,
Hsin-Fang Chiang,
Melissa L. Graham,
Russ Allbery,
Christine Banek,
Jonathan Sick,
Adam J. Thornton,
Jess Masciarelli,
Kian-Tat Lim,
Fritz Mueller,
Sergey Padolski,
Tim Jenness,
K. Simon Krughoff,
Michelle Gower,
Leanne P. Guy,
Gregory P. Dubois-Felsmann
Abstract:
We describe Rubin Observatory's experience with offering a data access facility (and associated services including our Science Platform) deployed on Google Cloud infrastructure as part of our pre-Operations Data Preview program.
We describe Rubin Observatory's experience with offering a data access facility (and associated services including our Science Platform) deployed on Google Cloud infrastructure as part of our pre-Operations Data Preview program.
△ Less
Submitted 29 November, 2021;
originally announced November 2021.
-
Cost Management on Commercial Cloud Platforms
Authors:
G. Bruce Berriman,
William O'Mullane,
Arik Mitschang,
Ivelina Momcheva
Abstract:
Commercial cloud platforms are a powerful technology for astronomical research. Despite the benefits of cloud computing -- such as on-demand scalability and reduction of systems management overhead -- confusion over how to manage costs remains, for many, one of the biggest barriers to entry. This confusion is exacerbated by the rapid growth in services offered by commercial providers, by the growt…
▽ More
Commercial cloud platforms are a powerful technology for astronomical research. Despite the benefits of cloud computing -- such as on-demand scalability and reduction of systems management overhead -- confusion over how to manage costs remains, for many, one of the biggest barriers to entry. This confusion is exacerbated by the rapid growth in services offered by commercial providers, by the growth in the number of these providers, and by storage, compute, and I/O metered at separate rates -- all of which can change without notice. As a rule, processing is very cheap, storage is more expensive, and downloading is very expensive. Thus, an application that produces large image data sets for download will be far more expensive than an application that performs extensive processing on a small data set. This Birds of a Feather (BoF) session aimed to quantify the above statement by presenting case studies of the costing of astronomy applications on commercial clouds that covered a range of processing scenarios; these presentations were the basis for discussion by the attendees.
△ Less
Submitted 10 December, 2020;
originally announced December 2020.
-
A Need for Dedicated Outreach Expertise and Online Programming: Astro2020 Science White Paper
Authors:
Amanda E Bauer,
Britt Lundgren,
William O'Mullane,
Lauren Corlies,
Megan E Schwamb,
Brian Nord,
Dara J Norman
Abstract:
Maximizing the public impact of astronomy projects in the next decade requires NSF-funded centers to support the development of online, mobile-friendly outreach and education activities. EPO teams with astronomy, education, and web development expertise should be in place to build accessible programs at scale and support astronomers doing outreach.
Maximizing the public impact of astronomy projects in the next decade requires NSF-funded centers to support the development of online, mobile-friendly outreach and education activities. EPO teams with astronomy, education, and web development expertise should be in place to build accessible programs at scale and support astronomers doing outreach.
△ Less
Submitted 30 October, 2019;
originally announced October 2019.
-
The demise of the filesystem and multi level service architecture
Authors:
William O'Mullane,
Niall Gaffney,
Frossie Economou,
Arfon M. Smith,
J. Ross Thomson,
Tim Jenness
Abstract:
Many astronomy data centres still work on filesystems. Industry has moved on; current practice in computing infrastructure is to achieve Big Data scalability using object stores rather than POSIX file systems. This presents us with opportunities for portability and reuse of software underlying processing and archive systems but it also causes problems for legacy implementations in current data cen…
▽ More
Many astronomy data centres still work on filesystems. Industry has moved on; current practice in computing infrastructure is to achieve Big Data scalability using object stores rather than POSIX file systems. This presents us with opportunities for portability and reuse of software underlying processing and archive systems but it also causes problems for legacy implementations in current data centers.
△ Less
Submitted 31 July, 2019; v1 submitted 26 July, 2019;
originally announced July 2019.
-
Future Uses of the LSST Facility: Input from the LSST Project Science Team
Authors:
S. Kahn,
Robert Blum,
Chuck Claver,
Andrew Connolly,
Leanne Guy,
Željko Ivezić,
Robert H. Lupton,
William O'Mullane,
Steven Ritz,
Michael Strauss,
Christopher Stubbs,
Sandrine J. Thomas,
J. Anthony Tyson,
Bo Xin
Abstract:
In this white paper, we discuss future uses of the LSST facility after the planned 10-year survey is complete. We expect the LSST survey to profoundly affect the scientific landscape over the next ten years, and it is likely that unexpected discoveries may drive its future scientific program. We discuss various operations and instrument options that could be considered for an extended LSST mission…
▽ More
In this white paper, we discuss future uses of the LSST facility after the planned 10-year survey is complete. We expect the LSST survey to profoundly affect the scientific landscape over the next ten years, and it is likely that unexpected discoveries may drive its future scientific program. We discuss various operations and instrument options that could be considered for an extended LSST mission beyond ten years.
△ Less
Submitted 21 July, 2019;
originally announced July 2019.
-
Astro2020 APC White Paper: Astronomy should be in the clouds
Authors:
Arfon M. Smith,
Rob Pike,
William O'Mullane,
Frossie Economou,
Adam Bolton,
Ivelina Momcheva,
Amanda E Bauer,
Bruce Becker,
Eric Bellm,
Andrew Connolly,
Steven M. Crawford,
Nimish Hathi,
Peter Melchior,
Joshua Peek,
Arif Solmaz,
Ross Thomson,
Erik TollerudI,
David W. Liska
Abstract:
Commodity cloud computing, as provided by commercial vendors such as Amazon, Google, and Microsoft, has revolutionized computing in many sectors. With the advent of a new class of big data, public access astronomical facility such as LSST, DKIST, and WFIRST, there exists a real opportunity to combine these missions with cloud computing platforms and fundamentally change the way astronomical data i…
▽ More
Commodity cloud computing, as provided by commercial vendors such as Amazon, Google, and Microsoft, has revolutionized computing in many sectors. With the advent of a new class of big data, public access astronomical facility such as LSST, DKIST, and WFIRST, there exists a real opportunity to combine these missions with cloud computing platforms and fundamentally change the way astronomical data is collected, processed, archived, and curated. Making these changes in a cross-mission, coordinated way can provide unprecedented economies of scale in personnel, data collection and management, archiving, algorithm and software development and, most importantly, science.
△ Less
Submitted 14 July, 2019;
originally announced July 2019.
-
Petabytes to Science
Authors:
Amanda E. Bauer,
Eric C. Bellm,
Adam S. Bolton,
Surajit Chaudhuri,
A. J. Connolly,
Kelle L. Cruz,
Vandana Desai,
Alex Drlica-Wagner,
Frossie Economou,
Niall Gaffney,
J. Kavelaars,
J. Kinney,
Ting S. Li,
B. Lundgren,
R. Margutti,
G. Narayan,
B. Nord,
Dara J. Norman,
W. O'Mullane,
S. Padhi,
J. E. G. Peek,
C. Schafer,
Megan E. Schwamb,
Arfon M. Smith,
Erik J. Tollerud
, et al. (2 additional authors not shown)
Abstract:
A Kavli foundation sponsored workshop on the theme \emph{Petabytes to Science} was held 12$^{th}$ to 14$^{th}$ of February 2019 in Las Vegas. The aim of the this workshop was to discuss important trends and technologies which may support astronomy. We also tackled how to better shape the workforce for the new trends and how we should approach education and public outreach. This document was coauth…
▽ More
A Kavli foundation sponsored workshop on the theme \emph{Petabytes to Science} was held 12$^{th}$ to 14$^{th}$ of February 2019 in Las Vegas. The aim of the this workshop was to discuss important trends and technologies which may support astronomy. We also tackled how to better shape the workforce for the new trends and how we should approach education and public outreach. This document was coauthored during the workshop and edited in the weeks after. It comprises the discussions and highlights many recommendations which came out of the workshop.
We shall distill parts of this document and formulate potential white papers for the decadal survey.
△ Less
Submitted 17 November, 2019; v1 submitted 13 May, 2019;
originally announced May 2019.
-
An Overview of the LSST Image Processing Pipelines
Authors:
James Bosch,
Yusra AlSayyad,
Robert Armstrong,
Eric Bellm,
Hsin-Fang Chiang,
Siegfried Eggl,
Krzysztof Findeisen,
Merlin Fisher-Levine,
Leanne P. Guy,
Augustin Guyonnet,
Željko Ivezić,
Tim Jenness,
Gábor Kovács,
K. Simon Krughoff,
Robert H. Lupton,
Nate B. Lust,
Lauren A. MacArthur,
Joshua Meyers,
Fred Moolekamp,
Christopher B. Morrison,
Timothy D. Morton,
William O'Mullane,
John K. Parejko,
Andrés A. Plazas,
Paul A. Price
, et al. (9 additional authors not shown)
Abstract:
The Large Synoptic Survey Telescope (LSST) is an ambitious astronomical survey with a similarly ambitious Data Management component. Data Management for LSST includes processing on both nightly and yearly cadences to generate transient alerts, deep catalogs of the static sky, and forced photometry light-curves for billions of objects at hundreds of epochs, spanning at least a decade. The algorithm…
▽ More
The Large Synoptic Survey Telescope (LSST) is an ambitious astronomical survey with a similarly ambitious Data Management component. Data Management for LSST includes processing on both nightly and yearly cadences to generate transient alerts, deep catalogs of the static sky, and forced photometry light-curves for billions of objects at hundreds of epochs, spanning at least a decade. The algorithms running in these pipelines are individually sophisticated and interact in subtle ways. This paper provides an overview of those pipelines, focusing more on those interactions than the details of any individual algorithm.
△ Less
Submitted 7 December, 2018;
originally announced December 2018.
-
Gaia Data Release 2: Observational Hertzsprung-Russell diagrams
Authors:
Gaia Collaboration,
C. Babusiaux,
F. van Leeuwen,
M. A. Barstow,
C. Jordi,
A. Vallenari,
D. Bossini,
A. Bressan,
T. Cantat-Gaudin,
M. van Leeuwen,
A. G. A. Brown,
T. Prusti,
J. H. J. de Bruijne,
C. A. L. Bailer-Jones,
M. Biermann,
D. W. Evans,
L. Eyer,
F. Jansen,
S. A. Klioner,
U. Lammers,
L. Lindegren,
X. Luri,
F. Mignard,
C. Panem,
D. Pourbaix
, et al. (428 additional authors not shown)
Abstract:
We highlight the power of the Gaia DR2 in studying many fine structures of the Hertzsprung-Russell diagram (HRD). Gaia allows us to present many different HRDs, depending in particular on stellar population selections. We do not aim here for completeness in terms of types of stars or stellar evolutionary aspects. Instead, we have chosen several illustrative examples. We describe some of the select…
▽ More
We highlight the power of the Gaia DR2 in studying many fine structures of the Hertzsprung-Russell diagram (HRD). Gaia allows us to present many different HRDs, depending in particular on stellar population selections. We do not aim here for completeness in terms of types of stars or stellar evolutionary aspects. Instead, we have chosen several illustrative examples. We describe some of the selections that can be made in Gaia DR2 to highlight the main structures of the Gaia HRDs. We select both field and cluster (open and globular) stars, compare the observations with previous classifications and with stellar evolutionary tracks, and we present variations of the Gaia HRD with age, metallicity, and kinematics. Late stages of stellar evolution such as hot subdwarfs, post-AGB stars, planetary nebulae, and white dwarfs are also analysed, as well as low-mass brown dwarf objects. The Gaia HRDs are unprecedented in both precision and coverage of the various Milky Way stellar populations and stellar evolutionary phases. Many fine structures of the HRDs are presented. The clear split of the white dwarf sequence into hydrogen and helium white dwarfs is presented for the first time in an HRD. The relation between kinematics and the HRD is nicely illustrated. Two different populations in a classical kinematic selection of the halo are unambiguously identified in the HRD. Membership and mean parameters for a selected list of open clusters are provided. They allow drawing very detailed cluster sequences, highlighting fine structures, and providing extremely precise empirical isochrones that will lead to more insight in stellar physics. Gaia DR2 demonstrates the potential of combining precise astrometry and photometry for large samples for studies in stellar evolution and stellar population and opens an entire new area for HRD-based studies.
△ Less
Submitted 13 August, 2018; v1 submitted 25 April, 2018;
originally announced April 2018.
-
Gaia Data Release 1: The archive visualisation service
Authors:
A. Moitinho,
A. Krone-Martins,
H. Savietto,
M. Barros,
C. Barata,
A. J. Falcão,
T. Fernandes,
J. Alves,
A. F. Silva,
M. Gomes,
J. Bakker,
A. G. A. Brown,
J. González-Núñez,
G. Gracia-Abril,
R. Gutiérrez-Sánchez,
J. Hernández,
S. Jordan,
X. Luri,
B. Merin,
F. Mignard,
A. Mora,
V. Navarro,
W. O'Mullane,
T. Sagristà Sellés,
J. Salgado
, et al. (9 additional authors not shown)
Abstract:
Context: The first Gaia data release (DR1) delivered a catalogue of astrometry and photometry for over a billion astronomical sources. Within the panoply of methods used for data exploration, visualisation is often the starting point and even the guiding reference for scientific thought. However, this is a volume of data that cannot be efficiently explored using traditional tools, techniques, and…
▽ More
Context: The first Gaia data release (DR1) delivered a catalogue of astrometry and photometry for over a billion astronomical sources. Within the panoply of methods used for data exploration, visualisation is often the starting point and even the guiding reference for scientific thought. However, this is a volume of data that cannot be efficiently explored using traditional tools, techniques, and habits.
Aims: We aim to provide a global visual exploration service for the Gaia archive, something that is not possible out of the box for most people. The service has two main goals. The first is to provide a software platform for interactive visual exploration of the archive contents, using common personal computers and mobile devices available to most users. The second aim is to produce intelligible and appealing visual representations of the enormous information content of the archive.
Methods: The interactive exploration service follows a client-server design. The server runs close to the data, at the archive, and is responsible for hiding as far as possible the complexity and volume of the Gaia data from the client. This is achieved by serving visual detail on demand. Levels of detail are pre-computed using data aggregation and subsampling techniques. For DR1, the client is a web application that provides an interactive multi-panel visualisation workspace as well as a graphical user interface.
Results: The Gaia archive Visualisation Service offers a web-based multi-panel interactive visualisation desktop in a browser tab. It currently provides highly configurable 1D histograms and 2D scatter plots of Gaia DR1 and the Tycho-Gaia Astrometric Solution (TGAS) with linked views. An innovative feature is the creation of ADQL queries from visually defined regions in plots. [abridged]
△ Less
Submitted 1 August, 2017;
originally announced August 2017.
-
Gaia Data Release 1. Testing the parallaxes with local Cepheids and RR Lyrae stars
Authors:
Gaia Collaboration,
G. Clementini,
L. Eyer,
V. Ripepi,
M. Marconi,
T. Muraveva,
A. Garofalo,
L. M. Sarro,
M. Palmer,
X. Luri,
R. Molinaro,
L. Rimoldini,
L. Szabados,
I. Musella,
R. I. Anderson,
T. Prusti,
J. H. J. de Bruijne,
A. G. A. Brown,
A. Vallenari,
C. Babusiaux,
C. A. L. Bailer-Jones,
U. Bastian,
M. Biermann,
D. W. Evans,
F. Jansen
, et al. (566 additional authors not shown)
Abstract:
Parallaxes for 331 classical Cepheids, 31 Type II Cepheids and 364 RR Lyrae stars in common between Gaia and the Hipparcos and Tycho-2 catalogues are published in Gaia Data Release 1 (DR1) as part of the Tycho-Gaia Astrometric Solution (TGAS). In order to test these first parallax measurements of the primary standard candles of the cosmological distance ladder, that involve astrometry collected by…
▽ More
Parallaxes for 331 classical Cepheids, 31 Type II Cepheids and 364 RR Lyrae stars in common between Gaia and the Hipparcos and Tycho-2 catalogues are published in Gaia Data Release 1 (DR1) as part of the Tycho-Gaia Astrometric Solution (TGAS). In order to test these first parallax measurements of the primary standard candles of the cosmological distance ladder, that involve astrometry collected by Gaia during the initial 14 months of science operation, we compared them with literature estimates and derived new period-luminosity ($PL$), period-Wesenheit ($PW$) relations for classical and Type II Cepheids and infrared $PL$, $PL$-metallicity ($PLZ$) and optical luminosity-metallicity ($M_V$-[Fe/H]) relations for the RR Lyrae stars, with zero points based on TGAS. The new relations were computed using multi-band ($V,I,J,K_{\mathrm{s}},W_{1}$) photometry and spectroscopic metal abundances available in the literature, and applying three alternative approaches: (i) by linear least squares fitting the absolute magnitudes inferred from direct transformation of the TGAS parallaxes, (ii) by adopting astrometric-based luminosities, and (iii) using a Bayesian fitting approach. TGAS parallaxes bring a significant added value to the previous Hipparcos estimates. The relations presented in this paper represent first Gaia-calibrated relations and form a "work-in-progress" milestone report in the wait for Gaia-only parallaxes of which a first solution will become available with Gaia's Data Release 2 (DR2) in 2018.
△ Less
Submitted 1 May, 2017;
originally announced May 2017.
-
Gaia Data Release 1. Open cluster astrometry: performance, limitations, and future prospects
Authors:
Gaia Collaboration,
F. van Leeuwen,
A. Vallenari,
C. Jordi,
L. Lindegren,
U. Bastian,
T. Prusti,
J. H. J. de Bruijne,
A. G. A. Brown,
C. Babusiaux,
C. A. L. Bailer-Jones,
M. Biermann,
D. W. Evans,
L. Eyer,
F. Jansen,
S. A. Klioner,
U. Lammers,
X. Luri,
F. Mignard,
C. Panem,
D. Pourbaix,
S. Randich,
P. Sartoretti,
H. I. Siddiqui,
C. Soubiran
, et al. (567 additional authors not shown)
Abstract:
Context. The first Gaia Data Release contains the Tycho-Gaia Astrometric Solution (TGAS). This is a subset of about 2 million stars for which, besides the position and photometry, the proper motion and parallax are calculated using Hipparcos and Tycho-2 positions in 1991.25 as prior information. Aims. We investigate the scientific potential and limitations of the TGAS component by means of the ast…
▽ More
Context. The first Gaia Data Release contains the Tycho-Gaia Astrometric Solution (TGAS). This is a subset of about 2 million stars for which, besides the position and photometry, the proper motion and parallax are calculated using Hipparcos and Tycho-2 positions in 1991.25 as prior information. Aims. We investigate the scientific potential and limitations of the TGAS component by means of the astrometric data for open clusters. Methods. Mean cluster parallax and proper motion values are derived taking into account the error correlations within the astrometric solutions for individual stars, an estimate of the internal velocity dispersion in the cluster, and, where relevant, the effects of the depth of the cluster along the line of sight. Internal consistency of the TGAS data is assessed. Results. Values given for standard uncertainties are still inaccurate and may lead to unrealistic unit-weight standard deviations of least squares solutions for cluster parameters. Reconstructed mean cluster parallax and proper motion values are generally in very good agreement with earlier Hipparcos-based determination, although the Gaia mean parallax for the Pleiades is a significant exception. We have no current explanation for that discrepancy. Most clusters are observed to extend to nearly 15 pc from the cluster centre, and it will be up to future Gaia releases to establish whether those potential cluster-member stars are still dynamically bound to the clusters. Conclusions. The Gaia DR1 provides the means to examine open clusters far beyond their more easily visible cores, and can provide membership assessments based on proper motions and parallaxes. A combined HR diagram shows the same features as observed before using the Hipparcos data, with clearly increased luminosities for older A and F dwarfs.
△ Less
Submitted 3 March, 2017;
originally announced March 2017.
-
COTS software in science operations, is it worth it?
Authors:
William O'Mullane,
Nana Bach,
Jose Hernandez,
Alexander Hutton,
Rosario Messineo
Abstract:
Often, perhaps not often enough, we choose Common Off the Shelf (COTS) software for integration in our systems. These range from repositories to databases and tools we use on a daily basis. It is very hard to assess the effectiveness of these choices. While none of us would consider a project specific word processing solution when LaTeX (or even Word) many will consider writing their own data mana…
▽ More
Often, perhaps not often enough, we choose Common Off the Shelf (COTS) software for integration in our systems. These range from repositories to databases and tools we use on a daily basis. It is very hard to assess the effectiveness of these choices. While none of us would consider a project specific word processing solution when LaTeX (or even Word) many will consider writing their own data management systems. We will look at some of the COTS we have used and attempt to explain how we came to the decision and if it was worth it.
△ Less
Submitted 11 November, 2016;
originally announced January 2017.
-
Gaia TGAS search for Large Magellanic Cloud runaway supergiant stars:Candidate hypervelocity star discovery, and the nature of R71
Authors:
Daniel J. Lennon,
Roeland P. van der Marel,
Mercedes Ramos Lerate,
William O'Mullane,
Johannes Sahlmann
Abstract:
We search for runaway stars in the Large Magellanic Cloud (LMC) by computing the space velocities of the visually brightest stars in the LMC that are included in the Gaia TGAS proper motion catalog. We compare with predictions from stellar dynamical models to obtain (peculiar) velocities relative to their local stellar environment. Two of the 31 stars have unusually high proper motions. Of the rem…
▽ More
We search for runaway stars in the Large Magellanic Cloud (LMC) by computing the space velocities of the visually brightest stars in the LMC that are included in the Gaia TGAS proper motion catalog. We compare with predictions from stellar dynamical models to obtain (peculiar) velocities relative to their local stellar environment. Two of the 31 stars have unusually high proper motions. Of the remaining 29 stars we find that most objects in this sample have velocities in very good agreement with model predictions of a circularly rotating disk model. Indeed the excellent fit to the model implies that the TGAS uncertainty estimates are likely overestimated. The fastest outliers in this subsample contain the LBV R71 and a few other well known emission line objects though in no case do we derive velocities consistent with fast (~100 km/s) runaways. Our results imply that R 71 in particular has a moderate deviation from the local stellar velocity field (40 km/s) lending support to the proposition that this object cannot have evolved as a normal single star since it lies too far from massive star forming complexes to have arrived at its current position during its lifetime. Our findings therefore strengthen the case for this LBV being the result of binary evolution. Of the two stars with unusually high proper motions we find that one, the isolated B1.5 Ia+ supergiant Sk-67 2 (HIP 22237), is a candidate hypervelocity star, the TGAS proper motion implying a very large peculiar transverse velocity (~360 km/s) directed radially away from the LMC centre. If confirmed, for example by Gaia Data Release 2, it would imply that this massive supergiant, on the periphery of the LMC, is leaving the galaxy where it will explode as a supernova.
△ Less
Submitted 11 March, 2017; v1 submitted 16 November, 2016;
originally announced November 2016.
-
Gaia Data Release 1: Astrometry - one billion positions, two million proper motions and parallaxes
Authors:
L. Lindegren,
U. Lammers,
U. Bastian,
J. Hernández,
S. Klioner,
D. Hobbs,
A. Bombrun,
D. Michalik,
M. Ramos-Lerate,
A. Butkevich,
G. Comoretto,
E. Joliet,
B. Holl,
A. Hutton,
P. Parsons,
H. Steidelmüller,
U. Abbas,
M. Altmann,
A. Andrei,
S. Anton,
N. Bach,
C. Barache,
U. Becciani,
J. Berthier,
L. Bianchi
, et al. (58 additional authors not shown)
Abstract:
Gaia Data Release 1 (Gaia DR1) contains astrometric results for more than 1 billion stars brighter than magnitude 20.7 based on observations collected by the Gaia satellite during the first 14 months of its operational phase. We give a brief overview of the astrometric content of the data release and of the model assumptions, data processing, and validation of the results. For stars in common with…
▽ More
Gaia Data Release 1 (Gaia DR1) contains astrometric results for more than 1 billion stars brighter than magnitude 20.7 based on observations collected by the Gaia satellite during the first 14 months of its operational phase. We give a brief overview of the astrometric content of the data release and of the model assumptions, data processing, and validation of the results. For stars in common with the Hipparcos and Tycho-2 catalogues, complete astrometric single-star solutions are obtained by incorporating positional information from the earlier catalogues. For other stars only their positions are obtained by neglecting their proper motions and parallaxes. The results are validated by an analysis of the residuals, through special validation runs, and by comparison with external data. Results. For about two million of the brighter stars (down to magnitude ~11.5) we obtain positions, parallaxes, and proper motions to Hipparcos-type precision or better. For these stars, systematic errors depending e.g. on position and colour are at a level of 0.3 milliarcsecond (mas). For the remaining stars we obtain positions at epoch J2015.0 accurate to ~10 mas. Positions and proper motions are given in a reference frame that is aligned with the International Celestial Reference Frame (ICRF) to better than 0.1 mas at epoch J2015.0, and non-rotating with respect to ICRF to within 0.03 mas/yr. The Hipparcos reference frame is found to rotate with respect to the Gaia DR1 frame at a rate of 0.24 mas/yr. Based on less than a quarter of the nominal mission length and on very provisional and incomplete calibrations, the quality and completeness of the astrometric data in Gaia DR1 are far from what is expected for the final mission products. The results nevertheless represent a huge improvement in the available fundamental stellar data and practical definition of the optical reference frame.
△ Less
Submitted 14 September, 2016;
originally announced September 2016.
-
On-orbit performance of the Gaia CCDs at L2
Authors:
C. Crowley,
R. Kohley,
N. C. Hambly,
M. Davidson,
A. Abreu,
F. van Leeuwen,
C. Fabricius,
G. Seabroke,
J. H. J. de Bruijne,
A. Short,
L. Lindegren,
A. G. A. Brown,
G. Sarri,
P. Gare,
T. Prusti,
T. Prod'homme,
A. Mora,
J. Martin-Fleitas,
F. Raison,
U. Lammers,
W. O'Mullane,
F. Jansen
Abstract:
The European Space Agency's Gaia satellite was launched into orbit around L2 in December 2013 with a payload containing 106 large-format scientific CCDs. The primary goal of the mission is to repeatedly obtain high-precision astrometric and photometric measurements of one thousand million stars over the course of five years. The scientific value of the down-linked data, and the operation of the on…
▽ More
The European Space Agency's Gaia satellite was launched into orbit around L2 in December 2013 with a payload containing 106 large-format scientific CCDs. The primary goal of the mission is to repeatedly obtain high-precision astrometric and photometric measurements of one thousand million stars over the course of five years. The scientific value of the down-linked data, and the operation of the onboard autonomous detection chain, relies on the high performance of the detectors. As Gaia slowly rotates and scans the sky, the CCDs are continuously operated in a mode where the line clock rate and the satellite rotation spin-rate are in synchronisation. Nominal mission operations began in July 2014 and the first data release is being prepared for release at the end of Summer 2016.
In this paper we present an overview of the focal plane, the detector system, and strategies for on-orbit performance monitoring of the system. This is followed by a presentation of the performance results based on analysis of data acquired during a two-year window beginning at payload switch-on. Results for parameters such as readout noise and electronic offset behaviour are presented and we pay particular attention to the effects of the L2 radiation environment on the devices. The radiation-induced degradation in the charge transfer efficiency (CTE) in the (parallel) scan direction is clearly diagnosed; however, an extrapolation shows that charge transfer inefficiency (CTI) effects at end of mission will be approximately an order of magnitude less than predicted pre-flight. It is shown that the CTI in the serial register (horizontal direction) is still dominated by the traps inherent to the manufacturing process and that the radiation-induced degradation so far is only a few per cent. Finally, we summarise some of the detector effects discovered on-orbit which are still being investigated.
△ Less
Submitted 14 September, 2016;
originally announced September 2016.
-
GAVIP: A Platform for Gaia Data Analysis
Authors:
Daniel Vagg,
Derek O'Callaghan,
Fionn Ó hÓgáin,
Sheila McBreen,
Lorraine Hanlon,
David Lynn,
William O'Mullane
Abstract:
Gaia is a major European Space Agency (ESA) astrophysics mission designed to map and analyse 10$^9$ stars, ultimately generating more than 1 PetaByte of data products. As Gaia data becomes publicly available and reaches a wider audience, there is an increasing need to facilitate the further use of Gaia products without needing to download large datasets. The Gaia Added Value Interface Platform (GA…
▽ More
Gaia is a major European Space Agency (ESA) astrophysics mission designed to map and analyse 10$^9$ stars, ultimately generating more than 1 PetaByte of data products. As Gaia data becomes publicly available and reaches a wider audience, there is an increasing need to facilitate the further use of Gaia products without needing to download large datasets. The Gaia Added Value Interface Platform (GAVIP) is designed to address this challenge by providing an innovative platform within which scientists can submit and deploy code, packaged as "Added Value Interfaces" (AVIs), which will be executed close to the data. Deployed AVIs and associated outputs may also be made available to other GAVIP platform users, thus providing a mechanism for scientific experiment reproducibility. This paper describes the capabilities and features of GAVIP.
△ Less
Submitted 30 May, 2016;
originally announced May 2016.
-
ESA Sky: a new Astronomy Multi-Mission Interface
Authors:
Bruno Merín,
Jesús Salgado,
Fabrizio Giordano,
Deborah Baines,
María-Henar Sarmiento,
Belén López Martí,
Elena Racero,
Raúl Gutiérrez,
Andy Pollock,
Michael Rosa,
Javier Castellanos,
Juan González,
Ignacio León,
Iñaki Ortiz de Landaluce,
Pilar de Teodoro,
Sara Nieto,
Daniel J. Lennon,
Christophe Arviset,
Guido de Marchi,
William O'Mullane
Abstract:
We present a science-driven discovery portal for all the ESA Astronomy Missions called ESA Sky that allow users to explore the multi-wavelength sky and to seamlessly retrieve science-ready data in all ESA Astronomy mission archives from a web application without prior-knowledge of any of the missions. The first public beta of the service has been released, currently featuring an interface for expl…
▽ More
We present a science-driven discovery portal for all the ESA Astronomy Missions called ESA Sky that allow users to explore the multi-wavelength sky and to seamlessly retrieve science-ready data in all ESA Astronomy mission archives from a web application without prior-knowledge of any of the missions. The first public beta of the service has been released, currently featuring an interface for exploration of the multi-wavelength sky and for single and/or multiple target searches of science-ready imaging data and catalogues. Future releases will enable retrieval of spectra and will have special time-domain exploration features. From a technical point of view, the system offers progressive multi-resolution all-sky projections of full mission datasets using a new generation of HEALPix projections called HiPS, developed at the CDS; detailed geometrical footprints to connect the all-sky mosaics to individual observations; and direct access to science-ready data at the underlying mission-specific science archives.
△ Less
Submitted 2 December, 2015;
originally announced December 2015.
-
MOC - HEALPix Multi-Order Coverage map Version 1.0
Authors:
Pierre Fernique,
Thomas Boch,
Tom Donaldson,
Daniel Durand,
Wil O'Mullane,
Martin Reinecke,
Mark Taylor
Abstract:
This document describes the Multi-Order Coverage map method (MOC) to specify arbitrary sky regions. The goal is to be able to provide a very fast comparison mechanism between coverage maps. The mechanism is based on the HEALPix sky tessellation algorithm. It is essentially a simple way to map regions of the sky into hierarchically grouped predefined cells.
This document describes the Multi-Order Coverage map method (MOC) to specify arbitrary sky regions. The goal is to be able to provide a very fast comparison mechanism between coverage maps. The mechanism is based on the HEALPix sky tessellation algorithm. It is essentially a simple way to map regions of the sky into hierarchically grouped predefined cells.
△ Less
Submitted 12 May, 2015;
originally announced May 2015.
-
The Past, Present and Future of Astronomical Data Formats
Authors:
Jessica Mink,
Robert G. Mann,
Robert Hanisch,
Arnold Rots,
Rob Seaman,
Tim Jenness,
Brian Thomas,
William O'Mullane
Abstract:
The future of astronomy is inextricably entwined with the care and feeding of astronomical data products. Community standards such as FITS and NDF have been instrumental in the success of numerous astronomy projects. Their very success challenges us to entertain pragmatic strategies to adapt and evolve the standards to meet the aggressive data-handling requirements of facilities now being designed…
▽ More
The future of astronomy is inextricably entwined with the care and feeding of astronomical data products. Community standards such as FITS and NDF have been instrumental in the success of numerous astronomy projects. Their very success challenges us to entertain pragmatic strategies to adapt and evolve the standards to meet the aggressive data-handling requirements of facilities now being designed and built. We discuss characteristics that have made standards successful in the past, as well as desirable features for the future, and an open discussion follows.
△ Less
Submitted 4 November, 2014;
originally announced November 2014.
-
Astronomy and Computing: a New Journal for the Astronomical Computing Community
Authors:
Alberto Accomazzi,
Tamás Budavári,
Christopher Fluke,
Norman Gray,
Robert G Mann,
William O'Mullane,
Andreas Wicenec,
Michael Wise
Abstract:
We introduce \emph{Astronomy and Computing}, a new journal for the growing population of people working in the domain where astronomy overlaps with computer science and information technology. The journal aims to provide a new communication channel within that community, which is not well served by current journals, and to help secure recognition of its true importance within modern astronomy. In…
▽ More
We introduce \emph{Astronomy and Computing}, a new journal for the growing population of people working in the domain where astronomy overlaps with computer science and information technology. The journal aims to provide a new communication channel within that community, which is not well served by current journals, and to help secure recognition of its true importance within modern astronomy. In this inaugural editorial, we describe the rationale for creating the journal, outline its scope and ambitions, and seek input from the community in defining in detail how the journal should work towards its high-level goals.
△ Less
Submitted 30 October, 2012;
originally announced October 2012.
-
The astrometric core solution for the Gaia mission. Overview of models, algorithms and software implementation
Authors:
Lennart Lindegren,
Uwe Lammers,
David Hobbs,
William O'Mullane,
Ulrich Bastian,
José Hernández
Abstract:
The Gaia satellite will observe about one billion stars and other point-like sources. The astrometric core solution will determine the astrometric parameters (position, parallax, and proper motion) for a subset of these sources, using a global solution approach which must also include a large number of parameters for the satellite attitude and optical instrument. The accurate and efficient impleme…
▽ More
The Gaia satellite will observe about one billion stars and other point-like sources. The astrometric core solution will determine the astrometric parameters (position, parallax, and proper motion) for a subset of these sources, using a global solution approach which must also include a large number of parameters for the satellite attitude and optical instrument. The accurate and efficient implementation of this solution is an extremely demanding task, but crucial for the outcome of the mission. We provide a comprehensive overview of the mathematical and physical models applicable to this solution, as well as its numerical and algorithmic framework. The astrometric core solution is a simultaneous least-squares estimation of about half a billion parameters, including the astrometric parameters for some 100 million well-behaved so-called primary sources. The global nature of the solution requires an iterative approach, which can be broken down into a small number of distinct processing blocks (source, attitude, calibration and global updating) and auxiliary processes (including the frame rotator and selection of primary sources). We describe each of these processes in some detail, formulate the underlying models, from which the observation equations are derived, and outline the adopted numerical solution methods with due consideration of robustness and the structure of the resulting system of equations. Appendices provide brief introductions to some important mathematical tools (quaternions and B-splines for the attitude representation, and a modified Cholesky algorithm for positive semidefinite problems) and discuss some complications expected in the real mission data.
△ Less
Submitted 18 December, 2011;
originally announced December 2011.
-
IVOA Recommendation: IVOA Registry Interfaces Version 1.0
Authors:
Kevin Benson,
Ray Plante,
Elizabeth Auden,
Matthew Graham,
Gretchen Greene,
Martin Hill,
Tony Linde,
Dave Morris,
Wil O'Mullane,
Guy Rixon,
Aurélien Stébé,
Kona Andrews
Abstract:
Registries provide a mechanism with which VO applications can discover and select resources--e.g. data and services--that are relevant for a particular scientific problem. This specification defines the interfaces that support interactions between applications and registries as well as between the registries themselves. It is based on a general, distributed model composed of so-called searchable a…
▽ More
Registries provide a mechanism with which VO applications can discover and select resources--e.g. data and services--that are relevant for a particular scientific problem. This specification defines the interfaces that support interactions between applications and registries as well as between the registries themselves. It is based on a general, distributed model composed of so-called searchable and publishing registries. The specification has two main components: an interface for searching and an interface for harvesting. All interfaces are defined by a standard Web Service Description Language (WSDL) document; however, harvesting is also supported through the existing Open Archives Initiative Protocol for Metadata Harvesting, defined as an HTTP REST interface. Finally, this specification details the metadata used to describe registries themselves as resources using an extension of the VOResource metadata schema.
△ Less
Submitted 3 October, 2011;
originally announced October 2011.
-
IVOA Recommendation: IVOA Astronomical Data Query Language Version 2.00
Authors:
Inaki Ortiz,
Jeff Lusted,
Pat Dowler,
Alexander Szalay,
Yuji Shirasaki,
Maria A. Nieto-Santisteban,
Masatoshi Ohishi,
William O'Mullane,
Pedro Osuna,
the VOQL-TEG Working Group,
the VOQL Working Group
Abstract:
This document describes the Astronomical Data Query Language (ADQL). ADQL has been developed based on SQL92. This document describes the subset of the SQL grammar supported by ADQL. Special restrictions and extensions to SQL92 have been defined in order to support generic and astronomy specific operations.
This document describes the Astronomical Data Query Language (ADQL). ADQL has been developed based on SQL92. This document describes the subset of the SQL grammar supported by ADQL. Special restrictions and extensions to SQL92 have been defined in order to support generic and astronomy specific operations.
△ Less
Submitted 3 October, 2011;
originally announced October 2011.
-
Implementing the Gaia Astrometric Global Iterative Solution (AGIS) in Java
Authors:
William O'Mullane,
Uwe Lammers,
Lennart Lindegren,
Jose Hernandez,
David Hobbs
Abstract:
This paper provides a description of the Java software framework which has been constructed to run the Astrometric Global Iterative Solution for the Gaia mission. This is the mathematical framework to provide the rigid reference frame for Gaia observations from the Gaia data itself. This process makes Gaia a self calibrated, and input catalogue independent, mission. The framework is highly distrib…
▽ More
This paper provides a description of the Java software framework which has been constructed to run the Astrometric Global Iterative Solution for the Gaia mission. This is the mathematical framework to provide the rigid reference frame for Gaia observations from the Gaia data itself. This process makes Gaia a self calibrated, and input catalogue independent, mission. The framework is highly distributed typically running on a cluster of machines with a database back end. All code is written in the Java language. We describe the overall architecture and some of the details of the implementation.
△ Less
Submitted 10 August, 2011;
originally announced August 2011.
-
Using Java for distributed computing in the Gaia satellite data processing
Authors:
William O'Mullane,
Xavier Luri,
Paul Parsons,
Uwe Lammers,
John Hoar,
Jose Hernandez
Abstract:
In recent years Java has matured to a stable easy-to-use language with the flexibility of an interpreter (for reflection etc.) but the performance and type checking of a compiled language. When we started using Java for astronomical applications around 1999 they were the first of their kind in astronomy. Now a great deal of astronomy software is written in Java as are many business applications.…
▽ More
In recent years Java has matured to a stable easy-to-use language with the flexibility of an interpreter (for reflection etc.) but the performance and type checking of a compiled language. When we started using Java for astronomical applications around 1999 they were the first of their kind in astronomy. Now a great deal of astronomy software is written in Java as are many business applications.
We discuss the current environment and trends concerning the language and present an actual example of scientific use of Java for high-performance distributed computing: ESA's mission Gaia. The Gaia scanning satellite will perform a galactic census of about 1000 million objects in our galaxy. The Gaia community has chosen to write its processing software in Java. We explore the manifold reasons for choosing Java for this large science collaboration.
Gaia processing is numerically complex but highly distributable, some parts being embarrassingly parallel. We describe the Gaia processing architecture and its realisation in Java. We delve into the astrometric solution which is the most advanced and most complex part of the processing. The Gaia simulator is also written in Java and is the most mature code in the system. This has been successfully running since about 2005 on the supercomputer "Marenostrum" in Barcelona. We relate experiences of using Java on a large shared machine.
Finally we discuss Java, including some of its problems, for scientific computing.
△ Less
Submitted 1 August, 2011;
originally announced August 2011.
-
LSST: from Science Drivers to Reference Design and Anticipated Data Products
Authors:
Željko Ivezić,
Steven M. Kahn,
J. Anthony Tyson,
Bob Abel,
Emily Acosta,
Robyn Allsman,
David Alonso,
Yusra AlSayyad,
Scott F. Anderson,
John Andrew,
James Roger P. Angel,
George Z. Angeli,
Reza Ansari,
Pierre Antilogus,
Constanza Araujo,
Robert Armstrong,
Kirk T. Arndt,
Pierre Astier,
Éric Aubourg,
Nicole Auza,
Tim S. Axelrod,
Deborah J. Bard,
Jeff D. Barr,
Aurelian Barrau,
James G. Bartlett
, et al. (288 additional authors not shown)
Abstract:
(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the…
▽ More
(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the Solar System, exploring the transient optical sky, and mapping the Milky Way. LSST will be a wide-field ground-based system sited at Cerro Pachón in northern Chile. The telescope will have an 8.4 m (6.5 m effective) primary mirror, a 9.6 deg$^2$ field of view, and a 3.2 Gigapixel camera. The standard observing sequence will consist of pairs of 15-second exposures in a given field, with two such visits in each pointing in a given night. With these repeats, the LSST system is capable of imaging about 10,000 square degrees of sky in a single filter in three nights. The typical 5$σ$ point-source depth in a single visit in $r$ will be $\sim 24.5$ (AB). The project is in the construction phase and will begin regular survey operations by 2022. The survey area will be contained within 30,000 deg$^2$ with $δ<+34.5^\circ$, and will be imaged multiple times in six bands, $ugrizy$, covering the wavelength range 320--1050 nm. About 90\% of the observing time will be devoted to a deep-wide-fast survey mode which will uniformly observe a 18,000 deg$^2$ region about 800 times (summed over all six bands) during the anticipated 10 years of operations, and yield a coadded map to $r\sim27.5$. The remaining 10\% of the observing time will be allocated to projects such as a Very Deep and Fast time domain survey. The goal is to make LSST data products, including a relational database of about 32 trillion observations of 40 billion objects, available to the public and scientists around the world.
△ Less
Submitted 23 May, 2018; v1 submitted 15 May, 2008;
originally announced May 2008.
-
Gaia: Organisation and challenges for the data processing
Authors:
F. Mignard,
C. Bailer-Jones,
U. Bastian,
R. Drimmel,
L. Eyer,
D. Katz,
F. van Leeuwen,
X. Luri,
W. O'Mullane,
X. Passot,
D. Pourbaix,
T. Prusti
Abstract:
Gaia is an ambitious space astrometry mission of ESA with a main objective to map the sky in astrometry and photometry down to a magnitude 20 by the end of the next decade. While the mission is built and operated by ESA and an industrial consortium, the data processing is entrusted to a consortium formed by the scientific community, which was formed in 2006 and formally selected by ESA one year…
▽ More
Gaia is an ambitious space astrometry mission of ESA with a main objective to map the sky in astrometry and photometry down to a magnitude 20 by the end of the next decade. While the mission is built and operated by ESA and an industrial consortium, the data processing is entrusted to a consortium formed by the scientific community, which was formed in 2006 and formally selected by ESA one year later. The satellite will downlink around 100 TB of raw telemetry data over a mission duration of 5 years from which a very complex iterative processing will lead to the final science output: astrometry with a final accuracy of a few tens of microarcseconds, epoch photometry in wide and narrow bands, radial velocity and spectra for the stars brighter than 17 mag. We discuss the general principles and main difficulties of this very large data processing and present the organisation of the European Consortium responsible for its design and implementation.
△ Less
Submitted 6 December, 2007;
originally announced December 2007.
-
Gaia Science Operations Centre
Authors:
William O'Mullane,
Uwe Lammers,
John Hoar,
Jose Hernandez
Abstract:
Brief outline of Science Operations Centre activities for Gaia.
Brief outline of Science Operations Centre activities for Gaia.
△ Less
Submitted 4 December, 2007;
originally announced December 2007.
-
ECSS in the eXtreme
Authors:
William O'Mullane,
John Hoar,
Uwe Lammers
Abstract:
The ESAC Gaia team engages in a form of eXtreme programming while the DPAC will follow a series of six month development cycles modeled on this approach. As a project within the European Space Agency the European Committee for Space Standardization (ECSS) standards are required. We present the bringing together of these realms.
The ESAC Gaia team engages in a form of eXtreme programming while the DPAC will follow a series of six month development cycles modeled on this approach. As a project within the European Space Agency the European Committee for Space Standardization (ECSS) standards are required. We present the bringing together of these realms.
△ Less
Submitted 3 December, 2007;
originally announced December 2007.
-
Gaia Data Processing Architecture
Authors:
W. O'Mullane,
U. Lammers,
C. Bailer-Jones,
U. Bastian,
A. Brown,
R. Drimmel,
L. Eyer,
C. Huc,
F. Jansen,
D. Katz,
L. Lindegren,
D. Pourbaix,
X. Luri,
F. Mignard,
J. Torra,
F. van Leeuwen
Abstract:
Gaia is ESA's ambitious space astrometry mission the main objective of which is to astrometrically and spectro-photometrically map 1000 Million celestial objects (mostly in our galaxy) with unprecedented accuracy. The announcement of opportunity for the data processing will be issued by ESA late in 2006. The Gaia Data Processing and Analysis Consortium (DPAC) has been formed recently and is prep…
▽ More
Gaia is ESA's ambitious space astrometry mission the main objective of which is to astrometrically and spectro-photometrically map 1000 Million celestial objects (mostly in our galaxy) with unprecedented accuracy. The announcement of opportunity for the data processing will be issued by ESA late in 2006. The Gaia Data Processing and Analysis Consortium (DPAC) has been formed recently and is preparing an answer. The satellite will downlink close to 100 TB of raw telemetry data over 5 years. To achieve its required accuracy of a few 10s of Microarcsecond astrometry, a highly involved processing of this data is required.
In addition to the main astrometric instrument Gaia will host a Radial Velocity instrument, two low-resolution dispersers for multi-color photometry and two Star Mappers. Gaia is a flying Giga Pixel camera. The various instruments each require relatively complex processing while at the same time being interdependent. We describe the overall composition of the DPAC and the envisaged overall architecture of the Gaia data processing system. We shall delve further into the core processing - one of the nine, so-called, coordination units comprising the Gaia processing system.
△ Less
Submitted 29 November, 2006;
originally announced November 2006.
-
When Database Systems Meet the Grid
Authors:
Maria A. Nieto-Santisteban,
Alexander S. Szalay,
Aniruddha R. Thakar,
William J. O'Mullane,
Jim Gray,
James Annis
Abstract:
We illustrate the benefits of combining database systems and Grid technologies for data-intensive applications. Using a cluster of SQL servers, we reimplemented an existing Grid application that finds galaxy clusters in a large astronomical database. The SQL implementation runs an order of magnitude faster than the earlier Tcl-C-file-based implementation. We discuss why and how Grid applications…
▽ More
We illustrate the benefits of combining database systems and Grid technologies for data-intensive applications. Using a cluster of SQL servers, we reimplemented an existing Grid application that finds galaxy clusters in a large astronomical database. The SQL implementation runs an order of magnitude faster than the earlier Tcl-C-file-based implementation. We discuss why and how Grid applications can take advantage of database systems.
△ Less
Submitted 3 February, 2005;
originally announced February 2005.
-
There Goes the Neighborhood: Relational Algebra for Spatial Data Search
Authors:
Jim Gray,
Alexander S. Szalay,
Aniruddha R. Thakar,
Gyorgy Fekete,
William O'Mullane,
Maria A. Nieto-Santisteban,
Gerd Heber,
Arnold H. Rots
Abstract:
We explored ways of doing spatial search within a relational database: (1) hierarchical triangular mesh (a tessellation of the sphere), (2) a zoned bucketing system, and (3) representing areas as disjunctive-normal form constraints. Each of these approaches has merits. They all allow efficient point-in-region queries. A relational representation for regions allows Boolean operations among them a…
▽ More
We explored ways of doing spatial search within a relational database: (1) hierarchical triangular mesh (a tessellation of the sphere), (2) a zoned bucketing system, and (3) representing areas as disjunctive-normal form constraints. Each of these approaches has merits. They all allow efficient point-in-region queries. A relational representation for regions allows Boolean operations among them and allows quick tests for point-in-region, regions-containing-point, and region-overlap. The speed of these algorithms is much improved by a zone and multi-scale zone-pyramid scheme. The approach has the virtue that the zone mechanism works well on B-Trees native to all SQL systems and integrates naturally with current query optimizers - rather than requiring a new spatial access method and concomitant query optimizer extensions. Over the last 5 years, we have used these techniques extensively in our work on SkyServer.sdss.org, and SkyQuery.net.
△ Less
Submitted 13 August, 2004;
originally announced August 2004.
-
Extending the SDSS Batch Query System to the National Virtual Observatory Grid
Authors:
Maria A. Nieto-Santisteban,
William O'Mullane,
Jim Gray,
Nolan Li,
Tamas Budavari,
Alexander S. Szalay,
Aniruddha R. Thakar
Abstract:
The Sloan Digital Sky Survey science database is approaching 2TB. While the vast majority of queries normally execute in seconds or minutes, this interactive execution time can be disproportionately increased by a small fraction of queries that take hours or days to run; either because they require non-index scans of the largest tables or because they request very large result sets. In response…
▽ More
The Sloan Digital Sky Survey science database is approaching 2TB. While the vast majority of queries normally execute in seconds or minutes, this interactive execution time can be disproportionately increased by a small fraction of queries that take hours or days to run; either because they require non-index scans of the largest tables or because they request very large result sets. In response to this, we added a multi-queue job submission and tracking system. The transfer of very large result sets from queries over the network is another serious problem. Statistics suggested that much of this data transfer is unnecessary; users would prefer to store results locally in order to allow further cross matching and filtering. To allow local analysis, we implemented a system that gives users their own personal database (MyDB) at the portal site. Users may transfer data to their MyDB, and then perform further analysis before extracting it to their own machine.
We intend to extend the MyDB and asynchronous query ideas to multiple NVO nodes. This implies development, in a distributed manner, of several features, which have been demonstrated for a single node in the SDSS Batch Query System (CasJobs). The generalization of asynchronous queries necessitates some form of MyDB storage as well as workflow tracking services on each node and coordination strategies among nodes.
△ Less
Submitted 12 March, 2004;
originally announced March 2004.
-
Preparing Red-Green-Blue (RGB) Images from CCD Data
Authors:
Robert Lupton,
Michael R. Blanton,
George Fekete,
David W. Hogg,
Wil O'Mullane,
Alex Szalay,
N. Wherry
Abstract:
We present a new, and we believe arguably correct, algorithm for producing Red-Green-Blue (RBG) composites from 3-band astronomical images. Our method ensures that an object with a specified astronomical color (e.g. g-r and r-i) has a unique color in the RGB image, as opposed to the burnt-out white stars to which we are accustomed. A natural consequence of this is that we can use the same colors…
▽ More
We present a new, and we believe arguably correct, algorithm for producing Red-Green-Blue (RBG) composites from 3-band astronomical images. Our method ensures that an object with a specified astronomical color (e.g. g-r and r-i) has a unique color in the RGB image, as opposed to the burnt-out white stars to which we are accustomed. A natural consequence of this is that we can use the same colors to code color-magnitude diagrams, providing a natural `index' to our images. We also introduce the use of an asinh stretch, which allows us to show faint objects while simultaneously preserving the structure of brighter objects in the field, such as the spiral arms of large galaxies. We believe that, in addition to their aesthetic value, our images convey far more information than do the traditional ones, and provide examples from Sloan Digital Sky Survey (SDSS) imaging, the Hubble Deep Field (HDF), and Chandra to support our claims. More examples are available at http://www.astro.princeton.edu/~rhl/PrettyPictures
△ Less
Submitted 18 December, 2003;
originally announced December 2003.
-
The angular power spectrum of radio emission at 2.3 GHz
Authors:
G. Giardino,
A. J. Banday,
P. Fosalba,
K. M. Gorski,
J. L. Jonas,
W. O'Mullane,
J. Tauber
Abstract:
We have analysed the Rhodes/HartRAO survey at 2326 MHz and derived the global angular power spectrum of Galactic continuum emission. In order to measure the angular power spectrum of the diffuse component, point sources were removed from the map by median filtering. A least-square fit to the angular power spectrum of the entire survey with a power law spectrum C_l proportional to l^{-alpha}, giv…
▽ More
We have analysed the Rhodes/HartRAO survey at 2326 MHz and derived the global angular power spectrum of Galactic continuum emission. In order to measure the angular power spectrum of the diffuse component, point sources were removed from the map by median filtering. A least-square fit to the angular power spectrum of the entire survey with a power law spectrum C_l proportional to l^{-alpha}, gives alpha = 2.43 +/- 0.01 for l = 2-100. The angular power spectrum of radio emission appears to steepen at high Galactic latitudes and for observed regions with |b| > 20 deg, the fitted spectral index is alpha = 2.92 +/- 0.07. We have extrapolated this result to 30 GHz (the lowest frequency channel of Planck) and estimate that no significant contribution to the sky temperature fluctuation is likely to come from synchrotron at degree-angular scales
△ Less
Submitted 15 March, 2001;
originally announced March 2001.
-
Analysis of CMB foregrounds using a database for Planck
Authors:
G. Giardino,
A. J. Banday,
K. Bennett,
P. Fosalba,
K. M. Gorski,
W. O'Mullane,
J. Tauber,
C. Vuerli
Abstract:
Within the scope of the Planck IDIS (Integrated Data Information System) project we have started to develop the data model for time-ordered data and full-sky maps. The data model is part of the Data Management Component (DMC), a software system designed according to a three-tier architecture which allows complete separation between data storage and processing. The DMC is already being used for s…
▽ More
Within the scope of the Planck IDIS (Integrated Data Information System) project we have started to develop the data model for time-ordered data and full-sky maps. The data model is part of the Data Management Component (DMC), a software system designed according to a three-tier architecture which allows complete separation between data storage and processing. The DMC is already being used for simulation activities and the modeling of some foreground components. We have ingested several Galactic surveys into the database and used the science data-access interface to process the data. The data structure for full-sky maps utilises the HEALPix tessellation of the sphere. We have been able to obtain consistent measures of the angular power spectrum of the Galactic radio continuum emission between 408 MHz and 2417 MHz.
△ Less
Submitted 3 November, 2000;
originally announced November 2000.
