-
Rotation and flipping invariant self-organizing maps with astronomical images: A cookbook and application to the VLA Sky Survey QuickLook images
Authors:
A. N. Vantyghem,
T. J. Galvin,
B. Sebastian,
C. P. O'Dea,
Y. A. Gordon,
M. Boyce,
L. Rudnick,
K. Polsterer,
Heinz Andernach,
M. Dionyssiou,
P. Venkataraman,
R. Norris,
S. A. Baum,
X. R. Wang,
M. Huynh
Abstract:
Modern wide field radio surveys typically detect millions of objects. Techniques based on machine learning are proving to be useful for classifying large numbers of objects. The self-organizing map (SOM) is an unsupervised machine learning algorithm that projects a many-dimensional dataset onto a two- or three-dimensional lattice of neurons. This dimensionality reduction allows the user to visuali…
▽ More
Modern wide field radio surveys typically detect millions of objects. Techniques based on machine learning are proving to be useful for classifying large numbers of objects. The self-organizing map (SOM) is an unsupervised machine learning algorithm that projects a many-dimensional dataset onto a two- or three-dimensional lattice of neurons. This dimensionality reduction allows the user to visualize common features of the data better and develop algorithms for classifying objects that are not otherwise possible with large datasets. To this aim, we use the PINK implementation of a SOM. PINK incorporates rotation and flipping invariance so that the SOM algorithm may be applied to astronomical images. In this cookbook we provide instructions for working with PINK, including preprocessing the input images, training the model, and offering lessons learned through experimentation. The problem of imbalanced classes can be improved by careful selection of the training sample and increasing the number of neurons in the SOM (chosen by the user). Because PINK is not scale-invariant, structure can be smeared in the neurons. This can also be improved by increasing the number of neurons in the SOM. We also introduce pyink, a Python package used to read and write PINK binary files, assist in common preprocessing operations, perform standard analyses, visualize the SOM and preprocessed images, and create image-based annotations using a graphical interface. A tutorial is also provided to guide the user through the entire process. We present an application of PINK to VLA Sky Survey (VLASS) images. We demonstrate that the PINK is generally able to group VLASS sources with similar morphology together. We use the results of PINK to estimate the probability that a given source in the VLASS QuickLook Catalogue is actually due to sidelobe contamination.
△ Less
Submitted 15 April, 2024;
originally announced April 2024.
-
Hydra II: Characterisation of Aegean, Caesar, ProFound, PyBDSF, and Selavy source finders
Authors:
M. M. Boyce,
A. M. Hopkins,
S. Riggi,
L. Rudnick,
M. Ramsay,
C. L. Hale,
J. Marvil,
M. Whiting,
P. Venkataraman,
C. P. O'Dea,
S. A. Baum,
Y. A. Gordon,
A. N. Vantyghem,
M. Dionyssiou,
H. Andernach,
J. D. Collier,
J. English,
B. S. Koribalski,
D. Leahy,
M. J. Michałowski,
S. Safi-Harb,
M. Vaccari,
E. Alexander,
M. Cowley,
A. D. Kapinska
, et al. (2 additional authors not shown)
Abstract:
We present a comparison between the performance of a selection of source finders using a new software tool called Hydra. The companion paper, Paper~I, introduced the Hydra tool and demonstrated its performance using simulated data. Here we apply Hydra to assess the performance of different source finders by analysing real observational data taken from the Evolutionary Map of the Universe (EMU) Pil…
▽ More
We present a comparison between the performance of a selection of source finders using a new software tool called Hydra. The companion paper, Paper~I, introduced the Hydra tool and demonstrated its performance using simulated data. Here we apply Hydra to assess the performance of different source finders by analysing real observational data taken from the Evolutionary Map of the Universe (EMU) Pilot Survey. EMU is a wide-field radio continuum survey whose primary goal is to make a deep ($20μ$Jy/beam RMS noise), intermediate angular resolution ($15^{\prime\prime}$), 1\,GHz survey of the entire sky south of $+30^{\circ}$ declination, and expecting to detect and catalogue up to 40 million sources. With the main EMU survey expected to begin in 2022 it is highly desirable to understand the performance of radio image source finder software and to identify an approach that optimises source detection capabilities. Hydra has been developed to refine this process, as well as to deliver a range of metrics and source finding data products from multiple source finders. We present the performance of the five source finders tested here in terms of their completeness and reliability statistics, their flux density and source size measurements, and an exploration of case studies to highlight finder-specific limitations.
△ Less
Submitted 27 April, 2023;
originally announced April 2023.
-
Hydra I: An extensible multi-source-finder comparison and cataloguing tool
Authors:
M. M. Boyce,
A. M. Hopkins,
S. Riggi,
L. Rudnick,
M. Ramsay,
C. L. Hale,
J. Marvil,
M. Whiting,
P. Venkataraman,
C. P. O'Dea,
S. A. Baum,
Y. A. Gordon,
A. N. Vantyghem,
M. Dionyssiou,
H. Andernach,
J. D. Collier,
J. English,
B. S. Koribalski,
D. Leahy,
M. J. Michałowski,
S. Safi-Harb,
M. Vaccari,
E. Alexander,
M. Cowley,
A. D. Kapinska
, et al. (2 additional authors not shown)
Abstract:
The latest generation of radio surveys are now producing sky survey images containing many millions of radio sources. In this context it is highly desirable to understand the performance of radio image source finder (SF) software and to identify an approach that optimises source detection capabilities. We have created Hydra to be an extensible multi-SF and cataloguing tool that can be used to comp…
▽ More
The latest generation of radio surveys are now producing sky survey images containing many millions of radio sources. In this context it is highly desirable to understand the performance of radio image source finder (SF) software and to identify an approach that optimises source detection capabilities. We have created Hydra to be an extensible multi-SF and cataloguing tool that can be used to compare and evaluate different SFs. Hydra, which currently includes the SFs Aegean, Caesar, ProFound, PyBDSF, and Selavy, provides for the addition of new SFs through containerisation and configuration files. The SF input RMS noise and island parameters are optimised to a 90\% ''percentage real detections'' threshold (calculated from the difference between detections in the real and inverted images), to enable comparison between SFs. Hydra provides completeness and reliability diagnostics through observed-deep ($\mathcal{D}$) and generated-shallow ($\mathcal{S}$) images, as well as other statistics. In addition, it has a visual inspection tool for comparing residual images through various selection filters, such as S/N bins in completeness or reliability. The tool allows the user to easily compare and evaluate different SFs in order to choose their desired SF, or a combination thereof. This paper is part one of a two part series. In this paper we introduce the Hydra software suite and validate its $\mathcal{D/S}$ metrics using simulated data. The companion paper demonstrates the utility of Hydra by comparing the performance of SFs using both simulated and real images.
△ Less
Submitted 27 April, 2023;
originally announced April 2023.
-
Asymmetric distribution of data products from WALLABY, an SKA precursor neutral hydrogen survey
Authors:
Manuel Parra-Royon,
Austin Shen,
Tristan Reynolds,
Parthasarathy Venkataraman,
María Angeles Mendoza,
Susana Sánchez-Exposito,
Julian Garrido,
Slava Kitaeff,
Lourdes Verdes-Montenegro
Abstract:
The Widefield ASKAP L-band Legacy All-sky Blind surveY (WALLABY) is a neutral hydrogen survey (HI) that is running on the Australian SKA Pathfinder (ASKAP), a precursor telescope for the Square Kilometre Array (SKA). The goal of WALLABY is to use ASKAP's powerful wide-field phased array feed technology to observe three quarters of the entire sky at the 21 cm neutral hydrogen line with an angular r…
▽ More
The Widefield ASKAP L-band Legacy All-sky Blind surveY (WALLABY) is a neutral hydrogen survey (HI) that is running on the Australian SKA Pathfinder (ASKAP), a precursor telescope for the Square Kilometre Array (SKA). The goal of WALLABY is to use ASKAP's powerful wide-field phased array feed technology to observe three quarters of the entire sky at the 21 cm neutral hydrogen line with an angular resolution of 30 arcseconds. Post-processing activities at the Australian SKA Regional Centre (AusSRC), Canadian Initiative for Radio Astronomy Data Analysis (CIRADA) and Spanish SKA Regional Centre prototype (SPSRC) will then produce publicly available advanced data products in the form of source catalogues, kinematic models and image cutouts, respectively. These advanced data products will be generated locally at each site and distributed across the network. Over the course of the full survey we expect to replicate data up to 10 MB per source detection, which could imply an ingestion of tens of GB to be consolidated in the other locations near real time. Here, we explore the use of an asymmetric database replication model and strategy, using PostgreSQL as the engine and Bucardo as the asynchronous replication service to enable robust multi-source pools operations with data products from WALLABY. This work would serve to evaluate this type of data distribution solution across globally distributed sites. Furthermore, a set of benchmarks have been developed to confirm that the deployed model is sufficient for future scalability and remote collaboration needs.
△ Less
Submitted 21 March, 2023;
originally announced March 2023.
-
WALLABY Pilot Survey: Public release of HI kinematic models for more than 100 galaxies from phase 1 of ASKAP pilot observations
Authors:
N. Deg,
K. Spekkens,
T. Westmeier,
T. N. Reynolds,
P. Venkataraman,
S. Goliath,
A. X. Shen,
R. Halloran,
A. Bosma,
B. Catinella,
W. J. G. de Blok,
H. Dénes,
E. M. Di Teodoro,
A. Elagali,
B. -Q. For,
C. Howlett,
G. I. G. Józsa,
P. Kamphuis,
D. Kleiner,
B. Koribalski,
K. Lee-Waddell,
F. Lelli,
X. Lin,
C. Murugeshan,
S. Oh
, et al. (7 additional authors not shown)
Abstract:
We present the Widefield ASKAP L-band Legacy All-sky Blind surveY (WALLABY) Pilot Phase I HI kinematic models. This first data release consists of HI observations of three fields in the direction of the Hydra and Norma clusters, and the NGC 4636 galaxy group. In this paper, we describe how we generate and publicly release flat-disk tilted-ring kinematic models for 109/592 unique HI detections in t…
▽ More
We present the Widefield ASKAP L-band Legacy All-sky Blind surveY (WALLABY) Pilot Phase I HI kinematic models. This first data release consists of HI observations of three fields in the direction of the Hydra and Norma clusters, and the NGC 4636 galaxy group. In this paper, we describe how we generate and publicly release flat-disk tilted-ring kinematic models for 109/592 unique HI detections in these fields. The modelling method adopted here - which we call the WALLABY Kinematic Analysis Proto-Pipeline (WKAPP) and for which the corresponding scripts are also publicly available - consists of combining results from the homogeneous application of the FAT and 3DBAROLO algorithms to the subset of 209 detections with sufficient resolution and S/N in order to generate optimized model parameters and uncertainties. The 109 models presented here tend to be gas rich detections resolved by at least 3-4 synthesized beams across their major axes, but there is no obvious environmental bias in the modelling. The data release described here is the first step towards the derivation of similar products for thousands of spatially-resolved WALLABY detections via a dedicated kinematic pipeline. Such a large publicly available and homogeneously analyzed dataset will be a powerful legacy product that that will enable a wide range of scientific studies.
△ Less
Submitted 14 November, 2022;
originally announced November 2022.
-
WALLABY Pilot Survey: Public release of HI data for almost 600 galaxies from phase 1 of ASKAP pilot observations
Authors:
T. Westmeier,
N. Deg,
K. Spekkens,
T. N. Reynolds,
A. X. Shen,
S. Gaudet,
S. Goliath,
M. T. Huynh,
P. Venkataraman,
X. Lin,
T. O'Beirne,
B. Catinella,
L. Cortese,
H. Dénes,
A. Elagali,
B. -Q. For,
G. I. G. Józsa,
C. Howlett,
J. M. van der Hulst,
R. J. Jurek,
P. Kamphuis,
V. A. Kilborn,
D. Kleiner,
B. S. Koribalski,
K. Lee-Waddell
, et al. (27 additional authors not shown)
Abstract:
We present WALLABY pilot data release 1, the first public release of HI pilot survey data from the Wide-field ASKAP L-band Legacy All-sky Blind Survey (WALLABY) on the Australian Square Kilometre Array Pathfinder. Phase 1 of the WALLABY pilot survey targeted three $60~{\rm deg}^2$ regions on the sky in the direction of the Hydra and Norma galaxy clusters and the NGC 4636 galaxy group, covering the…
▽ More
We present WALLABY pilot data release 1, the first public release of HI pilot survey data from the Wide-field ASKAP L-band Legacy All-sky Blind Survey (WALLABY) on the Australian Square Kilometre Array Pathfinder. Phase 1 of the WALLABY pilot survey targeted three $60~{\rm deg}^2$ regions on the sky in the direction of the Hydra and Norma galaxy clusters and the NGC 4636 galaxy group, covering the redshift range of z < 0.08. The source catalogue, images and spectra of nearly 600 extragalactic HI detections and kinematic models for 109 spatially resolved galaxies are available. As the pilot survey targeted regions containing nearby group and cluster environments, the median redshift of the sample of z ~ 0.014 is relatively low compared to the full WALLABY survey. The median galaxy HI mass is $2.3 \times 10^{9}~M_{\odot}$. The target noise level of 1.6 mJy per $30''$ beam and 18.5 kHz channel translates into a $5σ$ HI mass sensitivity for point sources of about $5.2 \times 10^{8} \, (D_{\rm L} / \mathrm{100~Mpc})^{2} \, M_{\odot}$ across 50 spectral channels (~200 km/s) and a $5σ$ HI column density sensitivity of about $8.6 \times 10^{19} \, (1 + z)^{4}~\mathrm{cm}^{-2}$ across 5 channels (~20 km/s) for emission filling the $30''$ beam. As expected for a pilot survey, several technical issues and artefacts are still affecting the data quality. Most notably, there are systematic flux errors of up to several 10% caused by uncertainties about the exact size and shape of each of the primary beams as well as the presence of sidelobes due to the finite deconvolution threshold. In addition, artefacts such as residual continuum emission and bandpass ripples have affected some of the data. The pilot survey has been highly successful in uncovering such technical problems, most of which are expected to be addressed and rectified before the start of the full WALLABY survey.
△ Less
Submitted 13 November, 2022;
originally announced November 2022.
-
Image Denoising Using Convolutional Autoencoder
Authors:
Prashanth Venkataraman
Abstract:
With the inexorable digitalisation of the modern world, every subset in the field of technology goes through major advancements constantly. One such subset is digital images which are ever so popular. Images can not always be as visually pleasing or clear as you would want them to be and are often distorted or obscured with noise. A number of techniques to enhance images have come up as the years…
▽ More
With the inexorable digitalisation of the modern world, every subset in the field of technology goes through major advancements constantly. One such subset is digital images which are ever so popular. Images can not always be as visually pleasing or clear as you would want them to be and are often distorted or obscured with noise. A number of techniques to enhance images have come up as the years passed, all with their own respective pros and cons. In this paper, we look at one such particular technique which accomplishes this task with the help of a neural network model commonly known as an autoencoder. We construct different architectures for the model and compare results in order to decide the one best suited for the task. The characteristics and working of the model are discussed briefly knowing which can help set a path for future research.
△ Less
Submitted 24 July, 2022;
originally announced July 2022.
-
Necessary and sufficient conditions for differentiability of a function of several variables
Authors:
R. P. Venkataraman
Abstract:
The necessary and sufficient conditions for differentiability of a function of several real variables stated and proved and its ramifications discussed.
The necessary and sufficient conditions for differentiability of a function of several real variables stated and proved and its ramifications discussed.
△ Less
Submitted 17 July, 2000; v1 submitted 3 July, 2000;
originally announced July 2000.
-
Is Schroedinger equation consistent with information theory?
Authors:
R. P. Venkataraman
Abstract:
It is shown that Schroedinger equation is not consistent with information theory. From the modified form of information which ensures that the most probable density function it yields tallies with a general form of continuous Riemann integrable density function that has real or imaginary zeros or singularities at end points of $[a,b] εR$, a new variational formulation for quantum mechanics is pr…
▽ More
It is shown that Schroedinger equation is not consistent with information theory. From the modified form of information which ensures that the most probable density function it yields tallies with a general form of continuous Riemann integrable density function that has real or imaginary zeros or singularities at end points of $[a,b] εR$, a new variational formulation for quantum mechanics is proposed that yields a system of Euler-Lagrange equations that are non-linear. It is proved that the solutions of this system are unique, orthonormal and complete. One dimensional harmonic oscillator has been solved.
△ Less
Submitted 3 July, 2000;
originally announced July 2000.
-
Inequivalence of ensembles in statistical mechanics
Authors:
R. P. Venkataraman
Abstract:
For studying the thermodynamic properties of systems using statistical mechanics we propose an ensemble that lies in between the familiar canonical and microcanonical ensembles. From a comparative study of these ensembles we conclude that all these ensembles may not yield the same results even in the thermodynamic limit except at high temperatures. An investigation of the coupling between system…
▽ More
For studying the thermodynamic properties of systems using statistical mechanics we propose an ensemble that lies in between the familiar canonical and microcanonical ensembles. From a comparative study of these ensembles we conclude that all these ensembles may not yield the same results even in the thermodynamic limit except at high temperatures. An investigation of the coupling between systems suggest that the state of thermodynamic equilibrium is a special case of statistical equilibrium. As a byproduct of this analysis we have obtained a general form for probability density function in an interval.
△ Less
Submitted 12 July, 2000;
originally announced July 2000.
