-
The ACCEL$^2$ project: simulating Lyman-$α$ forest in large-volume hydrodynamical simulations
Authors:
Solène Chabanier,
Corentin Ravoux,
Lucas Latrille,
Jean Sexton,
Éric Armengaud,
Julian Bautista,
Tyann Dumerchat,
Zarija Lukić
Abstract:
Cosmological information is usually extracted from the Lyman-$α$ forest correlations using only either large-scale information interpreted through linear theory or using small-scale information interpreted by means of expensive hydrodynamical simulations. A complete cosmological interpretation of the 3D correlations at all measurable scales is challenged by the need of more realistic models includ…
▽ More
Cosmological information is usually extracted from the Lyman-$α$ forest correlations using only either large-scale information interpreted through linear theory or using small-scale information interpreted by means of expensive hydrodynamical simulations. A complete cosmological interpretation of the 3D correlations at all measurable scales is challenged by the need of more realistic models including the complex growth of non-linear small scales that can only be studied within large hydrodynamical simulations. Past work were often limited by the trade off between the simulated cosmological volume and the resolution of the low-density intergalactic medium from which the Lyman-$α$ signal originates. We conduct a suite of hydrodynamical simulations of the intergalactic medium, including one of the largest Lyman-$α$ simulations ever performed in terms of volume (640 $h^{-1}\mathrm{Mpc}$), alongside simulations in smaller volumes with resolutions up to 25 $h^{-1}\mathrm{kpc}$. We compare the 3D Lyman-$α$ power spectra predicted by those simulations to different non-linear models. The inferred Lyman-$α$ bias and RSD parameters, $b_α$ and $β_α$ are in remarkable agreement with those measured in SDSS and DESI data. We find that, contrary to intuition, the convergence of large-scale modes of the 3D Lyman-$α$ power spectra, which determines $β_α$, is primarily influenced by the resolution of the simulation box through mode coupling, rather than the box size itself. Finally, we study the BAO signal encoded in the 3D Lyman-$α$ power spectra. For the first time with a hydrodynamical simulation, we clearly detect the BAO signal, however we only marginally detect its damping, associated with the non-linear growth of the structures.
△ Less
Submitted 5 July, 2024;
originally announced July 2024.
-
A High-Quality Workflow for Multi-Resolution Scientific Data Reduction and Visualization
Authors:
Daoce Wang,
Pascal Grosset,
Jesus Pulido,
Tushar M. Athawale,
Jiannan Tian,
Kai Zhao,
Zarija Lukić,
Axel Huebl,
Zhe Wang,
James Ahrens,
Dingwen Tao
Abstract:
Multi-resolution methods such as Adaptive Mesh Refinement (AMR) can enhance storage efficiency for HPC applications generating vast volumes of data. However, their applicability is limited and cannot be universally deployed across all applications. Furthermore, integrating lossy compression with multi-resolution techniques to further boost storage efficiency encounters significant barriers. To thi…
▽ More
Multi-resolution methods such as Adaptive Mesh Refinement (AMR) can enhance storage efficiency for HPC applications generating vast volumes of data. However, their applicability is limited and cannot be universally deployed across all applications. Furthermore, integrating lossy compression with multi-resolution techniques to further boost storage efficiency encounters significant barriers. To this end, we introduce an innovative workflow that facilitates high-quality multi-resolution data compression for both uniform and AMR simulations. Initially, to extend the usability of multi-resolution techniques, our workflow employs a compression-oriented Region of Interest (ROI) extraction method, transforming uniform data into a multi-resolution format. Subsequently, to bridge the gap between multi-resolution techniques and lossy compressors, we optimize three distinct compressors, ensuring their optimal performance on multi-resolution data. Lastly, we incorporate an advanced uncertainty visualization method into our workflow to understand the potential impacts of lossy compression. Experimental evaluation demonstrates that our workflow achieves significant compression quality improvements.
△ Less
Submitted 11 July, 2024; v1 submitted 5 July, 2024;
originally announced July 2024.
-
A Laplace transform-based test for the equality of positive semidefinite matrix distributions
Authors:
Žikica Lukić
Abstract:
In this paper, we present a novel test for determining equality in distribution of matrix distributions. Our approach is based on the integral squared difference of the empirical Laplace transforms with respect to the noncentral Wishart measure. We conduct an extensive power study to assess the performance of the test and determine the optimal choice of parameters. Furthermore, we demonstrate the…
▽ More
In this paper, we present a novel test for determining equality in distribution of matrix distributions. Our approach is based on the integral squared difference of the empirical Laplace transforms with respect to the noncentral Wishart measure. We conduct an extensive power study to assess the performance of the test and determine the optimal choice of parameters. Furthermore, we demonstrate the applicability of the test on financial and non-life insurance data, illustrating its effectiveness in practical scenarios.
△ Less
Submitted 15 June, 2024;
originally announced June 2024.
-
Maximum A Posteriori Ly-alpha Estimator (MAPLE): Band-power and covariance estimation of the 3D Ly-alpha forest power spectrum
Authors:
Benjamin Horowitz,
Roger de Belsunce,
Zarija Lukic
Abstract:
We present a novel maximum a posteriori estimator to jointly estimate band-powers and the covariance of the three-dimensional power spectrum (P3D) of Lyman-alpha forest flux fluctuations, called MAPLE. Our Wiener-filter based algorithm reconstructs a window-deconvolved P3D in the presence of complex survey geometries typical for Lyman-alpha surveys that are sparsely sampled transverse to and dense…
▽ More
We present a novel maximum a posteriori estimator to jointly estimate band-powers and the covariance of the three-dimensional power spectrum (P3D) of Lyman-alpha forest flux fluctuations, called MAPLE. Our Wiener-filter based algorithm reconstructs a window-deconvolved P3D in the presence of complex survey geometries typical for Lyman-alpha surveys that are sparsely sampled transverse to and densely sampled along the line-of-sight. We demonstrate our method on idealized Gaussian random fields with two selection functions: (i) a sparse sampling of 30 background sources per square degree designed to emulate the currently observing the Dark Energy Spectroscopic Instrument (DESI); (ii) a dense sampling of 900 background sources per square degree emulating the upcoming Prime Focus Spectrograph Galaxy Evolution Survey. Our proof-of-principle shows promise, especially since the algorithm can be extended to marginalize jointly over nuisance parameters and contaminants, i.e.offsets introduced by continuum fitting. Our code is implemented in JAX and is publicly available on GitHub.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Change point analysis -- the empirical Hankel transform approach
Authors:
Žikica Lukić,
Bojana Milošević
Abstract:
In this study, we introduce the first-of-its-kind class of tests for detecting change points in the distribution of a sequence of independent matrix-valued random variables. The tests are constructed using the weighted square integral difference of the empirical orthogonal Hankel transforms. The test statistics have a convenient closed-form expression, making them easy to implement in practice. We…
▽ More
In this study, we introduce the first-of-its-kind class of tests for detecting change points in the distribution of a sequence of independent matrix-valued random variables. The tests are constructed using the weighted square integral difference of the empirical orthogonal Hankel transforms. The test statistics have a convenient closed-form expression, making them easy to implement in practice. We present their limiting properties and demonstrate their quality through an extensive simulation study. We utilize these tests for change point detection in cryptocurrency markets to showcase their practical use. The detection of change points in this context can have various applications in constructing and analyzing novel trading systems.
△ Less
Submitted 31 December, 2023;
originally announced January 2024.
-
The impact of varying inhomogeneous reionization histories on metrics of Ly$α$ opacity
Authors:
Caitlin C. Doughty,
Joseph F. Hennawi,
Jose Oñorbe,
Frederick B. Davies,
Zarija Lukić
Abstract:
The epoch of hydrogen reionization is complete by $z=5$, but its progression at higher redshifts is uncertain. Measurements of Ly$α$ forest opacity show large scatter at $z<6$, suggestive of spatial fluctuations in neutral fraction ($x_\mathrm{HI}$), temperature, or ionizing background, either individually or in combination. However, these effects are degenerate, necessitating modeling these physi…
▽ More
The epoch of hydrogen reionization is complete by $z=5$, but its progression at higher redshifts is uncertain. Measurements of Ly$α$ forest opacity show large scatter at $z<6$, suggestive of spatial fluctuations in neutral fraction ($x_\mathrm{HI}$), temperature, or ionizing background, either individually or in combination. However, these effects are degenerate, necessitating modeling these physics in tandem in order to properly interpret the observations. We begin this process by developing a framework for modeling the reionization history and associated temperature fluctuations, with the intention of incorporating ionizing background fluctuations at a later time. To do this, we generate several reionization histories using semi-numerical code AMBER, selecting histories with volume-weighted neutral fractions that adhere to the observed CMB optical depth and dark pixel fractions. Implementing these histories in the \texttt{Nyx} cosmological hydrodynamics code, we examine the evolution of gas within the simulation, and the associated metrics of the Ly$α$ forest opacity. We find that the pressure smoothing scale within the IGM is strongly correlated with the adiabatic index of the temperature-density relation. We find that while models with 20,000 K photoheating at reionization are better able to reproduce the shape of the observed $z=5$ 1D flux power spectrum than those with 10,000 K, they fail to match the highest wavenumbers. The simulated autocorrelation function and optical depth distributions are systematically low and narrow, respectively, compared to the observed values, but are in better agreement when the reionization history is longer in duration, more symmetric in its distribution of reionization redshifts, or if there are remaining neutral regions at $z<6$. The systematically low variance likely requires the addition of a fluctuating UVB.
△ Less
Submitted 3 December, 2023;
originally announced December 2023.
-
Measurements of the Thermal and Ionization State of the Intergalactic Medium during the Cosmic Afternoon
Authors:
Teng Hu,
Vikram Khaire,
Joseph F. Hennawi,
Todd M. Tripp,
Jose Oñorbe,
Michael Walther,
Zarija Lukic
Abstract:
We perform the first measurement of the thermal and ionization state of the intergalactic medium (IGM) across 0.9 < z < 1.5 using 301 \lya absorption lines fitted from 12 HST STIS quasar spectra, with a total pathlength of Δz=2.1. We employ the machine-learning-based inference method that uses joint b-N distributions obtained from \lyaf decomposition. Our results show that the HI photoionization r…
▽ More
We perform the first measurement of the thermal and ionization state of the intergalactic medium (IGM) across 0.9 < z < 1.5 using 301 \lya absorption lines fitted from 12 HST STIS quasar spectra, with a total pathlength of Δz=2.1. We employ the machine-learning-based inference method that uses joint b-N distributions obtained from \lyaf decomposition. Our results show that the HI photoionization rates, Γ, are in good agreement with the recent UV background synthesis models, with \log (Γ/s^{-1})={-11.79}^{0.18}_{-0.15}, -11.98}^{0.09}_{-0.09}, and {-12.32}^{0.10}_{-0.12} at z=1.4, 1.2, and 1 respectively. We obtain the IGM temperature at the mean density, T_0, and the adiabatic index, γ, as [\log (T_0/K), γ]= [{4.13}^{+0.12}_{-0.10}, {1.34}^{+0.10}_{-0.15}], [{3.79}^{+0.11}_{-0.11}, {1.70}^{+0.09}_{-0.09}] and [{4.12}^{+0.15}_{-0.25}, {1.34}^{+0.21}_{-0.26}] at z=1.4, 1.2 and 1 respectively. Our measurements of T_0 at z=1.4 and 1.2 are consistent with the expected trend from z<3 temperature measurements as well as theoretical expectations that, in the absence of any non-standard heating, the IGM should cool down after HeII reionization. Whereas, our T_0 measurements at z=1 show unexpectedly high IGM temperature. However, because of the relatively large uncertainty in these measurements of the order of ΔT_0~5000 K, mostly emanating from the limited redshift path length of available data in these bins, we can not definitively conclude whether the IGM cools down at z<1.5. Lastly, we generate a mock dataset to test the constraining power of future measurement with larger datasets. The results demonstrate that, with redshift pathlength Δz \sim 2 for each redshift bin, three times the current dataset, we can constrain the T_0 of IGM within 1500K. Such precision would be sufficient to conclusively constrain the history of IGM thermal evolution at z < 1.5.
△ Less
Submitted 2 February, 2024; v1 submitted 29 November, 2023;
originally announced November 2023.
-
Measurement of the small-scale 3D Lyman-$α$ forest power spectrum
Authors:
Marie Lynn Abdul-Karim,
Eric Armengaud,
Guillaume Mention,
Solène Chabanier,
Corentin Ravoux,
Zarija Lukić
Abstract:
Small-scale correlations measured in the Lyman-$α$ (Ly$α$) forest encode information about the intergalactic medium and the primordial matter power spectrum. In this article, we present and implement a simple method to measure the 3-dimensional power spectrum, $P_{\rm 3D}$, of the Ly$α$ forest at wavenumbers $k$ corresponding to small, $\sim$ Mpc scales. In order to estimate $P_{\rm 3D}$ from spar…
▽ More
Small-scale correlations measured in the Lyman-$α$ (Ly$α$) forest encode information about the intergalactic medium and the primordial matter power spectrum. In this article, we present and implement a simple method to measure the 3-dimensional power spectrum, $P_{\rm 3D}$, of the Ly$α$ forest at wavenumbers $k$ corresponding to small, $\sim$ Mpc scales. In order to estimate $P_{\rm 3D}$ from sparsely and unevenly distributed data samples, we rely on averaging 1-dimensional Fourier Transforms, as previously carried out to estimate the 1-dimensional power spectrum of the Ly$α$ forest, $P_{\rm 1D}$. This methodology exhibits a very low computational cost. We confirm the validity of this approach through its application to Nyx cosmological hydrodynamical simulations. Subsequently, we apply our method to the eBOSS DR16 Ly$α$ forest sample, providing as a proof of principle, a first $P_{\rm 3D}$ measurement averaged over two redshift bins $z=2.2$ and $z=2.4$. This work highlights the potential for forthcoming $P_{\rm 3D}$ measurements, from upcoming large spectroscopic surveys, to untangle degeneracies in the cosmological interpretation of $P_{\rm 1D}$.
△ Less
Submitted 22 May, 2024; v1 submitted 13 October, 2023;
originally announced October 2023.
-
Forecasting constraints on the high-z IGM thermal state from the Lyman-$α$ forest flux auto-correlation function
Authors:
Molly Wolfson,
Joseph F. Hennawi,
Frederick B. Davies,
Zarija Lukić,
Jose Oñorbe
Abstract:
The auto-correlation function of the Lyman-$α$ (Ly$α$) forest flux from high-z quasars can statistically probe all scales of the intergalactic medium (IGM) just after the epoch of reionization. The thermal state of the IGM, which is determined by the physics of reionization, sets the amount of small-scale power seen in the \lya forest. To study the sensitivity of the auto-correlation function to t…
▽ More
The auto-correlation function of the Lyman-$α$ (Ly$α$) forest flux from high-z quasars can statistically probe all scales of the intergalactic medium (IGM) just after the epoch of reionization. The thermal state of the IGM, which is determined by the physics of reionization, sets the amount of small-scale power seen in the \lya forest. To study the sensitivity of the auto-correlation function to the thermal state of the IGM, we compute the auto-correlation function from cosmological hydrodynamical simulations with semi-numerical models of the thermal state of the IGM. We create mock data sets of 20 quasars to forecast constraints on $T_0$ and $γ$, which characterize a tight temperature-density relation in the IGM, at $5.4 \leq z \leq 6$. At $z = 5.4$ we find that an ideal data set constrains $T_0$ to 29\% and $γ$ to 9\%. In addition, we investigate four realistic reionization scenarios that combine temperature and ultra-violet background (UVB) fluctuations at $z = 5.8$. We find that, when using mock data generated from a model that includes temperature and UVB fluctuations, we can rule out a model with no temperature or UVB fluctuations at $>1σ$ level 50.5\% of the time.
△ Less
Submitted 11 September, 2023;
originally announced September 2023.
-
Impact of Self-shielding Minihalos on the Ly$α$ Forest at High Redshift
Authors:
Hyunbae Park,
Zarija Lukić,
Jean Sexton,
Marcelo Alvarez,
Paul R. Shapiro
Abstract:
Dense gas in minihalos with masses of $10^6-10^8~M_\odot$ can shield themselves from reionization for $\sim100$ Myr after being exposed to the UV background. These self-shielded systems, often unresolved in cosmological simulations, can introduce strong absorption in quasar spectra. This paper is the first systematic study on the impact of these systems on the Ly$α$ forest. We first derive the HI…
▽ More
Dense gas in minihalos with masses of $10^6-10^8~M_\odot$ can shield themselves from reionization for $\sim100$ Myr after being exposed to the UV background. These self-shielded systems, often unresolved in cosmological simulations, can introduce strong absorption in quasar spectra. This paper is the first systematic study on the impact of these systems on the Ly$α$ forest. We first derive the HI column density profile of photoevaporating minihalos by conducting 1D radiation-hydrodynamics simulations. We utilize these results to estimate the Ly$α$ opacity from minihalos in a large-scale simulation that cannot resolve self-shielding. When the ionization rate of the background radiation is $0.03\times10^{-12}~{\rm s}^{-1}$, as expected near the end of reionization at $z\sim5.5$, we find that the incidence rate of damped Ly$α$ absorbers increases by a factor of $\sim2-4$ compared to at $z=4.5$. The Ly$α$ flux is, on average, suppressed by $\sim 3\%$ of its mean due to minihalos. The absorption features enhance the 1D power spectrum up to $\sim5\%$ at $k\sim0.1~h~{\rm Mpc}^{-1}~({\rm or}~10^{-3}~{\rm km}^{-1}~{\rm s})$, which is comparable to the enhancement caused by inhomogeneous reionization. The flux is particularly suppressed in the vicinity of large halos along the line-of-sight direction at separations of up to $10~h^{-1}~{\rm Mpc}$ at $r_\perp\lesssim2~h^{-1}~{\rm Mpc}$. However, these effects become much smaller for higher ionizing rates ($\gtrsim0.3\times10^{-12}~{\rm s}^{-1}$) expected in the post-reionization Universe. Our findings highlight the need to consider minihalo absorption when interpreting the Ly$α$ forest at $z\gtrsim5.5$. Moreover, the sensitivity of these quantities to the ionizing background intensity can be exploited to constrain the intensity itself.
△ Less
Submitted 15 June, 2024; v1 submitted 8 September, 2023;
originally announced September 2023.
-
Measurements of the $z > 5$ Lyman-$α$ forest flux auto-correlation functions from the extended XQR-30 data set
Authors:
Molly Wolfson,
Joseph F. Hennawi,
Sarah E. I. Bosman,
Frederick B. Davies,
Zarija Lukić,
George D. Becker,
Huanqing Chen,
Guido Cupani,
Valentina D'Odorico,
Anna-Christina Eilers,
Martin G. Haehnelt,
Laura C. Keating,
Girish Kulkarni,
Samuel Lai,
Andrei Mesinger,
Fabian Walter,
Yongda Zhu
Abstract:
Recently, the Lyman-$α$ (Ly$α$) forest flux auto-correlation function has been shown to be sensitive to the mean free path of hydrogen-ionizing photons, $λ_{\text{mfp}}$, for simulations at $z \geq 5.4$. Measuring $λ_{\text{mfp}}$ at these redshifts will give vital information on the ending of reionization. Here we present the first observational measurements of the Ly$α$ forest flux auto-correlat…
▽ More
Recently, the Lyman-$α$ (Ly$α$) forest flux auto-correlation function has been shown to be sensitive to the mean free path of hydrogen-ionizing photons, $λ_{\text{mfp}}$, for simulations at $z \geq 5.4$. Measuring $λ_{\text{mfp}}$ at these redshifts will give vital information on the ending of reionization. Here we present the first observational measurements of the Ly$α$ forest flux auto-correlation functions in ten redshift bins from $5.1 \leq z \leq 6.0$. We use a sample of 35 quasar sightlines at $z > 5.7$ from the extended XQR-30 data set, this data has signal-to-noise ratios of $> 20$ per spectral pixel. We carefully account for systematic errors in continuum reconstruction, instrumentation, and contamination by damped Ly$α$ systems. With these measurements, we introduce software tools to generate auto-correlation function measurements from any simulation. For an initial comparison, we show our auto-correlation measurements with simulation models for recently measured $λ_{\text{mfp}}$ values and find good agreements. Further work in modeling and understanding the covariance matrices of the data is necessary to get robust measurements of $λ_{\text{mfp}}$ from this data.
△ Less
Submitted 6 September, 2023;
originally announced September 2023.
-
The Impact of the WHIM on the IGM Thermal State Determined from the Low-$z$ Lyman-$α$ Forest
Authors:
Teng Hu,
Vikram Khaire,
Joseph F. Hennawi,
Jose Onorbe,
Michael Walther,
Zarija Lukic,
Frederick Davies
Abstract:
At $z \lesssim 1$, shock heating caused by large-scale velocity flows and possibly violent feedback from galaxy formation, converts a significant fraction of the cool gas ($T\sim 10^4$ K) in the intergalactic medium (IGM) into warm-hot phase (WHIM) with $T >10^5$K, resulting in a significant deviation from the previously tight power-law IGM temperature-density relationship,…
▽ More
At $z \lesssim 1$, shock heating caused by large-scale velocity flows and possibly violent feedback from galaxy formation, converts a significant fraction of the cool gas ($T\sim 10^4$ K) in the intergalactic medium (IGM) into warm-hot phase (WHIM) with $T >10^5$K, resulting in a significant deviation from the previously tight power-law IGM temperature-density relationship, $T=T_0 (ρ/ {\barρ})^{γ-1}$. This study explores the impact of the WHIM on measurements of the low-$z$ IGM thermal state, $[T_0,γ]$, based on the $b$-$N_{H I}$ distribution of the Lyman-$α$ forest. Exploiting a machine learning-enabled simulation-based inference method trained on Nyx hydrodynamical simulations, we demonstrate that [$T_0$, $γ$] can still be reliably measured from the $b$-$N_{H I}$ distribution at $z=0.1$, notwithstanding the substantial WHIM in the IGM. To investigate the effects of different feedback, we apply this inference methodology to mock spectra derived from the IllustrisTNG and Illustris simulations at $z=0.1$. The results suggest that the underlying $[T_0,γ]$ of both simulations can be recovered with biases as low as $|Δ\log(T_0/\text{K})| \lesssim 0.05$ dex, $|Δγ| \lesssim 0.1$, smaller than the precision of a typical measurement. Given the large differences in the volume-weighted WHIM fractions between the three simulations (Illustris 38\%, IllustrisTNG 10\%, Nyx 4\%) we conclude that the $b$-$N_{H I}$ distribution is not sensitive to the WHIM under realistic conditions. Finally, we investigate the physical properties of the detectable Lyman-$α$ absorbers, and discover that although their $T$ and $Δ$ distributions remain mostly unaffected by feedback, they are correlated with the photoionization rate used in the simulation.
△ Less
Submitted 28 August, 2023;
originally announced August 2023.
-
A novel two-sample test within the space of symmetric positive definite matrix distributions and its application in finance
Authors:
Žikica Lukić,
Bojana Milošević
Abstract:
This paper introduces a novel two-sample test for a broad class of orthogonally equivalent positive definite symmetric matrix distributions. Our test is the first of its kind and we derive its asymptotic distribution. To estimate the test power, we use a warp-speed bootstrap method and consider the most common matrix distributions. We provide several real data examples, including the data for main…
▽ More
This paper introduces a novel two-sample test for a broad class of orthogonally equivalent positive definite symmetric matrix distributions. Our test is the first of its kind and we derive its asymptotic distribution. To estimate the test power, we use a warp-speed bootstrap method and consider the most common matrix distributions. We provide several real data examples, including the data for main cryptocurrencies and stock data of major US companies. The real data examples demonstrate the applicability of our test in the context closely related to algorithmic trading. The popularity of matrix distributions in many applications and the need for such a test in the literature are reconciled by our findings.
△ Less
Submitted 14 August, 2023;
originally announced August 2023.
-
Reconstructing Lyman-$α$ Fields from Low-Resolution Hydrodynamical Simulations with Deep Learning
Authors:
Cooper Jacobus,
Peter Harrington,
Zarija Lukić
Abstract:
Hydrodynamical cosmological simulations are a powerful tool for accurately predicting the properties of the intergalactic medium (IGM) and for producing mock skies that can be compared against observational data. However, the need to resolve density fluctuation in the IGM puts a stringent requirement on the resolution of such simulations which in turn limits the volumes which can be modelled, even…
▽ More
Hydrodynamical cosmological simulations are a powerful tool for accurately predicting the properties of the intergalactic medium (IGM) and for producing mock skies that can be compared against observational data. However, the need to resolve density fluctuation in the IGM puts a stringent requirement on the resolution of such simulations which in turn limits the volumes which can be modelled, even on most powerful supercomputers. In this work, we present a novel modeling method which combines physics-driven simulations with data-driven generative neural networks to produce outputs that are qualitatively and statistically close to the outputs of hydrodynamical simulations employing 8 times higher resolution. We show that the Ly-$α$ flux field, as well as the underlying hydrodynamic fields, have greatly improved statistical fidelity over a low-resolution simulation. Importantly, the design of our neural network allows for sampling multiple realizations from a given input, enabling us to quantify the model uncertainty. Using test data, we demonstrate that this model uncertainty correlates well with the true error of the Ly-$α$ flux prediction. Ultimately, our approach allows for training on small simulation volumes and applying it to much larger ones, opening the door to producing accurate Ly-$α$ mock skies in volumes of Hubble size, as will be probed with DESI and future spectroscopic sky surveys.
△ Less
Submitted 4 August, 2023;
originally announced August 2023.
-
Characterization-based approach for construction of goodness-of-fit test for Lévy distribution
Authors:
Žikica Lukić,
Bojana Milošević
Abstract:
The Lévy distribution, alongside the Normal and Cauchy distributions, is one of the only three stable distributions whose density can be obtained in a closed form. However, there are only a few specific goodness-of-fit tests for the Lévy distribution. In this paper, two novel classes of goodness-of-fit tests for the Lévy distribution are proposed. Both tests are based on V-empirical Laplace transf…
▽ More
The Lévy distribution, alongside the Normal and Cauchy distributions, is one of the only three stable distributions whose density can be obtained in a closed form. However, there are only a few specific goodness-of-fit tests for the Lévy distribution. In this paper, two novel classes of goodness-of-fit tests for the Lévy distribution are proposed. Both tests are based on V-empirical Laplace transforms. New tests are scale free under the null hypothesis, which makes them suitable for testing the composite hypothesis. The finite sample and limiting properties of test statistics are obtained. In addition, a generalization of the recent Bhati-Kattumannil goodness-of-fit test to the Lévy distribution is considered. For assessing the quality of novel and competitor tests, the local Bahadur efficiencies are computed, and a wide power study is conducted. Both criteria clearly demonstrate the quality of the new tests. The applicability of the novel tests is demonstrated with two real-data examples.
△ Less
Submitted 1 August, 2023;
originally announced August 2023.
-
AMRIC: A Novel In Situ Lossy Compression Framework for Efficient I/O in Adaptive Mesh Refinement Applications
Authors:
Daoce Wang,
Jesus Pulido,
Pascal Grosset,
Jiannan Tian,
Sian Jin,
Houjun Tang,
Jean Sexton,
Sheng Di,
Zarija Lukić,
Kai Zhao,
Bo Fang,
Franck Cappello,
James Ahrens,
Dingwen Tao
Abstract:
As supercomputers advance towards exascale capabilities, computational intensity increases significantly, and the volume of data requiring storage and transmission experiences exponential growth. Adaptive Mesh Refinement (AMR) has emerged as an effective solution to address these two challenges. Concurrently, error-bounded lossy compression is recognized as one of the most efficient approaches to…
▽ More
As supercomputers advance towards exascale capabilities, computational intensity increases significantly, and the volume of data requiring storage and transmission experiences exponential growth. Adaptive Mesh Refinement (AMR) has emerged as an effective solution to address these two challenges. Concurrently, error-bounded lossy compression is recognized as one of the most efficient approaches to tackle the latter issue. Despite their respective advantages, few attempts have been made to investigate how AMR and error-bounded lossy compression can function together. To this end, this study presents a novel in-situ lossy compression framework that employs the HDF5 filter to improve both I/O costs and boost compression quality for AMR applications. We implement our solution into the AMReX framework and evaluate on two real-world AMR applications, Nyx and WarpX, on the Summit supercomputer. Experiments with 4096 CPU cores demonstrate that AMRIC improves the compression ratio by up to 81X and the I/O performance by up to 39X over AMReX's original compression solution.
△ Less
Submitted 13 July, 2023;
originally announced July 2023.
-
SuperBench: A Super-Resolution Benchmark Dataset for Scientific Machine Learning
Authors:
Pu Ren,
N. Benjamin Erichson,
Shashank Subramanian,
Omer San,
Zarija Lukic,
Michael W. Mahoney
Abstract:
Super-Resolution (SR) techniques aim to enhance data resolution, enabling the retrieval of finer details, and improving the overall quality and fidelity of the data representation. There is growing interest in applying SR methods to complex spatiotemporal systems within the Scientific Machine Learning (SciML) community, with the hope of accelerating numerical simulations and/or improving forecasts…
▽ More
Super-Resolution (SR) techniques aim to enhance data resolution, enabling the retrieval of finer details, and improving the overall quality and fidelity of the data representation. There is growing interest in applying SR methods to complex spatiotemporal systems within the Scientific Machine Learning (SciML) community, with the hope of accelerating numerical simulations and/or improving forecasts in weather, climate, and related areas. However, the lack of standardized benchmark datasets for comparing and validating SR methods hinders progress and adoption in SciML. To address this, we introduce SuperBench, the first benchmark dataset featuring high-resolution datasets (up to $2048\times2048$ dimensions), including data from fluid flows, cosmology, and weather. Here, we focus on validating spatial SR performance from data-centric and physics-preserved perspectives, as well as assessing robustness to data degradation tasks. While deep learning-based SR methods (developed in the computer vision community) excel on certain tasks, despite relatively limited prior physics information, we identify limitations of these methods in accurately capturing intricate fine-scale features and preserving fundamental physical properties and constraints in scientific data. These shortcomings highlight the importance and subtlety of incorporating domain knowledge into ML models. We anticipate that SuperBench will significantly advance SR methods for scientific tasks.
△ Less
Submitted 24 June, 2023;
originally announced June 2023.
-
Optimal 1D Ly$α$ Forest Power Spectrum Estimation -- III. DESI early data
Authors:
Naim Göksel Karaçaylı,
Paul Martini,
Julien Guy,
Corentin Ravoux,
Marie Lynn Abdul Karim,
Eric Armengaud,
Michael Walther,
J. Aguilar,
S. Ahlen,
S. Bailey,
J. Bautista,
S. F. Beltran,
D. Brooks,
L. Cabayol-Garcia,
S. Chabanier,
E. Chaussidon,
J. Chaves-Montero,
K. Dawson,
R. de la Cruz,
A. de la Macorra,
P. Doel,
A. Font-Ribera,
J. E. Forero-Romero,
S. Gontcho A Gontcho,
A. X. Gonzalez-Morales
, et al. (37 additional authors not shown)
Abstract:
The one-dimensional power spectrum $P_{\mathrm{1D}}$ of the Ly$α$ forest provides important information about cosmological and astrophysical parameters, including constraints on warm dark matter models, the sum of the masses of the three neutrino species, and the thermal state of the intergalactic medium. We present the first measurement of $P_{\mathrm{1D}}$ with the quadratic maximum likelihood e…
▽ More
The one-dimensional power spectrum $P_{\mathrm{1D}}$ of the Ly$α$ forest provides important information about cosmological and astrophysical parameters, including constraints on warm dark matter models, the sum of the masses of the three neutrino species, and the thermal state of the intergalactic medium. We present the first measurement of $P_{\mathrm{1D}}$ with the quadratic maximum likelihood estimator (QMLE) from the Dark Energy Spectroscopic Instrument (DESI) survey early data sample. This early sample of $54~600$ quasars is already comparable in size to the largest previous studies, and we conduct a thorough investigation of numerous instrumental and analysis systematic errors to evaluate their impact on DESI data with QMLE. We demonstrate the excellent performance of the spectroscopic pipeline noise estimation and the impressive accuracy of the spectrograph resolution matrix with two-dimensional image simulations of raw DESI images that we processed with the DESI spectroscopic pipeline. We also study metal line contamination and noise calibration systematics with quasar spectra on the red side of the Ly$α$ emission line. In a companion paper, we present a similar analysis based on the Fast Fourier Transform estimate of the power spectrum. We conclude with a comparison of these two approaches and implications for the upcoming DESI Year 1 analysis.
△ Less
Submitted 12 January, 2024; v1 submitted 9 June, 2023;
originally announced June 2023.
-
The Lyman-$α$ forest catalog from the Dark Energy Spectroscopic Instrument Early Data Release
Authors:
César Ramírez-Pérez,
Ignasi Pérez-Ràfols,
Andreu Font-Ribera,
M. Abdul Karim,
E. Armengaud,
J. Bautista,
S. F. Beltran,
L. Cabayol-Garcia,
Z. Cai,
S. Chabanier,
E. Chaussidon,
J. Chaves-Montero,
A. Cuceu,
R. de la Cruz,
J. García-Bellido,
A. X. Gonzalez-Morales,
C. Gordon,
H. K. Herrera-Alcantar,
V. Iršič,
M. Ishak,
N. G. Karaçaylı,
Zarija Lukić,
C. J. Manser,
P. Montero-Camacho,
L. Napolitano
, et al. (45 additional authors not shown)
Abstract:
We present and validate the catalog of Lyman-$α$ forest fluctuations for 3D analyses using the Early Data Release (EDR) from the Dark Energy Spectroscopic Instrument (DESI) survey. We used 88,511 quasars collected from DESI Survey Validation (SV) data and the first two months of the main survey (M2). We present several improvements to the method used to extract the Lyman-$α$ absorption fluctuation…
▽ More
We present and validate the catalog of Lyman-$α$ forest fluctuations for 3D analyses using the Early Data Release (EDR) from the Dark Energy Spectroscopic Instrument (DESI) survey. We used 88,511 quasars collected from DESI Survey Validation (SV) data and the first two months of the main survey (M2). We present several improvements to the method used to extract the Lyman-$α$ absorption fluctuations performed in previous analyses from the Sloan Digital Sky Survey (SDSS). In particular, we modify the weighting scheme and show that it can improve the precision of the correlation function measurement by more than 20%. This catalog can be downloaded from https://data.desi.lbl.gov/public/edr/vac/edr/lya/fuji/v0.3 and it will be used in the near future for the first DESI measurements of the 3D correlations in the Lyman-$α$ forest.
△ Less
Submitted 25 December, 2023; v1 submitted 9 June, 2023;
originally announced June 2023.
-
The Dark Energy Spectroscopic Instrument: One-dimensional power spectrum from first Lyman-$α$ forest samples with Fast Fourier Transform
Authors:
Corentin Ravoux,
Marie Lynn Abdul Karim,
Eric Armengaud,
Michael Walther,
Naim Göksel Karaçaylı,
Paul Martini,
Julien Guy,
Jessica Nicole Aguilar,
Steven Ahlen,
Stephen Bailey,
Julian Bautista,
Sergio Felipe Beltran,
David Brooks,
Laura Cabayol-Garcia,
Solène Chabanier,
Edmond Chaussidon,
Jonás Chaves-Montero,
Kyle Dawson,
Rodrigo de la Cruz,
Axel de la Macorra,
Peter Doel,
Kevin Fanning,
Andreu Font-Ribera,
Jaime Forero-Romero,
Satya Gontcho A Gontcho
, et al. (41 additional authors not shown)
Abstract:
We present the one-dimensional Lyman-$α$ forest power spectrum measurement using the first data provided by the Dark Energy Spectroscopic Instrument (DESI). The data sample comprises $26,330$ quasar spectra, at redshift $z > 2.1$, contained in the DESI Early Data Release and the first two months of the main survey. We employ a Fast Fourier Transform (FFT) estimator and compare the resulting power…
▽ More
We present the one-dimensional Lyman-$α$ forest power spectrum measurement using the first data provided by the Dark Energy Spectroscopic Instrument (DESI). The data sample comprises $26,330$ quasar spectra, at redshift $z > 2.1$, contained in the DESI Early Data Release and the first two months of the main survey. We employ a Fast Fourier Transform (FFT) estimator and compare the resulting power spectrum to an alternative likelihood-based method in a companion paper. We investigate methodological and instrumental contaminants associated to the new DESI instrument, applying techniques similar to previous Sloan Digital Sky Survey (SDSS) measurements. We use synthetic data based on log-normal approximation to validate and correct our measurement. We compare our resulting power spectrum with previous SDSS and high-resolution measurements. With relatively small number statistics, we successfully perform the FFT measurement, which is already competitive in terms of the scale range. At the end of the DESI survey, we expect a five times larger Lyman-$α$ forest sample than SDSS, providing an unprecedented precise one-dimensional power spectrum measurement.
△ Less
Submitted 24 October, 2023; v1 submitted 9 June, 2023;
originally announced June 2023.
-
Convergence of small scale Ly$α$ structure at high-$z$ under different reionization scenarios
Authors:
Caitlin C. Doughty,
Joseph F. Hennawi,
Frederick B. Davies,
Zarija Lukić,
Jose Oñorbe
Abstract:
The Ly$α$ forest (LAF) at $z>5$ probes the thermal and reionization history of the intergalactic medium (IGM) and the nature of dark matter, but its interpretation requires comparison to cosmological hydrodynamical simulations. At high-$z$, convergence of these simulations is more exacting since transmission is dominated by underdense voids that are challenging to resolve. With evidence mounting f…
▽ More
The Ly$α$ forest (LAF) at $z>5$ probes the thermal and reionization history of the intergalactic medium (IGM) and the nature of dark matter, but its interpretation requires comparison to cosmological hydrodynamical simulations. At high-$z$, convergence of these simulations is more exacting since transmission is dominated by underdense voids that are challenging to resolve. With evidence mounting for a late end to reionization, small structures down to the sub-kpc level may survive to later times than conventionally thought due to the reduced time for pressure smoothing to impact the gas, further tightening simulation resolution requirements. We perform a suite of simulations using the Eulerian cosmological hydrodynamics code Nyx, spanning domain sizes of 1.25-10 $h^{-1}$ Mpc and 5-80 $h^{-1}$ kpc cells, and explore the interaction of these variables with the timing of reionization on the properties of the matter distribution and the simulated LAF at $z=5.5$. In observable Ly$α$ power, convergence within 10% is achieved for $k< 0.1$ s/km, but larger $k$ shows deviation of up to 20 percent. While a later reionization retains more small structure in the density field, because of the greater thermal broadening there is little difference in the convergence of LAF power between early ($z=9$) and later ($z=6$) reionizations. We conclude that at $z\sim5.5$, resolutions of 10 kpc are necessary for convergence of LAF power at $k<0.1$ s/km, while higher-$k$ modes require higher resolution, and that the timing of reionization does not significantly impact convergence given realistic photoheating.
△ Less
Submitted 25 May, 2023;
originally announced May 2023.
-
Measuring the thermal and ionization state of the low-$z$ IGM using likelihood free inference
Authors:
Teng Hu,
Vikram Khaire,
Joseph F. Hennawi,
Michael Walther,
Hector Hiss,
Justin Alsing,
Jose Oñorbe,
Zarija Lukic,
Frederick Davies
Abstract:
We present a new approach to measure the power-law temperature density relationship $T=T_0 (ρ/ \barρ)^{γ-1}$ and the UV background photoionization rate $Γ_{\rm HI}$ of the IGM based on the Voigt profile decomposition of the Ly$α$ forest into a set of discrete absorption lines with Doppler parameter $b$ and the neutral hydrogen column density $N_{\rm HI}$. Previous work demonstrated that the shape…
▽ More
We present a new approach to measure the power-law temperature density relationship $T=T_0 (ρ/ \barρ)^{γ-1}$ and the UV background photoionization rate $Γ_{\rm HI}$ of the IGM based on the Voigt profile decomposition of the Ly$α$ forest into a set of discrete absorption lines with Doppler parameter $b$ and the neutral hydrogen column density $N_{\rm HI}$. Previous work demonstrated that the shape of the $b$-$N_{\rm HI}$ distribution is sensitive to the IGM thermal parameters $T_0$ and $γ$, whereas our new inference algorithm also takes into account the normalization of the distribution, i.e. the line-density d$N$/d$z$, and we demonstrate that precise constraints can also be obtained on $Γ_{\rm HI}$. We use density-estimation likelihood-free inference (DELFI) to emulate the dependence of the $b$-$N_{\rm HI}$ distribution on IGM parameters trained on an ensemble of 624 Nyx hydrodynamical simulations at $z = 0.1$, which we combine with a Gaussian process emulator of the normalization. To demonstrate the efficacy of this approach, we generate hundreds of realizations of realistic mock HST/COS datasets, each comprising 34 quasar sightlines, and forward model the noise and resolution to match the real data. We use this large ensemble of mocks to extensively test our inference and empirically demonstrate that our posterior distributions are robust. Our analysis shows that by applying our new approach to existing Ly$α$ forest spectra at $z\simeq 0.1$, one can measure the thermal and ionization state of the IGM with very high precision ($σ_{\log T_0} \sim 0.08$ dex, $σ_γ\sim 0.06$, and $σ_{\log Γ_{\rm HI}} \sim 0.07$ dex).
△ Less
Submitted 14 July, 2022;
originally announced July 2022.
-
Modeling the Lyman-$α$ forest with Eulerian and SPH hydrodynamical methods
Authors:
Solène Chabanier,
J. D. Emberson,
Zarija Lukić,
Jesus Pulido,
Salman Habib,
Esteban Rangel,
Jean Sexton,
Nicholas Frontiere,
Michael Buehlmann
Abstract:
We compare two state-of-the-art numerical codes to study the overall accuracy in modeling the intergalactic medium and reproducing Lyman-$α$ forest observables for DESI and high-resolution data sets. The codes employ different approaches to solving both gravity and modeling the gas hydrodynamics. The first code, Nyx, solves the Poisson equation using the Particle-Mesh (PM) method and the Euler equ…
▽ More
We compare two state-of-the-art numerical codes to study the overall accuracy in modeling the intergalactic medium and reproducing Lyman-$α$ forest observables for DESI and high-resolution data sets. The codes employ different approaches to solving both gravity and modeling the gas hydrodynamics. The first code, Nyx, solves the Poisson equation using the Particle-Mesh (PM) method and the Euler equations using a finite volume method. The second code, \CRKHACC, uses a Tree-PM method to solve for gravity, and an improved Lagrangian smoothed particle hydrodynamics (SPH) technique, where fluid elements are modeled with particles, to treat the intergalactic gas. We compare the convergence behavior of the codes in flux statistics as well as the degree to which the codes agree in the converged limit. We find good agreement overall with differences being less than observational uncertainties, and a particularly notable $\lesssim$1\% agreement in the 1D flux power spectrum. This agreement was achieved by applying a tessellation methodology for reconstructing the density in \CRKHACC instead of using an SPH kernel as is standard practice. We show that use of the SPH kernel can lead to significant and unnecessary biases in flux statistics; this is especially prominent at high redshifts, $z \sim 5$, as the Lyman-$α$ forest mostly comes from lower-density regions which are intrinsically poorly sampled by SPH particles.
△ Less
Submitted 11 July, 2022;
originally announced July 2022.
-
Accelerating Parallel Write via Deeply Integrating Predictive Lossy Compression with HDF5
Authors:
Sian Jin,
Dingwen Tao,
Houjun Tang,
Sheng Di,
Suren Byna,
Zarija Lukic,
Franck Cappello
Abstract:
Lossy compression is one of the most efficient solutions to reduce storage overhead and improve I/O performance for HPC applications. However, existing parallel I/O libraries cannot fully utilize lossy compression to accelerate parallel write due to the lack of deep understanding on compression-write performance. To this end, we propose to deeply integrate predictive lossy compression with HDF5 to…
▽ More
Lossy compression is one of the most efficient solutions to reduce storage overhead and improve I/O performance for HPC applications. However, existing parallel I/O libraries cannot fully utilize lossy compression to accelerate parallel write due to the lack of deep understanding on compression-write performance. To this end, we propose to deeply integrate predictive lossy compression with HDF5 to significantly improve the parallel-write performance. Specifically, we propose analytical models to predict the time of compression and parallel write before the actual compression to enable compression-write overlapping. We also introduce an extra space in the process to handle possible data overflows resulting from prediction uncertainty in compression ratios. Moreover, we propose an optimization to reorder the compression tasks to increase the overlapping efficiency. Experiments with up to 4,096 cores from Summit show that our solution improves the write performance by up to 4.5X and 2.9X over the non-compression and lossy compression solutions, respectively, with only 1.5% storage overhead (compared to original data) on two real-world HPC applications.
△ Less
Submitted 29 June, 2022;
originally announced June 2022.
-
Snowmass2021 Cosmic Frontier White Paper: Prospects for obtaining Dark Matter Constraints with DESI
Authors:
Monica Valluri,
Solene Chabanier,
Vid Irsic,
Eric Armengaud,
Michael Walther,
Connie Rockosi,
Miguel A. Sanchez-Conde,
Leandro Beraldo e Silva,
Andrew P. Cooper,
Elise Darragh-Ford,
Kyle Dawson,
Alis J. Deason,
Simone Ferraro,
Jaime E. Forero-Romero,
Antonella Garzilli,
Ting Li,
Zarija Lukic,
Christopher J. Manser,
Nathalie Palanque-Delabrouille,
Corentin Ravoux,
Ting Tan,
Wenting Wang,
Risa Wechsler,
Andreia Carrillo,
Arjun Dey
, et al. (7 additional authors not shown)
Abstract:
Despite efforts over several decades, direct-detection experiments have not yet led to the discovery of the dark matter (DM) particle. This has led to increasing interest in alternatives to the Lambda CDM (LCDM) paradigm and alternative DM scenarios (including fuzzy DM, warm DM, self-interacting DM, etc.). In many of these scenarios, DM particles cannot be detected directly and constraints on thei…
▽ More
Despite efforts over several decades, direct-detection experiments have not yet led to the discovery of the dark matter (DM) particle. This has led to increasing interest in alternatives to the Lambda CDM (LCDM) paradigm and alternative DM scenarios (including fuzzy DM, warm DM, self-interacting DM, etc.). In many of these scenarios, DM particles cannot be detected directly and constraints on their properties can ONLY be arrived at using astrophysical observations. The Dark Energy Spectroscopic Instrument (DESI) is currently one of the most powerful instruments for wide-field surveys. The synergy of DESI with ESA's Gaia satellite and future observing facilities will yield datasets of unprecedented size and coverage that will enable constraints on DM over a wide range of physical and mass scales and across redshifts. DESI will obtain spectra of the Lyman-alpha forest out to z~5 by detecting about 1 million QSO spectra that will put constraints on clustering of the low-density intergalactic gas and DM halos at high redshift. DESI will obtain radial velocities of 10 million stars in the Milky Way (MW) and Local Group satellites enabling us to constrain their global DM distributions, as well as the DM distribution on smaller scales. The paradigm of cosmological structure formation has been extensively tested with simulations. However, the majority of simulations to date have focused on collisionless CDM. Simulations with alternatives to CDM have recently been gaining ground but are still in their infancy. While there are numerous publicly available large-box and zoom-in simulations in the LCDM framework, there are no comparable publicly available WDM, SIDM, FDM simulations. DOE support for a public simulation suite will enable a more cohesive community effort to compare observations from DESI (and other surveys) with numerical predictions and will greatly impact DM science.
△ Less
Submitted 1 July, 2022; v1 submitted 14 March, 2022;
originally announced March 2022.
-
Snowmass2021 Cosmic Frontier White Paper: Dark Matter Physics from Halo Measurements
Authors:
Keith Bechtol,
Simon Birrer,
Francis-Yan Cyr-Racine,
Katelin Schutz,
Susmita Adhikari,
Mustafa Amin,
Arka Banerjee,
Simeon Bird,
Nikita Blinov,
Kimberly K. Boddy,
Celine Boehm,
Kevin Bundy,
Malte Buschmann,
Sukanya Chakrabarti,
David Curtin,
Liang Dai,
Alex Drlica-Wagner,
Cora Dvorkin,
Adrienne L. Erickcek,
Daniel Gilman,
Saniya Heeba,
Stacy Kim,
Vid Iršič,
Alexie Leauthaud,
Mark Lovell
, et al. (19 additional authors not shown)
Abstract:
The non-linear process of cosmic structure formation produces gravitationally bound overdensities of dark matter known as halos. The abundances, density profiles, ellipticities, and spins of these halos can be tied to the underlying fundamental particle physics that governs dark matter at microscopic scales. Thus, macroscopic measurements of dark matter halos offer a unique opportunity to determin…
▽ More
The non-linear process of cosmic structure formation produces gravitationally bound overdensities of dark matter known as halos. The abundances, density profiles, ellipticities, and spins of these halos can be tied to the underlying fundamental particle physics that governs dark matter at microscopic scales. Thus, macroscopic measurements of dark matter halos offer a unique opportunity to determine the underlying properties of dark matter across the vast landscape of dark matter theories. This white paper summarizes the ongoing rapid development of theoretical and experimental methods, as well as new opportunities, to use dark matter halo measurements as a pillar of dark matter physics.
△ Less
Submitted 24 April, 2023; v1 submitted 14 March, 2022;
originally announced March 2022.
-
Snowmass2021 Computational Frontier White Paper: Cosmological Simulations and Modeling
Authors:
Marcelo A. Alvarez,
Arka Banerjee,
Simon Birrer,
Salman Habib,
Katrin Heitmann,
Zarija Lukić,
Julian B. Muñoz,
Yuuki Omori,
Hyunbae Park,
Annika H. G. Peter,
Jean Sexton,
Yi-Ming Zhong
Abstract:
Powerful new observational facilities will come online over the next decade, enabling a number of discovery opportunities in the "Cosmic Frontier", which targets understanding of the physics of the early universe, dark matter and dark energy, and cosmological probes of fundamental physics, such as neutrino masses and modifications of Einstein gravity. Synergies between different experiments will b…
▽ More
Powerful new observational facilities will come online over the next decade, enabling a number of discovery opportunities in the "Cosmic Frontier", which targets understanding of the physics of the early universe, dark matter and dark energy, and cosmological probes of fundamental physics, such as neutrino masses and modifications of Einstein gravity. Synergies between different experiments will be leveraged to present new classes of cosmic probes as well as to minimize systematic biases present in individual surveys. Success of this observational program requires actively pairing it with a well-matched state-of-the-art simulation and modeling effort. Next-generation cosmological modeling will increasingly focus on physically rich simulations able to model outputs of sky surveys spanning multiple wavebands. These simulations will have unprecedented resolution, volume coverage, and must deliver guaranteed high-fidelity results for individual surveys as well as for the cross-correlations across different surveys. The needed advances are as follows: (1) Development of scientifically rich and broadly-scoped simulations, which capture the relevant physics and correlations between probes (2) Accurate translation of simulation results into realistic image or spectral data to be directly compared with observations (3) Improved emulators and/or data-driven methods serving as surrogates for expensive simulations, constructed from a finite set of full-physics simulations (4) Detailed and transparent verification and validation programs for both simulations and analysis tools. (Abridged)
△ Less
Submitted 15 March, 2022; v1 submitted 14 March, 2022;
originally announced March 2022.
-
Self-supervised similarity search for large scientific datasets
Authors:
George Stein,
Peter Harrington,
Jacqueline Blaum,
Tomislav Medan,
Zarija Lukic
Abstract:
We present the use of self-supervised learning to explore and exploit large unlabeled datasets. Focusing on 42 million galaxy images from the latest data release of the Dark Energy Spectroscopic Instrument (DESI) Legacy Imaging Surveys, we first train a self-supervised model to distill low-dimensional representations that are robust to symmetries, uncertainties, and noise in each image. We then us…
▽ More
We present the use of self-supervised learning to explore and exploit large unlabeled datasets. Focusing on 42 million galaxy images from the latest data release of the Dark Energy Spectroscopic Instrument (DESI) Legacy Imaging Surveys, we first train a self-supervised model to distill low-dimensional representations that are robust to symmetries, uncertainties, and noise in each image. We then use the representations to construct and publicly release an interactive semantic similarity search tool. We demonstrate how our tool can be used to rapidly discover rare objects given only a single example, increase the speed of crowd-sourcing campaigns, and construct and improve training sets for supervised applications. While we focus on images from sky surveys, the technique is straightforward to apply to any scientific dataset of any dimensionality. The similarity search web app can be found at https://github.com/georgestein/galaxy_search
△ Less
Submitted 30 November, 2021; v1 submitted 25 October, 2021;
originally announced October 2021.
-
Improving IGM temperature constraints using wavelet analysis on high-redshift quasars
Authors:
Molly Wolfson,
Joseph F. Hennawi,
Frederick B. Davies,
Jose Oñorbe,
Hector Hiss,
Zarija Lukić
Abstract:
The thermal state of the intergalactic medium (IGM) contains vital information about the epoch of reionization, one of the most transformative yet poorly understood periods in the young universe. This thermal state is encoded in the small-scale structure of Lyman-$α$ (Ly$α$) absorption in quasar spectra. The 1D flux power spectrum measures the average small-scale structure along quasar sightlines.…
▽ More
The thermal state of the intergalactic medium (IGM) contains vital information about the epoch of reionization, one of the most transformative yet poorly understood periods in the young universe. This thermal state is encoded in the small-scale structure of Lyman-$α$ (Ly$α$) absorption in quasar spectra. The 1D flux power spectrum measures the average small-scale structure along quasar sightlines. At high redshifts, where the opacity is large, averaging mixes high signal-to-noise ratio transmission spikes with noisy absorption troughs. Wavelet amplitudes are an alternate statistic that maintains spatial information while quantifying fluctuations at the same spatial frequencies as the power spectrum, giving them the potential to more sensitively measure the small-scale structure. Previous Ly$α$ forest studies using wavelet amplitude probability density functions (PDFs) used limited spatial frequencies and neglected strong correlations between PDF bins and across wavelets scales, resulting in sub-optimal and unreliable parameter inference. Here we present a novel method for performing statistical inference using wavelet amplitude PDFs that spans the full range of spatial frequencies probed by the power spectrum and that fully accounts for these correlations. We applied this procedure to realistic mock data drawn from a simple thermal model parameterized by the temperature at mean density, $T_0$, and find that wavelets deliver 1$σ$ constraints on $T_0$ that are on average 7% more sensitive at $z=5$ (12% at $z=6$) than those from the power spectrum. We consider the possibility of combing wavelet PDFs with the power, but find that this does not lead to improved sensitivity.
△ Less
Submitted 6 October, 2021;
originally announced October 2021.
-
Mining for Strong Gravitational Lenses with Self-supervised Learning
Authors:
George Stein,
Jacqueline Blaum,
Peter Harrington,
Tomislav Medan,
Zarija Lukic
Abstract:
We employ self-supervised representation learning to distill information from 76 million galaxy images from the Dark Energy Spectroscopic Instrument Legacy Imaging Surveys' Data Release 9. Targeting the identification of new strong gravitational lens candidates, we first create a rapid similarity search tool to discover new strong lenses given only a single labelled example. We then show how train…
▽ More
We employ self-supervised representation learning to distill information from 76 million galaxy images from the Dark Energy Spectroscopic Instrument Legacy Imaging Surveys' Data Release 9. Targeting the identification of new strong gravitational lens candidates, we first create a rapid similarity search tool to discover new strong lenses given only a single labelled example. We then show how training a simple linear classifier on the self-supervised representations, requiring only a few minutes on a CPU, can automatically classify strong lenses with great efficiency. We present 1192 new strong lens candidates that we identified through a brief visual identification campaign, and release an interactive web-based similarity search tool and the top network predictions to facilitate crowd-sourcing rapid discovery of additional strong gravitational lenses and other rare objects: https://github.com/georgestein/ssl-legacysurvey.
△ Less
Submitted 21 June, 2022; v1 submitted 30 September, 2021;
originally announced October 2021.
-
HyPhy: Deep Generative Conditional Posterior Mapping of Hydrodynamical Physics
Authors:
Benjamin Horowitz,
Max Dornfest,
Zarija Lukić,
Peter Harrington
Abstract:
Generating large volume hydrodynamical simulations for cosmological observables is a computationally demanding task necessary for next generation observations. In this work, we construct a novel fully convolutional variational auto-encoder (VAE) to synthesize hydrodynamic fields conditioned on dark matter fields from N-body simulations. After training the model on a single hydrodynamical simulatio…
▽ More
Generating large volume hydrodynamical simulations for cosmological observables is a computationally demanding task necessary for next generation observations. In this work, we construct a novel fully convolutional variational auto-encoder (VAE) to synthesize hydrodynamic fields conditioned on dark matter fields from N-body simulations. After training the model on a single hydrodynamical simulation, we are able to probabilistically map new dark matter only simulations to corresponding full hydrodynamical outputs. By sampling over the latent space of our VAE, we can generate posterior samples and study the variance of the mapping. We find that our reconstructed field provides an accurate representation of the target hydrodynamical fields as well as a reasonable variance estimates. This approach has promise for the rapid generation of mocks as well as for implementation in a full Bayesian inverse model of observed data.
△ Less
Submitted 23 June, 2021;
originally announced June 2021.
-
Fast, high-fidelity Lyman $α$ forests with convolutional neural networks
Authors:
Peter Harrington,
Mustafa Mustafa,
Max Dornfest,
Benjamin Horowitz,
Zarija Lukić
Abstract:
Full-physics cosmological simulations are powerful tools for studying the formation and evolution of structure in the universe but require extreme computational resources. Here, we train a convolutional neural network to use a cheaper N-body-only simulation to reconstruct the baryon hydrodynamic variables (density, temperature, and velocity) on scales relevant to the Lyman-$α$ (Ly$α$) forest, usin…
▽ More
Full-physics cosmological simulations are powerful tools for studying the formation and evolution of structure in the universe but require extreme computational resources. Here, we train a convolutional neural network to use a cheaper N-body-only simulation to reconstruct the baryon hydrodynamic variables (density, temperature, and velocity) on scales relevant to the Lyman-$α$ (Ly$α$) forest, using data from Nyx simulations. We show that our method enables rapid estimation of these fields at a resolution of $\sim$20kpc, and captures the statistics of the Ly$α$ forest with much greater accuracy than existing approximations. Because our model is fully-convolutional, we can train on smaller simulation boxes and deploy on much larger ones, enabling substantial computational savings. Furthermore, as our method produces an approximation for the hydrodynamic fields instead of Ly$α$ flux directly, it is not limited to a particular choice of ionizing background or mean transmitted flux.
△ Less
Submitted 23 June, 2021;
originally announced June 2021.
-
Estimating Galactic Distances From Images Using Self-supervised Representation Learning
Authors:
Md Abul Hayat,
Peter Harrington,
George Stein,
Zarija Lukić,
Mustafa Mustafa
Abstract:
We use a contrastive self-supervised learning framework to estimate distances to galaxies from their photometric images. We incorporate data augmentations from computer vision as well as an application-specific augmentation accounting for galactic dust. We find that the resulting visual representations of galaxy images are semantically useful and allow for fast similarity searches, and can be succ…
▽ More
We use a contrastive self-supervised learning framework to estimate distances to galaxies from their photometric images. We incorporate data augmentations from computer vision as well as an application-specific augmentation accounting for galactic dust. We find that the resulting visual representations of galaxy images are semantically useful and allow for fast similarity searches, and can be successfully fine-tuned for the task of redshift estimation. We show that (1) pretraining on a large corpus of unlabeled data followed by fine-tuning on some labels can attain the accuracy of a fully-supervised model which requires 2-4x more labeled data, and (2) that by fine-tuning our self-supervised representations using all available data labels in the Main Galaxy Sample of the Sloan Digital Sky Survey (SDSS), we outperform the state-of-the-art supervised learning method.
△ Less
Submitted 11 January, 2021;
originally announced January 2021.
-
Self-Supervised Representation Learning for Astronomical Images
Authors:
Md Abul Hayat,
George Stein,
Peter Harrington,
Zarija Lukić,
Mustafa Mustafa
Abstract:
Sky surveys are the largest data generators in astronomy, making automated tools for extracting meaningful scientific information an absolute necessity. We show that, without the need for labels, self-supervised learning recovers representations of sky survey images that are semantically useful for a variety of scientific tasks. These representations can be directly used as features, or fine-tuned…
▽ More
Sky surveys are the largest data generators in astronomy, making automated tools for extracting meaningful scientific information an absolute necessity. We show that, without the need for labels, self-supervised learning recovers representations of sky survey images that are semantically useful for a variety of scientific tasks. These representations can be directly used as features, or fine-tuned, to outperform supervised methods trained only on labeled data. We apply a contrastive learning framework on multi-band galaxy photometry from the Sloan Digital Sky Survey (SDSS) to learn image representations. We then use them for galaxy morphology classification, and fine-tune them for photometric redshift estimation, using labels from the Galaxy Zoo 2 dataset and SDSS spectroscopy. In both downstream tasks, using the same learned representations, we outperform the supervised state-of-the-art results, and we show that our approach can achieve the accuracy of supervised models while using 2-4 times fewer labels for training.
△ Less
Submitted 8 April, 2021; v1 submitted 23 December, 2020;
originally announced December 2020.
-
Simulating intergalactic gas for DESI-like small scale Lymanα forest observations
Authors:
Michael Walther,
Eric Armengaud,
Corentin Ravoux,
Nathalie Palanque-Delabrouille,
Christophe Yèche,
Zarija Lukić
Abstract:
Measurements of the Ly$α$ forest based on large numbers of quasar spectra from sky surveys such as SDSS/eBOSS accurately probe the distribution of matter on small scales and thus provide important constraints on several ingredients of the cosmological model. A main summary statistic derived from those measurements is the one-dimensional power spectrum, P1D, of the Ly$α$ absorption. However, model…
▽ More
Measurements of the Ly$α$ forest based on large numbers of quasar spectra from sky surveys such as SDSS/eBOSS accurately probe the distribution of matter on small scales and thus provide important constraints on several ingredients of the cosmological model. A main summary statistic derived from those measurements is the one-dimensional power spectrum, P1D, of the Ly$α$ absorption. However, model predictions for P1D rely on expensive hydrodynamical simulations of the intergalactic medium, which was the limiting factor in previous analyses. Datasets from upcoming surveys such as DESI will push observational accuracy near the 1%-level and probe even smaller scales. This observational push mandate seven more accurate simulations as well as more careful exploration of parameter space. In this work we evaluate the robustness and accuracy of simulations and the statistical framework used to constrain cosmological parameters. We present a comparison between the grid-based simulation code Nyx and SPH-based code Gadget in the context ofP1D. In addition, we perform resolution and box-size convergence tests using Nyx code. We use a Gaussian process emulation scheme to reduce the number of simulations required for exploration of parameter space without sacrificing the model accuracy. We demonstrate the ability to produce unbiased parameter constraints in an end-to-end inference test using mock eBOSS- and DESI-like data, and we advocate for the usage of adaptive sampling schemes as opposed to using a fixed Latin hypercube design.
△ Less
Submitted 22 March, 2021; v1 submitted 7 December, 2020;
originally announced December 2020.
-
Report from the Tri-Agency Cosmological Simulation Task Force
Authors:
Nick Battaglia,
Andrew Benson,
Tim Eifler,
Andrew Hearin,
Katrin Heitmann,
Shirley Ho,
Alina Kiessling,
Zarija Lukic,
Michael Schneider,
Elena Sellentin,
Joachim Stadel
Abstract:
The Tri-Agency Cosmological Simulations (TACS) Task Force was formed when Program Managers from the Department of Energy (DOE), the National Aeronautics and Space Administration (NASA), and the National Science Foundation (NSF) expressed an interest in receiving input into the cosmological simulations landscape related to the upcoming DOE/NSF Vera Rubin Observatory (Rubin), NASA/ESA's Euclid, and…
▽ More
The Tri-Agency Cosmological Simulations (TACS) Task Force was formed when Program Managers from the Department of Energy (DOE), the National Aeronautics and Space Administration (NASA), and the National Science Foundation (NSF) expressed an interest in receiving input into the cosmological simulations landscape related to the upcoming DOE/NSF Vera Rubin Observatory (Rubin), NASA/ESA's Euclid, and NASA's Wide Field Infrared Survey Telescope (WFIRST). The Co-Chairs of TACS, Katrin Heitmann and Alina Kiessling, invited community scientists from the USA and Europe who are each subject matter experts and are also members of one or more of the surveys to contribute. The following report represents the input from TACS that was delivered to the Agencies in December 2018.
△ Less
Submitted 14 May, 2020;
originally announced May 2020.
-
Cosmic Inference: Constraining Parameters With Observations and Highly Limited Number of Simulations
Authors:
Timur Takhtaganov,
Zarija Lukic,
Juliane Mueller,
Dmitriy Morozov
Abstract:
Cosmological probes pose an inverse problem where the measurement result is obtained through observations, and the objective is to infer values of model parameters which characterize the underlying physical system -- our Universe. Modern cosmological probes increasingly rely on measurements of the small-scale structure, and the only way to accurately model physical behavior on those scales, roughl…
▽ More
Cosmological probes pose an inverse problem where the measurement result is obtained through observations, and the objective is to infer values of model parameters which characterize the underlying physical system -- our Universe. Modern cosmological probes increasingly rely on measurements of the small-scale structure, and the only way to accurately model physical behavior on those scales, roughly 65 Mpc/h or smaller, is via expensive numerical simulations. In this paper, we provide a detailed description of a novel statistical framework for obtaining accurate parameter constraints by combining observations with a very limited number of cosmological simulations. The proposed framework utilizes multi-output Gaussian process emulators that are adaptively constructed using Bayesian optimization methods. We compare several approaches for constructing multi-output emulators that enable us to take possible inter-output correlations into account while maintaining the efficiency needed for inference. Using Lyman alpha forest flux power spectrum, we demonstrate that our adaptive approach requires considerably fewer --- by a factor of a few in Lyman alpha P(k) case considered here --- simulations compared to the emulation based on Latin hypercube sampling, and that the method is more robust in reconstructing parameters and their Bayesian credible intervals.
△ Less
Submitted 17 May, 2019;
originally announced May 2019.
-
Inflation and Dark Energy from spectroscopy at $z > 2$
Authors:
Simone Ferraro,
Michael J. Wilson,
Muntazir Abidi,
David Alonso,
Behzad Ansarinejad,
Robert Armstrong,
Jacobo Asorey,
Arturo Avelino,
Carlo Baccigalupi,
Kevin Bandura,
Nicholas Battaglia,
Chetan Bavdhankar,
José Luis Bernal,
Florian Beutler,
Matteo Biagetti,
Guillermo A. Blanc,
Jonathan Blazek,
Adam S. Bolton,
Julian Borrill,
Brenda Frye,
Elizabeth Buckley-Geer,
Philip Bull,
Cliff Burgess,
Christian T. Byrnes,
Zheng Cai
, et al. (118 additional authors not shown)
Abstract:
The expansion of the Universe is understood to have accelerated during two epochs: in its very first moments during a period of Inflation and much more recently, at $z < 1$, when Dark Energy is hypothesized to drive cosmic acceleration. The undiscovered mechanisms behind these two epochs represent some of the most important open problems in fundamental physics. The large cosmological volume at…
▽ More
The expansion of the Universe is understood to have accelerated during two epochs: in its very first moments during a period of Inflation and much more recently, at $z < 1$, when Dark Energy is hypothesized to drive cosmic acceleration. The undiscovered mechanisms behind these two epochs represent some of the most important open problems in fundamental physics. The large cosmological volume at $2 < z < 5$, together with the ability to efficiently target high-$z$ galaxies with known techniques, enables large gains in the study of Inflation and Dark Energy. A future spectroscopic survey can test the Gaussianity of the initial conditions up to a factor of ~50 better than our current bounds, crossing the crucial theoretical threshold of $σ(f_{NL}^{\rm local})$ of order unity that separates single field and multi-field models. Simultaneously, it can measure the fraction of Dark Energy at the percent level up to $z = 5$, thus serving as an unprecedented test of the standard model and opening up a tremendous discovery space.
△ Less
Submitted 21 March, 2019;
originally announced March 2019.
-
Inhomogeneous Reionization Models in Cosmological Hydrodynamical Simulations
Authors:
Jose Oñorbe,
F. B. Davies,
Z. Lukić,
J. F. Hennawi,
D. Sorini
Abstract:
In this work we present a new hybrid method to simulate the thermal effects of the reionization in cosmological hydrodynamical simulations. The method improves upon the standard approach used in simulations of the intergalactic medium (IGM) and galaxy formation without a significant increase of the computational cost allowing for efficient exploration of the parameter space. The method uses a smal…
▽ More
In this work we present a new hybrid method to simulate the thermal effects of the reionization in cosmological hydrodynamical simulations. The method improves upon the standard approach used in simulations of the intergalactic medium (IGM) and galaxy formation without a significant increase of the computational cost allowing for efficient exploration of the parameter space. The method uses a small set of phenomenological input parameters and combines a semi-numerical reionization model to solve for the topology of reionization and an approximate model of how reionization heats the IGM, with the massively parallel \texttt{Nyx} hydrodynamics code, specifically designed to solve for the structure of diffuse IGM gas. We have produced several large-scale high resolution cosmological hydrodynamical simulations ($2048^3$, $L_{\rm box} = 40$ Mpc/h) with different instantaneous and inhomogeneous HI reionization models that use this new methodology. We study the IGM thermal properties of these models and find that large scale temperature fluctuations extend well beyond the end of reionization. Analyzing the 1D flux power spectrum of these models, we find up to $\sim 50\%$ differences in the large scale properties (low modes, $k\lesssim0.01$ s/km) of the post-reionization power spectrum due to the thermal fluctuations. We show that these differences could allow one to distinguish between different reionization scenarios already with existing Ly$α$ forest measurements. Finally, we explore the differences in the small-scale cutoff of the power spectrum and we find that, for the same heat input, models show very good agreement provided that the reionization redshift of the instantaneous reionization model happens at the midpoint of the inhomogeneous model.
△ Less
Submitted 7 June, 2019; v1 submitted 27 October, 2018;
originally announced October 2018.
-
Mapping quasar light echoes in 3D with Lyα forest tomography
Authors:
Tobias M. Schmidt,
Joseph F. Hennawi,
Khee-Gan Lee,
Zarija Lukic,
Jose Onorbe,
Martin White
Abstract:
The intense radiation emitted by luminous quasars dramatically alters the ionization state of their surrounding IGM. This so-called proximity effect extends out to tens of Mpc, and manifests as large coherent regions of enhanced Lyman-$α$ (Ly$α$) forest transmission in absorption spectra of background sightlines. Here we present a novel method based on Ly$α$ forest tomography, which is capable of…
▽ More
The intense radiation emitted by luminous quasars dramatically alters the ionization state of their surrounding IGM. This so-called proximity effect extends out to tens of Mpc, and manifests as large coherent regions of enhanced Lyman-$α$ (Ly$α$) forest transmission in absorption spectra of background sightlines. Here we present a novel method based on Ly$α$ forest tomography, which is capable of mapping these quasar `light echoes' in three dimensions. Using a dense grid (10-100) of faint ($m_r\approx24.7\,\mathrm{mag}$) background galaxies as absorption probes, one can measure the ionization state of the IGM in the vicinity of a foreground quasar, yielding detailed information about the quasar's radiative history and emission geometry. An end-to-end analysis - combining cosmological hydrodynamical simulations post-processed with a quasar emission model, realistic estimates of galaxy number densities, and instrument + telescope throughput - is conducted to explore the feasibility of detecting quasar light echoes. We present a new fully Bayesian statistical method that allows one to reconstruct quasar light echoes from thousands of individual low S/N transmission measurements. Armed with this machinery, we undertake an exhaustive parameter study and show that light echoes can be convincingly detected for luminous ($M_{1450} < -27.5\,\mathrm{mag}$ corresponding to $m_{1450} < 18.4\,\mathrm{mag}$ at $z\simeq 3.6$) quasars at redshifts $3<z_\mathrm{QSO}<5$, and that a relative precision better than $20\,\%$ on the quasar age can be achieved for individual objects, for the expected range of ages between 1 Myr and 100 Myr. The observational requirements are relatively modest - moderate resolution ($R\gtrsim750$) multi object spectroscopy at low $\rm{}S/N > 5$ is sufficient, requiring three hour integrations using existing instruments on 8m class telescopes.
△ Less
Submitted 11 October, 2018;
originally announced October 2018.
-
The Power Spectrum of the Lyman-$α$ Forest at z < 0.5
Authors:
Vikram Khaire,
Michael Walther,
Joseph F. Hennawi,
Jose Oñorbe,
Zarija Lukić,
J. Xavier Prochaska,
Todd M. Tripp,
Joseph N. Burchett,
Christian Rodriguez
Abstract:
We present new measurements of the flux power-spectrum P(k) of the $z<0.5$ HI Lyman-$α$ forest spanning scales k ~ 0.001-0.1 s/km. These results were derived from 65 far ultraviolet quasar spectra (resolution R~18000) observed with the Cosmic Origin Spectrograph (COS) on board the Hubble Space Telescope. The analysis required careful masking of all contaminating, coincident absorption from HI and…
▽ More
We present new measurements of the flux power-spectrum P(k) of the $z<0.5$ HI Lyman-$α$ forest spanning scales k ~ 0.001-0.1 s/km. These results were derived from 65 far ultraviolet quasar spectra (resolution R~18000) observed with the Cosmic Origin Spectrograph (COS) on board the Hubble Space Telescope. The analysis required careful masking of all contaminating, coincident absorption from HI and metal-line transitions of the Galactic interstellar medium and intervening absorbers as well as proper treatment of the complex COS line-spread function. From the P(k) measurements, we estimate the HI photoionization rate ($Γ_{\rm HI}$) in the z<0.5 intergalactic medium. Our results confirm most of the previous $Γ_{\rm HI}$ estimates. We conclude that previous concerns of a photon underproduction crisis are now resolved by demonstrating that the measured $Γ_{\rm HI}$ can be accounted for by ultraviolet emission from quasars alone. In a companion paper, we will present constraints on the thermal state of the $z<0.5$ intergalactic medium from the P(k) measurements presented here.
△ Less
Submitted 9 April, 2019; v1 submitted 16 August, 2018;
originally announced August 2018.
-
New Constraints on IGM Thermal Evolution from the Lyα Forest Power Spectrum
Authors:
Michael Walther,
Jose Oñorbe,
Joseph F. Hennawi,
Zarija Lukić
Abstract:
We determine the thermal evolution of the intergalactic medium (IGM) over $3\, \mathrm{Gyr}$ of cosmic time $1.8<z<5.4$ by comparing measurements of the Lyα forest power spectrum to a suite of $\sim70$ hydrodynamical simulations. We conduct Bayesian inference of IGM thermal parameters using an end-to-end forward modeling framework whereby mock spectra generated from our simulation grid are used to…
▽ More
We determine the thermal evolution of the intergalactic medium (IGM) over $3\, \mathrm{Gyr}$ of cosmic time $1.8<z<5.4$ by comparing measurements of the Lyα forest power spectrum to a suite of $\sim70$ hydrodynamical simulations. We conduct Bayesian inference of IGM thermal parameters using an end-to-end forward modeling framework whereby mock spectra generated from our simulation grid are used to build a custom emulator which interpolates the power spectrum between thermal grid points. The temperature at mean density $T_0$ rises steadily from $T_0\sim 6000\, \mathrm{K}$ at $z=5.4$, peaks at $14000\, \mathrm{K}$ for $z\sim 3.4$, and decreases at lower redshift reaching $T_0\sim 7000\, \mathrm{K}$ by $z\sim1.8$. This evolution provides conclusive evidence for photoionization heating resulting from the reionization of He II, as well as the subsequent cooling of the IGM due to the expansion of the Universe after all reionization events are complete. Our results are broadly consistent with previous measurements of thermal evolution based on a variety of approaches, but the sensitivity of the power spectrum, the combination of high precision BOSS measurements of large-scale modes ($k\lesssim 0.02\, \mathrm{s/km}$) with our recent determination of the small-scale power, our large grid of models, and our careful statistical analysis allow us to break the well known degeneracy between the temperature at mean density $T_0$ and the slope of the temperature density relation $γ$ that has plagued previous analyses. At the highest redshifts $z\geq5$ we infer lower temperatures than expected from the standard picture of IGM thermal evolution leaving little room for additional smoothing of the Lyα forest by free streaming of warm dark matter.
△ Less
Submitted 20 December, 2018; v1 submitted 13 August, 2018;
originally announced August 2018.
-
Quantitative Constraints on the Reionization History from the IGM Damping Wing Signature in Two Quasars at z > 7
Authors:
Frederick B. Davies,
Joseph F. Hennawi,
Eduardo Bañados,
Zarija Lukić,
Roberto Decarli,
Xiaohui Fan,
Emanuele P. Farina,
Chiara Mazzucchelli,
Hans-Walter Rix,
Bram P. Venemans,
Fabian Walter,
Feige Wang,
Jinyi Yang
Abstract:
During reionization, neutral hydrogen in the intergalactic medium (IGM) imprints a damping wing absorption feature on the spectrum of high-redshift quasars. A detection of this signature provides compelling evidence for a significantly neutral Universe, and enables measurements of the hydrogen neutral fraction $x_{\rm HI}(z)$ at that epoch. Obtaining reliable quantitative constraints from this tec…
▽ More
During reionization, neutral hydrogen in the intergalactic medium (IGM) imprints a damping wing absorption feature on the spectrum of high-redshift quasars. A detection of this signature provides compelling evidence for a significantly neutral Universe, and enables measurements of the hydrogen neutral fraction $x_{\rm HI}(z)$ at that epoch. Obtaining reliable quantitative constraints from this technique, however, is challenging due to stochasticity induced by the patchy inside-out topology of reionization, degeneracies with quasar lifetime, and the unknown unabsorbed quasar spectrum close to rest-frame Ly$α$. We combine a large-volume semi-numerical simulation of reionization topology with 1D radiative transfer through high-resolution hydrodynamical simulations of the high-redshift Universe to construct models of quasar transmission spectra during reionization. Our state-of-the-art approach captures the distribution of damping wing strengths in biased quasar halos that should have reionized earlier, as well as the erosion of neutral gas in the quasar environment caused by its own ionizing radiation. Combining this detailed model with our new technique for predicting the quasar continuum and its associated uncertainty, we introduce a Bayesian statistical method to jointly constrain the neutral fraction of the Universe and the quasar lifetime from individual quasar spectra. We apply this methodology to the spectra of the two highest redshift quasars known, ULAS J1120+0641 and ULAS J1342+0928, and measured volume-averaged neutral fractions $\langle x_{\rm HI} \rangle(z=7.09)=0.48^{+0.26}_{-0.26}$ and $\langle x_{\rm HI} \rangle(z=7.54)=0.60^{+0.20}_{-0.23}$ (posterior medians and 68% credible intervals) when marginalized over quasar lifetimes of $10^3 \leq t_{\rm q} \leq 10^8$ years.
△ Less
Submitted 16 February, 2018;
originally announced February 2018.
-
Modeling the HeII Transverse Proximity Effect: Constraints on Quasar Lifetime and Obscuration
Authors:
Tobias M. Schmidt,
Joseph F. Hennawi,
Gábor Worseck,
Frederick B. Davies,
Zarija Lukić,
Jose Oñorbe
Abstract:
The HeII transverse proximity effect - enhanced HeII Lyα transmission in a background sightline caused by the ionizing radiation of a foreground quasar - offers a unique opportunity to probe the emission properties of quasars, in particular the emission geometry (obscuration, beaming) and the quasar lifetime. Building on the foreground quasar survey published in Schmidt+2017, we present a detailed…
▽ More
The HeII transverse proximity effect - enhanced HeII Lyα transmission in a background sightline caused by the ionizing radiation of a foreground quasar - offers a unique opportunity to probe the emission properties of quasars, in particular the emission geometry (obscuration, beaming) and the quasar lifetime. Building on the foreground quasar survey published in Schmidt+2017, we present a detailed model of the HeII transverse proximity effect, specifically designed to include light travel time effects, finite quasar ages, and quasar obscuration. We post-process outputs from a cosmological hydrodynamical simulation with a fluctuating HeII UV background model, plus the added effect of the radiation from a single bright foreground quasar. We vary the age $t_\mathrm{age}$ and obscured sky fractions $Ω_\mathrm{obsc}$ of the foreground quasar, and explore the resulting effect on the HeII transverse proximity effect signal. Fluctuations in IGM density and the UV background, as well as the unknown orientation of the foreground quasar, result in a large variance of the HeII Lyα transmission along the background sightline. We develop a fully Bayesian statistical formalism to compare far UV HeII Lyα transmission spectra of the background quasars to our models, and extract joint constraints on $t_\mathrm{age}$ and $Ω_\mathrm{obsc}$ for the six Schmidt+2017 foreground quasars with the highest implied HeII photoionization rates. Our analysis suggests a bimodal distribution of quasar emission properties, whereby one foreground quasar, associated with a strong HeII transmission spike, is relatively old $(22\,\mathrm{Myr})$ and unobscured $Ω_\mathrm{obsc}<35\%$, whereas three others are either younger than $(10\,\mathrm{Myr})$ or highly obscured $(Ω_\mathrm{obsc}>70\%)$.
△ Less
Submitted 27 October, 2017; v1 submitted 12 October, 2017;
originally announced October 2017.
-
A Detection of $z$~2.3 Cosmic Voids from 3D Lyman-$α$ Forest Tomography in the COSMOS Field
Authors:
Alex Krolewski,
Khee-Gan Lee,
Martin White,
Joseph Hennawi,
David J. Schlegel,
Peter E. Nugent,
Zarija Lukić,
Casey W. Stark,
Olivier Le Fèvre,
Brian C. Lemaux,
Christian Maier,
Mara Salvato,
Lidia Tasca
Abstract:
We present the most distant detection of cosmic voids ($z \sim 2.3$) and the first detection of three-dimensional voids in the Lyman-$α$ forest. We used a 3D tomographic map of the absorption with effective comoving spatial resolution of $2.5\,h^{-1}\mathrm{Mpc}$ and volume of $3.15\times 10^5\,h^{-3}\mathrm{Mpc}^3$, which was reconstructed from moderate-resolution Keck-I/LRIS spectra of 240 backg…
▽ More
We present the most distant detection of cosmic voids ($z \sim 2.3$) and the first detection of three-dimensional voids in the Lyman-$α$ forest. We used a 3D tomographic map of the absorption with effective comoving spatial resolution of $2.5\,h^{-1}\mathrm{Mpc}$ and volume of $3.15\times 10^5\,h^{-3}\mathrm{Mpc}^3$, which was reconstructed from moderate-resolution Keck-I/LRIS spectra of 240 background Lyman-break galaxies and quasars in a $0.16\,\mathrm{deg}^2$ footprint in the COSMOS field. Voids were detected using a spherical overdensity finder calibrated from hydrodynamical simulations of the intergalactic medium. This allows us to identify voids in the IGM corresponding to voids in the underlying matter density field, yielding a consistent volume fraction of voids in both data (19.5%) and simulations (18.2%). We fit excursion set models to the void radius function and compare the radially-averaged stacked profiles of large voids ($r > 5$ $h^{-1}$ Mpc) to stacked voids in mock observations and the simulated density field. Comparing with 432 coeval galaxies with spectroscopic redshifts in the same volume as the tomographic map, we find that the tomography-identified voids are underdense in galaxies by 5.95$σ$ compared to random cells.
△ Less
Submitted 25 June, 2018; v1 submitted 6 October, 2017;
originally announced October 2017.
-
DESCQA: An Automated Validation Framework for Synthetic Sky Catalogs
Authors:
Yao-Yuan Mao,
Eve Kovacs,
Katrin Heitmann,
Thomas D. Uram,
Andrew J. Benson,
Duncan Campbell,
Sofía A. Cora,
Joseph DeRose,
Tiziana Di Matteo,
Salman Habib,
Andrew P. Hearin,
J. Bryce Kalmbach,
K. Simon Krughoff,
François Lanusse,
Zarija Lukić,
Rachel Mandelbaum,
Jeffrey A. Newman,
Nelson Padilla,
Enrique Paillas,
Adrian Pope,
Paul M. Ricker,
Andrés N. Ruiz,
Ananth Tenneti,
Cristian Vega-Martínez,
Risa H. Wechsler
, et al. (2 additional authors not shown)
Abstract:
The use of high-quality simulated sky catalogs is essential for the success of cosmological surveys. The catalogs have diverse applications, such as investigating signatures of fundamental physics in cosmological observables, understanding the effect of systematic uncertainties on measured signals and testing mitigation strategies for reducing these uncertainties, aiding analysis pipeline developm…
▽ More
The use of high-quality simulated sky catalogs is essential for the success of cosmological surveys. The catalogs have diverse applications, such as investigating signatures of fundamental physics in cosmological observables, understanding the effect of systematic uncertainties on measured signals and testing mitigation strategies for reducing these uncertainties, aiding analysis pipeline development and testing, and survey strategy optimization. The list of applications is growing with improvements in the quality of the catalogs and the details that they can provide. Given the importance of simulated catalogs, it is critical to provide rigorous validation protocols that enable both catalog providers and users to assess the quality of the catalogs in a straightforward and comprehensive way. For this purpose, we have developed the DESCQA framework for the Large Synoptic Survey Telescope Dark Energy Science Collaboration as well as for the broader community. The goal of DESCQA is to enable the inspection, validation, and comparison of an inhomogeneous set of synthetic catalogs via the provision of a common interface within an automated framework. In this paper, we present the design concept and first implementation of DESCQA. In order to establish and demonstrate its full functionality we use a set of interim catalogs and validation tests. We highlight several important aspects, both technical and scientific, that require thoughtful consideration when designing a validation framework, including validation metrics and how these metrics impose requirements on the synthetic sky catalogs.
△ Less
Submitted 8 February, 2018; v1 submitted 27 September, 2017;
originally announced September 2017.
-
A Fundamental Test for Galaxy Formation Models: Matching the Lyman-$α$ Absorption Profiles of Galactic Halos over Three Decades in Distance
Authors:
Daniele Sorini,
José Oñorbe,
Joseph F. Hennawi,
Zarija Lukić
Abstract:
Galaxy formation depends critically on the physical state of gas in the circumgalactic medium (CGM) and its interface with the intergalactic medium (IGM), determined by the complex interplay between inflows from the IGM and outflows from supernovae or AGN feedback. The average Lyman-alpha (Ly-a) absorption profile around galactic halos represents a powerful tool to probe their gaseous environments…
▽ More
Galaxy formation depends critically on the physical state of gas in the circumgalactic medium (CGM) and its interface with the intergalactic medium (IGM), determined by the complex interplay between inflows from the IGM and outflows from supernovae or AGN feedback. The average Lyman-alpha (Ly-a) absorption profile around galactic halos represents a powerful tool to probe their gaseous environments. We compare predictions from Illustris and Nyx hydrodynamical simulations with the observed absorption around foreground quasars, damped Ly-a systems, and Lyman-break galaxies. We show how large-scale BOSS and small-scale quasar pair measurements can be combined to precisely constrain the absorption profile over three decades in transverse distance 20kpc$\lesssim b\lesssim$20Mpc. Far from galaxies $\gtrsim2$Mpc, the simulations converge to the same profile and provide a reasonable match to the observations. This asymptotic agreement arises because the $Λ$CDM model successfully describes the ambient IGM, and represents a critical advantage of studying the mean absorption profile. However, significant differences between the simulations, and between simulations and observations are present on scales 20kpc$\lesssim b\lesssim$2Mpc, illustrating the challenges of accurately modeling and resolving galaxy formation physics. It is noteworthy that these differences are observed as far out as $\sim2$Mpc, indicating that the `sphere-of-influence' of galaxies could extend to approximately $\sim7$ times the halo virial radius ($\sim100$kpc). Current observations are very precise on these scales and can thus strongly discriminate between different galaxy formation models. We demonstrate that the Ly-a absorption profile is primarily sensitive to the underlying temperature-density relationship of diffuse gas around galaxies, and argue that it thus provides a fundamental test of galaxy formation models.
△ Less
Submitted 15 November, 2021; v1 submitted 12 September, 2017;
originally announced September 2017.
-
CosmoGAN: creating high-fidelity weak lensing convergence maps using Generative Adversarial Networks
Authors:
Mustafa Mustafa,
Deborah Bard,
Wahid Bhimji,
Zarija Lukić,
Rami Al-Rfou,
Jan M. Kratochvil
Abstract:
Inferring model parameters from experimental data is a grand challenge in many sciences, including cosmology. This often relies critically on high fidelity numerical simulations, which are prohibitively computationally expensive. The application of deep learning techniques to generative modeling is renewing interest in using high dimensional density estimators as computationally inexpensive emulat…
▽ More
Inferring model parameters from experimental data is a grand challenge in many sciences, including cosmology. This often relies critically on high fidelity numerical simulations, which are prohibitively computationally expensive. The application of deep learning techniques to generative modeling is renewing interest in using high dimensional density estimators as computationally inexpensive emulators of fully-fledged simulations. These generative models have the potential to make a dramatic shift in the field of scientific simulations, but for that shift to happen we need to study the performance of such generators in the precision regime needed for science applications. To this end, in this work we apply Generative Adversarial Networks to the problem of generating weak lensing convergence maps. We show that our generator network produces maps that are described by, with high statistical confidence, the same summary statistics as the fully simulated maps.
△ Less
Submitted 22 May, 2019; v1 submitted 7 June, 2017;
originally announced June 2017.
-
Measurement of the small-scale structure of the intergalactic medium using close quasar pairs
Authors:
Alberto Rorai,
Joseph F. Hennawi,
Jose Oñorbe,
Martin White,
J. Xavier Prochaska,
Girish Kulkarni,
Michael Walther,
Zarija Lukić,
Khee-Gan Lee
Abstract:
The distribution of diffuse gas in the intergalactic medium (IGM) imprints a series of hydrogen absorption lines on the spectra of distant background quasars known as the Lyman-$α$ forest. Cosmological hydrodynamical simulations predict that IGM density fluctuations are suppressed below a characteristic scale where thermal pressure balances gravity. We measured this pressure-smoothing scale by qua…
▽ More
The distribution of diffuse gas in the intergalactic medium (IGM) imprints a series of hydrogen absorption lines on the spectra of distant background quasars known as the Lyman-$α$ forest. Cosmological hydrodynamical simulations predict that IGM density fluctuations are suppressed below a characteristic scale where thermal pressure balances gravity. We measured this pressure-smoothing scale by quantifying absorption correlations in a sample of close quasar pairs. We compared our measurements to hydrodynamical simulations, where pressure smoothing is determined by the integrated thermal history of the IGM. Our findings are consistent with standard models for photoionization heating by the ultraviolet radiation backgrounds that reionized the universe.
△ Less
Submitted 26 April, 2017;
originally announced April 2017.
-
A New Method to Measure the Post-Reionization Ionizing Background from the Joint Distribution of Lyman-$α$ and Lyman-$β$ Forest Transmission
Authors:
Frederick B. Davies,
Joseph F. Hennawi,
Anna-Christina Eilers,
Zarija Lukić
Abstract:
The amplitude of the ionizing background that pervades the intergalactic medium (IGM) at the end of the epoch of reionization provides a valuable constraint on the emissivity of the sources which reionized the Universe. While measurements of the ionizing background at lower redshifts rely on a simulation-calibrated mapping between the photoionization rate and the mean transmission of the Ly$α$ for…
▽ More
The amplitude of the ionizing background that pervades the intergalactic medium (IGM) at the end of the epoch of reionization provides a valuable constraint on the emissivity of the sources which reionized the Universe. While measurements of the ionizing background at lower redshifts rely on a simulation-calibrated mapping between the photoionization rate and the mean transmission of the Ly$α$ forest, at $z\gtrsim6$ the IGM becomes increasingly opaque, and transmission arises solely in narrow spikes separated by saturated Gunn-Peterson troughs. In this regime, the traditional approach of measuring the average transmission over large $\sim 50$ Mpc$/h$ regions is less sensitive and sub-optimal. Additionally, the five times smaller oscillator strength of the Ly$β$ transition implies the Ly$β$ forest is considerably more transparent at $z\gtrsim6$, even in the presence of contamination by foreground $z\sim 5$ Ly$α$ forest absorption. In this work we present a novel statistical approach to analyze the joint distribution of transmission spikes in the co-spatial $z\sim 6$ Ly$α$ and Ly$β$ forests. Our method relies on Approximate Bayesian Computation (ABC), which circumvents the necessity of computing the intractable likelihood function describing the highly correlated Ly$α$ and Ly$β$ transmission. We apply ABC to mock data generated from a large-volume hydrodynamical simulation combined with a state-of-the-art model of ionizing background fluctuations in the post-reionization IGM, and show that it is sensitive to higher IGM neutral hydrogen fractions than previous techniques. As a proof of concept, we apply this methodology to a real spectrum of a $z=6.54$ quasar and measure the ionizing background from $5.4\leq z \leq 6.4$ along this sightline with $\sim0.2$ dex statistical uncertainties.
△ Less
Submitted 29 March, 2017;
originally announced March 2017.