-
Gadget3 on GPUs with OpenACC
Authors:
Antonio Ragagnin,
Klaus Dolag,
Mathias Wagner,
Claudio Gheller,
Conradin Roffler,
David Goz,
David Hubber,
Alexander Arth
Abstract:
We present preliminary results of a GPU porting of all main Gadget3 modules (gravity computation, SPH density computation, SPH hydrodynamic force, and thermal conduction) using OpenACC directives. Here we assign one GPU to each MPI rank and exploit both the host and accellerator capabilities by overlapping computations on the CPUs and GPUs: while GPUs asynchronously compute interactions between pa…
▽ More
We present preliminary results of a GPU porting of all main Gadget3 modules (gravity computation, SPH density computation, SPH hydrodynamic force, and thermal conduction) using OpenACC directives. Here we assign one GPU to each MPI rank and exploit both the host and accellerator capabilities by overlapping computations on the CPUs and GPUs: while GPUs asynchronously compute interactions between particles within their MPI ranks, CPUs perform tree-walks and MPI communications of neighbouring particles. We profile various portions of the code to understand the origin of our speedup, where we find that a peak speedup is not achieved because of time-steps with few active particles. We run a hydrodynamic cosmological simulation from the Magneticum project, with $2\cdot10^{7}$ particles, where we find a final total speedup of $\approx 2.$ We also present the results of an encouraging scaling test of a preliminary gravity-only OpenACC porting, run in the context of the EuroHack17 event, where the prototype of the porting proved to keep a constant speedup up to $1024$ GPUs.
△ Less
Submitted 24 March, 2020;
originally announced March 2020.
-
WVTICs -- SPH initial conditions for everyone
Authors:
Alexander Arth,
Julius Donnert,
Ulrich Steinwandel,
Ludwig Böss,
Timo Halbesma,
Martin Pütz,
David Hubber,
Klaus Dolag
Abstract:
We present a novel and fast application to generate glass-like initial conditions for Lagrangian hydrodynamic schemes (e.g. Smoothed Particle Hydrodynamics (SPH)) following arbitrary density models based on weighted Voronoi tessellations and combine it with improved initial configurations and an additional particle reshuffling scheme. We show our application's ability to sample different kinds of…
▽ More
We present a novel and fast application to generate glass-like initial conditions for Lagrangian hydrodynamic schemes (e.g. Smoothed Particle Hydrodynamics (SPH)) following arbitrary density models based on weighted Voronoi tessellations and combine it with improved initial configurations and an additional particle reshuffling scheme. We show our application's ability to sample different kinds of density features and to converge properly towards the given model density as well as a glass-like particle configuration. We analyse convergence with iterations as well as with varying particle number. Additionally, we demonstrate the versatility of the implemented algorithms by providing an extensive test suite for standard (magneto-) hydrodynamic test cases as well as a few common astrophysical applications. We indicate the potential to bridge further between observational astronomy and simulations as well as applicability to other fields of science by advanced features such as describing a density model using gridded data for exampling from an image file instead of an analytic model.
△ Less
Submitted 31 July, 2019; v1 submitted 25 July, 2019;
originally announced July 2019.
-
The Three Hundred Project: Ram pressure and gas content of haloes and subhaloes in the phase-space plane
Authors:
Jake Arthur,
Frazer R. Pearce,
Meghan E. Gray,
Alexander Knebe,
Weiguang Cui,
Pascal J. Elahi,
Chris Power,
Gustavo Yepes,
Alexander Arth,
Marco De Petris,
Klaus Dolag,
Lilian Garratt-Smithson,
Lyndsay J. Old,
Elena Rasia,
Adam R. H. Stevens
Abstract:
We use TheThreeHundred project, a suite of 324 resimulated massive galaxy clusters embedded in a broad range of environments, to investigate (i) how the gas content of surrounding haloes correlates with phase-space position at $z=0$, and (ii) to investigate the role that ram pressure plays in this correlation. By stacking all 324 normalised phase-space planes containing 169287 haloes and subhaloes…
▽ More
We use TheThreeHundred project, a suite of 324 resimulated massive galaxy clusters embedded in a broad range of environments, to investigate (i) how the gas content of surrounding haloes correlates with phase-space position at $z=0$, and (ii) to investigate the role that ram pressure plays in this correlation. By stacking all 324 normalised phase-space planes containing 169287 haloes and subhaloes, we show that the halo gas content is tightly correlated with phase-space position. At $\sim\,1.5-2\,\text{R}_{\text{200}}$ of the cluster dark matter halo, we find an extremely steep decline in the halo gas content of infalling haloes and subhaloes irrespective of cluster mass, possibly indicating the presence of an accretion shock. We also find that subhaloes are particularly gas-poor, even in the cluster outskirts, which could indicate active regions of ongoing pre-processing. By modelling the instantaneous ram pressure experienced by each halo and subhalo at $z=0$, we show that the ram pressure intensity is also well correlated with phase-space position, which is again irrespective of cluster mass. In fact, we show that regions in the phase-space plane with high differential velocity between a halo or subhalo and its local gas environment, are almost mutually exclusive with high halo gas content regions. This suggests a causal link between the gas content of objects and the instantaneous ram pressure they experience, where the dominant factor is the differential velocity.
△ Less
Submitted 17 January, 2019;
originally announced January 2019.
-
The Three Hundred Project: The evolution of galaxy cluster density profiles
Authors:
Robert Mostoghiu,
Alexander Knebe,
Weiguang Cui,
Frazer R. Pearce,
Gustavo Yepes,
Chris Power,
Romeel Dave,
Alexander Arth
Abstract:
Recent numerical studies of the dark matter density profiles of massive galaxy clusters ($M_{\rm halo} > 10^{15}$M$_{\odot}$) show that their median radial mass density profile remains unchanged up to $z > 1$, displaying a highly self-similar evolution. We verify this by using the data set of the THE THREE HUNDRED project, i.e. 324 cluster-sized haloes as found in full physics hydrodynamical simul…
▽ More
Recent numerical studies of the dark matter density profiles of massive galaxy clusters ($M_{\rm halo} > 10^{15}$M$_{\odot}$) show that their median radial mass density profile remains unchanged up to $z > 1$, displaying a highly self-similar evolution. We verify this by using the data set of the THE THREE HUNDRED project, i.e. 324 cluster-sized haloes as found in full physics hydrodynamical simulations. We track the progenitors of the mass-complete sample of clusters at $z=0$, and find that their median shape is already in place by $z=2.5$. However, selecting a dynamically relaxed subsample ($\sim16$ per cent of the clusters), we observe a shift of the scale radius $r_s$ towards larger values at earlier times. Classifying the whole sample by formation time, this evolution is understood as a result of a two-phase halo mass accretion process. Early-forming clusters -- identified as relaxed today -- have already entered their slow accretion phase, hence their mass growth occurs mostly at the outskirts. Late-forming clusters -- which are still unrelaxed today -- are in their fast accretion phase, thus the central region of the clusters is still growing. We conclude that the density profile of galaxy clusters shows a profound self-similarity out to redshifts $z\sim2.5$. This result holds for both gas and total density profiles when including baryonic physics, as reported here for two rather distinct sub-grid models.
△ Less
Submitted 10 December, 2018;
originally announced December 2018.
-
The Three Hundred Project: The influence of environment on simulated galaxy properties
Authors:
Yang Wang,
Frazer R. Pearce,
Alexander Knebe,
Gustavo Yepes,
Weiguang Cui,
Chris Power,
Alexander Arth,
Stefan Gottlober,
Marco De Petris,
Shaun Brown,
Longlong Feng
Abstract:
The relationship between galaxy properties and environment is a widely discussed topic within astrophysics. Here we use galaxy samples from hydrodynamical re-simulations to examine this relationship. We use the over-density ($δ_1$) within a $1 h^{-1}{\rm Mpc}$ sphere around a galaxy to evaluate its environment. Then the relations between galaxy properties, such as specific star formation rate(sSFR…
▽ More
The relationship between galaxy properties and environment is a widely discussed topic within astrophysics. Here we use galaxy samples from hydrodynamical re-simulations to examine this relationship. We use the over-density ($δ_1$) within a $1 h^{-1}{\rm Mpc}$ sphere around a galaxy to evaluate its environment. Then the relations between galaxy properties, such as specific star formation rate(sSFR), fraction of star forming galaxies, $g-r$ colour and $δ_1$ are examined within three galactic samples formed from galaxies within large clusters, those in the vicinity of large clusters and those in the field. We find tight environmental correlations for these galaxy properties. In brief, galaxies in denser environments tend to be redder and are more likely to be quenched. This is consistent with observations. We find that although the sSFR decreases with $δ_1$, this is mainly because that galaxies with higher stellar mass reside in environment with higher overdensity. At fixed over-density a galaxy's color is also independent of whether it lives within a cluster or within the field, but the relative fractions of the two samples varies dramatically with over-density and this drives an apparent evolution.
△ Less
Submitted 4 December, 2018; v1 submitted 13 September, 2018;
originally announced September 2018.
-
The Three Hundred project: a large catalogue of theoretically modelled galaxy clusters for cosmological and astrophysical applications
Authors:
Weiguang Cui,
Alexander Knebe,
Gustavo Yepes,
Frazer Pearce,
Chris Power,
Romeel Dave,
Alexander Arth,
Stefano Borgani,
Klaus Dolag,
Pascal Elahi,
Robert Mostoghiu,
Giuseppe Murante,
Elena Rasia,
Doris Stoppacher,
Jesus Vega-Ferrero,
Yang Wang,
Xiaohu Yang,
Andrew Benson,
Sofía A. Cora,
Darren J. Croton,
Manodeep Sinha,
Adam R. H. Stevens,
Cristian A. Vega-Martínez,
Jake Arthur,
Anna S. Baldi
, et al. (12 additional authors not shown)
Abstract:
We introduce the THE THREE HUNDRED project, an endeavour to model 324 large galaxy clusters with full-physics hydrodynamical re-simulations. Here we present the data set and study the differences to observations for fundamental galaxy cluster properties and scaling relations. We find that the modelled galaxy clusters are generally in reasonable agreement with observations with respect to baryonic…
▽ More
We introduce the THE THREE HUNDRED project, an endeavour to model 324 large galaxy clusters with full-physics hydrodynamical re-simulations. Here we present the data set and study the differences to observations for fundamental galaxy cluster properties and scaling relations. We find that the modelled galaxy clusters are generally in reasonable agreement with observations with respect to baryonic fractions and gas scaling relations at redshift z = 0. However, there are still some (model-dependent) differences, such as central galaxies being too massive, and galaxy colours (g - r) being bluer (about 0.2 dex lower at the peak position) than in observations. The agreement in gas scaling relations down to 10^{13} h^{-1} M_{\odot} between the simulations indicates that particulars of the sub-grid modelling of the baryonic physics only has a weak influence on these relations. We also include - where appropriate - a comparison to three semi-analytical galaxy formation models as applied to the same underlying dark-matter-only simulation. All simulations and derived data products are publicly available.
△ Less
Submitted 12 September, 2018;
originally announced September 2018.
-
Magnetic buoyancy in simulated galactic discs with a realistic circum galactic medium
Authors:
Ulrich P. Steinwandel,
Marcus C. Beck,
Alexander Arth,
Klaus Dolag,
Benjamin P. Moster,
Peter Nielaba
Abstract:
We present simulations of isolated disc galaxies in a realistic environment performed with the Tree-SPMHD-Code Gadget-3. Our simulations include a spherical circum-galactic medium (CGM) surrounding the galactic disc, motivated by observations and the results of cosmological simulations. We present three galactic models with different halo masses between 10e10 Msol and 10e12 Msol, and for each we u…
▽ More
We present simulations of isolated disc galaxies in a realistic environment performed with the Tree-SPMHD-Code Gadget-3. Our simulations include a spherical circum-galactic medium (CGM) surrounding the galactic disc, motivated by observations and the results of cosmological simulations. We present three galactic models with different halo masses between 10e10 Msol and 10e12 Msol, and for each we use two different approaches to seed the magnetic field, as well as a control simulation without a magnetic field. We find that the amplification of the magnetic field in the centre of the disc leads to a biconical magnetic outflow of gas that magnetizes the CGM. This biconical magnetic outflow reduces the star formation rate (SFR) of the galaxy by roughly 40 percent compared to the simulations without magnetic fields. As the key aspect of our simulations, we find that small scale turbulent motion of the gas in the disc leads to the amplification of the magnetic field up to tens of 10e-6 G, as long as the magnetic field strength is low. For stronger magnetic fields turbulent motion does not lead to significant amplification but is replaced by an alpha-omega dynamo. The occurance of a small scale turbulent dynamo becomes apparent through the magnetic power spectrum and analysis of the field lines' curvature. In accordance with recent observations we find an anti-correlation between the spiral structure in the gas density and in the magnetic field due to a diffusion term added to the induction equation.
△ Less
Submitted 29 August, 2018;
originally announced August 2018.
-
SPH to Grid: a new integral conserving method
Authors:
Bernhard Röttgers,
Alexander Arth
Abstract:
Analysing data from Smoothed Particle Hydrodynamics (SPH) simulations is about understanding global fluid properties rather than individual fluid elements. Therefore, in order to properly understand the outcome of such simulations it is crucial to transition from a particle to a grid based picture. In this paper we briefly summarise different methods of calculating a representative volume discreti…
▽ More
Analysing data from Smoothed Particle Hydrodynamics (SPH) simulations is about understanding global fluid properties rather than individual fluid elements. Therefore, in order to properly understand the outcome of such simulations it is crucial to transition from a particle to a grid based picture. In this paper we briefly summarise different methods of calculating a representative volume discretisation from SPH data and propose an improved version of commonly used techniques. We present a possibility to generate accurate 2D data directly without the CPU time and memory consuming detour over a 3D grid. We lay out the importance of an accurate algorithm to conserve integral fluid properties and to properly treat small scale structures using a typical galaxy simulation snapshot. For demonstration purposes we additionally calculate velocity power spectra and as expected find the main differences on small scales. Finally we propose two new multi-purpose analysis packages which utilise the new algorithms: Pygad and SPHMapper.
△ Less
Submitted 9 March, 2018;
originally announced March 2018.
-
Declining rotation curves at $z=2$ in $Λ$CDM galaxy formation simulations
Authors:
Adelheid F. Teklu,
Rhea-Silvia Remus,
Klaus Dolag,
Alexander Arth,
Andreas Burkert,
Aura Obreja,
Felix Schulze
Abstract:
Selecting disk galaxies from the cosmological, hydrodynamical simulation Magneticum Pathfinder we show that almost half of our poster child disk galaxies at $z=2$ show significantly declining rotation curves and low dark matter fractions, very similar to recently reported observations. These galaxies do not show any anomalous behavior, reside in standard dark matter halos and typically grow signif…
▽ More
Selecting disk galaxies from the cosmological, hydrodynamical simulation Magneticum Pathfinder we show that almost half of our poster child disk galaxies at $z=2$ show significantly declining rotation curves and low dark matter fractions, very similar to recently reported observations. These galaxies do not show any anomalous behavior, reside in standard dark matter halos and typically grow significantly in mass until $z = 0$, where they span all morphological classes, including disk galaxies matching present day rotation curves and observed dark matter fractions. Our findings demonstrate that declining rotation curves and low dark matter fractions in rotation dominated galaxies at $z=2$ appear naturally within the $Λ$CDM paradigm and reflect the complex baryonic physics, which plays a role at the peak epoch of star-formation. In addition, we find some dispersion dominated galaxies at $z=2$ which host a significant gas disk and exhibit similar shaped rotation curves as the disk galaxy population, rendering it difficult to differentiate between these two populations with currently available observation techniques.
△ Less
Submitted 14 February, 2018; v1 submitted 21 November, 2017;
originally announced November 2017.
-
An improved SPH scheme for cosmological simulations
Authors:
A. M. Beck,
G. Murante,
A. Arth,
R. -S. Remus,
A. F. Teklu,
J. M. F. Donnert,
S. Planelles,
M. C. Beck,
P. Foerster,
M. Imgrund,
K. Dolag,
S. Borgani
Abstract:
We present an implementation of smoothed particle hydrodynamics (SPH) with improved accuracy for simulations of galaxies and the large-scale structure. In particular, we combine, implement, modify and test a vast majority of SPH improvement techniques in the latest instalment of the GADGET code. We use the Wendland kernel functions, a particle wake-up time-step limiting mechanism and a time-depend…
▽ More
We present an implementation of smoothed particle hydrodynamics (SPH) with improved accuracy for simulations of galaxies and the large-scale structure. In particular, we combine, implement, modify and test a vast majority of SPH improvement techniques in the latest instalment of the GADGET code. We use the Wendland kernel functions, a particle wake-up time-step limiting mechanism and a time-dependent scheme for artificial viscosity, which includes a high-order gradient computation and shear flow limiter. Additionally, we include a novel prescription for time-dependent artificial conduction, which corrects for gravitationally induced pressure gradients and largely improves the SPH performance in capturing the development of gas-dynamical instabilities. We extensively test our new implementation in a wide range of hydrodynamical standard tests including weak and strong shocks as well as shear flows, turbulent spectra, gas mixing, hydrostatic equilibria and self-gravitating gas clouds. We jointly employ all modifications; however, when necessary we study the performance of individual code modules. We approximate hydrodynamical states more accurately and with significantly less noise than standard SPH. Furthermore, the new implementation promotes the mixing of entropy between different fluid phases, also within cosmological simulations. Finally, we study the performance of the hydrodynamical solver in the context of radiative galaxy formation and non-radiative galaxy cluster formation. We find galactic disks to be colder, thinner and more extended and our results on galaxy clusters show entropy cores instead of steadily declining entropy profiles. In summary, we demonstrate that our improved SPH implementation overcomes most of the undesirable limitations of standard SPH, thus becoming the core of an efficient code for large cosmological simulations.
△ Less
Submitted 20 October, 2015; v1 submitted 25 February, 2015;
originally announced February 2015.
-
Anisotropic thermal conduction in galaxy clusters with MHD in Gadget
Authors:
Alexander Arth,
Klaus Dolag,
Alexander M. Beck,
Margarita Petkova,
Harald Lesch
Abstract:
We present an implementation of thermal conduction including the anisotropic effects of magnetic fields for SPH. The anisotropic thermal conduction is mainly proceeding parallel to magnetic fields and suppressed perpendicular to the fields. We derive the SPH formalism for the anisotropic heat transport and solve the corresponding equation with an implicit conjugate gradient scheme. We discuss seve…
▽ More
We present an implementation of thermal conduction including the anisotropic effects of magnetic fields for SPH. The anisotropic thermal conduction is mainly proceeding parallel to magnetic fields and suppressed perpendicular to the fields. We derive the SPH formalism for the anisotropic heat transport and solve the corresponding equation with an implicit conjugate gradient scheme. We discuss several issues of unphysical heat transport in the cases of extreme ansiotropies or unmagnetized regions and present possible numerical workarounds. We implement our algorithm into the GADGET code and study its behaviour in several test cases. In general, we reproduce the analytical solutions of our idealised test problems, and obtain good results in cosmological simulations of galaxy cluster formations. Within galaxy clusters, the anisotropic conduction produces a net heat transport similar to an isotropic Spitzer conduction model with an efficiency of one per cent. In contrast to isotropic conduction our new formalism allows small-scale structure in the temperature distribution to remain stable, because of their decoupling caused by magnetic field lines. Compared to observations, isotropic conduction with more than 10 per cent of the Spitzer value leads to an oversmoothed temperature distribution within clusters, while the results obtained with anisotropic thermal conduction reproduce the observed temperature fluctuations well. A proper treatment of heat transport is crucial especially in the outskirts of clusters and also in high density regions. It's connection to the local dynamical state of the cluster also might contribute to the observed bimodal distribution of cool core and non cool core clusters. Our new scheme significantly advances the modelling of thermal conduction in numerical simulations and overall gives better results compared to observations.
△ Less
Submitted 15 September, 2017; v1 submitted 19 December, 2014;
originally announced December 2014.
