Showing 1–50 of 230 results for author: Jacobs, D

An Update on the External Calibrator for Hydrogen Observatories (ECHO)

Authors: Yifan Zhao, Daniel C. Jacobs, Titu Samson, Mrudula Gopal Krishna, Michael Horn, Marc-Olivier R. Lalonde, Raven Braithwaite, Logan Skabelund

Abstract: Precision measurements of the beam pattern response are needed to predict the response of a radio telescope. Mapping the beam of a low frequency radio array presents a unique challenge and science cases such as the observation of the 21\,cm line at high redshift have demanding requirements. Drone-based systems offer the unique potential for a measurement which is entirely under experimenter contro… ▽ More Precision measurements of the beam pattern response are needed to predict the response of a radio telescope. Mapping the beam of a low frequency radio array presents a unique challenge and science cases such as the observation of the 21\,cm line at high redshift have demanding requirements. Drone-based systems offer the unique potential for a measurement which is entirely under experimenter control, but progress has been paced by practical implementation challenges. Previously, a prototype drone system, called the External Calibrator for Hydrogen Observatories (ECHO), demonstrated good performance in making a complete hemispherical beam measurement. This paper reports updates to the system focusing on performance of a new drone platform, minimizing interference from the drone, and a new transmitter. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Magic Insert: Style-Aware Drag-and-Drop

Authors: Nataniel Ruiz, Yuanzhen Li, Neal Wadhwa, Yael Pritch, Michael Rubinstein, David E. Jacobs, Shlomi Fruchter

Abstract: We present Magic Insert, a method for dragging-and-dropping subjects from a user-provided image into a target image of a different style in a physically plausible manner while matching the style of the target image. This work formalizes the problem of style-aware drag-and-drop and presents a method for tackling it by addressing two sub-problems: style-aware personalization and realistic object ins… ▽ More We present Magic Insert, a method for dragging-and-dropping subjects from a user-provided image into a target image of a different style in a physically plausible manner while matching the style of the target image. This work formalizes the problem of style-aware drag-and-drop and presents a method for tackling it by addressing two sub-problems: style-aware personalization and realistic object insertion in stylized images. For style-aware personalization, our method first fine-tunes a pretrained text-to-image diffusion model using LoRA and learned text tokens on the subject image, and then infuses it with a CLIP representation of the target style. For object insertion, we use Bootstrapped Domain Adaption to adapt a domain-specific photorealistic object insertion model to the domain of diverse artistic styles. Overall, the method significantly outperforms traditional approaches such as inpainting. Finally, we present a dataset, SubjectPlop, to facilitate evaluation and future progress in this area. Project page: https://magicinsert.github.io/ △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: Project page: https://magicinsert.github.io/

Drone-Based Antenna Beam Calibration in the High Arctic

Authors: Lawrence Herman, Christopher Barbarie, Mohan Agrawal, Vlad Calinescu, Simon Chen, H. Cynthia Chiang, Cherie K. Day, Eamon Egan, Stephen Fay, Kit Gerodias, Maya Goss, Michael Hétu, Daniel C. Jacobs, Marc-Olivier R. Lalonde, Francis McGee, Loïc Miara, John Orlowski-Scherer, Jonathan Sievers

Abstract: The development of low-frequency radio astronomy experiments for detecting 21-cm line emission from hydrogen presents new opportunities for creative solutions to the challenge of characterizing an antenna beam pattern. The Array of Long Baseline Antennas for Taking Radio Observations from the Seventy-ninth parallel (ALBATROS) is a new radio interferometer sited in the Canadian high Arctic that aim… ▽ More The development of low-frequency radio astronomy experiments for detecting 21-cm line emission from hydrogen presents new opportunities for creative solutions to the challenge of characterizing an antenna beam pattern. The Array of Long Baseline Antennas for Taking Radio Observations from the Seventy-ninth parallel (ALBATROS) is a new radio interferometer sited in the Canadian high Arctic that aims to map Galactic foregrounds at frequencies below $\sim$30 MHz. We present PteroSoar, a custom-built hexacopter outfitted with a transmitter, that will be used to characterize the beam patterns of ALBATROS and other experiments. The PteroSoar drone hardware is motivated by the need for user-servicing at remote sites and environmental factors that are unique to the high Arctic. In particular, magnetic heading is unreliable because the magnetic field lines near the north pole are almost vertical. We therefore implement moving baseline real time kinematic (RTK) positioning with two GPS units to obtain heading solutions with $\sim$1$^\circ$ accuracy. We present a preliminary beam map of an ALBATROS antenna, thus demonstrating successful PteroSoar operation in the high Arctic. △ Less

Submitted 30 June, 2024; originally announced July 2024.

Rethinking Score Distillation as a Bridge Between Image Distributions

Authors: David McAllister, Songwei Ge, Jia-Bin Huang, David W. Jacobs, Alexei A. Efros, Aleksander Holynski, Angjoo Kanazawa

Abstract: Score distillation sampling (SDS) has proven to be an important tool, enabling the use of large-scale diffusion priors for tasks operating in data-poor domains. Unfortunately, SDS has a number of characteristic artifacts that limit its usefulness in general-purpose applications. In this paper, we make progress toward understanding the behavior of SDS and its variants by viewing them as solving an… ▽ More Score distillation sampling (SDS) has proven to be an important tool, enabling the use of large-scale diffusion priors for tasks operating in data-poor domains. Unfortunately, SDS has a number of characteristic artifacts that limit its usefulness in general-purpose applications. In this paper, we make progress toward understanding the behavior of SDS and its variants by viewing them as solving an optimal-cost transport path from a source distribution to a target distribution. Under this new interpretation, these methods seek to transport corrupted images (source) to the natural image distribution (target). We argue that current methods' characteristic artifacts are caused by (1) linear approximation of the optimal path and (2) poor estimates of the source distribution. We show that calibrating the text conditioning of the source distribution can produce high-quality generation and translation results with little extra overhead. Our method can be easily applied across many domains, matching or beating the performance of specialized methods. We demonstrate its utility in text-to-2D, text-based NeRF optimization, translating paintings to real images, optical illusion generation, and 3D sketch-to-real. We compare our method to existing approaches for score distillation sampling and show that it can produce high-frequency details with realistic colors. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Comments: Project webpage: https://sds-bridge.github.io/

Investigating Mutual Coupling in the Hydrogen Epoch of Reionization Array and Mitigating its Effects on the 21-cm Power Spectrum

Authors: E. Rath, R. Pascua, A. T. Josaitis, A. Ewall-Wice, N. Fagnoni, E. de Lera Acedo, Z. E. Martinot, Z. Abdurashidova, T. Adams, J. E. Aguirre, R. Baartman, A. P. Beardsley, L. M. Berkhout, G. Bernardi, T. S. Billings, J. D. Bowman, P. Bull, J. Burba, R. Byrne, S. Carey, K. -F. Chen, S. Choudhuri, T. Cox, D. R. DeBoer, M. Dexter , et al. (56 additional authors not shown)

Abstract: Interferometric experiments designed to detect the highly redshifted 21-cm signal from neutral hydrogen are producing increasingly stringent constraints on the 21-cm power spectrum, but some k-modes remain systematics-dominated. Mutual coupling is a major systematic that must be overcome in order to detect the 21-cm signal, and simulations that reproduce effects seen in the data can guide strategi… ▽ More Interferometric experiments designed to detect the highly redshifted 21-cm signal from neutral hydrogen are producing increasingly stringent constraints on the 21-cm power spectrum, but some k-modes remain systematics-dominated. Mutual coupling is a major systematic that must be overcome in order to detect the 21-cm signal, and simulations that reproduce effects seen in the data can guide strategies for mitigating mutual coupling. In this paper, we analyse 12 nights of data from the Hydrogen Epoch of Reionization Array and compare the data against simulations that include a computationally efficient and physically motivated semi-analytic treatment of mutual coupling. We find that simulated coupling features qualitatively agree with coupling features in the data; however, coupling features in the data are brighter than the simulated features, indicating the presence of additional coupling mechanisms not captured by our model. We explore the use of fringe-rate filters as mutual coupling mitigation tools and use our simulations to investigate the effects of mutual coupling on a simulated cosmological 21-cm power spectrum in a "worst case" scenario where the foregrounds are particularly bright. We find that mutual coupling contaminates a large portion of the "EoR Window", and the contamination is several orders-of-magnitude larger than our simulated cosmic signal across a wide range of cosmological Fourier modes. While our fiducial fringe-rate filtering strategy reduces mutual coupling by roughly a factor of 100 in power, a non-negligible amount of coupling cannot be excised with fringe-rate filters, so more sophisticated mitigation strategies are required. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 19 pages, 12 figures, submitted to MNRAS

CinePile: A Long Video Question Answering Dataset and Benchmark

Authors: Ruchit Rawal, Khalid Saifullah, Ronen Basri, David Jacobs, Gowthami Somepalli, Tom Goldstein

Abstract: Current datasets for long-form video understanding often fall short of providing genuine long-form comprehension challenges, as many tasks derived from these datasets can be successfully tackled by analyzing just one or a few random frames from a video. To address this issue, we present a novel dataset and benchmark, CinePile, specifically designed for authentic long-form video understanding. This… ▽ More Current datasets for long-form video understanding often fall short of providing genuine long-form comprehension challenges, as many tasks derived from these datasets can be successfully tackled by analyzing just one or a few random frames from a video. To address this issue, we present a novel dataset and benchmark, CinePile, specifically designed for authentic long-form video understanding. This paper details our innovative approach for creating a question-answer dataset, utilizing advanced LLMs with human-in-the-loop and building upon human-generated raw data. Our comprehensive dataset comprises 305,000 multiple-choice questions (MCQs), covering various visual and multimodal aspects, including temporal comprehension, understanding human-object interactions, and reasoning about events or actions within a scene. Additionally, we evaluate recent video-centric LLMs, both open-source and proprietary, on the test split of our dataset. The findings reveal that even state-of-the-art video-centric LLMs significantly lag behind human performance in these tasks, highlighting the complexity and challenge inherent in video understanding. The dataset is available at https://hf.co/datasets/tomg-group-umd/cinepile △ Less

Submitted 14 June, 2024; v1 submitted 14 May, 2024; originally announced May 2024.

Comments: Project page with all the artifacts - https://ruchitrawal.github.io/cinepile/. Updated version with results on Gemini Flash model and additional related work

GaNI: Global and Near Field Illumination Aware Neural Inverse Rendering

Authors: Jiaye Wu, Saeed Hadadan, Geng Lin, Matthias Zwicker, David Jacobs, Roni Sengupta

Abstract: In this paper, we present GaNI, a Global and Near-field Illumination-aware neural inverse rendering technique that can reconstruct geometry, albedo, and roughness parameters from images of a scene captured with co-located light and camera. Existing inverse rendering techniques with co-located light-camera focus on single objects only, without modeling global illumination and near-field lighting mo… ▽ More In this paper, we present GaNI, a Global and Near-field Illumination-aware neural inverse rendering technique that can reconstruct geometry, albedo, and roughness parameters from images of a scene captured with co-located light and camera. Existing inverse rendering techniques with co-located light-camera focus on single objects only, without modeling global illumination and near-field lighting more prominent in scenes with multiple objects. We introduce a system that solves this problem in two stages; we first reconstruct the geometry powered by neural volumetric rendering NeuS, followed by inverse neural radiosity that uses the previously predicted geometry to estimate albedo and roughness. However, such a naive combination fails and we propose multiple technical contributions that enable this two-stage approach. We observe that NeuS fails to handle near-field illumination and strong specular reflections from the flashlight in a scene. We propose to implicitly model the effects of near-field illumination and introduce a surface angle loss function to handle specular reflections. Similarly, we observe that invNeRad assumes constant illumination throughout the capture and cannot handle moving flashlights during capture. We propose a light position-aware radiance cache network and additional smoothness priors on roughness to reconstruct reflectance. Experimental evaluation on synthetic and real data shows that our method outperforms the existing co-located light-camera-based inverse rendering techniques. Our approach produces significantly better reflectance and slightly better geometry than capture strategies that do not require a dark room. △ Less

Submitted 22 March, 2024; originally announced March 2024.

LoDIP: Low light phase retrieval with deep image prior

Authors: Raunak Manekar, Elisa Negrini, Minh Pham, Daniel Jacobs, Jaideep Srivastava

Abstract: Phase retrieval (PR) is a fundamental challenge in scientific imaging, enabling nanoscale techniques like coherent diffractive imaging (CDI). Imaging at low radiation doses becomes important in applications where samples are susceptible to radiation damage. However, most PR methods struggle in low dose scenario due to the presence of very high shot noise. Advancements in the optical data acquisiti… ▽ More Phase retrieval (PR) is a fundamental challenge in scientific imaging, enabling nanoscale techniques like coherent diffractive imaging (CDI). Imaging at low radiation doses becomes important in applications where samples are susceptible to radiation damage. However, most PR methods struggle in low dose scenario due to the presence of very high shot noise. Advancements in the optical data acquisition setup, exemplified by in-situ CDI, have shown potential for low-dose imaging. But these depend on a time series of measurements, rendering them unsuitable for single-image applications. Similarly, on the computational front, data-driven phase retrieval techniques are not readily adaptable to the single-image context. Deep learning based single-image methods, such as deep image prior, have been effective for various imaging tasks but have exhibited limited success when applied to PR. In this work, we propose LoDIP which combines the in-situ CDI setup with the power of implicit neural priors to tackle the problem of single-image low-dose phase retrieval. Quantitative evaluations demonstrate the superior performance of LoDIP on this task as well as applicability to real experimental scenarios. △ Less

Submitted 27 February, 2024; originally announced February 2024.

MSC Class: 68T10 68T07 78A46

A demonstration of the effect of fringe-rate filtering in the Hydrogen Epoch of Reionization Array delay power spectrum pipeline

Authors: Hugh Garsden, Philip Bull, Mike Wilensky, Zuhra Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Lindsay M. Berkhout, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Steven Carey, Chris L. Carilli, Kai-Feng Chen, Carina Cheng, Samir Choudhuri, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter , et al. (72 additional authors not shown)

Abstract: Radio interferometers targeting the 21cm brightness temperature fluctuations at high redshift are subject to systematic effects that operate over a range of different timescales. These can be isolated by designing appropriate Fourier filters that operate in fringe-rate (FR) space, the Fourier pair of local sidereal time (LST). Applications of FR filtering include separating effects that are correl… ▽ More Radio interferometers targeting the 21cm brightness temperature fluctuations at high redshift are subject to systematic effects that operate over a range of different timescales. These can be isolated by designing appropriate Fourier filters that operate in fringe-rate (FR) space, the Fourier pair of local sidereal time (LST). Applications of FR filtering include separating effects that are correlated with the rotating sky vs. those relative to the ground, down-weighting emission in the primary beam sidelobes, and suppressing noise. FR filtering causes the noise contributions to the visibility data to become correlated in time however, making interpretation of subsequent averaging and error estimation steps more subtle. In this paper, we describe fringe rate filters that are implemented using discrete prolate spheroidal sequences, and designed for two different purposes -- beam sidelobe/horizon suppression (the `mainlobe' filter), and ground-locked systematics removal (the `notch' filter). We apply these to simulated data, and study how their properties affect visibilities and power spectra generated from the simulations. Included is an introduction to fringe-rate filtering and a demonstration of fringe-rate filters applied to simple situations to aid understanding. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Comments: 21 pages, 18 figures, submitted to Monthly Notices of the Royal Astronomical Society

Hydrogen Epoch of Reionization Array (HERA) Phase II Deployment and Commissioning

Authors: Lindsay M. Berkhout, Daniel C. Jacobs, Zuhra Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Jacob Burba, Steven Carey, Chris L. Carilli, Kai-Feng Chen, Carina Cheng, Samir Choudhuri, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Joshua S. Dillon , et al. (71 additional authors not shown)

Abstract: This paper presents the design and deployment of the Hydrogen Epoch of Reionization Array (HERA) phase II system. HERA is designed as a staged experiment targeting 21 cm emission measurements of the Epoch of Reionization. First results from the phase I array are published as of early 2022, and deployment of the phase II system is nearing completion. We describe the design of the phase II system an… ▽ More This paper presents the design and deployment of the Hydrogen Epoch of Reionization Array (HERA) phase II system. HERA is designed as a staged experiment targeting 21 cm emission measurements of the Epoch of Reionization. First results from the phase I array are published as of early 2022, and deployment of the phase II system is nearing completion. We describe the design of the phase II system and discuss progress on commissioning and future upgrades. As HERA is a designated Square Kilometer Array (SKA) pathfinder instrument, we also show a number of "case studies" that investigate systematics seen while commissioning the phase II system, which may be of use in the design and operation of future arrays. Common pathologies are likely to manifest in similar ways across instruments, and many of these sources of contamination can be mitigated once the source is identified. △ Less

Submitted 8 January, 2024; originally announced January 2024.

Journal ref: PASP 2024 136 045002

matvis: A matrix-based visibility simulator for fast forward modelling of many-element 21 cm arrays

Authors: Piyanat Kittiwisit, Steven G. Murray, Hugh Garsden, Philip Bull, Christopher Cain, Aaron R. Parsons, Jackson Sipple, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Lindsay M. Berkhout, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Steven Carey, Chris L. Carilli, Kai-Feng Chen, Carina Cheng , et al. (73 additional authors not shown)

Abstract: Detection of the faint 21 cm line emission from the Cosmic Dawn and Epoch of Reionisation will require not only exquisite control over instrumental calibration and systematics to achieve the necessary dynamic range of observations but also validation of analysis techniques to demonstrate their statistical properties and signal loss characteristics. A key ingredient in achieving this is the ability… ▽ More Detection of the faint 21 cm line emission from the Cosmic Dawn and Epoch of Reionisation will require not only exquisite control over instrumental calibration and systematics to achieve the necessary dynamic range of observations but also validation of analysis techniques to demonstrate their statistical properties and signal loss characteristics. A key ingredient in achieving this is the ability to perform high-fidelity simulations of the kinds of data that are produced by the large, many-element, radio interferometric arrays that have been purpose-built for these studies. The large scale of these arrays presents a computational challenge, as one must simulate a detailed sky and instrumental model across many hundreds of frequency channels, thousands of time samples, and tens of thousands of baselines for arrays with hundreds of antennas. In this paper, we present a fast matrix-based method for simulating radio interferometric measurements (visibilities) at the necessary scale. We achieve this through judicious use of primary beam interpolation, fast approximations for coordinate transforms, and a vectorised outer product to expand per-antenna quantities to per-baseline visibilities, coupled with standard parallelisation techniques. We validate the results of this method, implemented in the publicly-available matvis code, against a high-precision reference simulator, and explore its computational scaling on a variety of problems. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Comments: 25 pages, 20 figures, submitted to RAS Techniques and Instruments, matvis is publicly available at https://github.com/HERA-Team/matvis

Bayesian estimation of cross-coupling and reflection systematics in 21cm array visibility data

Authors: Geoff G. Murphy, Philip Bull, Mario G. Santos, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee Billings, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Christopher Cain, Steven Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Joshua S. Dillon, Nico Eksteen , et al. (54 additional authors not shown)

Abstract: Observations with radio arrays that target the 21-cm signal originating from the early Universe suffer from a variety of systematic effects. An important class of these are reflections and spurious couplings between antennas. We apply a Hamiltonian Monte Carlo sampler to the modelling and mitigation of these systematics in simulated Hydrogen Epoch of Reionisation Array (HERA) data. This method all… ▽ More Observations with radio arrays that target the 21-cm signal originating from the early Universe suffer from a variety of systematic effects. An important class of these are reflections and spurious couplings between antennas. We apply a Hamiltonian Monte Carlo sampler to the modelling and mitigation of these systematics in simulated Hydrogen Epoch of Reionisation Array (HERA) data. This method allows us to form statistical uncertainty estimates for both our models and the recovered visibilities, which is an important ingredient in establishing robust upper limits on the Epoch of Reionisation (EoR) power spectrum. In cases where the noise is large compared to the EoR signal, this approach can constrain the systematics well enough to mitigate them down to the noise level for both systematics studied. Where the noise is smaller than the EoR, our modelling can mitigate the majority of the reflections with there being only a minor level of residual systematics, while cross-coupling sees essentially complete mitigation. Our approach performs similarly to existing filtering/fitting techniques used in the HERA pipeline, but with the added benefit of rigorously propagating uncertainties. In all cases it does not significantly attenuate the underlying signal. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Comments: 19 pages, 14 figures, submitted to MNRAS

Direct Optimal Mapping Image Power Spectrum and its Window Functions

Authors: Zhilei Xu, Honggeun Kim, Jacqueline N. Hewitt, Kai-Feng Chen, Nicholas S. Kern, Eleanor Rath, Ruby Byrne, Adélie Gorce, Robert Pascua, Zachary E. Martinot, Joshua S. Dillon, Bryna J. Hazelton, Adrian Liu, Miguel F. Morales, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman , et al. (57 additional authors not shown)

Abstract: The key to detecting neutral hydrogen during the epoch of reionization (EoR) is to separate the cosmological signal from the dominating foreground radiation. We developed direct optimal mapping (DOM) to map interferometric visibilities; it contains only linear operations, with full knowledge of point spread functions from visibilities to images. Here, we demonstrate a fast Fourier transform-based… ▽ More The key to detecting neutral hydrogen during the epoch of reionization (EoR) is to separate the cosmological signal from the dominating foreground radiation. We developed direct optimal mapping (DOM) to map interferometric visibilities; it contains only linear operations, with full knowledge of point spread functions from visibilities to images. Here, we demonstrate a fast Fourier transform-based image power spectrum and its window functions computed from the DOM images. We use noiseless simulation, based on the Hydrogen Epoch of Reionization Array Phase I configuration, to study the image power spectrum properties. The window functions show $<10^{-11}$ of the integrated power leaks from the foreground-dominated region into the EoR window; the 2D and 1D power spectra also verify the separation between the foregrounds and the EoR. △ Less

Submitted 5 July, 2024; v1 submitted 17 November, 2023; originally announced November 2023.

Comments: Published in ApJ

Towards Combinatorial Generalization for Catalysts: A Kohn-Sham Charge-Density Approach

Authors: Phillip Pope, David Jacobs

Abstract: The Kohn-Sham equations underlie many important applications such as the discovery of new catalysts. Recent machine learning work on catalyst modeling has focused on prediction of the energy, but has so far not yet demonstrated significant out-of-distribution generalization. Here we investigate another approach based on the pointwise learning of the Kohn-Sham charge-density. On a new dataset of bu… ▽ More The Kohn-Sham equations underlie many important applications such as the discovery of new catalysts. Recent machine learning work on catalyst modeling has focused on prediction of the energy, but has so far not yet demonstrated significant out-of-distribution generalization. Here we investigate another approach based on the pointwise learning of the Kohn-Sham charge-density. On a new dataset of bulk catalysts with charge densities, we show density models can generalize to new structures with combinations of elements not seen at train time, a form of combinatorial generalization. We show that over 80% of binary and ternary test cases achieve faster convergence than standard baselines in Density Functional Theory, amounting to an average reduction of 13% in the number of iterations required to reach convergence, which may be of independent interest. Our results suggest that density learning is a viable alternative, trading greater inference costs for a step towards combinatorial generalization, a key property for applications. △ Less

Submitted 28 October, 2023; originally announced October 2023.

Comments: Published at NeurIPS 2023

Evidence of Ultra-faint Radio Frequency Interference in Deep 21~cm Epoch of Reionization Power Spectra with the Murchison Widefield Array

Authors: Michael J. Wilensky, Miguel F. Morales, Bryna J. Hazelton, Pyxie L. Star, Nichole Barry, Ruby Byrne, C. H. Jordan, Daniel C. Jacobs, Jonathan C. Pober, C. M. Trott

Abstract: We present deep upper limits from the 2014 Murchison Widefield Array (MWA) Phase I observing season, with a particular emphasis on identifying the spectral fingerprints of extremely faint radio frequency interference (RFI) contamination in the 21~cm power spectra (PS). After meticulous RFI excision involving a combination of the \textsc{SSINS} RFI flagger and a series of PS-based jackknife tests,… ▽ More We present deep upper limits from the 2014 Murchison Widefield Array (MWA) Phase I observing season, with a particular emphasis on identifying the spectral fingerprints of extremely faint radio frequency interference (RFI) contamination in the 21~cm power spectra (PS). After meticulous RFI excision involving a combination of the \textsc{SSINS} RFI flagger and a series of PS-based jackknife tests, our lowest upper limit on the Epoch of Reionization (EoR) 21~cm PS signal is $Δ^2 \leq 1.61\cdot10^4 \text{ mK}^2$ at $k=0.258\text{ h Mpc}^{-1}$ at a redshift of 7.1 using 14.7 hours of data. By leveraging our understanding of how even fainter RFI is likely to contaminate the EoR PS, we are able to identify ultra-faint RFI signals in the cylindrical PS. Surprisingly this signature is most obvious in PS formed with less than an hour of data, but is potentially subdominant to other systematics in multiple-hour integrations. Since the total RFI budget in a PS detection is quite strict, this nontrivial integration behavior suggests a need to more realistically model coherently integrated ultra-faint RFI in PS measurements so that its potential contribution to a future detection can be diagnosed. △ Less

Submitted 7 November, 2023; v1 submitted 5 October, 2023; originally announced October 2023.

Comments: Update acknowledgements, author metadata, and attach journal doi

Journal ref: The Astrophysical Journal, 2023, Volume 957, Number 2, p. 78

RealFill: Reference-Driven Generation for Authentic Image Completion

Authors: Luming Tang, Nataniel Ruiz, Qinghao Chu, Yuanzhen Li, Aleksander Holynski, David E. Jacobs, Bharath Hariharan, Yael Pritch, Neal Wadhwa, Kfir Aberman, Michael Rubinstein

Abstract: Recent advances in generative imagery have brought forth outpainting and inpainting models that can produce high-quality, plausible image content in unknown regions. However, the content these models hallucinate is necessarily inauthentic, since they are unaware of the true scene. In this work, we propose RealFill, a novel generative approach for image completion that fills in missing regions of a… ▽ More Recent advances in generative imagery have brought forth outpainting and inpainting models that can produce high-quality, plausible image content in unknown regions. However, the content these models hallucinate is necessarily inauthentic, since they are unaware of the true scene. In this work, we propose RealFill, a novel generative approach for image completion that fills in missing regions of an image with the content that should have been there. RealFill is a generative inpainting model that is personalized using only a few reference images of a scene. These reference images do not have to be aligned with the target image, and can be taken with drastically varying viewpoints, lighting conditions, camera apertures, or image styles. Once personalized, RealFill is able to complete a target image with visually compelling contents that are faithful to the original scene. We evaluate RealFill on a new image completion benchmark that covers a set of diverse and challenging scenarios, and find that it outperforms existing approaches by a large margin. Project page: https://realfill.github.io △ Less

Submitted 14 May, 2024; v1 submitted 28 September, 2023; originally announced September 2023.

Comments: SIGGRAPH 2024 (Journal Track). Project page: https://realfill.github.io

Demodulation demonstration using the LightCube CubeSat

Authors: Lindsay M. Berkhout, Christopher McCormick, Daniel C. Jacobs, Jaime Sanchez de la Vega

Abstract: LightCube is a 1U educational CubeSat which had the goal of connecting the public with space by producing a flash visible to the naked eye on command by a public user. The spacecraft could be triggered via HAM radio communications by those with an amateur license. LightCube is commanded with a DTMF sequence, and reports telemetry using RTTY, an AFSK modulation scheme and is decoded with a custom G… ▽ More LightCube is a 1U educational CubeSat which had the goal of connecting the public with space by producing a flash visible to the naked eye on command by a public user. The spacecraft could be triggered via HAM radio communications by those with an amateur license. LightCube is commanded with a DTMF sequence, and reports telemetry using RTTY, an AFSK modulation scheme and is decoded with a custom GNURadio-companion flowgraph. Several radio applications were written, including a from-scratch decoder written for educational purposes and one optimized to be compatible with the SatNOGS environment. Lightcube deployed from the International Space Station on April 24th 2023 and operated for 24 hours before suffering a battery failure. During this time it was tracked by many amateurs around the world with observations reported to the SatNOGs database. Audio observations of the beacons were subsequently decoded by the student team and by amateurs. Having received many observations from around the world, the team has been able to reconstruct the sequence of events leading to loss of communications. △ Less

Submitted 15 September, 2023; originally announced September 2023.

Development and validation of an interpretable machine learning-based calculator for predicting 5-year weight trajectories after bariatric surgery: a multinational retrospective cohort SOPHIA study

Authors: Patrick Saux, Pierre Bauvin, Violeta Raverdy, Julien Teigny, Hélène Verkindt, Tomy Soumphonphakdy, Maxence Debert, Anne Jacobs, Daan Jacobs, Valerie Monpellier, Phong Ching Lee, Chin Hong Lim, Johanna C Andersson-Assarsson, Lena Carlsson, Per-Arne Svensson, Florence Galtier, Guelareh Dezfoulian, Mihaela Moldovanu, Severine Andrieux, Julien Couster, Marie Lepage, Erminia Lembo, Ornella Verrastro, Maud Robert, Paulina Salminen , et al. (9 additional authors not shown)

Abstract: Background Weight loss trajectories after bariatric surgery vary widely between individuals, and predicting weight loss before the operation remains challenging. We aimed to develop a model using machine learning to provide individual preoperative prediction of 5-year weight loss trajectories after surgery. Methods In this multinational retrospective observational study we enrolled adult participa… ▽ More Background Weight loss trajectories after bariatric surgery vary widely between individuals, and predicting weight loss before the operation remains challenging. We aimed to develop a model using machine learning to provide individual preoperative prediction of 5-year weight loss trajectories after surgery. Methods In this multinational retrospective observational study we enrolled adult participants (aged $\ge$18 years) from ten prospective cohorts (including ABOS [NCT01129297], BAREVAL [NCT02310178], the Swedish Obese Subjects study, and a large cohort from the Dutch Obesity Clinic [Nederlandse Obesitas Kliniek]) and two randomised trials (SleevePass [NCT00793143] and SM-BOSS [NCT00356213]) in Europe, the Americas, and Asia, with a 5 year followup after Roux-en-Y gastric bypass, sleeve gastrectomy, or gastric band. Patients with a previous history of bariatric surgery or large delays between scheduled and actual visits were excluded. The training cohort comprised patients from two centres in France (ABOS and BAREVAL). The primary outcome was BMI at 5 years. A model was developed using least absolute shrinkage and selection operator to select variables and the classification and regression trees algorithm to build interpretable regression trees. The performances of the model were assessed through the median absolute deviation (MAD) and root mean squared error (RMSE) of BMI. Findings10 231 patients from 12 centres in ten countries were included in the analysis, corresponding to 30 602 patient-years. Among participants in all 12 cohorts, 7701 (75$\bullet$3%) were female, 2530 (24$\bullet$7%) were male. Among 434 baseline attributes available in the training cohort, seven variables were selected: height, weight, intervention type, age, diabetes status, diabetes duration, and smoking status. At 5 years, across external testing cohorts the overall mean MAD BMI was 2$\bullet$8 kg/m${}^2$ (95% CI 2$\bullet$6-3$\bullet$0) and mean RMSE BMI was 4$\bullet$7 kg/m${}^2$ (4$\bullet$4-5$\bullet$0), and the mean difference between predicted and observed BMI was-0$\bullet$3 kg/m${}^2$ (SD 4$\bullet$7). This model is incorporated in an easy to use and interpretable web-based prediction tool to help inform clinical decision before surgery. InterpretationWe developed a machine learning-based model, which is internationally validated, for predicting individual 5-year weight loss trajectories after three common bariatric interventions. △ Less

Submitted 31 August, 2023; originally announced August 2023.

Comments: The Lancet Digital Health, 2023

Fitting for the energy levels of hydrogen

Authors: David M. Jacobs, Marko Horbatsch

Abstract: Atomic hydrogen energy levels calculated to high precision are required to assist experimental researchers working on spectroscopy in the pursuit of testing quantum electrodynamics (QED) and probing for physics beyond the Standard Model. There are two important parts to the problem of computing these levels: an accurate evaluation of contributions from QED and using an accurate value for the proto… ▽ More Atomic hydrogen energy levels calculated to high precision are required to assist experimental researchers working on spectroscopy in the pursuit of testing quantum electrodynamics (QED) and probing for physics beyond the Standard Model. There are two important parts to the problem of computing these levels: an accurate evaluation of contributions from QED and using an accurate value for the proton charge radius as an input. Recent progress on QED corrections to the fine structure, as well as increasing evidence that a proton charge radius in the range of 0.84 fm is favored over the previously adopted larger value in the 0.88 fm range, has advanced the field, yet several state-of-the-art measurements remain in contradiction with this smaller value. Motivated by on-going and future work in this area, we present here a simple parameterization for the energy levels of hydrogen at the level of hyperfine structure using the so-called relativistic Ritz approach. The fitting of a finite sample of QED-generated levels at low to intermediate principal quantum number, $n$, gives a generally applicable formula for \emph{all} values of $n$ for each distinct angular momentum channel, given in this work up to orbital angular momentum number $\ell=30$. We also provide a simple linear parameterization for the shift in hydrogen energy levels as a function of the proton radius, providing a useful cross check for extant and future measured energy intervals. △ Less

Submitted 27 August, 2023; originally announced August 2023.

Comments: 6 pages of main text, 3 figures. Accepted by J. Phys. B

Journal ref: J. Phys. B: At. Mol. Opt. Phys. 56 185002 (2023)

Computational Long Exposure Mobile Photography

Authors: Eric Tabellion, Nikhil Karnad, Noa Glaser, Ben Weiss, David E. Jacobs, Yael Pritch

Abstract: Long exposure photography produces stunning imagery, representing moving elements in a scene with motion-blur. It is generally employed in two modalities, producing either a foreground or a background blur effect. Foreground blur images are traditionally captured on a tripod-mounted camera and portray blurred moving foreground elements, such as silky water or light trails, over a perfectly sharp b… ▽ More Long exposure photography produces stunning imagery, representing moving elements in a scene with motion-blur. It is generally employed in two modalities, producing either a foreground or a background blur effect. Foreground blur images are traditionally captured on a tripod-mounted camera and portray blurred moving foreground elements, such as silky water or light trails, over a perfectly sharp background landscape. Background blur images, also called panning photography, are captured while the camera is tracking a moving subject, to produce an image of a sharp subject over a background blurred by relative motion. Both techniques are notoriously challenging and require additional equipment and advanced skills. In this paper, we describe a computational burst photography system that operates in a hand-held smartphone camera app, and achieves these effects fully automatically, at the tap of the shutter button. Our approach first detects and segments the salient subject. We track the scene motion over multiple frames and align the images in order to preserve desired sharpness and to produce aesthetically pleasing motion streaks. We capture an under-exposed burst and select the subset of input frames that will produce blur trails of controlled length, regardless of scene or camera motion velocity. We predict inter-frame motion and synthesize motion-blur to fill the temporal gaps between the input frames. Finally, we composite the blurred image with the sharp regular exposure to protect the sharpness of faces or areas of the scene that are barely moving, and produce a final high resolution and high dynamic range (HDR) photograph. Our system democratizes a capability previously reserved to professionals, and makes this creative style accessible to most casual photographers. More information and supplementary material can be found on our project webpage: https://motion-mode.github.io/ △ Less

Submitted 2 August, 2023; originally announced August 2023.

Comments: 15 pages, 17 figures

ACM Class: I.4; I.3.3; I.2.10

Journal ref: ACM Trans. Graph. 42, 4, Article 48 (August 2023)

Why soft contacts are stickier when breaking than when making them

Authors: Antoine Sanner, Nityanshu Kumar, Ali Dhinojwala, Tevis D. B. Jacobs, Lars Pastewka

Abstract: Insects, pick-and-place manufacturing, engineered adhesives, and soft robots employ soft materials to stick to surfaces even in the presence of roughness. Experiments show that the force required for making contact is lower than for releasing it, a phenomenon known as the adhesion hysteresis. The common explanation for this hysteresis is either contact aging or viscoelasticity. Here, we show that… ▽ More Insects, pick-and-place manufacturing, engineered adhesives, and soft robots employ soft materials to stick to surfaces even in the presence of roughness. Experiments show that the force required for making contact is lower than for releasing it, a phenomenon known as the adhesion hysteresis. The common explanation for this hysteresis is either contact aging or viscoelasticity. Here, we show that adhesion hysteresis emerges even for perfectly elastic contacts and in the absence of contact aging and viscoelasticity because of surface roughness. We present a crack-perturbation model and experimental observations that reveal discrete jumps of the contact perimeter. These stick-slip instabilities are triggered by local differences in fracture energy between roughness peaks and valleys. Pinning of the contact perimeter retards both its advancement when coming into contact and its retraction when pulling away. Our model quantitatively reproduces the hysteresis observed in experiments and allows us to derive analytical predictions for its magnitude, accounting for realistic rough geometries across orders of magnitude in length scale. Our results explain why adhesion hysteresis is ubiquitous and reveal why soft pads in nature and engineering are efficient in adhering even to surfaces with significant roughness. △ Less

Submitted 26 July, 2023; originally announced July 2023.

Comments: 38 pages, 11 figures

The Completely Hackable Amateur Radio Telescope (CHART) Project

Authors: Lindsay M. Berkhout, Adam P. Beardsley, Daniel C. Jacobs, Raven Braithwaite, Bryanna Gutierrez-Coatney, Arib Islam, Ahlea Wright

Abstract: We present the Completely Hackable Amateur Radio Telescope (CHART), a project that provides hands-on radio instrumentation and design experience to undergraduates while bringing accessible radio astronomy experiments to high school students and teachers. Here we describe a system which can detect 21-cm emission from the Milky Way which is optimized for cost and simplicity of construction. Software… ▽ More We present the Completely Hackable Amateur Radio Telescope (CHART), a project that provides hands-on radio instrumentation and design experience to undergraduates while bringing accessible radio astronomy experiments to high school students and teachers. Here we describe a system which can detect 21-cm emission from the Milky Way which is optimized for cost and simplicity of construction. Software, documentation, and tutorials are all completely open source to improve the user experience and facilitate community involvement. We demonstrate the design with several observations which we compare with state-of-the-art surveys. The system is shown to detect galactic 21-cm emission in both rural and urban settings. △ Less

Submitted 27 November, 2023; v1 submitted 20 July, 2023; originally announced July 2023.

Journal ref: Phys. Educ. 59 015020 (2024)

Measured Albedo in the Wild: Filling the Gap in Intrinsics Evaluation

Authors: Jiaye Wu, Sanjoy Chowdhury, Hariharmano Shanmugaraja, David Jacobs, Soumyadip Sengupta

Abstract: Intrinsic image decomposition and inverse rendering are long-standing problems in computer vision. To evaluate albedo recovery, most algorithms report their quantitative performance with a mean Weighted Human Disagreement Rate (WHDR) metric on the IIW dataset. However, WHDR focuses only on relative albedo values and often fails to capture overall quality of the albedo. In order to comprehensively… ▽ More Intrinsic image decomposition and inverse rendering are long-standing problems in computer vision. To evaluate albedo recovery, most algorithms report their quantitative performance with a mean Weighted Human Disagreement Rate (WHDR) metric on the IIW dataset. However, WHDR focuses only on relative albedo values and often fails to capture overall quality of the albedo. In order to comprehensively evaluate albedo, we collect a new dataset, Measured Albedo in the Wild (MAW), and propose three new metrics that complement WHDR: intensity, chromaticity and texture metrics. We show that existing algorithms often improve WHDR metric but perform poorly on other metrics. We then finetune different algorithms on our MAW dataset to significantly improve the quality of the reconstructed albedo both quantitatively and qualitatively. Since the proposed intensity, chromaticity, and texture metrics and the WHDR are all complementary we further introduce a relative performance measure that captures average performance. By analysing existing algorithms we show that there is significant room for improvement. Our dataset and evaluation metrics will enable researchers to develop algorithms that improve albedo reconstruction. Code and Data available at: https://measuredalbedo.github.io/ △ Less

Submitted 29 June, 2023; v1 submitted 27 June, 2023; originally announced June 2023.

Comments: Accepted into ICCP2023

Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models

Authors: Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, Yogesh Balaji

Abstract: Despite tremendous progress in generating high-quality images using diffusion models, synthesizing a sequence of animated frames that are both photorealistic and temporally coherent is still in its infancy. While off-the-shelf billion-scale datasets for image generation are available, collecting similar video data of the same scale is still challenging. Also, training a video diffusion model is co… ▽ More Despite tremendous progress in generating high-quality images using diffusion models, synthesizing a sequence of animated frames that are both photorealistic and temporally coherent is still in its infancy. While off-the-shelf billion-scale datasets for image generation are available, collecting similar video data of the same scale is still challenging. Also, training a video diffusion model is computationally much more expensive than its image counterpart. In this work, we explore finetuning a pretrained image diffusion model with video data as a practical solution for the video synthesis task. We find that naively extending the image noise prior to video noise prior in video diffusion leads to sub-optimal performance. Our carefully designed video noise prior leads to substantially better performance. Extensive experimental validation shows that our model, Preserve Your Own Correlation (PYoCo), attains SOTA zero-shot text-to-video results on the UCF-101 and MSR-VTT benchmarks. It also achieves SOTA video generation quality on the small-scale UCF-101 benchmark with a $10\times$ smaller model using significantly less computation than the prior art. △ Less

Submitted 25 March, 2024; v1 submitted 17 May, 2023; originally announced May 2023.

Comments: ICCV 2023. Project webpage: https://research.nvidia.com/labs/dir/pyoco

HaLP: Hallucinating Latent Positives for Skeleton-based Self-Supervised Learning of Actions

Authors: Anshul Shah, Aniket Roy, Ketul Shah, Shlok Kumar Mishra, David Jacobs, Anoop Cherian, Rama Chellappa

Abstract: Supervised learning of skeleton sequence encoders for action recognition has received significant attention in recent times. However, learning such encoders without labels continues to be a challenging problem. While prior works have shown promising results by applying contrastive learning to pose sequences, the quality of the learned representations is often observed to be closely tied to data au… ▽ More Supervised learning of skeleton sequence encoders for action recognition has received significant attention in recent times. However, learning such encoders without labels continues to be a challenging problem. While prior works have shown promising results by applying contrastive learning to pose sequences, the quality of the learned representations is often observed to be closely tied to data augmentations that are used to craft the positives. However, augmenting pose sequences is a difficult task as the geometric constraints among the skeleton joints need to be enforced to make the augmentations realistic for that action. In this work, we propose a new contrastive learning approach to train models for skeleton-based action recognition without labels. Our key contribution is a simple module, HaLP - to Hallucinate Latent Positives for contrastive learning. Specifically, HaLP explores the latent space of poses in suitable directions to generate new positives. To this end, we present a novel optimization formulation to solve for the synthetic positives with an explicit control on their hardness. We propose approximations to the objective, making them solvable in closed form with minimal overhead. We show via experiments that using these generated positives within a standard contrastive learning framework leads to consistent improvements across benchmarks such as NTU-60, NTU-120, and PKU-II on tasks like linear evaluation, transfer learning, and kNN evaluation. Our code will be made available at https://github.com/anshulbshah/HaLP. △ Less

Submitted 1 April, 2023; originally announced April 2023.

Comments: To be presented at CVPR 2023

LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentation

Authors: Koutilya Pnvr, Bharat Singh, Pallabi Ghosh, Behjat Siddiquie, David Jacobs

Abstract: Large-scale pre-training tasks like image classification, captioning, or self-supervised techniques do not incentivize learning the semantic boundaries of objects. However, recent generative foundation models built using text-based latent diffusion techniques may learn semantic boundaries. This is because they have to synthesize intricate details about all objects in an image based on a text descr… ▽ More Large-scale pre-training tasks like image classification, captioning, or self-supervised techniques do not incentivize learning the semantic boundaries of objects. However, recent generative foundation models built using text-based latent diffusion techniques may learn semantic boundaries. This is because they have to synthesize intricate details about all objects in an image based on a text description. Therefore, we present a technique for segmenting real and AI-generated images using latent diffusion models (LDMs) trained on internet-scale datasets. First, we show that the latent space of LDMs (z-space) is a better input representation compared to other feature representations like RGB images or CLIP encodings for text-based image segmentation. By training the segmentation models on the latent z-space, which creates a compressed representation across several domains like different forms of art, cartoons, illustrations, and photographs, we are also able to bridge the domain gap between real and AI-generated images. We show that the internal features of LDMs contain rich semantic information and present a technique in the form of LD-ZNet to further boost the performance of text-based segmentation. Overall, we show up to 6% improvement over standard baselines for text-to-image segmentation on natural images. For AI-generated imagery, we show close to 20% improvement compared to state-of-the-art techniques. The project is available at https://koutilya-pnvr.github.io/LD-ZNet/. △ Less

Submitted 23 August, 2023; v1 submitted 22 March, 2023; originally announced March 2023.

Comments: Supplementary material is included in the paper following the references section

Search for the Epoch of Reionisation with HERA: Upper Limits on the Closure Phase Delay Power Spectrum

Authors: Pascal M. Keller, Bojan Nikolic, Nithyanandan Thyagarajan, Chris L. Carilli, Gianni Bernardi, Ntsikelelo Charles, Landman Bester, Oleg M. Smirnov, Nicholas S. Kern, Joshua S. Dillon, Bryna J. Hazelton, Miguel F. Morales, Daniel C. Jacobs, Aaron R. Parsons, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley , et al. (58 additional authors not shown)

Abstract: Radio interferometers aiming to measure the power spectrum of the redshifted 21 cm line during the Epoch of Reionisation (EoR) need to achieve an unprecedented dynamic range to separate the weak signal from overwhelming foreground emissions. Calibration inaccuracies can compromise the sensitivity of these measurements to the effect that a detection of the EoR is precluded. An alternative to standa… ▽ More Radio interferometers aiming to measure the power spectrum of the redshifted 21 cm line during the Epoch of Reionisation (EoR) need to achieve an unprecedented dynamic range to separate the weak signal from overwhelming foreground emissions. Calibration inaccuracies can compromise the sensitivity of these measurements to the effect that a detection of the EoR is precluded. An alternative to standard analysis techniques makes use of the closure phase, which allows one to bypass antenna-based direction-independent calibration. Similarly to standard approaches, we use a delay spectrum technique to search for the EoR signal. Using 94 nights of data observed with Phase I of the Hydrogen Epoch of Reionization Array (HERA), we place approximate constraints on the 21 cm power spectrum at $z=7.7$. We find at 95% confidence that the 21 cm EoR brightness temperature is $\le$(372)$^2$ "pseudo" mK$^2$ at 1.14 "pseudo" $h$ Mpc$^{-1}$, where the "pseudo" emphasises that these limits are to be interpreted as approximations to the actual distance scales and brightness temperatures. Using a fiducial EoR model, we demonstrate the feasibility of detecting the EoR with the full array. Compared to standard methods, the closure phase processing is relatively simple, thereby providing an important independent check on results derived using visibility intensities, or related. △ Less

Submitted 15 February, 2023; originally announced February 2023.

Comments: 16 pages, 14 figures, accepted for publication by MNRAS

Hyperbolic Contrastive Learning for Visual Representations beyond Objects

Authors: Songwei Ge, Shlok Mishra, Simon Kornblith, Chun-Liang Li, David Jacobs

Abstract: Although self-/un-supervised methods have led to rapid progress in visual representation learning, these methods generally treat objects and scenes using the same lens. In this paper, we focus on learning representations for objects and scenes that preserve the structure among them. Motivated by the observation that visually similar objects are close in the representation space, we argue that th… ▽ More Although self-/un-supervised methods have led to rapid progress in visual representation learning, these methods generally treat objects and scenes using the same lens. In this paper, we focus on learning representations for objects and scenes that preserve the structure among them. Motivated by the observation that visually similar objects are close in the representation space, we argue that the scenes and objects should instead follow a hierarchical structure based on their compositionality. To exploit such a structure, we propose a contrastive learning framework where a Euclidean loss is used to learn object representations and a hyperbolic loss is used to encourage representations of scenes to lie close to representations of their constituent objects in a hyperbolic space. This novel hyperbolic objective encourages the scene-object hypernymy among the representations by optimizing the magnitude of their norms. We show that when pretraining on the COCO and OpenImages datasets, the hyperbolic loss improves downstream performance of several baselines across multiple datasets and tasks, including image classification, object detection, and semantic segmentation. We also show that the properties of the learned representations allow us to solve various vision tasks that involve the interaction between scenes and objects in a zero-shot fashion. Our code can be found at \url{https://github.com/shlokk/HCL/tree/main/HCL}. △ Less

Submitted 1 December, 2022; originally announced December 2022.

The Star-Planet Activity Research CubeSat (SPARCS): Determining Inputs to Planetary Habitability

Authors: David R. Ardila, Evgenya Shkolnik, Paul Scowen, Daniel Jacobs, Dawn Gregory, Travis Barman, Christopher Basset, Judd Bowman, Samuel Cheng, Jonathan Gamaut, Logan Jensen, April Jewell, Mary Knapp, Matthew Kolopanis, Joseph Llama, R. O. Parke Loyd, Victoria Meadows, Shouleh Nikzad, Sara Peacock, Tahina Ramiaramanantsoa, Nathaniel Struebel, Mark Swain

Abstract: Seventy-five billion low-mass stars in our galaxy host at least one small planet in their habitable zone (HZ). The stellar ultraviolet (UV) radiation received by the planets is strong and highly variable, and has consequences for atmospheric loss, composition, and habitability. SPARCS is a NASA-funded mission to characterize the quiescent and flare UV emission from low-mass stars, by observing 1… ▽ More Seventy-five billion low-mass stars in our galaxy host at least one small planet in their habitable zone (HZ). The stellar ultraviolet (UV) radiation received by the planets is strong and highly variable, and has consequences for atmospheric loss, composition, and habitability. SPARCS is a NASA-funded mission to characterize the quiescent and flare UV emission from low-mass stars, by observing 10 to 20 low-mass stars, over timescales of days, simultaneously in two UV bands: 153-171 nm and 260-300 nm. SPARCS Sun-synchronous terminator orbit allows for long periods of uninterrupted observations, reaching 10s of days for some targets. The payload consists of a 10 cm-class telescope, a dichroic element, UV detectors and associated electronics, a thermal control system, and an on-board processor. The payload is hosted on a Blue Canyon Technologies 6U CubeSat. SPARCS hosts several technology innovations that have broad applicability to other missions. The payload demonstrates the use of "2D-doped" (i.e., delta- and superlattice-doped) detectors and detector-integrated metal dielectric filters in space. This detector technology provides ~5x larger quantum efficiency than NASA's GALEX detectors. In addition, SPARCS' payload processor provides dynamic exposure control, automatically adjusting the exposure time to avoid flare saturation and to time-resolve the strongest stellar flares. A simple passive cooling system maintains the detector temperature under 238K to minimize dark current. The spacecraft bus provides pointing jitter smaller than 6", minimizing the impact of flat-field errors, dark current, and read-noise. All these elements enable competitive astrophysics science within a CubeSat platform. SPARCS is currently in the final design and fabrication phase (Phase C in the NASA context). It will be launched in 2024, for a primary science mission of one year. △ Less

Submitted 10 November, 2022; originally announced November 2022.

Comments: Presented at the 73rd International Astronautical Congress, 18-22 September 2022, Paris, France

A simple, efficient and scalable contrastive masked autoencoder for learning visual representations

Authors: Shlok Mishra, Joshua Robinson, Huiwen Chang, David Jacobs, Aaron Sarna, Aaron Maschinot, Dilip Krishnan

Abstract: We introduce CAN, a simple, efficient and scalable method for self-supervised learning of visual representations. Our framework is a minimal and conceptually clean synthesis of (C) contrastive learning, (A) masked autoencoders, and (N) the noise prediction approach used in diffusion models. The learning mechanisms are complementary to one another: contrastive learning shapes the embedding space ac… ▽ More We introduce CAN, a simple, efficient and scalable method for self-supervised learning of visual representations. Our framework is a minimal and conceptually clean synthesis of (C) contrastive learning, (A) masked autoencoders, and (N) the noise prediction approach used in diffusion models. The learning mechanisms are complementary to one another: contrastive learning shapes the embedding space across a batch of image samples; masked autoencoders focus on reconstruction of the low-frequency spatial correlations in a single image sample; and noise prediction encourages the reconstruction of the high-frequency components of an image. The combined approach results in a robust, scalable and simple-to-implement algorithm. The training process is symmetric, with 50% of patches in both views being masked at random, yielding a considerable efficiency improvement over prior contrastive learning methods. Extensive empirical studies demonstrate that CAN achieves strong downstream performance under both linear and finetuning evaluations on transfer learning and robustness tasks. CAN outperforms MAE and SimCLR when pre-training on ImageNet, but is especially useful for pre-training on larger uncurated datasets such as JFT-300M: for linear probe on ImageNet, CAN achieves 75.4% compared to 73.4% for SimCLR and 64.1% for MAE. The finetuned performance on ImageNet of our ViT-L model is 86.1%, compared to 85.5% for SimCLR, and 85.4% for MAE. The overall FLOPs load of SimCLR is 70% higher than CAN for ViT-L models. △ Less

Submitted 30 October, 2022; originally announced October 2022.

Comments: Mishra and Robinson contributed equally

Characterization Of Inpaint Residuals In Interferometric Measurements of the Epoch Of Reionization

Authors: Michael Pagano, Jing Liu, Adrian Liu, Nicholas S. Kern, Aaron Ewall-Wice, Philip Bull, Robert Pascua, Siamak Ravanbakhsh, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Steven Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer , et al. (53 additional authors not shown)

Abstract: Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum du… ▽ More Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that capable of inpainting RFI corrupted data in interferometric instruments. We train our network on simulated data and show that our network is capable at inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modeling are best suited for inpainting over narrowband RFI. We also show that with our fiducial parameters Discrete Prolate Spheroidal Sequences (DPSS) and CLEAN provide the best performance for intermittent ``narrowband'' RFI while Gaussian Progress Regression (GPR) and Least Squares Spectral Analysis (LSSA) provide the best performance for larger RFI gaps. However we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We find these results to be consistent in both simulated and real visibilities. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that in the future, as the noise level of the data comes down, CLEAN and DPSS are most capable of reproducing the fine frequency structure in the visibilities of HERA data. △ Less

Submitted 20 February, 2023; v1 submitted 26 October, 2022; originally announced October 2022.

Comments: 21 pages, 13 figures

New EoR Power Spectrum Limits From MWA Phase II Using the Delay Spectrum Method and Novel Systematic Rejection

Authors: Matthew Kolopanis, Jonathan Pober, Daniel C. Jacobs, Samantha McGraw

Abstract: We present an analysis of Epoch of Reionization data from Phase II of the Murchison Widefield Array using the \texttt{simpleDS} delay spectrum pipeline. Prior work analyzed the same observations using the FHD/$\varepsilon$ppsilon imaging pipeline, and so the present analysis represents the first time that both principal types of 21 cm cosmology power spectrum estimation approaches have been applie… ▽ More We present an analysis of Epoch of Reionization data from Phase II of the Murchison Widefield Array using the \texttt{simpleDS} delay spectrum pipeline. Prior work analyzed the same observations using the FHD/$\varepsilon$ppsilon imaging pipeline, and so the present analysis represents the first time that both principal types of 21 cm cosmology power spectrum estimation approaches have been applied to the same data set. Our limits on the 21 cm power spectrum amplitude span a range in $k$ space of $|k| < 1~h_{100}{\rm Mpc}^{-1}$ with a lowest measurement of $Δ^2(k) \leq$ $4.58\times10^3$ mK$^2$ at $k = 0.190 h_{100}\rm{Mpc}^{-1}$ and $z = 7.14$. In order to achieve these limits, we need to mitigate a previously unidentified common mode systematic in the data set. If not accounted for, this systematic introduces an overall \emph{negative} bias that can make foreground contaminated measurements appear as stringent, noise-limited constraints on the 21 cm signal amplitude. The identification of this systematic highlights the risk in modeling systematics as positive-definite contributions to the power spectrum and in ``conservatively'' interpreting all measurements as upper limits. △ Less

Submitted 4 April, 2023; v1 submitted 19 October, 2022; originally announced October 2022.

Comments: 19 pages, 12 figures, Accepted by MNRAS; Updated to match published version

Journal ref: MNRAS Volume 521, Issue 4, June 2023, Page 5120

Improved Constraints on the 21 cm EoR Power Spectrum and the X-Ray Heating of the IGM with HERA Phase I Observations

Authors: The HERA Collaboration, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Rennan Barkana, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Daniela Breitman, Philip Bull, Jacob Burba, Steve Carey, Chris L. Carilli, Carina Cheng, Samir Choudhuri, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Joshua S. Dillon , et al. (70 additional authors not shown)

Abstract: We report the most sensitive upper limits to date on the 21 cm epoch of reionization power spectrum using 94 nights of observing with Phase I of the Hydrogen Epoch of Reionization Array (HERA). Using similar analysis techniques as in previously reported limits (HERA Collaboration 2022a), we find at 95% confidence that $Δ^2(k = 0.34$ $h$ Mpc$^{-1}$) $\leq 457$ mK$^2$ at $z = 7.9$ and that… ▽ More We report the most sensitive upper limits to date on the 21 cm epoch of reionization power spectrum using 94 nights of observing with Phase I of the Hydrogen Epoch of Reionization Array (HERA). Using similar analysis techniques as in previously reported limits (HERA Collaboration 2022a), we find at 95% confidence that $Δ^2(k = 0.34$ $h$ Mpc$^{-1}$) $\leq 457$ mK$^2$ at $z = 7.9$ and that $Δ^2 (k = 0.36$ $h$ Mpc$^{-1}) \leq 3,496$ mK$^2$ at $z = 10.4$, an improvement by a factor of 2.1 and 2.6 respectively. These limits are mostly consistent with thermal noise over a wide range of $k$ after our data quality cuts, despite performing a relatively conservative analysis designed to minimize signal loss. Our results are validated with both statistical tests on the data and end-to-end pipeline simulations. We also report updated constraints on the astrophysics of reionization and the cosmic dawn. Using multiple independent modeling and inference techniques previously employed by HERA Collaboration (2022b), we find that the intergalactic medium must have been heated above the adiabatic cooling limit at least as early as $z = 10.4$, ruling out a broad set of so-called "cold reionization" scenarios. If this heating is due to high-mass X-ray binaries during the cosmic dawn, as is generally believed, our result's 99% credible interval excludes the local relationship between soft X-ray luminosity and star formation and thus requires heating driven by evolved low-metallicity stars. △ Less

Submitted 19 January, 2023; v1 submitted 10 October, 2022; originally announced October 2022.

Comments: 57 pages, 37 figures. Updated to match the accepted ApJ version. Corresponding author: Joshua S. Dillon

Journal ref: 2023 ApJ 945 124

Impact of instrument and data characteristics in the interferometric reconstruction of the 21 cm power spectrum

Authors: Adélie Gorce, Samskruthi Ganjam, Adrian Liu, Steven G. Murray, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Jacob Burba, Steven Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Joshua S. Dillon , et al. (53 additional authors not shown)

Abstract: Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand t… ▽ More Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand the power measured by an interferometer, we assess the impact of instrument characteristics and analysis choices on these window functions. Focusing on the Hydrogen Epoch of Reionization Array (HERA) as a case study, we find that long-baseline observations correspond to enhanced low-k tails of the window functions, which facilitate foreground leakage, whilst an informed choice of bandwidth and frequency taper can reduce said tails. With simple test cases and realistic simulations, we show that, apart from tracing mode mixing, the window functions help accurately reconstruct the power spectrum estimator of simulated visibilities. The window functions depend strongly on the beam chromaticity, and less on its spatial structure - a Gaussian approximation, ignoring side lobes, is sufficient. Finally, we investigate the potential of asymmetric window functions, down-weighting the contribution of low-k power to avoid foreground leakage. The window functions presented here correspond to the latest HERA upper limits for the full Phase I data. They allow an accurate reconstruction of the power spectrum measured by the instrument and will be used in future analyses to confront theoretical models and data directly in cylindrical space. △ Less

Submitted 11 January, 2023; v1 submitted 7 October, 2022; originally announced October 2022.

Comments: 18 pages, 19 figures, accepted for publication in MNRAS

Improved accuracy and reproducibility of coronary artery cal-cification features using deconvolution

Authors: Yingnan Song, Ammar Hoori, Hao Wu, Mani Vembar, Sadeer Al-Kindi, Leslie Ciancibello, James G. Terry, David R. Jacobs Jr, John Jeffrey Carr, David L. Wilson

Abstract: Our long-range goal is to improve current whole-heart CT calcium score by extracting quantitative features from individual calcifications. We performed deconvolution to improve small calcifications assessment which challenge conventional CT calcium score scanning resolution. We analyzed features of individual calcifications on repeated standard (2.5-mm) and thin (1.25-mm) slice scans from QRM-Card… ▽ More Our long-range goal is to improve current whole-heart CT calcium score by extracting quantitative features from individual calcifications. We performed deconvolution to improve small calcifications assessment which challenge conventional CT calcium score scanning resolution. We analyzed features of individual calcifications on repeated standard (2.5-mm) and thin (1.25-mm) slice scans from QRM-Cardio phantom, cadaver hearts, and CARDIA study participants. Pre-processing to improve resolution involved of Lucy-Richardson deconvolution with a measured PSF or 3D blind deconvolution where the PSF was iteratively optimized on high detail structures like calcifications in the images. Using QRM with inserts having known mg-calcium, we determined that both blind and conventional deconvolution improved mass measurements nearly equally well on standard images. Further, de-convolved thin images gave excellent recovery of actual mass scores, suggesting that such processing could be our gold standard. For CARDIA images, blind deconvolution greatly improved results on standard slices. Accuracy across 33 calcifications (without, with deconvolution) was (23%,9%), (18%,1%), and (-19%,-1%), for Agatston, volume, and mass scores, respectively. Reproducibility was (0.13,0.10), (0.12,0.08), and (0.11,0.06), respectively. Mass scores were more reproducible than Agatston scores or vol-ume scores. Cadaver volumes showed similar improvements in accuracy/reproducibility and slightly better results with a measured PSF. For many other calcification features in CARDIA data, blind deconvolution improved reproducibility in 21 out of 24 features. Deconvolution improves accuracy and reproducibility of multiple features extracted from individual calcifications in CT calcium score exam. Blind deconvolution improves feature assessments of coronary calcification in archived datasets. △ Less

Submitted 12 July, 2022; originally announced July 2022.

Comments: submitted to Journal of Medical Imaging

The surface-topography challenge: Problem definition

Authors: Tevis D. B. Jacobs, Nathaniel Miller, Martin H. Müser, Lars Pastewka

Abstract: We present to the community a surface-definition problem, whose solution we consider to be critical for the proper description of contacts between nominally flat surfaces [1,2]. In 2015, Müser and Dapp issued the Contact Mechanics Challenge, which provided complete topography data for a fictional surface and asked theorists and modelers to compute the expected contact parameters for such a surface… ▽ More We present to the community a surface-definition problem, whose solution we consider to be critical for the proper description of contacts between nominally flat surfaces [1,2]. In 2015, Müser and Dapp issued the Contact Mechanics Challenge, which provided complete topography data for a fictional surface and asked theorists and modelers to compute the expected contact parameters for such a surface. This effort was a success, but exposed one glaring flaw in the community's understanding of the nature of contact: these models require as input a complete description of surface topography, which is rarely or never available for real-world surfaces [3-6]. The present challenge is to experimentalists: we will send you samples of two materials (one smoother and one rougher); you determine the surface topography of these materials. We call on you to measure such surfaces however you wish, using contact-based techniques, light scattering, microscopy, or other techniques. Examples of quantities of interest are: root-mean-square (RMS) parameters; the power spectral density (PSD); or the autocorrelation function (ACF). For the material, we have chosen chromium nitride, a wear- and corrosion-resistant coating used in industrial applications including automotive components, cutting tools, and die-casting. To participate, simply go to: https://contact.engineering/challenge to provide your shipping address and other information, then samples will be shipped out to you. The only requirement of participation is that your raw topography measurements are deposited on the free contact.engineering web app to facilitate data sharing. The purpose of this challenge is for our community to move towards: (a) better agreement on how to describe the multi-scale topography of experimental surfaces; and (b) better understanding of how to apply the well-developed models and theories to real-world surfaces. △ Less

Submitted 27 June, 2022; originally announced June 2022.

Comments: 5 pages, 0 figures

Relativistic Ritz approach to hydrogen-like atoms II: spectral analysis of hydrogen and deuterium

Authors: David M. Jacobs

Abstract: A long-distance effective theory of hydrogen-like atoms, dubbed the relativistic Ritz approach was recently introduced and some its theoretical consequences were explored. In this article, the relativistic Ritz approach is used to fit extant measurements of atomic hydrogen and deuterium transitions using information-theoretic analyses. As a result, the fine-structure constant ($α$), a fundamental… ▽ More A long-distance effective theory of hydrogen-like atoms, dubbed the relativistic Ritz approach was recently introduced and some its theoretical consequences were explored. In this article, the relativistic Ritz approach is used to fit extant measurements of atomic hydrogen and deuterium transitions using information-theoretic analyses. As a result, the fine-structure constant ($α$), a fundamental parameter of the Standard Model, may be determined simultaneously with the ionization energies of hydrogen and deuterium, $E_I^{\text{(H)}}$ and $E_I^{\text{(D)}}$. The best hydrogen analysis yields $α^{-1} = 137.035\,999\,185(25)$, in good agreement with the value obtained by other methods and without relying on a separately determined Rydberg constant. From the same analysis, I find that $E_I^{\text{(H)}} = 13.598\,434\,599\,684(25)\,\text{eV}$, an improvement of two orders of magnitude in precision compared to previous determinations and in agreement with the Standard Model prediction at 1.8 parts per trillion. The best deuterium analysis yields $E_I^{\text{(D)}} = 13.602\,134\,636\,543(31)\,\text{eV}$, agreeing with the Standard Model at 2.3 parts per trillion. This study demonstrates how the relativistic Ritz approach can be used for testing the Standard Model with the spectra of hydrogen-like atoms. △ Less

Submitted 10 November, 2022; v1 submitted 9 June, 2022; originally announced June 2022.

Comments: v2:14 pages of main text, 4 figures, no appendices; v1: 5 pages of main text, 5 pages of appendices

Autoregressive Perturbations for Data Poisoning

Authors: Pedro Sandoval-Segura, Vasu Singla, Jonas Geiping, Micah Goldblum, Tom Goldstein, David W. Jacobs

Abstract: The prevalence of data scraping from social media as a means to obtain datasets has led to growing concerns regarding unauthorized use of data. Data poisoning attacks have been proposed as a bulwark against scraping, as they make data "unlearnable" by adding small, imperceptible perturbations. Unfortunately, existing methods require knowledge of both the target architecture and the complete datase… ▽ More The prevalence of data scraping from social media as a means to obtain datasets has led to growing concerns regarding unauthorized use of data. Data poisoning attacks have been proposed as a bulwark against scraping, as they make data "unlearnable" by adding small, imperceptible perturbations. Unfortunately, existing methods require knowledge of both the target architecture and the complete dataset so that a surrogate network can be trained, the parameters of which are used to generate the attack. In this work, we introduce autoregressive (AR) poisoning, a method that can generate poisoned data without access to the broader dataset. The proposed AR perturbations are generic, can be applied across different datasets, and can poison different architectures. Compared to existing unlearnable methods, our AR poisons are more resistant against common defenses such as adversarial training and strong data augmentations. Our analysis further provides insight into what makes an effective data poison. △ Less

Submitted 13 October, 2022; v1 submitted 8 June, 2022; originally announced June 2022.

Comments: Accepted to NeurIPS 2022. Code available at https://github.com/psandovalsegura/autoregressive-poisoning

Stochastic estimation of Green's functions with application to diffusion and advection-diffusion-reaction problems

Authors: Russell G. Keanini, Jerry Dahlberg, Philip Brown, Mehdi Morovati, Hamidreza Moradi, Donald Jacobs, Peter T. Tkacik

Abstract: A stochastic method is described for estimating Green's functions (GF's), appropriate to linear advection-diffusion-reaction transport problems, evolving in arbitrary geometries. By allowing straightforward construction of approximate, though high-accuracy GF's, within any geometry, the technique solves the central challenge in obtaining Green's function solutions. In contrast to Monte Carlo solut… ▽ More A stochastic method is described for estimating Green's functions (GF's), appropriate to linear advection-diffusion-reaction transport problems, evolving in arbitrary geometries. By allowing straightforward construction of approximate, though high-accuracy GF's, within any geometry, the technique solves the central challenge in obtaining Green's function solutions. In contrast to Monte Carlo solutions of individual transport problems, subject to specific sets of conditions and forcing, the proposed technique produces approximate GF's that can be used: a) to obtain (infinite) sets of solutions, subject to any combination of (random and deterministic) boundary, initial, and internal forcing, b) as high fidelity direct models in inverse problems, and c) as high quality process models in thermal and mass transport design, optimization, and process control problems. △ Less

Submitted 11 May, 2022; originally announced June 2022.

Comments: 47 pages, 8 figures

MSC Class: 35Qxx (Primary); 60Gxx (Secondary) ACM Class: G.1; G.2; J.2

Relativistic Ritz approach to hydrogen-like atoms I: theoretical considerations

Authors: David M. Jacobs

Abstract: The Rydberg formula along with the Ritz quantum defect ansatz has been a standard theoretical tool used in atomic physics since before the advent of quantum mechanics, yet this approach has remained limited by its non-relativistic foundation. Here I present a long-distance relativistic effective theory describing hydrogen-like systems with arbitrary mass ratios, thereby extending the canonical Rit… ▽ More The Rydberg formula along with the Ritz quantum defect ansatz has been a standard theoretical tool used in atomic physics since before the advent of quantum mechanics, yet this approach has remained limited by its non-relativistic foundation. Here I present a long-distance relativistic effective theory describing hydrogen-like systems with arbitrary mass ratios, thereby extending the canonical Ritz-like approach. Fitting the relativistic theory to the hydrogen energy levels predicted by bound-state QED indicates that it is superior to the canonical, nonrelativistic approach. An analytic analysis reveals nonlinear consistency relations within the bound-state QED level predictions that relate higher-order corrections to those at lower order, providing guideposts for future perturbative calculations as well as insights into the asymptotic behavior of Bethe logarithms. Applications of the approach include fitting to atomic spectroscopic data, allowing for the determination the fine-structure constant from large spectral data sets and also to check for internal consistency of the data independently from bound-state QED. △ Less

Submitted 28 September, 2022; v1 submitted 6 June, 2022; originally announced June 2022.

Comments: v2: 11 pages of main text, 14 figures, 2 appendices

Journal ref: Phys. Rev. A 106, 062810 (2022)

Poisons that are learned faster are more effective

Authors: Pedro Sandoval-Segura, Vasu Singla, Liam Fowl, Jonas Geiping, Micah Goldblum, David Jacobs, Tom Goldstein

Abstract: Imperceptible poisoning attacks on entire datasets have recently been touted as methods for protecting data privacy. However, among a number of defenses preventing the practical use of these techniques, early-stopping stands out as a simple, yet effective defense. To gauge poisons' vulnerability to early-stopping, we benchmark error-minimizing, error-maximizing, and synthetic poisons in terms of p… ▽ More Imperceptible poisoning attacks on entire datasets have recently been touted as methods for protecting data privacy. However, among a number of defenses preventing the practical use of these techniques, early-stopping stands out as a simple, yet effective defense. To gauge poisons' vulnerability to early-stopping, we benchmark error-minimizing, error-maximizing, and synthetic poisons in terms of peak test accuracy over 100 epochs and make a number of surprising observations. First, we find that poisons that reach a low training loss faster have lower peak test accuracy. Second, we find that a current state-of-the-art error-maximizing poison is 7 times less effective when poison training is stopped at epoch 8. Third, we find that stronger, more transferable adversarial attacks do not make stronger poisons. We advocate for evaluating poisons in terms of peak test accuracy. △ Less

Submitted 18 April, 2022; originally announced April 2022.

Comments: 8 pages, 4 figures. Accepted to CVPR 2022 Art of Robustness Workshop

Direct Optimal Mapping for 21cm Cosmology: A Demonstration with the Hydrogen Epoch of Reionization Array

Authors: Zhilei Xu, Jacqueline N. Hewitt, Kai-Feng Chen, Honggeun Kim, Joshua S. Dillon, Nicholas S. Kern, Miguel F. Morales, Bryna J. Hazelton, Ruby Byrne, Nicolas Fagnoni, Eloy de Lera Acedo, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Jacob Burba , et al. (56 additional authors not shown)

Abstract: Motivated by the desire for wide-field images with well-defined statistical properties for 21cm cosmology, we implement an optimal mapping pipeline that computes a maximum likelihood estimator for the sky using the interferometric measurement equation. We demonstrate this direct optimal mapping with data from the Hydrogen Epoch of Reionization (HERA) Phase I observations. After validating the pipe… ▽ More Motivated by the desire for wide-field images with well-defined statistical properties for 21cm cosmology, we implement an optimal mapping pipeline that computes a maximum likelihood estimator for the sky using the interferometric measurement equation. We demonstrate this direct optimal mapping with data from the Hydrogen Epoch of Reionization (HERA) Phase I observations. After validating the pipeline with simulated data, we develop a maximum likelihood figure-of-merit for comparing four sky models at 166MHz with a bandwidth of 100kHz. The HERA data agree with the GLEAM catalogs to <10%. After subtracting the GLEAM point sources, the HERA data discriminate between the different continuum sky models, providing most support for the model of Byrne et al. 2021. We report the computation cost for mapping the HERA Phase I data and project the computation for the HERA 320-antenna data; both are feasible with a modern server. The algorithm is broadly applicable to other interferometers and is valid for wide-field and non-coplanar arrays. △ Less

Submitted 26 October, 2022; v1 submitted 12 April, 2022; originally announced April 2022.

Comments: 16 pages, 10 figures, 2 tables, published on ApJ

Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer

Authors: Songwei Ge, Thomas Hayes, Harry Yang, Xi Yin, Guan Pang, David Jacobs, Jia-Bin Huang, Devi Parikh

Abstract: Videos are created to express emotion, exchange information, and share experiences. Video synthesis has intrigued researchers for a long time. Despite the rapid progress driven by advances in visual synthesis, most existing studies focus on improving the frames' quality and the transitions between them, while little progress has been made in generating longer videos. In this paper, we present a me… ▽ More Videos are created to express emotion, exchange information, and share experiences. Video synthesis has intrigued researchers for a long time. Despite the rapid progress driven by advances in visual synthesis, most existing studies focus on improving the frames' quality and the transitions between them, while little progress has been made in generating longer videos. In this paper, we present a method that builds on 3D-VQGAN and transformers to generate videos with thousands of frames. Our evaluation shows that our model trained on 16-frame video clips from standard benchmarks such as UCF-101, Sky Time-lapse, and Taichi-HD datasets can generate diverse, coherent, and high-quality long videos. We also showcase conditional extensions of our approach for generating meaningful long videos by incorporating temporal information with text and audio. Videos and code can be found at https://songweige.github.io/projects/tats/index.html. △ Less

Submitted 24 September, 2022; v1 submitted 7 April, 2022; originally announced April 2022.

Comments: In ECCV 2022

Measurements of one-point statistics in 21 cm intensity maps via foreground avoidance strategy

Authors: Piyanat Kittiwisit, Judd D. Bowman, Steven G. Murray, Bharat K. Gehlot, Daniel C. Jacobs, Adam P. Beardsley

Abstract: Measurements of the one-point probability distribution function and higher-order moments (variance, skewness, and kurtosis) of the high-redshift 21 cm fluctuations are among the most direct statistical probes of the non-Gaussian nature of structure formation and evolution during reionization. However, contamination from astrophysical foregrounds and instrument systematics pose significant challeng… ▽ More Measurements of the one-point probability distribution function and higher-order moments (variance, skewness, and kurtosis) of the high-redshift 21 cm fluctuations are among the most direct statistical probes of the non-Gaussian nature of structure formation and evolution during reionization. However, contamination from astrophysical foregrounds and instrument systematics pose significant challenges in measuring these statistics in real observations. In this work, we use forward modelling to investigate the feasibility of measuring 21 cm one-point statistics through a foreground avoidance strategy. Leveraging the characteristic wedge-shape of the foregrounds in k-space, we apply a wedge-cut filter that removes the foreground contaminated modes from a mock data set based on the Hydrogen Epoch of Reionization Array (HERA) instrument, and measure the one-point statistics from the image-space representation of the remaining non-contaminated modes. We experiment with varying degrees of wedge-cutting over different frequency bandwidths and find that the centre of the band is the least susceptible to bias from wedge-cutting. Based on this finding, we introduce a rolling filter method that allows reconstruction of an optimal wedge-cut 21~cm intensity map over the full bandwidth using outputs from wedge-cutting over multiple sub-bands. We perform Monte Carlo simulations to show that HERA should be able to measure the rise in skewness and kurtosis near the end of reionization with the rolling wedge-cut method if foreground leakage from the Fourier transform window function can be controlled. △ Less

Submitted 29 September, 2022; v1 submitted 3 April, 2022; originally announced April 2022.

Comments: 13 pages, 10 figures, Accepted to MNRAS

Fast Light-Weight Near-Field Photometric Stereo

Authors: Daniel Lichy, Soumyadip Sengupta, David W. Jacobs

Abstract: We introduce the first end-to-end learning-based solution to near-field Photometric Stereo (PS), where the light sources are close to the object of interest. This setup is especially useful for reconstructing large immobile objects. Our method is fast, producing a mesh from 52 512$\times$384 resolution images in about 1 second on a commodity GPU, thus potentially unlocking several AR/VR applicatio… ▽ More We introduce the first end-to-end learning-based solution to near-field Photometric Stereo (PS), where the light sources are close to the object of interest. This setup is especially useful for reconstructing large immobile objects. Our method is fast, producing a mesh from 52 512$\times$384 resolution images in about 1 second on a commodity GPU, thus potentially unlocking several AR/VR applications. Existing approaches rely on optimization coupled with a far-field PS network operating on pixels or small patches. Using optimization makes these approaches slow and memory intensive (requiring 17GB GPU and 27GB of CPU memory) while using only pixels or patches makes them highly susceptible to noise and calibration errors. To address these issues, we develop a recursive multi-resolution scheme to estimate surface normal and depth maps of the whole image at each step. The predicted depth map at each scale is then used to estimate `per-pixel lighting' for the next scale. This design makes our approach almost 45$\times$ faster and 2$^{\circ}$ more accurate (11.3$^{\circ}$ vs. 13.3$^{\circ}$ Mean Angular Error) than the state-of-the-art near-field PS reconstruction technique, which uses iterative optimization. △ Less

Submitted 30 March, 2022; originally announced March 2022.

Comments: Accepted to CVPR 2022

contact.engineering -- Create, analyze and publish digital surface twins from topography measurements across many scales

Authors: Michael C. Röttger, Antoine Sanner, Luke A. Thimons, Till Junge, Abhijeet Gujrati, Joseph M. Monti, Wolfram G. Nöhring, Tevis D. B. Jacobs, Lars Pastewka

Abstract: The optimization of surface finish to improve performance occurs largely through trial and error, despite significant advancements in the relevant science. There are three central challenges that account for this disconnect: (1) the challenge of integration of many different types of measurement for the same surface to capture the multi-scale nature of roughness; (2) the technical complexity of im… ▽ More The optimization of surface finish to improve performance occurs largely through trial and error, despite significant advancements in the relevant science. There are three central challenges that account for this disconnect: (1) the challenge of integration of many different types of measurement for the same surface to capture the multi-scale nature of roughness; (2) the technical complexity of implementing spectral analysis methods, and of applying mechanical or numerical models to describe surface performance; (3) a lack of consistency between researchers and industries in how surfaces are measured, quantified, and communicated. Here we present a freely-available internet-based application which attempts to overcome all three challenges. First, the application enables the user to upload many different topography measurements taken from a single surface, including using different techniques, and then integrates all of them together to create a digital surface twin. Second, the application calculates many of the commonly used topography metrics, such as root-mean-square parameters, power spectral density (PSD), and autocorrelation function (ACF), as well as implementing analytical and numerical calculations, such as boundary element modeling (BEM) for elastic and plastic deformation. Third, the application serves as a repository for users to securely store surfaces, and if they choose, to share these with collaborators or even publish them (with a digital object identifier) for all to access. The primary goal of this application is to enable researchers and manufacturers to quickly and easily apply cutting-edge tools for the characterization and properties-modeling of real-world surfaces. An additional goal is to advance the use of open-science principles in surface engineering by providing a FAIR database where researchers can choose to publish surface measurements for all to use. △ Less

Submitted 18 September, 2022; v1 submitted 25 March, 2022; originally announced March 2022.

Comments: 19 pages, 6 figures

Journal ref: Surf. Topogr.: Metrol. Prop. 10, 035032 (2022)

On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels

Authors: Amnon Geifman, Meirav Galun, David Jacobs, Ronen Basri

Abstract: We study the properties of various over-parametrized convolutional neural architectures through their respective Gaussian process and neural tangent kernels. We prove that, with normalized multi-channel input and ReLU activation, the eigenfunctions of these kernels with the uniform measure are formed by products of spherical harmonics, defined over the channels of the different pixels. We next use… ▽ More We study the properties of various over-parametrized convolutional neural architectures through their respective Gaussian process and neural tangent kernels. We prove that, with normalized multi-channel input and ReLU activation, the eigenfunctions of these kernels with the uniform measure are formed by products of spherical harmonics, defined over the channels of the different pixels. We next use hierarchical factorizable kernels to bound their respective eigenvalues. We show that the eigenvalues decay polynomially, quantify the rate of decay, and derive measures that reflect the composition of hierarchical features in these networks. Our results provide concrete quantitative characterization of over-parameterized convolutional network architectures. △ Less

Submitted 17 March, 2022; originally announced March 2022.

Estimating the Feasibility of 21cm-Ly$α$ Synergies using the Hydrogen Epoch of Reionization Array

Authors: Tyler A. Cox, Daniel C. Jacobs, Steven G. Murray

Abstract: Cross-correlating 21cm and Ly$α$ intensity maps of the Epoch of Reionization (EoR) promises to be a powerful tool for exploring the properties of the first galaxies. Next-generation intensity mapping experiments such as the Hydrogen Epoch of Reionization Array (HERA) and SPHEREx will individually probe reionization through the power spectra of the 21cm and Ly$α$ lines respectively, but will be lim… ▽ More Cross-correlating 21cm and Ly$α$ intensity maps of the Epoch of Reionization (EoR) promises to be a powerful tool for exploring the properties of the first galaxies. Next-generation intensity mapping experiments such as the Hydrogen Epoch of Reionization Array (HERA) and SPHEREx will individually probe reionization through the power spectra of the 21cm and Ly$α$ lines respectively, but will be limited by bright foregrounds and instrumental systematics. Cross-correlating these measurements could reduce systematics, potentially tightening constraints on the inferred astrophysical parameters. In this study, we present forecasts of cross-correlation taking into account the effects of exact uv-sampling and foreground filtering to estimate the feasibility of HERAxSPHEREx making a detection of the 21cm-Ly$α$ cross-power spectrum. We also project the sensitivity of a cross-power spectrum between HERA and the proposed next-generation Cosmic Dawn Intensity Mapper. By isolating the sources of uncertainty, we explore the impacts of experimental limitations such as foreground filtering and Ly$α$ thermal noise uncertainty have on making a detection of the cross-power spectrum. We then implement this strategy in a simulation of the cross-power spectrum and observational error to identify redshifts where fiducial 21cmFAST models predict the highest signal-to-noise detection ($z \sim 8$). We conclude that detection of the SPHEREx-HERA cross-correlation will require an optimistic level of 21cm foreground filtering, as well as deeper thermal noise integrations due to a lack of overlapping sensitive modes but for CDIM with its larger range of scales and lower noise forecast detection levels, may be possible even with stricter 21cm foreground filtering. △ Less

Submitted 17 February, 2022; originally announced February 2022.

Comments: 11 pages, 6 figures, Accepted by MNRAS

Rapid Mass Parameter Estimation of Binary Black Hole Coalescences Using Deep Learning

Authors: Alistair McLeod, Daniel Jacobs, Chayan Chatterjee, Linqing Wen, Fiona Panther

Abstract: Deep learning can be used to drastically decrease the processing time of parameter estimation for coalescing binaries of compact objects including black holes and neutron stars detected in gravitational waves (GWs). As a first step, we present two neural network models trained to rapidly estimate the posterior distributions of the chirp mass and mass ratio of a detected binary black hole system fr… ▽ More Deep learning can be used to drastically decrease the processing time of parameter estimation for coalescing binaries of compact objects including black holes and neutron stars detected in gravitational waves (GWs). As a first step, we present two neural network models trained to rapidly estimate the posterior distributions of the chirp mass and mass ratio of a detected binary black hole system from the GW strain data of LIGO Hanford and Livingston Observatories. Using these parameters the component masses can be predicted, which has implications for the prediction of the likelihood that a merger contains a neutron star. The results are compared to the 'gold standard' of parameter estimation of gravitational waves used by the LIGO-Virgo Collaboration (LVC), LALInference. Our models predict posterior distributions consistent with that from LALInference while using orders of magnitude less processing time once the models are trained. The median predictions are within the 90% credible intervals of LALInference for all predicted parameters when tested on real binary black hole events detected during the LVC's first and second observing runs. We argue that deep learning has strong potential for low-latency high-accuracy parameter estimation suitable for real-time GW search pipelines. △ Less

Submitted 26 January, 2022; originally announced January 2022.

Comments: 17 pages, 16 figures

Biased Hypothesis Formation From Projection Pursuit

Authors: John Patterson, Chris Avery, Tyler Grear, Donald J. Jacobs

Abstract: The effect of bias on hypothesis formation is characterized for an automated data-driven projection pursuit neural network to extract and select features for binary classification of data streams. This intelligent exploratory process partitions a complete vector state space into disjoint subspaces to create working hypotheses quantified by similarities and differences observed between two groups o… ▽ More The effect of bias on hypothesis formation is characterized for an automated data-driven projection pursuit neural network to extract and select features for binary classification of data streams. This intelligent exploratory process partitions a complete vector state space into disjoint subspaces to create working hypotheses quantified by similarities and differences observed between two groups of labeled data streams. Data streams are typically time sequenced, and may exhibit complex spatio-temporal patterns. For example, given atomic trajectories from molecular dynamics simulation, the machine's task is to quantify dynamical mechanisms that promote function by comparing protein mutants, some known to function while others are nonfunctional. Utilizing synthetic two-dimensional molecules that mimic the dynamics of functional and nonfunctional proteins, biases are identified and controlled in both the machine learning model and selected training data under different contexts. The refinement of a working hypothesis converges to a statistically robust multivariate perception of the data based on a context-dependent perspective. Including diverse perspectives during data exploration enhances interpretability of the multivariate characterization of similarities and differences. △ Less

Submitted 3 January, 2022; originally announced January 2022.

Comments: 12 pages, 5 figures

Journal ref: Advances in Artificial Intelligence and Machine Learning. 2021;3:213-224