Showing 1–50 of 241 results for author: Hewitt, J

Learning Translations via Matrix Completion

Authors: Derry Wijaya, Brendan Callahan, John Hewitt, Jie Gao, Xiao Ling, Marianna Apidianaki, Chris Callison-Burch

Abstract: Bilingual Lexicon Induction is the task of learning word translations without bilingual parallel corpora. We model this task as a matrix completion problem, and present an effective and extendable framework for completing the matrix. This method harnesses diverse bilingual and monolingual signals, each of which may be incomplete or noisy. Our model achieves state-of-the-art performance for both hi… ▽ More Bilingual Lexicon Induction is the task of learning word translations without bilingual parallel corpora. We model this task as a matrix completion problem, and present an effective and extendable framework for completing the matrix. This method harnesses diverse bilingual and monolingual signals, each of which may be incomplete or noisy. Our model achieves state-of-the-art performance for both high and low resource languages. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: This is a late posting of an old paper as Google Scholar somehow misses indexing the ACL anthology version of the paper

ACM Class: I.2.7

Journal ref: Volume: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Year: 2017, Pages: 1452-1463

Investigating Mutual Coupling in the Hydrogen Epoch of Reionization Array and Mitigating its Effects on the 21-cm Power Spectrum

Authors: E. Rath, R. Pascua, A. T. Josaitis, A. Ewall-Wice, N. Fagnoni, E. de Lera Acedo, Z. E. Martinot, Z. Abdurashidova, T. Adams, J. E. Aguirre, R. Baartman, A. P. Beardsley, L. M. Berkhout, G. Bernardi, T. S. Billings, J. D. Bowman, P. Bull, J. Burba, R. Byrne, S. Carey, K. -F. Chen, S. Choudhuri, T. Cox, D. R. DeBoer, M. Dexter , et al. (56 additional authors not shown)

Abstract: Interferometric experiments designed to detect the highly redshifted 21-cm signal from neutral hydrogen are producing increasingly stringent constraints on the 21-cm power spectrum, but some k-modes remain systematics-dominated. Mutual coupling is a major systematic that must be overcome in order to detect the 21-cm signal, and simulations that reproduce effects seen in the data can guide strategi… ▽ More Interferometric experiments designed to detect the highly redshifted 21-cm signal from neutral hydrogen are producing increasingly stringent constraints on the 21-cm power spectrum, but some k-modes remain systematics-dominated. Mutual coupling is a major systematic that must be overcome in order to detect the 21-cm signal, and simulations that reproduce effects seen in the data can guide strategies for mitigating mutual coupling. In this paper, we analyse 12 nights of data from the Hydrogen Epoch of Reionization Array and compare the data against simulations that include a computationally efficient and physically motivated semi-analytic treatment of mutual coupling. We find that simulated coupling features qualitatively agree with coupling features in the data; however, coupling features in the data are brighter than the simulated features, indicating the presence of additional coupling mechanisms not captured by our model. We explore the use of fringe-rate filters as mutual coupling mitigation tools and use our simulations to investigate the effects of mutual coupling on a simulated cosmological 21-cm power spectrum in a "worst case" scenario where the foregrounds are particularly bright. We find that mutual coupling contaminates a large portion of the "EoR Window", and the contamination is several orders-of-magnitude larger than our simulated cosmic signal across a wide range of cosmological Fourier modes. While our fiducial fringe-rate filtering strategy reduces mutual coupling by roughly a factor of 100 in power, a non-negligible amount of coupling cannot be excised with fringe-rate filters, so more sophisticated mitigation strategies are required. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 19 pages, 12 figures, submitted to MNRAS

Modelling the evolution of an ice sheet's weathering crust

Authors: Tilly Woods, Ian J. Hewitt

Abstract: The weathering crust is a layer of porous ice that can form at the surface of an ice sheet. It grows and decays in response changing weather and climate conditions, affecting the albedo, the melt rate, and the transport of meltwater across the surface. To understand this behaviour, we seek time-dependent solutions to a continuum, thermodynamic model for the porosity, temperature and thickness of t… ▽ More The weathering crust is a layer of porous ice that can form at the surface of an ice sheet. It grows and decays in response changing weather and climate conditions, affecting the albedo, the melt rate, and the transport of meltwater across the surface. To understand this behaviour, we seek time-dependent solutions to a continuum, thermodynamic model for the porosity, temperature and thickness of the weathering crust, and the internal and surface melt rates. We find solutions using a numerical enthalpy method, presented in this study. We use idealised `switching' and sinusoidal forcings to explore the different dynamics exhibited during growth and decay, the timescales involved, and the impact of diurnal vs. annual variations. The results demonstrate qualitative agreement with observations, and provide insight into the relative importance of different surface heat fluxes during the growth and decay of the crust. The model therefore provides a useful tool for exploring the response of the weathering crust to climate change. △ Less

Submitted 3 May, 2024; originally announced May 2024.

Comments: 36 pages, 11 figures

A demonstration of the effect of fringe-rate filtering in the Hydrogen Epoch of Reionization Array delay power spectrum pipeline

Authors: Hugh Garsden, Philip Bull, Mike Wilensky, Zuhra Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Lindsay M. Berkhout, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Steven Carey, Chris L. Carilli, Kai-Feng Chen, Carina Cheng, Samir Choudhuri, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter , et al. (72 additional authors not shown)

Abstract: Radio interferometers targeting the 21cm brightness temperature fluctuations at high redshift are subject to systematic effects that operate over a range of different timescales. These can be isolated by designing appropriate Fourier filters that operate in fringe-rate (FR) space, the Fourier pair of local sidereal time (LST). Applications of FR filtering include separating effects that are correl… ▽ More Radio interferometers targeting the 21cm brightness temperature fluctuations at high redshift are subject to systematic effects that operate over a range of different timescales. These can be isolated by designing appropriate Fourier filters that operate in fringe-rate (FR) space, the Fourier pair of local sidereal time (LST). Applications of FR filtering include separating effects that are correlated with the rotating sky vs. those relative to the ground, down-weighting emission in the primary beam sidelobes, and suppressing noise. FR filtering causes the noise contributions to the visibility data to become correlated in time however, making interpretation of subsequent averaging and error estimation steps more subtle. In this paper, we describe fringe rate filters that are implemented using discrete prolate spheroidal sequences, and designed for two different purposes -- beam sidelobe/horizon suppression (the `mainlobe' filter), and ground-locked systematics removal (the `notch' filter). We apply these to simulated data, and study how their properties affect visibilities and power spectra generated from the simulations. Included is an introduction to fringe-rate filtering and a demonstration of fringe-rate filters applied to simple situations to aid understanding. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Comments: 21 pages, 18 figures, submitted to Monthly Notices of the Royal Astronomical Society

Model Editing with Canonical Examples

Authors: John Hewitt, Sarah Chen, Lanruo Lora Xie, Edward Adams, Percy Liang, Christopher D. Manning

Abstract: We introduce model editing with canonical examples, a setting in which (1) a single learning example is provided per desired behavior, (2) evaluation is performed exclusively out-of-distribution, and (3) deviation from an initial model is strictly limited. A canonical example is a simple instance of good behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g., An aspect of re… ▽ More We introduce model editing with canonical examples, a setting in which (1) a single learning example is provided per desired behavior, (2) evaluation is performed exclusively out-of-distribution, and (3) deviation from an initial model is strictly limited. A canonical example is a simple instance of good behavior, e.g., The capital of Mauritius is Port Louis) or bad behavior, e.g., An aspect of researchers is coldhearted). The evaluation set contains more complex examples of each behavior (like a paragraph in which the capital of Mauritius is called for.) We create three datasets and modify three more for model editing with canonical examples, covering knowledge-intensive improvements, social bias mitigation, and syntactic edge cases. In our experiments on Pythia language models, we find that LoRA outperforms full finetuning and MEMIT. We then turn to the Backpack language model architecture because it is intended to enable targeted improvement. The Backpack defines a large bank of sense vectors--a decomposition of the different uses of each word--which are weighted and summed to form the output logits of the model. We propose sense finetuning, which selects and finetunes a few ($\approx$ 10) sense vectors for each canonical example, and find that it outperforms other finetuning methods, e.g., 4.8% improvement vs 0.3%. Finally, we improve GPT-J-6B by an inference-time ensemble with just the changes from sense finetuning of a 35x smaller Backpack, in one setting outperforming editing GPT-J itself (4.1% vs 1.0%). △ Less

Submitted 8 February, 2024; originally announced February 2024.

A method for characterizing disease emergence curves from paired pathogen detection and serology data

Authors: Joshua Hewitt, Grete Wilson-Henjum, Derek T. Collins, Jourdan M. Ringenberg, Christopher A. Quintanal, Robert Pleszewski, Jeffrey C. Chandler, Thomas J. DeLiberto, Kim M. Pepin

Abstract: Wildlife disease surveillance programs and research studies track infection and identify risk factors for wild populations, humans, and agriculture. Often, several types of samples are collected from individuals to provide more complete information about an animal's infection history. Methods that jointly analyze multiple data streams to study disease emergence and drivers of infection via epidemi… ▽ More Wildlife disease surveillance programs and research studies track infection and identify risk factors for wild populations, humans, and agriculture. Often, several types of samples are collected from individuals to provide more complete information about an animal's infection history. Methods that jointly analyze multiple data streams to study disease emergence and drivers of infection via epidemiological process models remain underdeveloped. Joint-analysis methods can more thoroughly analyze all available data, more precisely quantifying epidemic processes, outbreak status, and risks. We contribute a paired data modeling approach that analyzes multiple samples from individuals. We use "characterization maps" to link paired data to epidemiological processes through a hierarchical statistical observation model. Our approach can provide both Bayesian and frequentist estimates of epidemiological parameters and state. We motivate our approach through the need to use paired pathogen and antibody detection tests to estimate parameters and infection trajectories for the widely applicable susceptible, infectious, recovered (SIR) model. We contribute general formulas to link characterization maps to arbitrary process models and datasets and an extended SIR model that better accommodates paired data. We find via simulation that paired data can more efficiently estimate SIR parameters than unpaired data, requiring samples from 5-10 times fewer individuals. We then study SARS-CoV-2 in wild White-tailed deer (Odocoileus virginianus) from three counties in the United States. Estimates for average infectious times corroborate captive animal studies. Our methods use general statistical theory to let applications extend beyond the SIR model we consider, and to more complicated examples of paired data. △ Less

Submitted 13 May, 2024; v1 submitted 18 January, 2024; originally announced January 2024.

Comments: 22 pages, 5 figures, 1 table

Hydrogen Epoch of Reionization Array (HERA) Phase II Deployment and Commissioning

Authors: Lindsay M. Berkhout, Daniel C. Jacobs, Zuhra Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Jacob Burba, Steven Carey, Chris L. Carilli, Kai-Feng Chen, Carina Cheng, Samir Choudhuri, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Joshua S. Dillon , et al. (71 additional authors not shown)

Abstract: This paper presents the design and deployment of the Hydrogen Epoch of Reionization Array (HERA) phase II system. HERA is designed as a staged experiment targeting 21 cm emission measurements of the Epoch of Reionization. First results from the phase I array are published as of early 2022, and deployment of the phase II system is nearing completion. We describe the design of the phase II system an… ▽ More This paper presents the design and deployment of the Hydrogen Epoch of Reionization Array (HERA) phase II system. HERA is designed as a staged experiment targeting 21 cm emission measurements of the Epoch of Reionization. First results from the phase I array are published as of early 2022, and deployment of the phase II system is nearing completion. We describe the design of the phase II system and discuss progress on commissioning and future upgrades. As HERA is a designated Square Kilometer Array (SKA) pathfinder instrument, we also show a number of "case studies" that investigate systematics seen while commissioning the phase II system, which may be of use in the design and operation of future arrays. Common pathologies are likely to manifest in similar ways across instruments, and many of these sources of contamination can be mitigated once the source is identified. △ Less

Submitted 8 January, 2024; originally announced January 2024.

Journal ref: PASP 2024 136 045002

From Whole-slide Image to Biomarker Prediction: A Protocol for End-to-End Deep Learning in Computational Pathology

Authors: Omar S. M. El Nahhas, Marko van Treeck, Georg Wölflein, Michaela Unger, Marta Ligero, Tim Lenz, Sophia J. Wagner, Katherine J. Hewitt, Firas Khader, Sebastian Foersch, Daniel Truhn, Jakob Nikolas Kather

Abstract: Hematoxylin- and eosin (H&E) stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision onco… ▽ More Hematoxylin- and eosin (H&E) stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision oncology. This protocol describes a practical workflow for solid tumor associative modeling in pathology (STAMP), enabling prediction of biomarkers directly from WSIs using deep learning. The STAMP workflow is biomarker agnostic and allows for genetic- and clinicopathologic tabular data to be included as an additional input, together with histopathology images. The protocol consists of five main stages which have been successfully applied to various research problems: formal problem definition, data preprocessing, modeling, evaluation and clinical translation. The STAMP workflow differentiates itself through its focus on serving as a collaborative framework that can be used by clinicians and engineers alike for setting up research projects in the field of computational pathology. As an example task, we applied STAMP to the prediction of microsatellite instability (MSI) status in colorectal cancer, showing accurate performance for the identification of MSI-high tumors. Moreover, we provide an open-source codebase which has been deployed at several hospitals across the globe to set up computational pathology workflows. The STAMP workflow requires one workday of hands-on computational execution and basic command line knowledge. △ Less

Submitted 18 December, 2023; originally announced December 2023.

matvis: A matrix-based visibility simulator for fast forward modelling of many-element 21 cm arrays

Authors: Piyanat Kittiwisit, Steven G. Murray, Hugh Garsden, Philip Bull, Christopher Cain, Aaron R. Parsons, Jackson Sipple, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Lindsay M. Berkhout, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Steven Carey, Chris L. Carilli, Kai-Feng Chen, Carina Cheng , et al. (73 additional authors not shown)

Abstract: Detection of the faint 21 cm line emission from the Cosmic Dawn and Epoch of Reionisation will require not only exquisite control over instrumental calibration and systematics to achieve the necessary dynamic range of observations but also validation of analysis techniques to demonstrate their statistical properties and signal loss characteristics. A key ingredient in achieving this is the ability… ▽ More Detection of the faint 21 cm line emission from the Cosmic Dawn and Epoch of Reionisation will require not only exquisite control over instrumental calibration and systematics to achieve the necessary dynamic range of observations but also validation of analysis techniques to demonstrate their statistical properties and signal loss characteristics. A key ingredient in achieving this is the ability to perform high-fidelity simulations of the kinds of data that are produced by the large, many-element, radio interferometric arrays that have been purpose-built for these studies. The large scale of these arrays presents a computational challenge, as one must simulate a detailed sky and instrumental model across many hundreds of frequency channels, thousands of time samples, and tens of thousands of baselines for arrays with hundreds of antennas. In this paper, we present a fast matrix-based method for simulating radio interferometric measurements (visibilities) at the necessary scale. We achieve this through judicious use of primary beam interpolation, fast approximations for coordinate transforms, and a vectorised outer product to expand per-antenna quantities to per-baseline visibilities, coupled with standard parallelisation techniques. We validate the results of this method, implemented in the publicly-available matvis code, against a high-precision reference simulator, and explore its computational scaling on a variety of problems. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Comments: 25 pages, 20 figures, submitted to RAS Techniques and Instruments, matvis is publicly available at https://github.com/HERA-Team/matvis

Bayesian estimation of cross-coupling and reflection systematics in 21cm array visibility data

Authors: Geoff G. Murphy, Philip Bull, Mario G. Santos, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee Billings, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Christopher Cain, Steven Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Joshua S. Dillon, Nico Eksteen , et al. (54 additional authors not shown)

Abstract: Observations with radio arrays that target the 21-cm signal originating from the early Universe suffer from a variety of systematic effects. An important class of these are reflections and spurious couplings between antennas. We apply a Hamiltonian Monte Carlo sampler to the modelling and mitigation of these systematics in simulated Hydrogen Epoch of Reionisation Array (HERA) data. This method all… ▽ More Observations with radio arrays that target the 21-cm signal originating from the early Universe suffer from a variety of systematic effects. An important class of these are reflections and spurious couplings between antennas. We apply a Hamiltonian Monte Carlo sampler to the modelling and mitigation of these systematics in simulated Hydrogen Epoch of Reionisation Array (HERA) data. This method allows us to form statistical uncertainty estimates for both our models and the recovered visibilities, which is an important ingredient in establishing robust upper limits on the Epoch of Reionisation (EoR) power spectrum. In cases where the noise is large compared to the EoR signal, this approach can constrain the systematics well enough to mitigate them down to the noise level for both systematics studied. Where the noise is smaller than the EoR, our modelling can mitigate the majority of the reflections with there being only a minor level of residual systematics, while cross-coupling sees essentially complete mitigation. Our approach performs similarly to existing filtering/fitting techniques used in the HERA pipeline, but with the added benefit of rigorously propagating uncertainties. In all cases it does not significantly attenuate the underlying signal. △ Less

Submitted 6 December, 2023; originally announced December 2023.

Comments: 19 pages, 14 figures, submitted to MNRAS

Direct Optimal Mapping Image Power Spectrum and its Window Functions

Authors: Zhilei Xu, Honggeun Kim, Jacqueline N. Hewitt, Kai-Feng Chen, Nicholas S. Kern, Eleanor Rath, Ruby Byrne, Adélie Gorce, Robert Pascua, Zachary E. Martinot, Joshua S. Dillon, Bryna J. Hazelton, Adrian Liu, Miguel F. Morales, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman , et al. (57 additional authors not shown)

Abstract: The key to detecting neutral hydrogen during the epoch of reionization (EoR) is to separate the cosmological signal from the dominating foreground radiation. We developed direct optimal mapping (DOM) to map interferometric visibilities; it contains only linear operations, with full knowledge of point spread functions from visibilities to images. Here, we demonstrate a fast Fourier transform-based… ▽ More The key to detecting neutral hydrogen during the epoch of reionization (EoR) is to separate the cosmological signal from the dominating foreground radiation. We developed direct optimal mapping (DOM) to map interferometric visibilities; it contains only linear operations, with full knowledge of point spread functions from visibilities to images. Here, we demonstrate a fast Fourier transform-based image power spectrum and its window functions computed from the DOM images. We use noiseless simulation, based on the Hydrogen Epoch of Reionization Array Phase I configuration, to study the image power spectrum properties. The window functions show $<10^{-11}$ of the integrated power leaks from the foreground-dominated region into the EoR window; the 2D and 1D power spectra also verify the separation between the foregrounds and the EoR. △ Less

Submitted 5 July, 2024; v1 submitted 17 November, 2023; originally announced November 2023.

Comments: Published in ApJ

Detection of large-scale synchrotron radiation from the molecular envelope of the Sgr B cloud complex at the Galactic center

Authors: F. Yusef-Zadeh, M. Wardle, R. Arendt, J. W. Hewitt, Y. Hu, A. Lazarian, N. Kassim, S. Hyman, I. Heywood

Abstract: We present highly sensitive measurements taken with MeerKAT at 1280 MHz as well as archival GBT, MWA and VLA images at 333, 88 and 74 MHz. We report the detection of synchrotron radio emission from the infrared dark cloud (IRDC) associated with the halo of the Sgr B complex on a scale of ~60 pc. A strong spatial correlation between low-frequency radio continuum emission and dense molecular gas, co… ▽ More We present highly sensitive measurements taken with MeerKAT at 1280 MHz as well as archival GBT, MWA and VLA images at 333, 88 and 74 MHz. We report the detection of synchrotron radio emission from the infrared dark cloud (IRDC) associated with the halo of the Sgr B complex on a scale of ~60 pc. A strong spatial correlation between low-frequency radio continuum emission and dense molecular gas, combined with spectral index measurements, indicates enhanced synchrotron emission by cosmic-ray electrons. Correlation of the FeI 6.4 keV Kalpha line and synchrotron emission provides compelling evidence that the low energy cosmic-ray electrons are responsible for producing the Kalpha line emission. The observed synchrotron emission within the halo of the Sgr B cloud complex has mean spectral index alpha -1+/-1 gives the magnetic field strength ~100 muG for cloud densities nH = 10^4-10^5 cm-3 and estimate cosmic-ray ionization rates between 10^-13 and 10^-14 s^-1. Furthermore, the energy spectrum of primary cosmic-ray electrons is constrained to be E^-3 +/-1 for typical energies of few hundred MeV. The extrapolation of this spectrum to higher energies is consistent with X-ray and gamma-ray emission detected from this cloud. These measurements have important implications on the role that high cosmic-ray electron fluxes at the Galactic center play in production of radio synchrotron emission, the FeI Kalpha line emission at 6.4 keV and ~GeV gamma-ray emission throughout the central molecular zone (CMZ). △ Less

Submitted 30 October, 2023; originally announced October 2023.

Comments: 10 pages, 4 figures, MN (in press)

Character-level Chinese Backpack Language Models

Authors: Hao Sun, John Hewitt

Abstract: The Backpack is a Transformer alternative shown to improve interpretability in English language modeling by decomposing predictions into a weighted sum of token sense components. However, Backpacks' reliance on token-defined meaning raises questions as to their potential for languages other than English, a language for which subword tokenization provides a reasonable approximation for lexical item… ▽ More The Backpack is a Transformer alternative shown to improve interpretability in English language modeling by decomposing predictions into a weighted sum of token sense components. However, Backpacks' reliance on token-defined meaning raises questions as to their potential for languages other than English, a language for which subword tokenization provides a reasonable approximation for lexical items. In this work, we train, evaluate, interpret, and control Backpack language models in character-tokenized Chinese, in which words are often composed of many characters. We find that our (134M parameter) Chinese Backpack language model performs comparably to a (104M parameter) Transformer, and learns rich character-level meanings that log-additively compose to form word meanings. In SimLex-style lexical semantic evaluations, simple averages of Backpack character senses outperform input embeddings from a Transformer. We find that complex multi-character meanings are often formed by using the same per-character sense weights consistently across context. Exploring interpretability-through control, we show that we can localize a source of gender bias in our Backpacks to specific character senses and intervene to reduce the bias. △ Less

Submitted 19 October, 2023; originally announced October 2023.

Comments: BlackboxNLP 2023 Camera-Ready

Closing the Curious Case of Neural Text Degeneration

Authors: Matthew Finlayson, John Hewitt, Alexander Koller, Swabha Swayamdipta, Ashish Sabharwal

Abstract: Despite their ubiquity in language generation, it remains unknown why truncation sampling heuristics like nucleus sampling are so effective. We provide a theoretical explanation for the effectiveness of the truncation sampling by proving that truncation methods that discard tokens below some probability threshold (the most common type of truncation) can guarantee that all sampled tokens have nonze… ▽ More Despite their ubiquity in language generation, it remains unknown why truncation sampling heuristics like nucleus sampling are so effective. We provide a theoretical explanation for the effectiveness of the truncation sampling by proving that truncation methods that discard tokens below some probability threshold (the most common type of truncation) can guarantee that all sampled tokens have nonzero true probability. However, thresholds are a coarse heuristic, and necessarily discard some tokens with nonzero true probability as well. In pursuit of a more precise sampling strategy, we show that we can leverage a known source of model errors, the softmax bottleneck, to prove that certain tokens have nonzero true probability, without relying on a threshold. Based on our findings, we develop an experimental truncation strategy and the present pilot studies demonstrating the promise of this type of algorithm. Our evaluations show that our method outperforms its threshold-based counterparts under automatic and human evaluation metrics for low-entropy (i.e., close to greedy) open-ended text generation. Our theoretical findings and pilot experiments provide both insight into why truncation sampling works, and make progress toward more expressive sampling algorithms that better surface the generative capabilities of large language models. △ Less

Submitted 2 October, 2023; originally announced October 2023.

MSC Class: 68T50 ACM Class: I.2.7

Soft matter physics of the ground beneath our feet

Authors: Anne Voigtländer, Morgane Houssais, Karol A. Bacik, Ian C. Bourg, Justin C. Burton, Karen E. Daniels, Sujit S. Datta, Emanuela Del Gado, Nakul S. Deshpande, Olivier Devauchelle, Behrooz Ferdowsi, Rachel Glade, Lucas Goehring, Ian J. Hewitt, Douglas Jerolmack, Ruben Juanes, Arshad Kudrolli, Ching-Yao Lai, Wei Li, Claire Masteller, Kavinda Nissanka, Allan M. Rubin, Howard A. Stone, Jenny Suckale, Nathalie M. Vriend , et al. (2 additional authors not shown)

Abstract: Inspired by presentations by the authors during a workshop organized at the Princeton Center for Theoretical Science (PCTS) in January 2022, we present a perspective on some of the outstanding questions related to the "physics of the ground beneath our feet." These identified challenges are intrinsically shared with the field of Soft Matter but also have unique aspects when the natural environment… ▽ More Inspired by presentations by the authors during a workshop organized at the Princeton Center for Theoretical Science (PCTS) in January 2022, we present a perspective on some of the outstanding questions related to the "physics of the ground beneath our feet." These identified challenges are intrinsically shared with the field of Soft Matter but also have unique aspects when the natural environment is studied. △ Less

Submitted 31 July, 2023; originally announced August 2023.

Comments: Perspective Paper, 30 pages, 15 figures

The Impact of Beam Variations on Power Spectrum Estimation for 21 cm Cosmology II: Mitigation of Foreground Systematics for HERA

Authors: Honggeun Kim, Nicholas S. Kern, Jacqueline N. Hewitt, Bang D. Nhan, Joshua S. Dillon, Eloy de Lera Acedo, Scott B. C. Dynes, Nivedita Mahesh, Nicolas Fagnoni, David R. DeBoer

Abstract: One key challenge in detecting 21 cm cosmological signal at z > 6 is to separate the cosmological signal from foreground emission. This can be studied in a power spectrum space where the foreground is confined to low delay modes whereas the cosmological signal can spread out to high delay modes. When there is a calibration error, however, chromaticity of gain errors propagates to the power spectru… ▽ More One key challenge in detecting 21 cm cosmological signal at z > 6 is to separate the cosmological signal from foreground emission. This can be studied in a power spectrum space where the foreground is confined to low delay modes whereas the cosmological signal can spread out to high delay modes. When there is a calibration error, however, chromaticity of gain errors propagates to the power spectrum estimate and contaminates the modes for cosmological detection. The Hydrogen Epoch of Reionization Array (HERA) employs a high-precision calibration scheme using redundancy in measurements. In this study, we focus on the gain errors induced by nonredundancies arising from feed offset relative to the HERA's 14 meter parabolic dish element, and investigate how to mitigate the chromatic gain errors using three different methods: restricting baseline lengths for calibration, smoothing the antenna gains, and applying a temporal filter prior to calibration. With 2 cm/2 degree perturbations for translation/tilting motions, a level achievable under normal HERA operating conditions, the combination of the baseline cut and temporal filtering indicates that the spurious gain feature due to nonredundancies is significantly reduced, and the power spectrum recovers the clean foreground-free region. We found that the mitigation technique works even for large feed motions but in order to keep a stable calibration process, the feed positions need to be constrained to 2 cm for translation motions and 2 degree for tilting offset relative to the dish's vertex. △ Less

Submitted 24 July, 2023; originally announced July 2023.

Comments: Accepted for publication in ApJ

The Third Fermi Large Area Telescope Catalog of Gamma-ray Pulsars

Authors: David A. Smith, Philippe Bruel, Colin J. Clark, Lucas Guillemot, Matthew T. Kerr, Paul Ray, Soheila Abdollahi, Marco Ajello, Luca Baldini, Jean Ballet, Matthew Baring, Cees Bassa, Josefa Becerra Gonzalez, Ronaldo Bellazzini, Alessandra Berretta, Bhaswati Bhattacharyya, Elisabetta Bissaldi, Raffaella Bonino, Eugenio Bottacini, Johan Bregeon, Marta Burgay, Toby Burnett, Rob Cameron, Fernando Camilo, Regina Caputo , et al. (134 additional authors not shown)

Abstract: We present 294 pulsars found in GeV data from the Large Area Telescope (LAT) on the Fermi Gamma-ray Space Telescope. Another 33 millisecond pulsars (MSPs) discovered in deep radio searches of LAT sources will likely reveal pulsations once phase-connected rotation ephemerides are achieved. A further dozen optical and/or X-ray binary systems co-located with LAT sources also likely harbor gamma-ray M… ▽ More We present 294 pulsars found in GeV data from the Large Area Telescope (LAT) on the Fermi Gamma-ray Space Telescope. Another 33 millisecond pulsars (MSPs) discovered in deep radio searches of LAT sources will likely reveal pulsations once phase-connected rotation ephemerides are achieved. A further dozen optical and/or X-ray binary systems co-located with LAT sources also likely harbor gamma-ray MSPs. This catalog thus reports roughly 340 gamma-ray pulsars and candidates, 10% of all known pulsars, compared to $\leq 11$ known before Fermi. Half of the gamma-ray pulsars are young. Of these, the half that are undetected in radio have a broader Galactic latitude distribution than the young radio-loud pulsars. The others are MSPs, with 6 undetected in radio. Overall, >235 are bright enough above 50 MeV to fit the pulse profile, the energy spectrum, or both. For the common two-peaked profiles, the gamma-ray peak closest to the magnetic pole crossing generally has a softer spectrum. The spectral energy distributions tend to narrow as the spindown power $\dot E$ decreases to its observed minimum near $10^{33}$ erg s$^{-1}$, approaching the shape for synchrotron radiation from monoenergetic electrons. We calculate gamma-ray luminosities when distances are available. Our all-sky gamma-ray sensitivity map is useful for population syntheses. The electronic catalog version provides gamma-ray pulsar ephemerides, properties and fit results to guide and be compared with modeling results. △ Less

Submitted 20 July, 2023; originally announced July 2023.

Comments: 142 pages. Accepted by the Astrophysical Journal Supplement

Lost in the Middle: How Language Models Use Long Contexts

Authors: Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang

Abstract: While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing t… ▽ More While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context. We analyze the performance of language models on two tasks that require identifying relevant information in their input contexts: multi-document question answering and key-value retrieval. We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts. In particular, we observe that performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models. Our analysis provides a better understanding of how language models use their input context and provides new evaluation protocols for future long-context language models. △ Less

Submitted 20 November, 2023; v1 submitted 6 July, 2023; originally announced July 2023.

Comments: 18 pages, 16 figures. Accepted for publication in Transactions of the Association for Computational Linguistics (TACL), 2023

Backpack Language Models

Authors: John Hewitt, John Thickstun, Christopher D. Manning, Percy Liang

Abstract: We present Backpacks: a new neural architecture that marries strong modeling performance with an interface for interpretability and control. Backpacks learn multiple non-contextual sense vectors for each word in a vocabulary, and represent a word in a sequence as a context-dependent, non-negative linear combination of sense vectors in this sequence. We find that, after training, sense vectors spec… ▽ More We present Backpacks: a new neural architecture that marries strong modeling performance with an interface for interpretability and control. Backpacks learn multiple non-contextual sense vectors for each word in a vocabulary, and represent a word in a sequence as a context-dependent, non-negative linear combination of sense vectors in this sequence. We find that, after training, sense vectors specialize, each encoding a different aspect of a word. We can interpret a sense vector by inspecting its (non-contextual, linear) projection onto the output space, and intervene on these interpretable hooks to change the model's behavior in predictable ways. We train a 170M-parameter Backpack language model on OpenWebText, matching the loss of a GPT-2 small (124Mparameter) Transformer. On lexical similarity evaluations, we find that Backpack sense vectors outperform even a 6B-parameter Transformer LM's word embeddings. Finally, we present simple algorithms that intervene on sense vectors to perform controllable text generation and debiasing. For example, we can edit the sense vocabulary to tend more towards a topic, or localize a source of gender bias to a sense vector and globally suppress that sense. △ Less

Submitted 26 May, 2023; originally announced May 2023.

Comments: ACL 2023 Camera-Ready

Regression-based Deep-Learning predicts molecular biomarkers from pathology slides

Authors: Omar S. M. El Nahhas, Chiara M. L. Loeffler, Zunamys I. Carrero, Marko van Treeck, Fiona R. Kolbinger, Katherine J. Hewitt, Hannah S. Muti, Mara Graziani, Qinghe Zeng, Julien Calderaro, Nadina Ortiz-Brüchle, Tanwei Yuan, Michael Hoffmeister, Hermann Brenner, Alexander Brobeil, Jorge S. Reis-Filho, Jakob Nikolas Kather

Abstract: Deep Learning (DL) can predict biomarkers from cancer histopathology. Several clinically approved applications use this technology. Most approaches, however, predict categorical labels, whereas biomarkers are often continuous measurements. We hypothesized that regression-based DL outperforms classification-based DL. Therefore, we developed and evaluated a new self-supervised attention-based weakly… ▽ More Deep Learning (DL) can predict biomarkers from cancer histopathology. Several clinically approved applications use this technology. Most approaches, however, predict categorical labels, whereas biomarkers are often continuous measurements. We hypothesized that regression-based DL outperforms classification-based DL. Therefore, we developed and evaluated a new self-supervised attention-based weakly supervised regression method that predicts continuous biomarkers directly from images in 11,671 patients across nine cancer types. We tested our method for multiple clinically and biologically relevant biomarkers: homologous repair deficiency (HRD) score, a clinically used pan-cancer biomarker, as well as markers of key biological processes in the tumor microenvironment. Using regression significantly enhances the accuracy of biomarker prediction, while also improving the interpretability of the results over classification. In a large cohort of colorectal cancer patients, regression-based prediction scores provide a higher prognostic value than classification-based scores. Our open-source regression approach offers a promising alternative for continuous biomarker analysis in computational pathology. △ Less

Submitted 11 April, 2023; originally announced April 2023.

Fermi-GBM Discovery of GRB 221009A: An Extraordinarily Bright GRB from Onset to Afterglow

Authors: S. Lesage, P. Veres, M. S. Briggs, A. Goldstein, D. Kocevski, E. Burns, C. A. Wilson-Hodge, P. N. Bhat, D. Huppenkothen, C. L. Fryer, R. Hamburg, J. Racusin, E. Bissaldi, W. H. Cleveland, S. Dalessi, C. Fletcher, M. M. Giles, B. A. Hristov, C. M. Hui, B. Mailyan, C. Malacaria, S. Poolakkil, O. J. Roberts, A. von Kienlin, J. Wood , et al. (115 additional authors not shown)

Abstract: We report the discovery of GRB 221009A, the highest flux gamma-ray burst ever observed by the Fermi Gamma-ray Burst Monitor (GBM). This GRB has continuous prompt emission lasting more than 600 seconds which smoothly transitions to afterglow visible in the GBM energy range (8 keV--40 MeV), and total energetics higher than any other burst in the GBM sample. By using a variety of new and existing ana… ▽ More We report the discovery of GRB 221009A, the highest flux gamma-ray burst ever observed by the Fermi Gamma-ray Burst Monitor (GBM). This GRB has continuous prompt emission lasting more than 600 seconds which smoothly transitions to afterglow visible in the GBM energy range (8 keV--40 MeV), and total energetics higher than any other burst in the GBM sample. By using a variety of new and existing analysis techniques we probe the spectral and temporal evolution of GRB 221009A. We find no emission prior to the GBM trigger time (t0; 2022 October 9 at 13:16:59.99 UTC), indicating that this is the time of prompt emission onset. The triggering pulse exhibits distinct spectral and temporal properties suggestive of the thermal, photospheric emission of shock-breakout, with significant emission up to $\sim$15 MeV. We characterize the onset of external shock at t0+600 s and find evidence of a plateau region in the early-afterglow phase which transitions to a slope consistent with Swift-XRT afterglow measurements. We place the total energetics of GRB 221009A in context with the rest of the GBM sample and find that this GRB has the highest total isotropic-equivalent energy ($\textrm{E}_{γ,\textrm{iso}}=1.0\times10^{55}$ erg) and second highest isotropic-equivalent luminosity ($\textrm{L}_{γ,\textrm{iso}}=9.9\times10^{53}$ erg/s) based on redshift of z = 0.151. These extreme energetics are what allowed us to observe the continuously emitting central engine of GBM from the beginning of the prompt emission phase through the onset of early afterglow. △ Less

Submitted 12 July, 2023; v1 submitted 24 March, 2023; originally announced March 2023.

Comments: 26 pages 7 figures - accepted for publication in ApJL

Search for the Epoch of Reionisation with HERA: Upper Limits on the Closure Phase Delay Power Spectrum

Authors: Pascal M. Keller, Bojan Nikolic, Nithyanandan Thyagarajan, Chris L. Carilli, Gianni Bernardi, Ntsikelelo Charles, Landman Bester, Oleg M. Smirnov, Nicholas S. Kern, Joshua S. Dillon, Bryna J. Hazelton, Miguel F. Morales, Daniel C. Jacobs, Aaron R. Parsons, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley , et al. (58 additional authors not shown)

Abstract: Radio interferometers aiming to measure the power spectrum of the redshifted 21 cm line during the Epoch of Reionisation (EoR) need to achieve an unprecedented dynamic range to separate the weak signal from overwhelming foreground emissions. Calibration inaccuracies can compromise the sensitivity of these measurements to the effect that a detection of the EoR is precluded. An alternative to standa… ▽ More Radio interferometers aiming to measure the power spectrum of the redshifted 21 cm line during the Epoch of Reionisation (EoR) need to achieve an unprecedented dynamic range to separate the weak signal from overwhelming foreground emissions. Calibration inaccuracies can compromise the sensitivity of these measurements to the effect that a detection of the EoR is precluded. An alternative to standard analysis techniques makes use of the closure phase, which allows one to bypass antenna-based direction-independent calibration. Similarly to standard approaches, we use a delay spectrum technique to search for the EoR signal. Using 94 nights of data observed with Phase I of the Hydrogen Epoch of Reionization Array (HERA), we place approximate constraints on the 21 cm power spectrum at $z=7.7$. We find at 95% confidence that the 21 cm EoR brightness temperature is $\le$(372)$^2$ "pseudo" mK$^2$ at 1.14 "pseudo" $h$ Mpc$^{-1}$, where the "pseudo" emphasises that these limits are to be interpreted as approximations to the actual distance scales and brightness temperatures. Using a fiducial EoR model, we demonstrate the feasibility of detecting the EoR with the full array. Compared to standard methods, the closure phase processing is relatively simple, thereby providing an important independent check on results derived using visibility intensities, or related. △ Less

Submitted 15 February, 2023; originally announced February 2023.

Comments: 16 pages, 14 figures, accepted for publication by MNRAS

A Machine Learning Approach for Player and Position Adjusted Expected Goals in Football (Soccer)

Authors: James H. Hewitt, Oktay Karakuş

Abstract: Football is a very result-driven industry, with goals being rarer than in most sports, so having further parameters to judge the performance of teams and individuals is key. Expected Goals (xG) allow further insight than just a scoreline. To tackle the need for further analysis in football, this paper uses machine learning applications that are developed and applied to Football Event data. From th… ▽ More Football is a very result-driven industry, with goals being rarer than in most sports, so having further parameters to judge the performance of teams and individuals is key. Expected Goals (xG) allow further insight than just a scoreline. To tackle the need for further analysis in football, this paper uses machine learning applications that are developed and applied to Football Event data. From the concept, a Binary Classification problem is created whereby a probabilistic valuation is outputted using Logistic Regression and Gradient Boosting based approaches. The model successfully predicts xGs probability values for football players based on 15,575 shots. The proposed solution utilises StatsBomb as the data provider and an industry benchmark to tune the models in the right direction. The proposed ML solution for xG is further used to tackle the age-old cliche of: 'the ball has fallen to the wrong guy there'. The development of the model is used to adjust and gain more realistic values of expected goals than the general models show. To achieve this, this paper tackles Positional Adjusted xG, splitting the training data into Forward, Midfield, and Defence with the aim of providing insight into player qualities based on their positional sub-group. Positional Adjusted xG successfully predicts and proves that more attacking players are better at accumulating xG. The highest value belonged to Forwards followed by Midfielders and Defenders. Finally, this study has further developments into Player Adjusted xG with the aim of proving that Messi is statistically at a higher efficiency level than the average footballer. This is achieved by using Messi subset samples to quantify his qualities in comparison to the average xG models finding that Messi xG performs 347 xG higher than the general model outcome. △ Less

Submitted 2 May, 2023; v1 submitted 19 January, 2023; originally announced January 2023.

Comments: 16 pages, 8 tables, 6 figures

Topological Data Analysis Detects Percolation Thresholds in Arctic Melt-Pond Evolution

Authors: Wilfred Offord, Michael Coughlan, Ian J. Hewitt, Heather A. Harrington, Gillian Grindstaff

Abstract: During the summer melt period, ponds form on the surface of Arctic sea ice as it melts, with important consequences for ice evolution and marine ecology. Due to the ice-albedo feedback, these melt ponds experience uneven heat absorption, and exhibit complex patterns, which has motivated the development of modelling and data analysis to understand their particular dynamics. We provide a multiscale… ▽ More During the summer melt period, ponds form on the surface of Arctic sea ice as it melts, with important consequences for ice evolution and marine ecology. Due to the ice-albedo feedback, these melt ponds experience uneven heat absorption, and exhibit complex patterns, which has motivated the development of modelling and data analysis to understand their particular dynamics. We provide a multiscale shape analysis using tools from computational algebraic topology, simultaneously capturing convexity, proximity, integrity, and feature size complementing existing single-scale quantification. Of particular interest in modelling the ponds is a percolation threshold at which local pond structure begins merging into macroscopic features. This percolation threshold has previously been observed using fractal dimension techniques. The signed Euclidean distance transform (SEDT) is a topological encoding of heterogeneous shape in binary images, and has been previously applied to porous media for percolation as well as other material behaviours. Here we adapt the SEDT for Arctic melt pond data to give a rich characterization and computation of shape, quantifying overall melt pond development in several complementary ways, and from which classical percolation and dimension results can be extracted. This orientation-invariant topological approach distinguishes different dynamical network models of melt pond evolution of varying complexity. △ Less

Submitted 15 December, 2022; originally announced December 2022.

Comments: 13 pages, 13 figures

MSC Class: 86A40

JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset

Authors: Ruth-Ann Armstrong, John Hewitt, Christopher Manning

Abstract: JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois. Many of the most-spoken low-resource languages are creoles. These languages commonly have a lexicon derived from a major world language and a distinctive grammar reflecting the languages of the original speakers and the process of language birth by creolization. This gives them a distincti… ▽ More JamPatoisNLI provides the first dataset for natural language inference in a creole language, Jamaican Patois. Many of the most-spoken low-resource languages are creoles. These languages commonly have a lexicon derived from a major world language and a distinctive grammar reflecting the languages of the original speakers and the process of language birth by creolization. This gives them a distinctive place in exploring the effectiveness of transfer from large monolingual or multilingual pretrained models. While our work, along with previous work, shows that transfer from these models to low-resource languages that are unrelated to languages in their training set is not very effective, we would expect stronger results from transfer to creoles. Indeed, our experiments show considerably better results from few-shot learning of JamPatoisNLI than for such unrelated languages, and help us begin to understand how the unique relationship between creoles and their high-resource base languages affect cross-lingual transfer. JamPatoisNLI, which consists of naturally-occurring premises and expert-written hypotheses, is a step towards steering research into a traditionally underserved language and a useful benchmark for understanding cross-lingual NLP. △ Less

Submitted 6 December, 2022; originally announced December 2022.

Comments: 14 pages, 3 figures, Findings of EMNLP 2022

ACM Class: I.2.7

The Impact of Beam Variations on Power Spectrum Estimation for 21-cm Cosmology I: Simulations of Foreground Contamination for HERA

Authors: Honggeun Kim, Bang D. Nhan, Jacqueline N. Hewitt, Nicholas S. Kern, Joshua S. Dillon, Eloy de Lera Acedo, Scott Dynes, Nivedita Mahesh, Nicolas Fagnoni, David R. DeBoer

Abstract: Detecting cosmological signals from the Epoch of Reionization (EoR) requires high-precision calibration to isolate the cosmological signals from foreground emission. In radio interferometery, perturbed primary beams of antenna elements can disrupt the precise calibration, which results in contaminating the foreground-free region, or the EoR window, in the cylindrically averaged power spectrum. For… ▽ More Detecting cosmological signals from the Epoch of Reionization (EoR) requires high-precision calibration to isolate the cosmological signals from foreground emission. In radio interferometery, perturbed primary beams of antenna elements can disrupt the precise calibration, which results in contaminating the foreground-free region, or the EoR window, in the cylindrically averaged power spectrum. For Hydrogen Epoch of Reionization Array (HERA), we simulate and characterize the perturbed primary beams induced by feed motions such as axial, lateral, and tilting motions, above the 14-meter dish. To understand the effect of the perturbed beams, visibility measurements are modeled with two different foreground components, point sources and diffuse sources, and we find different feed motions present a different reaction to each type of sky source. HERA's redundant-baseline calibration in the presence of non-redundant antenna beams due to feed motions introduces chromatic errors in gain solutions, which produces foreground power leakage into the EoR window. The observed leakage from vertical feed motions comes predominately from point sources around zenith. Furthermore, the observed leakage from horizontal and tilting feed motion comes predominately from the diffuse components near the horizon. Mitigation of chromatic gain errors will be necessary for robust detection of the EoR signals with minimal foreground bias, and this will be discussed in the subsequent paper. △ Less

Submitted 28 October, 2022; originally announced October 2022.

Comments: Accepted for publication in ApJ

Truncation Sampling as Language Model Desmoothing

Authors: John Hewitt, Christopher D. Manning, Percy Liang

Abstract: Long samples of text from neural language models can be of poor quality. Truncation sampling algorithms--like top-$p$ or top-$k$ -- address this by setting some words' probabilities to zero at each step. This work provides framing for the aim of truncation, and an improved algorithm for that aim. We propose thinking of a neural language model as a mixture of a true distribution and a smoothing dis… ▽ More Long samples of text from neural language models can be of poor quality. Truncation sampling algorithms--like top-$p$ or top-$k$ -- address this by setting some words' probabilities to zero at each step. This work provides framing for the aim of truncation, and an improved algorithm for that aim. We propose thinking of a neural language model as a mixture of a true distribution and a smoothing distribution that avoids infinite perplexity. In this light, truncation algorithms aim to perform desmoothing, estimating a subset of the support of the true distribution. Finding a good subset is crucial: we show that top-$p$ unnecessarily truncates high-probability words, for example causing it to truncate all words but Trump for a document that starts with Donald. We introduce $η$-sampling, which truncates words below an entropy-dependent probability threshold. Compared to previous algorithms, $η$-sampling generates more plausible long English documents according to humans, is better at breaking out of repetition, and behaves more reasonably on a battery of test distributions. △ Less

Submitted 27 October, 2022; originally announced October 2022.

Comments: Findings of EMNLP, + small fixes

Characterization Of Inpaint Residuals In Interferometric Measurements of the Epoch Of Reionization

Authors: Michael Pagano, Jing Liu, Adrian Liu, Nicholas S. Kern, Aaron Ewall-Wice, Philip Bull, Robert Pascua, Siamak Ravanbakhsh, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Jacob Burba, Steven Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer , et al. (53 additional authors not shown)

Abstract: Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum du… ▽ More Radio Frequency Interference (RFI) is one of the systematic challenges preventing 21cm interferometric instruments from detecting the Epoch of Reionization. To mitigate the effects of RFI on data analysis pipelines, numerous inpaint techniques have been developed to restore RFI corrupted data. We examine the qualitative and quantitative errors introduced into the visibilities and power spectrum due to inpainting. We perform our analysis on simulated data as well as real data from the Hydrogen Epoch of Reionization Array (HERA) Phase 1 upper limits. We also introduce a convolutional neural network that capable of inpainting RFI corrupted data in interferometric instruments. We train our network on simulated data and show that our network is capable at inpainting real data without requiring to be retrained. We find that techniques that incorporate high wavenumbers in delay space in their modeling are best suited for inpainting over narrowband RFI. We also show that with our fiducial parameters Discrete Prolate Spheroidal Sequences (DPSS) and CLEAN provide the best performance for intermittent ``narrowband'' RFI while Gaussian Progress Regression (GPR) and Least Squares Spectral Analysis (LSSA) provide the best performance for larger RFI gaps. However we caution that these qualitative conclusions are sensitive to the chosen hyperparameters of each inpainting technique. We find these results to be consistent in both simulated and real visibilities. We show that all inpainting techniques reliably reproduce foreground dominated modes in the power spectrum. Since the inpainting techniques should not be capable of reproducing noise realizations, we find that the largest errors occur in the noise dominated delay modes. We show that in the future, as the noise level of the data comes down, CLEAN and DPSS are most capable of reproducing the fine frequency structure in the visibilities of HERA data. △ Less

Submitted 20 February, 2023; v1 submitted 26 October, 2022; originally announced October 2022.

Comments: 21 pages, 13 figures

Improved Constraints on the 21 cm EoR Power Spectrum and the X-Ray Heating of the IGM with HERA Phase I Observations

Authors: The HERA Collaboration, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Rennan Barkana, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Daniela Breitman, Philip Bull, Jacob Burba, Steve Carey, Chris L. Carilli, Carina Cheng, Samir Choudhuri, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Joshua S. Dillon , et al. (70 additional authors not shown)

Abstract: We report the most sensitive upper limits to date on the 21 cm epoch of reionization power spectrum using 94 nights of observing with Phase I of the Hydrogen Epoch of Reionization Array (HERA). Using similar analysis techniques as in previously reported limits (HERA Collaboration 2022a), we find at 95% confidence that $Δ^2(k = 0.34$ $h$ Mpc$^{-1}$) $\leq 457$ mK$^2$ at $z = 7.9$ and that… ▽ More We report the most sensitive upper limits to date on the 21 cm epoch of reionization power spectrum using 94 nights of observing with Phase I of the Hydrogen Epoch of Reionization Array (HERA). Using similar analysis techniques as in previously reported limits (HERA Collaboration 2022a), we find at 95% confidence that $Δ^2(k = 0.34$ $h$ Mpc$^{-1}$) $\leq 457$ mK$^2$ at $z = 7.9$ and that $Δ^2 (k = 0.36$ $h$ Mpc$^{-1}) \leq 3,496$ mK$^2$ at $z = 10.4$, an improvement by a factor of 2.1 and 2.6 respectively. These limits are mostly consistent with thermal noise over a wide range of $k$ after our data quality cuts, despite performing a relatively conservative analysis designed to minimize signal loss. Our results are validated with both statistical tests on the data and end-to-end pipeline simulations. We also report updated constraints on the astrophysics of reionization and the cosmic dawn. Using multiple independent modeling and inference techniques previously employed by HERA Collaboration (2022b), we find that the intergalactic medium must have been heated above the adiabatic cooling limit at least as early as $z = 10.4$, ruling out a broad set of so-called "cold reionization" scenarios. If this heating is due to high-mass X-ray binaries during the cosmic dawn, as is generally believed, our result's 99% credible interval excludes the local relationship between soft X-ray luminosity and star formation and thus requires heating driven by evolved low-metallicity stars. △ Less

Submitted 19 January, 2023; v1 submitted 10 October, 2022; originally announced October 2022.

Comments: 57 pages, 37 figures. Updated to match the accepted ApJ version. Corresponding author: Joshua S. Dillon

Journal ref: 2023 ApJ 945 124

Impact of instrument and data characteristics in the interferometric reconstruction of the 21 cm power spectrum

Authors: Adélie Gorce, Samskruthi Ganjam, Adrian Liu, Steven G. Murray, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Jacob Burba, Steven Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Joshua S. Dillon , et al. (53 additional authors not shown)

Abstract: Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand t… ▽ More Combining the visibilities measured by an interferometer to form a cosmological power spectrum is a complicated process. In a delay-based analysis, the mapping between instrumental and cosmological space is not a one-to-one relation. Instead, neighbouring modes contribute to the power measured at one point, with their respective contributions encoded in the window functions. To better understand the power measured by an interferometer, we assess the impact of instrument characteristics and analysis choices on these window functions. Focusing on the Hydrogen Epoch of Reionization Array (HERA) as a case study, we find that long-baseline observations correspond to enhanced low-k tails of the window functions, which facilitate foreground leakage, whilst an informed choice of bandwidth and frequency taper can reduce said tails. With simple test cases and realistic simulations, we show that, apart from tracing mode mixing, the window functions help accurately reconstruct the power spectrum estimator of simulated visibilities. The window functions depend strongly on the beam chromaticity, and less on its spatial structure - a Gaussian approximation, ignoring side lobes, is sufficient. Finally, we investigate the potential of asymmetric window functions, down-weighting the contribution of low-k power to avoid foreground leakage. The window functions presented here correspond to the latest HERA upper limits for the full Phase I data. They allow an accurate reconstruction of the power spectrum measured by the instrument and will be used in future analyses to confront theoretical models and data directly in cylindrical space. △ Less

Submitted 11 January, 2023; v1 submitted 7 October, 2022; originally announced October 2022.

Comments: 18 pages, 19 figures, accepted for publication in MNRAS

Test Time Transform Prediction for Open Set Histopathological Image Recognition

Authors: Adrian Galdran, Katherine J. Hewitt, Narmin L. Ghaffari, Jakob N. Kather, Gustavo Carneiro, Miguel A. González Ballester

Abstract: Tissue typology annotation in Whole Slide histological images is a complex and tedious, yet necessary task for the development of computational pathology models. We propose to address this problem by applying Open Set Recognition techniques to the task of jointly classifying tissue that belongs to a set of annotated classes, e.g. clinically relevant tissue categories, while rejecting in test time… ▽ More Tissue typology annotation in Whole Slide histological images is a complex and tedious, yet necessary task for the development of computational pathology models. We propose to address this problem by applying Open Set Recognition techniques to the task of jointly classifying tissue that belongs to a set of annotated classes, e.g. clinically relevant tissue categories, while rejecting in test time Open Set samples, i.e. images that belong to categories not present in the training set. To this end, we introduce a new approach for Open Set histopathological image recognition based on training a model to accurately identify image categories and simultaneously predict which data augmentation transform has been applied. In test time, we measure model confidence in predicting this transform, which we expect to be lower for images in the Open Set. We carry out comprehensive experiments in the context of colorectal cancer assessment from histological images, which provide evidence on the strengths of our approach to automatically identify samples from unknown categories. Code is released at https://github.com/agaldran/t3po . △ Less

Submitted 27 June, 2022; v1 submitted 20 June, 2022; originally announced June 2022.

Comments: Accepted to MICCAI 2022

Search for new cosmic-ray acceleration sites within the 4FGL catalog Galactic plane sources

Authors: Fermi-LAT Collaboration, S. Abdollahi, F. Acero, M. Ackermann, L. Baldini, J. Ballet, G. Barbiellini, D. Bastieri, R. Bellazzini, B. Berenji, A. Berretta, E. Bissaldi, R. D. Blandford, R. Bonino, P. Bruel, S. Buson, R. A. Cameron, R. Caputo, P. A. Caraveo, D. Castro, G. Chiaro, N. Cibrario, S. Ciprini, J. Coronado-Blázquez, M. Crnogorcevic , et al. (95 additional authors not shown)

Abstract: Cosmic rays are mostly composed of protons accelerated to relativistic speeds. When those protons encounter interstellar material, they produce neutral pions which in turn decay into gamma rays. This offers a compelling way to identify the acceleration sites of protons. A characteristic hadronic spectrum, with a low-energy break around 200 MeV, was detected in the gamma-ray spectra of four Superno… ▽ More Cosmic rays are mostly composed of protons accelerated to relativistic speeds. When those protons encounter interstellar material, they produce neutral pions which in turn decay into gamma rays. This offers a compelling way to identify the acceleration sites of protons. A characteristic hadronic spectrum, with a low-energy break around 200 MeV, was detected in the gamma-ray spectra of four Supernova Remnants (SNRs), IC 443, W44, W49B and W51C, with the Fermi Large Area Telescope. This detection provided direct evidence that cosmic-ray protons are (re-)accelerated in SNRs. Here, we present a comprehensive search for low-energy spectral breaks among 311 4FGL catalog sources located within 5 degrees from the Galactic plane. Using 8 years of data from the Fermi Large Area Telescope between 50 MeV and 1 GeV, we find and present the spectral characteristics of 56 sources with a spectral break confirmed by a thorough study of systematic uncertainty. Our population of sources includes 13 SNRs for which the proton-proton interaction is enhanced by the dense target material; the high-mass gamma-ray binary LS~I +61 303; the colliding wind binary eta Carinae; and the Cygnus star-forming region. This analysis better constrains the origin of the gamma-ray emission and enlarges our view to potential new cosmic-ray acceleration sites. △ Less

Submitted 6 May, 2022; originally announced May 2022.

Comments: Accepted for publication in The Astrophysical Journal

Direct Optimal Mapping for 21cm Cosmology: A Demonstration with the Hydrogen Epoch of Reionization Array

Authors: Zhilei Xu, Jacqueline N. Hewitt, Kai-Feng Chen, Honggeun Kim, Joshua S. Dillon, Nicholas S. Kern, Miguel F. Morales, Bryna J. Hazelton, Ruby Byrne, Nicolas Fagnoni, Eloy de Lera Acedo, Zara Abdurashidova, Tyrone Adams, James E. Aguirre, Paul Alexander, Zaki S. Ali, Rushelle Baartman, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Jacob Burba , et al. (56 additional authors not shown)

Abstract: Motivated by the desire for wide-field images with well-defined statistical properties for 21cm cosmology, we implement an optimal mapping pipeline that computes a maximum likelihood estimator for the sky using the interferometric measurement equation. We demonstrate this direct optimal mapping with data from the Hydrogen Epoch of Reionization (HERA) Phase I observations. After validating the pipe… ▽ More Motivated by the desire for wide-field images with well-defined statistical properties for 21cm cosmology, we implement an optimal mapping pipeline that computes a maximum likelihood estimator for the sky using the interferometric measurement equation. We demonstrate this direct optimal mapping with data from the Hydrogen Epoch of Reionization (HERA) Phase I observations. After validating the pipeline with simulated data, we develop a maximum likelihood figure-of-merit for comparing four sky models at 166MHz with a bandwidth of 100kHz. The HERA data agree with the GLEAM catalogs to <10%. After subtracting the GLEAM point sources, the HERA data discriminate between the different continuum sky models, providing most support for the model of Byrne et al. 2021. We report the computation cost for mapping the HERA Phase I data and project the computation for the HERA 320-antenna data; both are feasible with a modern server. The algorithm is broadly applicable to other interferometers and is valid for wide-field and non-coplanar arrays. △ Less

Submitted 26 October, 2022; v1 submitted 12 April, 2022; originally announced April 2022.

Comments: 16 pages, 10 figures, 2 tables, published on ApJ

A Gamma-ray Pulsar Timing Array Constrains the Nanohertz Gravitational Wave Background

Authors: M. Ajello, W. B. Atwood, L. Baldini, J. Ballet, G. Barbiellini, D. Bastieri, R. Bellazzini, A. Berretta, B. Bhattacharyya, E. Bissaldi, R. D. Blandford, E. Bloom, R. Bonino, P. Bruel, R. Buehler, E. Burns, S. Buson, R. A. Cameron, P. A. Caraveo, E. Cavazzuti, N. Cibrario, S. Ciprini, C. J. Clark, I. Cognard, J. Coronado-Blázquez , et al. (107 additional authors not shown)

Abstract: After large galaxies merge, their central supermassive black holes are expected to form binary systems whose orbital motion generates a gravitational wave background (GWB) at nanohertz frequencies. Searches for this background utilize pulsar timing arrays, which perform long-term monitoring of millisecond pulsars (MSPs) at radio wavelengths. We use 12.5 years of Fermi Large Area Telescope data to… ▽ More After large galaxies merge, their central supermassive black holes are expected to form binary systems whose orbital motion generates a gravitational wave background (GWB) at nanohertz frequencies. Searches for this background utilize pulsar timing arrays, which perform long-term monitoring of millisecond pulsars (MSPs) at radio wavelengths. We use 12.5 years of Fermi Large Area Telescope data to form a gamma-ray pulsar timing array. Results from 35 bright gamma-ray pulsars place a 95\% credible limit on the GWB characteristic strain of $1.0\times10^{-14}$ at 1 yr$^{-1}$, which scales as the observing time span $t_{\mathrm{obs}}^{-13/6}$. This direct measurement provides an independent probe of the GWB while offering a check on radio noise models. △ Less

Submitted 11 April, 2022; originally announced April 2022.

Comments: 3 figures in the main text. 3 figures and 8 tables are in the supplementary material

Incremental Fermi Large Area Telescope Fourth Source Catalog

Authors: Fermi-LAT collaboration, :, Soheila Abdollahi, Fabio Acero, Luca Baldini, Jean Ballet, Denis Bastieri, Ronaldo Bellazzini, Bijan Berenji, Alessandra Berretta, Elisabetta Bissaldi, Roger D. Blandford, Elliott Bloom, Raffaella Bonino, Ari Brill, Richard J. Britto, Philippe Bruel, Toby H. Burnett, Sara Buson, Rob A. Cameron, Regina Caputo, Patrizia A. Caraveo, Daniel Castro, Sylvain Chaty, Teddy C. Cheung , et al. (116 additional authors not shown)

Abstract: We present an incremental version (4FGL-DR3, for Data Release 3) of the fourth Fermi-LAT catalog of gamma-ray sources. Based on the first twelve years of science data in the energy range from 50 MeV to 1 TeV, it contains 6658 sources. The analysis improves on that used for the 4FGL catalog over eight years of data: more sources are fit with curved spectra, we introduce a more robust spectral param… ▽ More We present an incremental version (4FGL-DR3, for Data Release 3) of the fourth Fermi-LAT catalog of gamma-ray sources. Based on the first twelve years of science data in the energy range from 50 MeV to 1 TeV, it contains 6658 sources. The analysis improves on that used for the 4FGL catalog over eight years of data: more sources are fit with curved spectra, we introduce a more robust spectral parameterization for pulsars, and we extend the spectral points to 1 TeV. The spectral parameters, spectral energy distributions, and associations are updated for all sources. Light curves are rebuilt for all sources with 1 yr intervals (not 2 month intervals). Among the 5064 original 4FGL sources, 16 were deleted, 112 are formally below the detection threshold over 12 yr (but are kept in the list), while 74 are newly associated, 10 have an improved association, and seven associations were withdrawn. Pulsars are split explicitly between young and millisecond pulsars. Pulsars and binaries newly detected in LAT sources, as well as more than 100 newly classified blazars, are reported. We add three extended sources and 1607 new point sources, mostly just above the detection threshold, among which eight are considered identified, and 699 have a plausible counterpart at other wavelengths. We discuss degree-scale residuals to the global sky model and clusters of soft unassociated point sources close to the Galactic plane, which are possibly related to limitations of the interstellar emission model and missing extended sources. △ Less

Submitted 10 May, 2022; v1 submitted 26 January, 2022; originally announced January 2022.

Comments: accepted in ApJS; follow-up paper to 1902.10045

Journal ref: ApJS 260, 53 (2022)

Bendocapillary Instability of Liquid in a Flexible-Walled Channel

Authors: Alexander T. Bradley, Ian J. Hewitt, Dominic Vella

Abstract: We study the bendocapillary instability of a liquid droplet that part fills a flexible walled channel. Inspired by experiments in which a `weaving' pattern emerges as droplets of liquid are condensed slowly into deformable microchannels, we develop a mathematical model of this instability. We describe equilibria of the system, and use a combination of numerical methods, and asymptotic analysis in… ▽ More We study the bendocapillary instability of a liquid droplet that part fills a flexible walled channel. Inspired by experiments in which a `weaving' pattern emerges as droplets of liquid are condensed slowly into deformable microchannels, we develop a mathematical model of this instability. We describe equilibria of the system, and use a combination of numerical methods, and asymptotic analysis in the limit of small channel wall deflections, to elucidate the key features of this instability. We find that configurations are always unstable to perturbations of sufficiently small wavenumber, that the growth rate of the instability is highly sensitive to the volume of liquid in the channel, and that both wetting and non-wetting configurations are susceptible to the instability in the same channel. Insight into novel interfacial instabilities opens the possibility for their control and thus exploitation in processes such as microfabrication. △ Less

Submitted 7 December, 2022; v1 submitted 4 January, 2022; originally announced January 2022.

Comments: 23 pages, 8 figures

Journal ref: J. Fluid Mech. 955, A26 (2023)

Numerical approximation of viscous contact problems applied to glacial sliding

Authors: Gonzalo G. de Diego, Patrick E. Farrell, Ian J. Hewitt

Abstract: Viscous contact problems describe the time evolution of fluid flows in contact with a surface from which they can detach and reattach. These problems are of particular importance in glaciology, where they arise in the study of grounding lines and subglacial cavities. In this work, we propose a novel numerical method for solving viscous contact problems based on a mixed formulation with Lagrange mu… ▽ More Viscous contact problems describe the time evolution of fluid flows in contact with a surface from which they can detach and reattach. These problems are of particular importance in glaciology, where they arise in the study of grounding lines and subglacial cavities. In this work, we propose a novel numerical method for solving viscous contact problems based on a mixed formulation with Lagrange multipliers of a variational inequality involving the Stokes equation. The advection equation for evolving the geometry of the domain occupied by the fluid is then solved via a specially-built upwinding scheme, leading to a robust and accurate algorithm for viscous contact problems. We first verify the method by comparing the numerical results to analytical results obtained by a linearised method. Then, we use this numerical scheme to reconstruct friction laws for glacial sliding with cavitation. Finally, we compute the evolution of cavities from a steady state under oscillating water pressures. The results depend strongly on the location of the initial steady state along the friction law. In particular, we find that if the steady state is located on the downsloping or rate-weakening part of the friction law, the cavity evolves towards the upsloping section, indicating that the downsloping part is unstable. △ Less

Submitted 21 January, 2022; v1 submitted 10 November, 2021; originally announced November 2021.

MSC Class: 86A40; 65K15 ACM Class: G.1.8; G.1.10

Automated Detection of Antenna Malfunctions in Large-N Interferometers: A Case Study with the Hydrogen Epoch of Reionization Array

Authors: Dara Storer, Joshua S. Dillon, Daniel C. Jacobs, Miguel F. Morales, Bryna J. Hazelton, Aaron Ewall-Wice, Zara Abdurashidova, James E. Aguirre, Paul Alexander, Zaki S. Ali, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Jacob Burba, Steven Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer, Eloy de Lera Acedo, Matt Dexter, Scott Dynes , et al. (53 additional authors not shown)

Abstract: We present a framework for identifying and flagging malfunctioning antennas in large radio interferometers. We outline two distinct categories of metrics designed to detect outliers along known failure modes of large arrays: cross-correlation metrics, based on all antenna pairs, and auto-correlation metrics, based solely on individual antennas. We define and motivate the statistical framework for… ▽ More We present a framework for identifying and flagging malfunctioning antennas in large radio interferometers. We outline two distinct categories of metrics designed to detect outliers along known failure modes of large arrays: cross-correlation metrics, based on all antenna pairs, and auto-correlation metrics, based solely on individual antennas. We define and motivate the statistical framework for all metrics used, and present tailored visualizations that aid us in clearly identifying new and existing systematics. We implement these techniques using data from 105 antennas in the Hydrogen Epoch of Reionization Array (HERA) as a case study. Finally, we provide a detailed algorithm for implementing these metrics as flagging tools on real data sets. △ Less

Submitted 4 May, 2022; v1 submitted 26 September, 2021; originally announced September 2021.

Comments: 31 pages, 17 figures

Journal ref: Radio Science, vol. 57, no. 1, 2022

Conditional probing: measuring usable information beyond a baseline

Authors: John Hewitt, Kawin Ethayarajh, Percy Liang, Christopher D. Manning

Abstract: Probing experiments investigate the extent to which neural representations make properties -- like part-of-speech -- predictable. One suggests that a representation encodes a property if probing that representation produces higher accuracy than probing a baseline representation like non-contextual word embeddings. Instead of using baselines as a point of comparison, we're interested in measuring i… ▽ More Probing experiments investigate the extent to which neural representations make properties -- like part-of-speech -- predictable. One suggests that a representation encodes a property if probing that representation produces higher accuracy than probing a baseline representation like non-contextual word embeddings. Instead of using baselines as a point of comparison, we're interested in measuring information that is contained in the representation but not in the baseline. For example, current methods can detect when a representation is more useful than the word identity (a baseline) for predicting part-of-speech; however, they cannot detect when the representation is predictive of just the aspects of part-of-speech not explainable by the word identity. In this work, we extend a theory of usable information called $\mathcal{V}$-information and propose conditional probing, which explicitly conditions on the information in the baseline. In a case study, we find that after conditioning on non-contextual word embeddings, properties like part-of-speech are accessible at deeper layers of a network than previously thought. △ Less

Submitted 19 September, 2021; originally announced September 2021.

Comments: EMNLP 2021 + typo fixes

HERA Phase I Limits on the Cosmic 21-cm Signal: Constraints on Astrophysics and Cosmology During the Epoch of Reionization

Authors: The HERA Collaboration, Zara Abdurashidova, James E. Aguirre, Paul Alexander, Zaki Ali, Yanga Balfour, Rennan Barkana, Adam Beardsley, Gianni Bernardi, Tashalee Billings, Judd Bowman, Richard Bradley, Phillip Bull, Jacob Burba, Steven Carey, Christopher Carilli, Carina Cheng, David DeBoer, Matthew Dexter, Eloy de Lera Acedo, Joshua Dillon, John Ely, Aaron Ewall-Wice, Nicolas Fagnoni, Anastasia Fialkov , et al. (59 additional authors not shown)

Abstract: Recently, the Hydrogen Epoch of Reionization Array (HERA) collaboration has produced the experiment's first upper limits on the power spectrum of 21-cm fluctuations at z~8 and 10. Here, we use several independent theoretical models to infer constraints on the intergalactic medium (IGM) and galaxies during the epoch of reionization (EoR) from these limits. We find that the IGM must have been heated… ▽ More Recently, the Hydrogen Epoch of Reionization Array (HERA) collaboration has produced the experiment's first upper limits on the power spectrum of 21-cm fluctuations at z~8 and 10. Here, we use several independent theoretical models to infer constraints on the intergalactic medium (IGM) and galaxies during the epoch of reionization (EoR) from these limits. We find that the IGM must have been heated above the adiabatic cooling threshold by z~8, independent of uncertainties about the IGM ionization state and the nature of the radio background. Combining HERA limits with galaxy and EoR observations constrains the spin temperature of the z~8 neutral IGM to 27 K < T_S < 630 K (2.3 K < T_S < 640 K) at 68% (95%) confidence. They therefore also place a lower bound on X-ray heating, a previously unconstrained aspects of early galaxies. For example, if the CMB dominates the z~8 radio background, the new HERA limits imply that the first galaxies produced X-rays more efficiently than local ones (with soft band X-ray luminosities per star formation rate constrained to L_X/SFR = { 10^40.2, 10^41.9 } erg/s/(M_sun/yr) at 68% confidence), consistent with expectations of X-ray binaries in low-metallicity environments. The z~10 limits require even earlier heating if dark-matter interactions (e.g., through millicharges) cool down the hydrogen gas. Using a model in which an extra radio background is produced by galaxies, we rule out (at 95% confidence) the combination of high radio and low X-ray luminosities of L_{r,ν}/SFR > 3.9 x 10^24 W/Hz/(M_sun/yr) and L_X/SFR<10^40 erg/s/(M_sun/yr). The new HERA upper limits neither support nor disfavor a cosmological interpretation of the recent EDGES detection. The analysis framework described here provides a foundation for the interpretation of future HERA results. △ Less

Submitted 20 December, 2022; v1 submitted 16 August, 2021; originally announced August 2021.

Comments: 40 pages, 19 figures, accepted to ApJ

On the Opportunities and Risks of Foundation Models

Authors: Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh , et al. (89 additional authors not shown)

Abstract: AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their cap… ▽ More AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles(e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on standard deep learning and transfer learning, their scale results in new emergent capabilities,and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature. △ Less

Submitted 12 July, 2022; v1 submitted 16 August, 2021; originally announced August 2021.

Comments: Authored by the Center for Research on Foundation Models (CRFM) at the Stanford Institute for Human-Centered Artificial Intelligence (HAI). Report page with citation guidelines: https://crfm.stanford.edu/report.html

Hybrid cosmic ray measurements using the IceAct telescopes in coincidence with the IceCube and IceTop detectors

Authors: Larissa Paul, Matthias Plum, Merlin Schaufel, Thomas Bretz, Giang Do, John W. Hewitt, Frank Maslowski, Florian Rehbein, Johannes Schäfer, Adrian Zink

Abstract: IceAct is a proposed surface array of compact (50 cm diameter) and cost-effective Imaging Air Cherenkov Telescopes installed at the site of the IceCube Neutrino Observatory at the geographic South Pole. Since January 2019, two IceAct telescope demonstrators, featuring 61 silicon pho- tomultiplier (SiPM) pixels have been taking data in the center of the IceTop surface array during the austral winte… ▽ More IceAct is a proposed surface array of compact (50 cm diameter) and cost-effective Imaging Air Cherenkov Telescopes installed at the site of the IceCube Neutrino Observatory at the geographic South Pole. Since January 2019, two IceAct telescope demonstrators, featuring 61 silicon pho- tomultiplier (SiPM) pixels have been taking data in the center of the IceTop surface array during the austral winter. We present the first analysis of hybrid cosmic ray events detected by the IceAct imaging air-Cherenkov telescopes in coincidence with the IceCube Neutrino Observatory, includ- ing the IceTop surface array and the IceCube in-ice array. By featuring an energy threshold of about 10 TeV and a wide field-of-view, the IceAct telescopes show promising capabilities of im- proving current cosmic ray composition studies: measuring the Cherenkov light emissions in the atmosphere adds new information about the shower development not accessible with the current detectors, enabling significantly better primary particle type discrimination on a statistical basis. The hybrid measurement also allows for detailed feasibility studies of detector cross-calibration and of cosmic ray veto capabilities for neutrino analyses. We present the performance of the telescopes, the results from the analysis of two years of data, and an outlook of a hybrid simulation for a future telescope array. △ Less

Submitted 12 August, 2021; originally announced August 2021.

Comments: Presented at the 37th International Cosmic Ray Conference (ICRC 2021). See arXiv:2107.06966 for all IceCube contributions

Report number: PoS-ICRC2021-276

First Results from HERA Phase I: Upper Limits on the Epoch of Reionization 21 cm Power Spectrum

Authors: The HERA Collaboration, Zara Abdurashidova, James E. Aguirre, Paul Alexander, Zaki S. Ali, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Jacob Burba, Steve Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer, Matt Dexter, Eloy de Lera Acedo, Taylor Dibblee-Barkman, Joshua S. Dillon, John Ely, Aaron Ewall-Wice, Nicolas Fagnoni, Randall Fritz , et al. (52 additional authors not shown)

Abstract: We report upper-limits on the Epoch of Reionization (EoR) 21 cm power spectrum at redshifts 7.9 and 10.4 with 18 nights of data ($\sim36$ hours of integration) from Phase I of the Hydrogen Epoch of Reionization Array (HERA). The Phase I data show evidence for systematics that can be largely suppressed with systematic models down to a dynamic range of $\sim10^9$ with respect to the peak foreground… ▽ More We report upper-limits on the Epoch of Reionization (EoR) 21 cm power spectrum at redshifts 7.9 and 10.4 with 18 nights of data ($\sim36$ hours of integration) from Phase I of the Hydrogen Epoch of Reionization Array (HERA). The Phase I data show evidence for systematics that can be largely suppressed with systematic models down to a dynamic range of $\sim10^9$ with respect to the peak foreground power. This yields a 95% confidence upper limit on the 21 cm power spectrum of $Δ^2_{21} \le (30.76)^2\ {\rm mK}^2$ at $k=0.192\ h\ {\rm Mpc}^{-1}$ at $z=7.9$, and also $Δ^2_{21} \le (95.74)^2\ {\rm mK}^2$ at $k=0.256\ h\ {\rm Mpc}^{-1}$ at $z=10.4$. At $z=7.9$, these limits are the most sensitive to-date by over an order of magnitude. While we find evidence for residual systematics at low line-of-sight Fourier $k_\parallel$ modes, at high $k_\parallel$ modes we find our data to be largely consistent with thermal noise, an indicator that the system could benefit from deeper integrations. The observed systematics could be due to radio frequency interference, cable sub-reflections, or residual instrumental cross-coupling, and warrant further study. This analysis emphasizes algorithms that have minimal inherent signal loss, although we do perform a careful accounting in a companion paper of the small forms of loss or bias associated with the pipeline. Overall, these results are a promising first step in the development of a tuned, instrument-specific analysis pipeline for HERA, particularly as Phase II construction is completed en route to reaching the full sensitivity of the experiment. △ Less

Submitted 4 August, 2021; originally announced August 2021.

Comments: Accepted to ApJ. https://reionization.org/science/public-data-release-1/

On the finite element approximation of a semicoercive Stokes variational inequality arising in glaciology

Authors: Gonzalo G. de Diego, Patrick E. Farrell, Ian J. Hewitt

Abstract: Stokes variational inequalities arise in the formulation of glaciological problems involving contact. We consider the problem of a two-dimensional marine ice sheet with a grounding line, although the analysis presented here is extendable to other contact problems in glaciology, such as that of subglacial cavitation. The analysis of this problem and its discretisation is complicated by the nonlinea… ▽ More Stokes variational inequalities arise in the formulation of glaciological problems involving contact. We consider the problem of a two-dimensional marine ice sheet with a grounding line, although the analysis presented here is extendable to other contact problems in glaciology, such as that of subglacial cavitation. The analysis of this problem and its discretisation is complicated by the nonlinear rheology commonly used for modelling ice, the enforcement of a friction boundary condition given by a power law, and the presence of rigid modes in the velocity space, which render the variational inequality semicoercive. In this work, we consider a mixed formulation of this variational inequality involving a Lagrange multiplier and provide an analysis of its finite element approximation. Error estimates in the presence of rigid modes are obtained by means of a specially-built projection operator onto the subspace of rigid modes and a Korn-type inequality. These proofs rely on the fact that the subspace of rigid modes is at most one-dimensional. Numerical results are reported to validate the error estimates. △ Less

Submitted 11 October, 2022; v1 submitted 30 July, 2021; originally announced August 2021.

MSC Class: 65N12; 65N15; 65N30; 86A40

Catalog of Long-Term Transient Sources in the First 10 Years of Fermi-LAT Data

Authors: L. Baldini, J. Ballet, D. Bastieri, J. Becerra Gonzalez, R. Bellazzini, A. Berretta, E. Bissaldi, R. D. Blandford, E. D. Bloom, R. Bonino, E. Bottacini, P. Bruel, S. Buson, R. A. Cameron, P. A. Caraveo, E. Cavazzuti, S. Chen, G. Chiaro, D. Ciangottini, S. Ciprini, P. Cristarella Orestano, M. Crnogorcevic, S. Cutini, F. D'Ammando, P. de la Torre Luque , et al. (90 additional authors not shown)

Abstract: We present the first Fermi Large Area Telescope (LAT) catalog of long-term $γ$-ray transient sources (1FLT). This comprises sources that were detected on monthly time intervals during the first decade of Fermi-LAT operations. The monthly time scale allows us to identify transient and variable sources that were not yet reported in other Fermi-LAT catalogs. The monthly datasets were analyzed using a… ▽ More We present the first Fermi Large Area Telescope (LAT) catalog of long-term $γ$-ray transient sources (1FLT). This comprises sources that were detected on monthly time intervals during the first decade of Fermi-LAT operations. The monthly time scale allows us to identify transient and variable sources that were not yet reported in other Fermi-LAT catalogs. The monthly datasets were analyzed using a wavelet-based source detection algorithm that provided the candidate new transient sources. The search was limited to the extragalactic regions of the sky to avoid the dominance of the Galactic diffuse emission at low Galactic latitudes. The transient candidates were then analyzed using the standard Fermi-LAT Maximum Likelihood analysis method. All sources detected with a statistical significance above 4$σ$ in at least one monthly bin were listed in the final catalog. The 1FLT catalog contains 142 transient $γ$-ray sources that are not included in the 4FGL-DR2 catalog. Many of these sources (102) have been confidently associated with Active Galactic Nuclei (AGN): 24 are associated with Flat Spectrum Radio Quasars; 1 with a BL Lac object; 70 with Blazars of Uncertain Type; 3 with Radio Galaxies; 1 with a Compact Steep Spectrum radio source; 1 with a Steep Spectrum Radio Quasar; 2 with AGN of other types. The remaining 40 sources have no candidate counterparts at other wavelengths. The median $γ$-ray spectral index of the 1FLT-AGN sources is softer than that reported in the latest Fermi-LAT AGN general catalog. This result is consistent with the hypothesis that detection of the softest $γ$-ray emitters is less efficient when the data are integrated over year-long intervals. △ Less

Submitted 31 May, 2021; originally announced June 2021.

Comments: 41 pages, 17 figures, 7 tables; Accepted by ApJS on 24 May 2021; Contact Authors: I. Mereu, S. Cutini, E. Cavazzuti, G. Tosti

The DP Color Function of Joins and Vertex-Gluings of Graphs

Authors: Jack Becker, Jade Hewitt, Hemanshu Kaul, Michael Maxfield, Jeffrey A. Mudrock, David Spivey, Seth Thomason, Tim Wagstrom

Abstract: DP-coloring (also called correspondence coloring) is a generalization of list coloring that has been widely studied in recent years after its introduction by Dvořák and Postle in 2015. As the analogue of the chromatic polynomial $P(G,m)$, the DP color function of a graph $G$, denoted $P_{DP}(G,m)$, counts the minimum number of DP-colorings over all possible $m$-fold covers. Chromatic polynomials f… ▽ More DP-coloring (also called correspondence coloring) is a generalization of list coloring that has been widely studied in recent years after its introduction by Dvořák and Postle in 2015. As the analogue of the chromatic polynomial $P(G,m)$, the DP color function of a graph $G$, denoted $P_{DP}(G,m)$, counts the minimum number of DP-colorings over all possible $m$-fold covers. Chromatic polynomials for joins and vertex-gluings of graphs are well understood, but the effect of these graph operations on the DP color function is not known. In this paper we make progress on understanding the DP color function of the join of a graph with a complete graph and vertex-gluings of certain graphs. We also develop tools to study the DP color function under these graph operations, and we study the threshold (smallest $m$) beyond which the DP color function of a graph constructed with these operations equals its chromatic polynomial. △ Less

Submitted 1 July, 2022; v1 submitted 25 April, 2021; originally announced April 2021.

Comments: 26 pages, 1 figure

MSC Class: 05C15; 05C30; 05C69

Effects of model incompleteness on the drift-scan calibration of radio telescopes

Authors: Bharat K. Gehlot, Daniel C. Jacobs, Judd D. Bowman, Nivedita Mahesh, Steven G. Murray, Matthew Kolopanis, Adam P. Beardsley, Zara Abdurashidova, James E. Aguirre, Paul Alexander, Zaki S. Ali, Yanga Balfour, Gianni Bernardi, Tashalee S. Billings, Richard F. Bradley, Phil Bull, Jacob Burba, Steve Carey, Chris L. Carilli, Carina Cheng, David R. DeBoer, Matt Dexter, Eloy de Lera Acedo, Joshua S. Dillon, John Ely , et al. (54 additional authors not shown)

Abstract: Precision calibration poses challenges to experiments probing the redshifted 21-cm signal of neutral hydrogen from the Cosmic Dawn and Epoch of Reionization (z~30-6). In both interferometric and global signal experiments, systematic calibration is the leading source of error. Though many aspects of calibration have been studied, the overlap between the two types of instruments has received less at… ▽ More Precision calibration poses challenges to experiments probing the redshifted 21-cm signal of neutral hydrogen from the Cosmic Dawn and Epoch of Reionization (z~30-6). In both interferometric and global signal experiments, systematic calibration is the leading source of error. Though many aspects of calibration have been studied, the overlap between the two types of instruments has received less attention. We investigate the sky based calibration of total power measurements with a HERA dish and an EDGES style antenna to understand the role of auto-correlations in the calibration of an interferometer and the role of sky in calibrating a total power instrument. Using simulations we study various scenarios such as time variable gain, incomplete sky calibration model, and primary beam model. We find that temporal gain drifts, sky model incompleteness, and beam inaccuracies cause biases in the receiver gain amplitude and the receiver temperature estimates. In some cases, these biases mix spectral structure between beam and sky resulting in spectrally variable gain errors. Applying the calibration method to the HERA and EDGES data, we find good agreement with calibration via the more standard methods. Although instrumental gains are consistent with beam and sky errors similar in scale to those simulated, the receiver temperatures show significant deviations from expected values. While we show that it is possible to partially mitigate biases due to model inaccuracies by incorporating a time-dependent gain model in calibration, the resulting errors on calibration products are larger and more correlated. Completely addressing these biases will require more accurate sky and primary beam models. △ Less

Submitted 15 July, 2021; v1 submitted 25 April, 2021; originally announced April 2021.

Comments: 16 pages, 13 figures, 1 table; accepted for publication in MNRAS main journal

Droplet trapping in bendotaxis caused by contact angle hysteresis

Authors: Alexander T. Bradley, Ian J. Hewitt, Dominic Vella

Abstract: Passive droplet transport mechanisms, in which continuous external energy input is not required for motion, have received significant attention in recent years. Experimental studies of such mechanisms often ignore, or use careful treatments to minimize, contact angle hysteresis, which can impede droplet motion, or even arrest it completely. Here, we consider the effect of contact angle hysteresis… ▽ More Passive droplet transport mechanisms, in which continuous external energy input is not required for motion, have received significant attention in recent years. Experimental studies of such mechanisms often ignore, or use careful treatments to minimize, contact angle hysteresis, which can impede droplet motion, or even arrest it completely. Here, we consider the effect of contact angle hysteresis on bendotaxis, a mechanism in which droplets spontaneously deform an elastic channel via capillary pressure and thereby move. Here, we seek to understand when contact angle hysteresis prevents bendotaxis. We supplement a previous mathematical model of the dynamics of bendotaxis with a simple model of contact angle hysteresis, and show that this model predicts droplet trapping when hysteresis is sufficiently strong. By identifying the equilibrium configurations adopted by these trapped droplets and assessing their linear stability, we uncover a sensitive dependence of bendotaxis on contact angle hysteresis and develop criteria to describe when droplets will be trapped. △ Less

Submitted 6 January, 2022; v1 submitted 20 April, 2021; originally announced April 2021.

Journal ref: Phys. Rev. Fluids 6 114003 (2021)

Refining Targeted Syntactic Evaluation of Language Models

Authors: Benjamin Newman, Kai-Siang Ang, Julia Gong, John Hewitt

Abstract: Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models' syntactic knowledge using hand-crafted minimal pairs of sentences that differ only in the main verb's conjugation. The method evaluates whether language models rate each grammatical sentence as more likely than its ungrammatical counterpart. We identify two distinct goals for TSE. First, eval… ▽ More Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models' syntactic knowledge using hand-crafted minimal pairs of sentences that differ only in the main verb's conjugation. The method evaluates whether language models rate each grammatical sentence as more likely than its ungrammatical counterpart. We identify two distinct goals for TSE. First, evaluating the systematicity of a language model's syntactic knowledge: given a sentence, can it conjugate arbitrary verbs correctly? Second, evaluating a model's likely behavior: given a sentence, does the model concentrate its probability mass on correctly conjugated verbs, even if only on a subset of the possible verbs? We argue that current implementations of TSE do not directly capture either of these goals, and propose new metrics to capture each goal separately. Under our metrics, we find that TSE overestimates systematicity of language models, but that models score up to 40% better on verbs that they predict are likely in context. △ Less

Submitted 19 April, 2021; originally announced April 2021.

Comments: 14 pages, 5 figures, 3 tables. To appear at NAACL 2021

ACM Class: I.2.7

Validation of the HERA Phase I Epoch of Reionization 21 cm Power Spectrum Software Pipeline

Authors: James E. Aguirre, Steven G. Murray, Robert Pascua, Zachary E. Martinot, Jacob Burba, Joshua S. Dillon, Daniel C. Jacobs, Nicholas S. Kern, Piyanat Kittiwisit, Matthew Kolopanis, Adam Lanman, Adrian Liu, Lily Whitler, Zara Abdurashidova, Paul Alexander, Zaki S. Ali, Yanga Balfour, Adam P. Beardsley, Gianni Bernardi, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Philip Bull, Steve Carey, Chris L. Carilli , et al. (51 additional authors not shown)

Abstract: We describe the validation of the HERA Phase I software pipeline by a series of modular tests, building up to an end-to-end simulation. The philosophy of this approach is to validate the software and algorithms used in the Phase I upper limit analysis on wholly synthetic data satisfying the assumptions of that analysis, not addressing whether the actual data meet these assumptions. We discuss the… ▽ More We describe the validation of the HERA Phase I software pipeline by a series of modular tests, building up to an end-to-end simulation. The philosophy of this approach is to validate the software and algorithms used in the Phase I upper limit analysis on wholly synthetic data satisfying the assumptions of that analysis, not addressing whether the actual data meet these assumptions. We discuss the organization of this validation approach, the specific modular tests performed, and the construction of the end-to-end simulations. We explicitly discuss the limitations in scope of the current simulation effort. With mock visibility data generated from a known analytic power spectrum and a wide range of realistic instrumental effects and foregrounds, we demonstrate that the current pipeline produces power spectrum estimates that are consistent with known analytic inputs to within thermal noise levels (at the 2 sigma level) for k > 0.2 h/Mpc for both bands and fields considered. Our input spectrum is intentionally amplified to enable a strong `detection' at k ~0.2 h/Mpc -- at the level of ~25 sigma -- with foregrounds dominating on larger scales, and thermal noise dominating at smaller scales. Our pipeline is able to detect this amplified input signal after suppressing foregrounds with a dynamic range (foreground to noise ratio) of > 10^7. Our validation test suite uncovered several sources of scale-independent signal loss throughout the pipeline, whose amplitude is well-characterized and accounted for in the final estimates. We conclude with a discussion of the steps required for the next round of data analysis. △ Less

Submitted 19 April, 2021; originally announced April 2021.

Comments: 32 pages, 20 figures. Submitted to the Astrophysical Journal