Showing 1–50 of 279 results for author: Davis, L

Multivariate Representations of Univariate Marked Hawkes Processes

Authors: Louis Davis, Conor Kresin, Boris Baeumer, Ting Wang

Abstract: Univariate marked Hawkes processes are used to model a range of real-world phenomena including earthquake aftershock sequences, contagious disease spread, content diffusion on social media platforms, and order book dynamics. This paper illustrates a fundamental connection between univariate marked Hawkes processes and multivariate Hawkes processes. Exploiting this connection renders a framework th… ▽ More Univariate marked Hawkes processes are used to model a range of real-world phenomena including earthquake aftershock sequences, contagious disease spread, content diffusion on social media platforms, and order book dynamics. This paper illustrates a fundamental connection between univariate marked Hawkes processes and multivariate Hawkes processes. Exploiting this connection renders a framework that can be built upon for expressive and flexible inference on diverse data. Specifically, multivariate unmarked Hawkes representations are introduced as a tool to parameterize univariate marked Hawkes processes. We show that such multivariate representations can asymptotically approximate a large class of univariate marked Hawkes processes, are stationary given the approximated process is stationary, and that resultant conditional intensity parameters are identifiable. A simulation study demonstrates the efficacy of this approach, and provides heuristic bounds for error induced by the relatively larger parameter space of multivariate Hawkes processes. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Comments: 26 pages, 3 figures, submitted to the Annals of Statistics

Identification of Intermediate-mass Black Hole Candidates Among a Sample of Sd Galaxies

Authors: Benjamin L. Davis, Alister W. Graham, Roberto Soria, Zehao Jin, Igor D. Karachentsev, Valentina E. Karachentseva, Elena D'Onghia

Abstract: We analyzed images of every northern hemisphere Sd galaxy listed in the Third Reference Catalogue of Bright Galaxies (RC3) with a relatively face-on inclination ($θ\leq30°$). Specifically, we measured the spiral arms' winding angle, $φ$, in 85 galaxies. We applied a novel black hole mass planar scaling relation involving the rotational velocities (from the literature) and pitch angles of each gala… ▽ More We analyzed images of every northern hemisphere Sd galaxy listed in the Third Reference Catalogue of Bright Galaxies (RC3) with a relatively face-on inclination ($θ\leq30°$). Specifically, we measured the spiral arms' winding angle, $φ$, in 85 galaxies. We applied a novel black hole mass planar scaling relation involving the rotational velocities (from the literature) and pitch angles of each galaxy to predict central black hole masses. This yielded 23 galaxies, each having at least a 50% chance of hosting a central intermediate-mass black hole (IMBH), $10^2<M_\mathrm{BH}\leq10^5\,\mathrm{M}_\odot$. These 23 nearby ($\lesssim$50 Mpc) targets may be suitable for an array of follow-up observations to check for active nuclei. Based on our full sample of 85 Sd galaxies, we estimate that the typical Sd galaxy (which tends to be bulgeless) harbors a black hole with $\log(M_\mathrm{BH}/\mathrm{M}_\odot)=6.00\pm0.14$, but with a 27.7% chance of hosting an IMBH, making this morphological type of galaxy fertile ground for hunting elusive IMBHs. Thus, we find that a $\sim$$10^6\,\mathrm{M}_\odot$ black hole corresponds roughly to the onset of bulge development and serves as a conspicuous waypoint along the galaxy-SMBH coevolution journey. Our survey suggests that $>$1.22% of bright galaxies ($B_{\rm T}\lesssim15.5$ mag) in the local Universe host an IMBH (i.e., the "occupation fraction"), which implies a number density $>$$4.96\times10^{-6}$ Mpc$^{-3}$ for central IMBHs. Finally, we observe that Sd galaxies exhibit an unexpected diversity of properties that resemble the general population of spiral galaxies, albeit with an enhanced signature of the eponymous prototypical traits (i.e., low masses, loosely wound spiral arms, and smaller rotational velocities). △ Less

Submitted 26 June, 2024; v1 submitted 9 June, 2024; originally announced June 2024.

Comments: Unedited manuscript (30 pages, 5 figures, and 1 table) accepted by The Astrophysical Journal on June 7, 2024

A Multidimensional Fractional Hawkes Process for Multiple Earthquake Mainshock Aftershock Sequences

Authors: Louis Davis, Boris Baeumer, Ting Wang

Abstract: Most point process models for earthquakes currently in the literature assume the magnitude distribution is i.i.d. potentially hindering the ability of the model to describe the main features of data sets containing multiple earthquake mainshock aftershock sequences in succession. This study presents a novel multidimensional fractional Hawkes process model designed to capture magnitude dependent tr… ▽ More Most point process models for earthquakes currently in the literature assume the magnitude distribution is i.i.d. potentially hindering the ability of the model to describe the main features of data sets containing multiple earthquake mainshock aftershock sequences in succession. This study presents a novel multidimensional fractional Hawkes process model designed to capture magnitude dependent triggering behaviour by incorporating history dependence into the magnitude distribution. This is done by discretising the magnitude range into disjoint intervals and modelling events with magnitude in these ranges as the subprocesses of a mutually exciting Hawkes process using the Mittag-Leffler density as the kernel function. We demonstrate this model's use by applying it to two data sets, Japan and the Middle America Trench, both containing multiple mainshock aftershock sequences and compare it to the existing ETAS model by using information criteria, residual diagnostics and retrospective prediction performance. We find that for both data sets all metrics indicate that the multidimensional fractional Hawkes process performs favourably against the ETAS model. Furthermore, using the multidimensional fractional Hawkes process we are able to infer characteristics of the data sets that are consistent with results currently in the literature and that cannot be found by using the ETAS model. △ Less

Submitted 1 April, 2024; originally announced April 2024.

Comments: 37 pages, 10 tables, 3 figures

Towards Modeling Learner Performance with Large Language Models

Authors: Seyed Parsa Neshaei, Richard Lee Davis, Adam Hazimeh, Bojan Lazarevski, Pierre Dillenbourg, Tanja Käser

Abstract: Recent work exploring the capabilities of pre-trained large language models (LLMs) has demonstrated their ability to act as general pattern machines by completing complex token sequences representing a wide array of tasks, including time-series prediction and robot control. This paper investigates whether the pattern recognition and sequence modeling capabilities of LLMs can be extended to the dom… ▽ More Recent work exploring the capabilities of pre-trained large language models (LLMs) has demonstrated their ability to act as general pattern machines by completing complex token sequences representing a wide array of tasks, including time-series prediction and robot control. This paper investigates whether the pattern recognition and sequence modeling capabilities of LLMs can be extended to the domain of knowledge tracing, a critical component in the development of intelligent tutoring systems (ITSs) that tailor educational experiences by predicting learner performance over time. In an empirical evaluation across multiple real-world datasets, we compare two approaches to using LLMs for this task, zero-shot prompting and model fine-tuning, with existing, non-LLM approaches to knowledge tracing. While LLM-based approaches do not achieve state-of-the-art performance, fine-tuned LLMs surpass the performance of naive baseline models and perform on par with standard Bayesian Knowledge Tracing approaches across multiple metrics. These findings suggest that the pattern recognition capabilities of LLMs can be used to model complex learning trajectories, opening a novel avenue for applying LLMs to educational contexts. The paper concludes with a discussion of the implications of these findings for future research, suggesting that further refinements and a deeper understanding of LLMs' predictive mechanisms could lead to enhanced performance in knowledge tracing tasks. △ Less

Submitted 29 February, 2024; originally announced March 2024.

Comments: 12 pages, 4 figures

HELLO project: High-$z$ Evolution of Large and Luminous Objects

Authors: Stefan Waterval, Andrea V. Macciò, Tobias Buck, Aura Obreja, Changhyun Cho, Zehao Jin, Benjamin L. Davis, Xi Kang

Abstract: We present the High-$z$ Evolution of Large and Luminous Objects (HELLO) project, a set of more than 30 high-resolution hydrodynamical cosmological simulations aimed to study Milky Way analogues ($M_\star\sim10^{10-11}$ $\mathrm{M}_\odot$) at high redshift, namely at $z=3.6$ (age $\sim$ 1.7 Gyr) and $z=2$ (age $\sim$ 3.3 Gyr). The HELLO project features an updated scheme for chemical enrichment and… ▽ More We present the High-$z$ Evolution of Large and Luminous Objects (HELLO) project, a set of more than 30 high-resolution hydrodynamical cosmological simulations aimed to study Milky Way analogues ($M_\star\sim10^{10-11}$ $\mathrm{M}_\odot$) at high redshift, namely at $z=3.6$ (age $\sim$ 1.7 Gyr) and $z=2$ (age $\sim$ 3.3 Gyr). The HELLO project features an updated scheme for chemical enrichment and the addition of local photoionization feedback processes. Independently of redshift and stellar mass, all galaxies follow a similar evolutionary path: (i) first a smooth progression along the star formation main sequence, where galaxies grow in both stellar mass and size, (ii) a (short) period of intense star formation, which causes a contraction phase in the stellar size, until the galaxies reach their peak star formation rate (SFR), during this period we also witness a significant black hole growth, and (iii) the onset of declining SFRs, which is due to a mix of gas consumption, stellar feedback, and AGN feedback, but with AGN feedback still being subdominant with respect to stellar feedback for energy deposition. The exact phase in which a galaxy in our mass range can be found at a given redshift is set by its gas reservoir and assembly history. Finally, our galaxies are in excellent agreement with several various scaling relations observed with the Hubble Space Telescope and the James Webb Space Telescope, and hence can be used to provide the theoretical framework to interpret current and future observations from these facilities and shed light on the transition from star-forming to quiescent galaxies. △ Less

Submitted 6 March, 2024; originally announced March 2024.

Comments: 22 pages, 16 figures, 3 tables, submitted to MNRAS

Ultralight vector dark matter search using data from the KAGRA O3GK run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, A. G. Abac, R. Abbott, H. Abe, I. Abouelfettouh, F. Acernese, K. Ackley, C. Adamcewicz, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, I. Aguilar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi , et al. (1778 additional authors not shown)

Abstract: Among the various candidates for dark matter (DM), ultralight vector DM can be probed by laser interferometric gravitational wave detectors through the measurement of oscillating length changes in the arm cavities. In this context, KAGRA has a unique feature due to differing compositions of its mirrors, enhancing the signal of vector DM in the length change in the auxiliary channels. Here we prese… ▽ More Among the various candidates for dark matter (DM), ultralight vector DM can be probed by laser interferometric gravitational wave detectors through the measurement of oscillating length changes in the arm cavities. In this context, KAGRA has a unique feature due to differing compositions of its mirrors, enhancing the signal of vector DM in the length change in the auxiliary channels. Here we present the result of a search for $U(1)_{B-L}$ gauge boson DM using the KAGRA data from auxiliary length channels during the first joint observation run together with GEO600. By applying our search pipeline, which takes into account the stochastic nature of ultralight DM, upper bounds on the coupling strength between the $U(1)_{B-L}$ gauge boson and ordinary matter are obtained for a range of DM masses. While our constraints are less stringent than those derived from previous experiments, this study demonstrates the applicability of our method to the lower-mass vector DM search, which is made difficult in this measurement by the short observation time compared to the auto-correlation time scale of DM. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Comments: 20 pages, 5 figures

Report number: LIGO-P2300250

A Fractional Model for Earthquakes

Authors: Louis Davis, Boris Baeumer, Ting Wang

Abstract: This paper extends the existing fractional Hawkes process to better model mainshock-aftershock sequences of earthquakes. The fractional Hawkes process is a self-exciting point process model with temporal decay kernel being a Mittag-Leffler function. A maximum likelihood estimation scheme is developed and its consistency is checked. It is then compared to the ETAS model on three earthquake sequence… ▽ More This paper extends the existing fractional Hawkes process to better model mainshock-aftershock sequences of earthquakes. The fractional Hawkes process is a self-exciting point process model with temporal decay kernel being a Mittag-Leffler function. A maximum likelihood estimation scheme is developed and its consistency is checked. It is then compared to the ETAS model on three earthquake sequences in Southern California. The fractional Hawkes process performs favourably against the ETAS model. Additionally, two parameters in the fractional Hawkes process may have a fixed geophysical meaning dependent on the study zone and the stage of the seismic cycle the zone is in. △ Less

Submitted 29 February, 2024; originally announced March 2024.

Comments: 16 pages, 7 figure, submitted to the Journal of the Royal Statistical Society Series C

Causa prima: cosmology meets causal discovery for the first time

Authors: Mario Pasquato, Zehao Jin, Pablo Lemos, Benjamin L. Davis, Andrea V. Macciò

Abstract: In astrophysics, experiments are impossible. We thus must rely exclusively on observational data. Other observational sciences increasingly leverage causal inference methods, but this is not yet the case in astrophysics. Here we attempt causal discovery for the first time to address an important open problem in astrophysics: the (co)evolution of supermassive black holes (SMBHs) and their host gala… ▽ More In astrophysics, experiments are impossible. We thus must rely exclusively on observational data. Other observational sciences increasingly leverage causal inference methods, but this is not yet the case in astrophysics. Here we attempt causal discovery for the first time to address an important open problem in astrophysics: the (co)evolution of supermassive black holes (SMBHs) and their host galaxies. We apply the Peter-Clark (PC) algorithm to a comprehensive catalog of galaxy properties to obtain a completed partially directed acyclic graph (CPDAG), representing a Markov equivalence class over directed acyclic graphs (DAGs). Central density and velocity dispersion are found to cause SMBH mass. We test the robustness of our analysis by random sub-sampling, recovering similar results. We also apply the Fast Causal Inference (FCI) algorithm to our dataset to relax the hypothesis of causal sufficiency, admitting unobserved confounds. Hierarchical SMBH assembly may provide a physical explanation for our findings. △ Less

Submitted 25 November, 2023; originally announced November 2023.

Comments: ML4PS NeurIPS workshop 2023 accepted

Model-independent extraction of form factors and $|V_{cb}|$ in $\overline{B} \rightarrow D \ell^- \overlineν_\ell$ with hadronic tagging at BaBar

Authors: BaBar Collaboration, J. P. Lees, V. Poireau, V. Tisserand, E. Grauges, A. Palano, G. Eigen, D. N. Brown, Yu. G. Kolomensky, M. Fritsch, H. Koch, R. Cheaib, C. Hearty, T. S. Mattison, J. A. McKenna, R. Y. So, V. E. Blinov, A. R. Buzykaev, V. P. Druzhinin, E. A. Kozyrev, E. A. Kravchenko, S. I. Serednyakov, Yu. I. Skovpen, E. P. Solodov, K. Yu. Todyshev , et al. (186 additional authors not shown)

Abstract: Using the entire BaBar $Υ(4S)$ data set, the first two-dimensional unbinned angular analysis of the semileptonic decay $\overline{B} \rightarrow D \ell^- \overlineν_\ell$ is performed, employing hadronic reconstruction of the tag-side $B$ meson from $Υ(4S)\to B\overline{B}$. Here, $\ell$ denotes the light charged leptons $e$ and $μ$. A novel data-driven signal-background separation procedure with… ▽ More Using the entire BaBar $Υ(4S)$ data set, the first two-dimensional unbinned angular analysis of the semileptonic decay $\overline{B} \rightarrow D \ell^- \overlineν_\ell$ is performed, employing hadronic reconstruction of the tag-side $B$ meson from $Υ(4S)\to B\overline{B}$. Here, $\ell$ denotes the light charged leptons $e$ and $μ$. A novel data-driven signal-background separation procedure with minimal dependence on simulation is developed. This procedure preserves all multi-dimensional correlations present in the data. The expected $\sin^2θ_\ell$ dependence of the differential decay rate in the Standard Model is demonstrated, where $θ_\ell$ is the lepton helicity angle. Including input from the latest lattice QCD calculations and previously available experimental data, the underlying form factors are extracted using both model-independent (BGL) and dependent (CLN) methods. Comparisons with lattice calculations show flavor SU(3) symmetry to be a good approximation in the $B_{(s)}\to D_{(s)}$ sector. Using the BGL results, the CKM matrix element $|V_{cb}|=(41.09\pm 1.16)\times 10^{-3}$ and the Standard Model prediction of the lepton-flavor universality violation variable $\mathcal{R}(D)=0.300\pm 0.004$, are extracted. The value of $|V_{cb}|$ from $\overline{B} \rightarrow D \ell^- \overlineν_\ell$ tends to be higher than that extracted using $\overline{B} \rightarrow D \ell^- \overlineν_\ell$. The Standard Model $\mathcal{R}(D)$ calculation is at a $1.97σ$ tension with the latest HFLAV experimental average. △ Less

Submitted 25 November, 2023; originally announced November 2023.

Discovering Black Hole Mass Scaling Relations with Symbolic Regression

Authors: Zehao Jin, Benjamin L. Davis

Abstract: Our knowledge of supermassive black holes (SMBHs) and their relation to their host galaxies is still limited, and there are only around 150 SMBHs that have their masses directly measured and confirmed. Better black hole mass scaling relations will help us reveal the physics of black holes, as well as predict black hole masses that are not yet measured. Here, we apply symbolic regression, combined… ▽ More Our knowledge of supermassive black holes (SMBHs) and their relation to their host galaxies is still limited, and there are only around 150 SMBHs that have their masses directly measured and confirmed. Better black hole mass scaling relations will help us reveal the physics of black holes, as well as predict black hole masses that are not yet measured. Here, we apply symbolic regression, combined with random forest to those directly-measured black hole masses and host galaxy properties, and find a collection of higher-dimensional (N-D) black hole mass scaling relations. These N-D black hole mass scaling relations have scatter smaller than any of the existing black hole mass scaling relations. One of the best among them involves the parameters of central stellar velocity dispersion, bulge-to-total ratio, and density at the black hole's sphere-of-influence with an intrinsic scatter of $ε=0.083\,\ \text{dex}$, significantly lower than $ε\sim 0.3\,\ \text{dex}$ for the M-$σ$ relation. These relations will inspire black hole physics, test black hole models implemented in simulations, and estimate unknown black hole masses on an unprecedented precision. △ Less

Submitted 20 November, 2023; v1 submitted 30 October, 2023; originally announced October 2023.

Comments: 9 pages, 3 figures, accepted by NeurIPS 2023 workshop on Machine Learning and the Physical Sciences

Pump Pulse Bandwidth-Activated Nonlinear Phononic Coupling in CdWO$_4$

Authors: Megan F. Biggs, Brittany E. Knighton, Aldair Alejandro, Lauren M. Davis, Claire Rader, Jeremy A. Johnson

Abstract: To control structure-function relationships in solids with light, we must harness the shape of the potential energy surface, as expressed in anharmonic coupling coefficients. We use two-dimensional terahertz (THz) spectroscopy to identify trilinear coupling between sets of vibrational modes in CdWO$_4$. It is generally understood that efficient trilinear coupling occurs when the frequencies of two… ▽ More To control structure-function relationships in solids with light, we must harness the shape of the potential energy surface, as expressed in anharmonic coupling coefficients. We use two-dimensional terahertz (THz) spectroscopy to identify trilinear coupling between sets of vibrational modes in CdWO$_4$. It is generally understood that efficient trilinear coupling occurs when the frequencies of two coupled modes add or subtract to the frequency of the third mode. Interestingly, we observe that this condition is not necessary: the THz driving-pulse itself can activate the coupling by contributing broad frequency content to the initial motion of the excited modes. Understanding that the bandwidth of the driving force can activate energy-flow pathways has broad implications for coherent control of collective modes using intense THz light pulses. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Comments: 27 Pages, 15 Figures

Building an Open-Vocabulary Video CLIP Model with Better Architectures, Optimization and Data

Authors: Zuxuan Wu, Zejia Weng, Wujian Peng, Xitong Yang, Ang Li, Larry S. Davis, Yu-Gang Jiang

Abstract: Despite significant results achieved by Contrastive Language-Image Pretraining (CLIP) in zero-shot image recognition, limited effort has been made exploring its potential for zero-shot video recognition. This paper presents Open-VCLIP++, a simple yet effective framework that adapts CLIP to a strong zero-shot video classifier, capable of identifying novel actions and events during testing. Open-VCL… ▽ More Despite significant results achieved by Contrastive Language-Image Pretraining (CLIP) in zero-shot image recognition, limited effort has been made exploring its potential for zero-shot video recognition. This paper presents Open-VCLIP++, a simple yet effective framework that adapts CLIP to a strong zero-shot video classifier, capable of identifying novel actions and events during testing. Open-VCLIP++ minimally modifies CLIP to capture spatial-temporal relationships in videos, thereby creating a specialized video classifier while striving for generalization. We formally demonstrate that training Open-VCLIP++ is tantamount to continual learning with zero historical data. To address this problem, we introduce Interpolated Weight Optimization, a technique that leverages the advantages of weight interpolation during both training and testing. Furthermore, we build upon large language models to produce fine-grained video descriptions. These detailed descriptions are further aligned with video features, facilitating a better transfer of CLIP to the video domain. Our approach is evaluated on three widely used action recognition datasets, following a variety of zero-shot evaluation protocols. The results demonstrate that our method surpasses existing state-of-the-art techniques by significant margins. Specifically, we achieve zero-shot accuracy scores of 88.1%, 58.7%, and 81.2% on UCF, HMDB, and Kinetics-600 datasets respectively, outpacing the best-performing alternative methods by 8.5%, 8.2%, and 12.3%. We also evaluate our approach on the MSR-VTT video-text retrieval dataset, where it delivers competitive video-to-text and text-to-video retrieval performance, while utilizing substantially less fine-tuning data compared to other methods. Code is released at https://github.com/wengzejia1/Open-VCLIP. △ Less

Submitted 8 October, 2023; originally announced October 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2302.00624

Discovery of a Planar Black Hole Mass Scaling Relation for Spiral Galaxies

Authors: Benjamin L. Davis, Zehao Jin

Abstract: Supermassive black holes (SMBHs) are tiny in comparison to the galaxies they inhabit, yet they manage to influence and coevolve along with their hosts. Evidence of this mutual development is observed in the structure and dynamics of galaxies and their correlations with black hole mass ($M_\mathrm{BH}$). For our study, we focus on relative parameters that are unique to only disk galaxies. As such,… ▽ More Supermassive black holes (SMBHs) are tiny in comparison to the galaxies they inhabit, yet they manage to influence and coevolve along with their hosts. Evidence of this mutual development is observed in the structure and dynamics of galaxies and their correlations with black hole mass ($M_\mathrm{BH}$). For our study, we focus on relative parameters that are unique to only disk galaxies. As such, we quantify the structure of spiral galaxies via their logarithmic spiral-arm pitch angles ($φ$) and their dynamics through the maximum rotational velocities of their galactic disks ($v_\mathrm{max}$). In the past, we have studied black hole mass scaling relations between $M_\mathrm{BH}$ and $φ$ or $v_\mathrm{max}$, separately. Now, we combine the three parameters into a trivariate $M_\mathrm{BH}$-$φ$-$v_\mathrm{max}$ relationship that yields best-in-class accuracy in prediction of black hole masses in spiral galaxies. Because most black hole mass scaling relations have been created from samples of the largest SMBHs within the most massive galaxies, they lack certainty when extrapolated to low-mass spiral galaxies. Thus, it is difficult to confidently use existing scaling relations when trying to identify galaxies that might harbor the elusive class of intermediate-mass black holes (IMBHs). Therefore, we offer our novel relationship as an ideal predictor to search for IMBHs and probe the low-mass end of the black hole mass function by utilizing spiral galaxies. Already with rotational velocities widely available for a large population of galaxies and pitch angles readily measurable from uncalibrated images, we expect that the $M_\mathrm{BH}$-$φ$-$v_\mathrm{max}$ fundamental plane will be a useful tool for estimating black hole masses, even at high redshifts. △ Less

Submitted 16 September, 2023; originally announced September 2023.

Comments: Unedited manuscript (12 pages & 4 figures), accepted for publication by The Astrophysical Journal Letters on September 15, 2023

Journal ref: ApJL 956 L22 (2023)

Low-latency gravitational wave alert products and their performance at the time of the fourth LIGO-Virgo-KAGRA observing run

Authors: Sushant Sharma Chaudhary, Andrew Toivonen, Gaurav Waratkar, Geoffrey Mo, Deep Chatterjee, Sarah Antier, Patrick Brockill, Michael W. Coughlin, Reed Essick, Shaon Ghosh, Soichiro Morisaki, Pratyusava Baral, Amanda Baylor, Naresh Adhikari, Patrick Brady, Gareth Cabourn Davies, Tito Dal Canton, Marco Cavaglià, Jolien Creighton, Sunil Choudhary, Yu-Kuang Chu, Patrick Clearwater, Luke Davis, Thomas Dent, Marco Drago , et al. (28 additional authors not shown)

Abstract: Multi-messenger searches for BNS and NSBH mergers are currently one of the most exciting areas of astronomy. The search for joint electromagnetic and neutrino counterparts to GWs has resumed with O4. To support this effort, public semi-automated data products are sent in near real-time and include localization and source properties to guide complementary observations. In preparation for O4, we hav… ▽ More Multi-messenger searches for BNS and NSBH mergers are currently one of the most exciting areas of astronomy. The search for joint electromagnetic and neutrino counterparts to GWs has resumed with O4. To support this effort, public semi-automated data products are sent in near real-time and include localization and source properties to guide complementary observations. In preparation for O4, we have conducted a study using a simulated population of compact binaries and a MDC in the form of a real-time replay to optimize and profile the software infrastructure and scientific deliverables. End-to-end performance was tested, including data ingestion, running online search pipelines, performing annotations, and issuing alerts to the astrophysics community. We present an overview of the low-latency infrastructure and the performance of the data products that are now being released during O4 based on the MDC. We report the expected median latency for the preliminary alert of full bandwidth searches (29.5s) and show consistency and accuracy of released data products using the MDC. For the first time, we report the expected median latency for triggers from early warning searches (-3.1s), which are new in O4 and target neutron star mergers during inspiral phase. This paper provides a performance overview for LVK low-latency alert infrastructure and data products using the MDC and serves as a useful reference for the interpretation of O4 detections. △ Less

Submitted 27 May, 2024; v1 submitted 8 August, 2023; originally announced August 2023.

Journal ref: PNAS 121 (18) e2316474121 (2024)

Population Age Group Sensitivity for COVID-19 Infections with Deep Learning

Authors: Md Khairul Islam, Tyler Valentine, Royal Wang, Levi Davis, Matt Manner, Judy Fox

Abstract: The COVID-19 pandemic has created unprecedented challenges for governments and healthcare systems worldwide, highlighting the critical importance of understanding the factors that contribute to virus transmission. This study aimed to identify the most influential age groups in COVID-19 infection rates at the US county level using the Modified Morris Method and deep learning for time series. Our ap… ▽ More The COVID-19 pandemic has created unprecedented challenges for governments and healthcare systems worldwide, highlighting the critical importance of understanding the factors that contribute to virus transmission. This study aimed to identify the most influential age groups in COVID-19 infection rates at the US county level using the Modified Morris Method and deep learning for time series. Our approach involved training the state-of-the-art time-series model Temporal Fusion Transformer on different age groups as a static feature and the population vaccination status as the dynamic feature. We analyzed the impact of those age groups on COVID-19 infection rates by perturbing individual input features and ranked them based on their Morris sensitivity scores, which quantify their contribution to COVID-19 transmission rates. The findings are verified using ground truth data from the CDC and US Census, which provide the true infection rates for each age group. The results suggest that young adults were the most influential age group in COVID-19 transmission at the county level between March 1, 2020, and November 27, 2021. Using these results can inform public health policies and interventions, such as targeted vaccination strategies, to better control the spread of the virus. Our approach demonstrates the utility of feature sensitivity analysis in identifying critical factors contributing to COVID-19 transmission and can be applied in other public health domains. △ Less

Submitted 3 July, 2023; originally announced July 2023.

Mitigating Molecular Aggregation in Drug Discovery with Predictive Insights from Explainable AI

Authors: Hunter Sturm, Jonas Teufel, Kaitlin A. Isfeld, Pascal Friederich, Rebecca L. Davis

Abstract: As the importance of high-throughput screening (HTS) continues to grow due to its value in early stage drug discovery and data generation for training machine learning models, there is a growing need for robust methods for pre-screening compounds to identify and prevent false-positive hits. Small, colloidally aggregating molecules are one of the primary sources of false-positive hits in high-throu… ▽ More As the importance of high-throughput screening (HTS) continues to grow due to its value in early stage drug discovery and data generation for training machine learning models, there is a growing need for robust methods for pre-screening compounds to identify and prevent false-positive hits. Small, colloidally aggregating molecules are one of the primary sources of false-positive hits in high-throughput screens, making them an ideal candidate to target for removal from libraries using predictive pre-screening tools. However, a lack of understanding of the causes of molecular aggregation introduces difficulty in the development of predictive tools for detecting aggregating molecules. Herein, we present an examination of the molecular features differentiating datasets of aggregating and non-aggregating molecules, as well as a machine learning approach to predicting molecular aggregation. Our method uses explainable graph neural networks and counterfactuals to reliably predict and explain aggregation, giving additional insights and design rules for future screening. The integration of this method in HTS approaches will help combat false positives, providing better lead molecules more rapidly and thus accelerating drug discovery cycles. △ Less

Submitted 3 June, 2023; originally announced June 2023.

Comments: 17 pages, plus SI

Active matter under control: Insights from response theory

Authors: Luke K. Davis, Karel Proesmans, Étienne Fodor

Abstract: Active constituents burn fuel to sustain individual motion, giving rise to collective effects that are not seen in systems at thermal equilibrium, such as phase separation with purely repulsive interactions. There is a great potential in harnessing the striking phenomenology of active matter to build novel controllable and responsive materials that surpass passive ones. Yet, we currently lack a sy… ▽ More Active constituents burn fuel to sustain individual motion, giving rise to collective effects that are not seen in systems at thermal equilibrium, such as phase separation with purely repulsive interactions. There is a great potential in harnessing the striking phenomenology of active matter to build novel controllable and responsive materials that surpass passive ones. Yet, we currently lack a systematic roadmap to predict the protocols driving active systems between different states in a way that is thermodynamically optimal. Equilibrium thermodynamics is an inadequate foundation to this end, due to the dissipation rate arising from the constant fuel consumption in active matter. Here, we derive and implement a versatile framework for the thermodynamic control of active matter. Combining recent developments in stochastic thermodynamics and nonequilibrium response theory, our approach shows how to find the optimal control for either continuous- or discrete-state active systems operating arbitrarily far from equilibrium. Our results open the door to designing novel active materials which are not only built to stabilize specific nonequilibrium collective states, but are also optimized to switch between different states at minimum dissipation. △ Less

Submitted 15 February, 2024; v1 submitted 18 May, 2023; originally announced May 2023.

Journal ref: Phys. Rev. X 14, 011012 (2024)

What is missing in autonomous discovery: Open challenges for the community

Authors: Phillip M. Maffettone, Pascal Friederich, Sterling G. Baird, Ben Blaiszik, Keith A. Brown, Stuart I. Campbell, Orion A. Cohen, Tantum Collins, Rebecca L. Davis, Ian T. Foster, Navid Haghmoradi, Mark Hereld, Nicole Jung, Ha-Kyung Kwon, Gabriella Pizzuto, Jacob Rintamaki, Casper Steinmann, Luca Torresi, Shijing Sun

Abstract: Self-driving labs (SDLs) leverage combinations of artificial intelligence, automation, and advanced computing to accelerate scientific discovery. The promise of this field has given rise to a rich community of passionate scientists, engineers, and social scientists, as evidenced by the development of the Acceleration Consortium and recent Accelerate Conference. Despite its strengths, this rapidly… ▽ More Self-driving labs (SDLs) leverage combinations of artificial intelligence, automation, and advanced computing to accelerate scientific discovery. The promise of this field has given rise to a rich community of passionate scientists, engineers, and social scientists, as evidenced by the development of the Acceleration Consortium and recent Accelerate Conference. Despite its strengths, this rapidly developing field presents numerous opportunities for growth, challenges to overcome, and potential risks of which to remain aware. This community perspective builds on a discourse instantiated during the first Accelerate Conference, and looks to the future of self-driving labs with a tempered optimism. Incorporating input from academia, government, and industry, we briefly describe the current status of self-driving labs, then turn our attention to barriers, opportunities, and a vision for what is possible. Our field is delivering solutions in technology and infrastructure, artificial intelligence and knowledge generation, and education and workforce development. In the spirit of community, we intend for this work to foster discussion and drive best practices as our field grows. △ Less

Submitted 2 May, 2023; v1 submitted 21 April, 2023; originally announced April 2023.

Josephson-like tunnel resonance and large Coulomb drag in GaAs-based electron-hole bilayers

Authors: M. L. Davis, S. Parolo, C. Reichl, W. Dietsche, W. Wegscheider

Abstract: Bilayers consisting of two-dimensional (2D) electron and hole gases separated by a 10 nm thick AlGaAs barrier are formed by charge accumulation in epitaxially grown GaAs. Both vertical and lateral electric transport are measured in the millikelvin temperature range. The conductivity between the layers shows a sharp tunnel resonance at a density of $1.1 \cdot 10^{10} \text{ cm}^{-2}$, which is cons… ▽ More Bilayers consisting of two-dimensional (2D) electron and hole gases separated by a 10 nm thick AlGaAs barrier are formed by charge accumulation in epitaxially grown GaAs. Both vertical and lateral electric transport are measured in the millikelvin temperature range. The conductivity between the layers shows a sharp tunnel resonance at a density of $1.1 \cdot 10^{10} \text{ cm}^{-2}$, which is consistent with a Josephson-like enhanced tunnel conductance. The tunnel resonance disappears with increasing densities and the two 2D charge gases start to show 2D-Fermi-gas behavior. Interlayer interactions persist causing a positive drag voltage that is very large at small densities. The transition from the Josephson-like tunnel resonance to the Fermi-gas behavior is interpreted as a phase transition from an exciton gas in the Bose-Einstein-condensate state to a degenerate electron-hole Fermi gas. △ Less

Submitted 11 September, 2023; v1 submitted 13 April, 2023; originally announced April 2023.

Comments: Updated after reviewer comments

Exploring celebrity influence on public attitude towards the COVID-19 pandemic: social media shared sentiment analysis

Authors: Brianna M White, Chad A Melton, Parya Zareie, Robert L Davis, Robert A Bednarczyk, Arash Shaban-Nejad

Abstract: The COVID-19 pandemic has introduced new opportunities for health communication, including an increase in the public use of online outlets for health-related emotions. People have turned to social media networks to share sentiments related to the impacts of the COVID-19 pandemic. In this paper we examine the role of social messaging shared by Persons in the Public Eye (i.e. athletes, politicians,… ▽ More The COVID-19 pandemic has introduced new opportunities for health communication, including an increase in the public use of online outlets for health-related emotions. People have turned to social media networks to share sentiments related to the impacts of the COVID-19 pandemic. In this paper we examine the role of social messaging shared by Persons in the Public Eye (i.e. athletes, politicians, news personnel) in determining overall public discourse direction. We harvested approximately 13 million tweets ranging from 1 January 2020 to 1 March 2022. The sentiment was calculated for each tweet using a fine-tuned DistilRoBERTa model, which was used to compare COVID-19 vaccine-related Twitter posts (tweets) that co-occurred with mentions of People in the Public Eye. Our findings suggest the presence of consistent patterns of emotional content co-occurring with messaging shared by Persons in the Public Eye for the first two years of the COVID-19 pandemic influenced public opinion and largely stimulated online public discourse. We demonstrate that as the pandemic progressed, public sentiment shared on social networks was shaped by risk perceptions, political ideologies and health-protective behaviours shared by Persons in the Public Eye, often in a negative light. △ Less

Submitted 23 February, 2023; originally announced March 2023.

Comments: 7 Pages, 4 Figures

ACM Class: I.2.7

Journal ref: BMJ Health & Care Informatics 2023;30:e100665

FlexNeRF: Photorealistic Free-viewpoint Rendering of Moving Humans from Sparse Views

Authors: Vinoj Jayasundara, Amit Agrawal, Nicolas Heron, Abhinav Shrivastava, Larry S. Davis

Abstract: We present FlexNeRF, a method for photorealistic freeviewpoint rendering of humans in motion from monocular videos. Our approach works well with sparse views, which is a challenging scenario when the subject is exhibiting fast/complex motions. We propose a novel approach which jointly optimizes a canonical time and pose configuration, with a pose-dependent motion field and pose-independent tempora… ▽ More We present FlexNeRF, a method for photorealistic freeviewpoint rendering of humans in motion from monocular videos. Our approach works well with sparse views, which is a challenging scenario when the subject is exhibiting fast/complex motions. We propose a novel approach which jointly optimizes a canonical time and pose configuration, with a pose-dependent motion field and pose-independent temporal deformations complementing each other. Thanks to our novel temporal and cyclic consistency constraints along with additional losses on intermediate representation such as segmentation, our approach provides high quality outputs as the observed views become sparser. We empirically demonstrate that our method significantly outperforms the state-of-the-art on public benchmark datasets as well as a self-captured fashion dataset. The project page is available at: https://flex-nerf.github.io/ △ Less

Submitted 25 March, 2023; originally announced March 2023.

Comments: CVPR 2023

Search for $B$ Mesogenesis at BABAR

Authors: BABAR Collaboration, J. P. Lees, V. Poireau, V. Tisserand, E. Grauges, A. Palano, G. Eigen, D. N. Brown, Yu. G. Kolomensky, M. Fritsch, H. Koch, R. Cheaib, C. Hearty, T. S. Mattison, J. A. McKenna, R. Y. So, V. E. Blinov, A. R. Buzykaev, V. P. Druzhinin, V. B. Golubev, E. A. Kozyrev, E. A. Kravchenko, A. P. Onuchin, S. I. Serednyakov, Yu. I. Skovpen , et al. (218 additional authors not shown)

Abstract: A new mechanism has been proposed to simultaneously explain the presence of dark matter and the matter-antimatter asymmetry in the universe. This scenario predicts exotic $B$ meson decays into a baryon and a dark sector anti-baryon ($ψ_D$) with branching fractions accessible at $B$ factories. We present a search for $B \rightarrow Λψ_D$ decays using data collected by the $BABAR$ experiment at SLAC… ▽ More A new mechanism has been proposed to simultaneously explain the presence of dark matter and the matter-antimatter asymmetry in the universe. This scenario predicts exotic $B$ meson decays into a baryon and a dark sector anti-baryon ($ψ_D$) with branching fractions accessible at $B$ factories. We present a search for $B \rightarrow Λψ_D$ decays using data collected by the $BABAR$ experiment at SLAC. This reaction is identified by fully reconstructing the accompanying $B$ meson and requiring the presence of a single $Λ$ baryon in the remaining particles. No significant signal is observed, and bounds on the $B \rightarrow Λψ_D$ branching fraction are derived in the range $0.13 - 5.2\times 10^{-5}$ for $1.0 < m_{ψ_D} < 4.2$ GeV/$c^{2}$. These results set strong constraints on the parameter space allowed by the theory. △ Less

Submitted 31 January, 2023; originally announced February 2023.

Journal ref: PHYS. REV. D 107, 092001 (2023)

Material vs. structure: Topological origins of band-gap truncation resonances in periodic structures

Authors: Matheus I. N. Rosa, Bruce L. Davis, Liao Liu, Massimo Ruzzene, Mahmoud I. Hussein

Abstract: While resonant modes do not exist within band gaps in infinite periodic materials, they may appear as in-gap localized edge modes once the material is truncated to form a finite periodic structure. Here, we provide an analysis framework that reveals the topological origins of truncation resonances, elucidating formally the conditions that influence their existence and properties. Elastic beams wit… ▽ More While resonant modes do not exist within band gaps in infinite periodic materials, they may appear as in-gap localized edge modes once the material is truncated to form a finite periodic structure. Here, we provide an analysis framework that reveals the topological origins of truncation resonances, elucidating formally the conditions that influence their existence and properties. Elastic beams with sinusoidal and step-wise property modulations are considered as classical examples of periodic structures. Their non-trivial topological characteristics stem from the consideration of a phason parameter that produces spatial shifts of the property modulation while continuously varying how the boundaries are truncated. In this context, non-trivial band gaps are characterized by an integer topological invariant, the Chern number, which is equal to the number of truncation resonances that traverse a band gap as the phason is varied. We highlight the existence of multiple chiral edge states that may be localized at opposite boundaries, and illustrate how these can be independently tuned by modified boundary-specific phason parameters. Furthermore, we show that the frequency location of a truncation resonance is influenced by the modulation volume fraction, boundary conditions, and number of cells comprising the finite structure, thus quantifying its robustness to these factors. Non-topological in-gap resonances induced by a defect are also demonstrated, showing that these can be coupled with topological modes when the defect is located at an edge. Finally, experimental investigations on bi-material phononic-crystal beams are conducted to support these findings. The tunability of truncation resonances by material-property modulation may be exploited in applications ranging from vibration attenuation and thermal conductivity reduction to filtering and flow control by phononic subsurfaces. △ Less

Submitted 30 December, 2022; originally announced January 2023.

Journal ref: Physical Review Materials 7, 124201 (2023)

Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection

Authors: Junke Wang, Zhenxin Li, Chao Zhang, Jingjing Chen, Zuxuan Wu, Larry S. Davis, Yu-Gang Jiang

Abstract: Online media data, in the forms of images and videos, are becoming mainstream communication channels. However, recent advances in deep learning, particularly deep generative models, open the doors for producing perceptually convincing images and videos at a low cost, which not only poses a serious threat to the trustworthiness of digital information but also has severe societal implications. This… ▽ More Online media data, in the forms of images and videos, are becoming mainstream communication channels. However, recent advances in deep learning, particularly deep generative models, open the doors for producing perceptually convincing images and videos at a low cost, which not only poses a serious threat to the trustworthiness of digital information but also has severe societal implications. This motivates a growing interest of research in media tampering detection, i.e., using deep learning techniques to examine whether media data have been maliciously manipulated. Depending on the content of the targeted images, media forgery could be divided into image tampering and Deepfake techniques. The former typically moves or erases the visual elements in ordinary images, while the latter manipulates the expressions and even the identity of human faces. Accordingly, the means of defense include image tampering detection and Deepfake detection, which share a wide variety of properties. In this paper, we provide a comprehensive review of the current media tampering detection approaches, and discuss the challenges and trends in this field for future research. △ Less

Submitted 11 December, 2022; originally announced December 2022.

Fine-tuned Sentiment Analysis of COVID-19 Vaccine-Related Social Media Data: Comparative Study

Authors: Chad A Melton, Brianna M White, Robert L Davis, Robert A Bednarczyk, Arash Shaban-Nejad

Abstract: This study investigated and compared public sentiment related to COVID-19 vaccines expressed on two popular social media platforms, Reddit and Twitter, harvested from January 1, 2020, to March 1, 2022. To accomplish this task, we created a fine-tuned DistilRoBERTa model to predict sentiments of approximately 9.5 million Tweets and 70 thousand Reddit comments. To fine-tune our model, our team manua… ▽ More This study investigated and compared public sentiment related to COVID-19 vaccines expressed on two popular social media platforms, Reddit and Twitter, harvested from January 1, 2020, to March 1, 2022. To accomplish this task, we created a fine-tuned DistilRoBERTa model to predict sentiments of approximately 9.5 million Tweets and 70 thousand Reddit comments. To fine-tune our model, our team manually labeled the sentiment of 3600 Tweets and then augmented our dataset by the method of back-translation. Text sentiment for each social media platform was then classified with our fine-tuned model using Python and the Huggingface sentiment analysis pipeline. Our results determined that the average sentiment expressed on Twitter was more negative (52% positive) than positive and the sentiment expressed on Reddit was more positive than negative (53% positive). Though average sentiment was found to vary between these social media platforms, both displayed similar behavior related to sentiment shared at key vaccine-related developments during the pandemic. Considering this similar trend in shared sentiment demonstrated across social media platforms, Twitter and Reddit continue to be valuable data sources that public health officials can utilize to strengthen vaccine confidence and combat misinformation. As the spread of misinformation poses a range of psychological and psychosocial risks (anxiety, fear, etc.), there is an urgency in understanding the public perspective and attitude toward shared falsities. Comprehensive educational delivery systems tailored to the population's expressed sentiments that facilitate digital literacy, health information-seeking behavior, and precision health promotion could aid in clarifying such misinformation. △ Less

Submitted 17 October, 2022; originally announced November 2022.

Comments: 11 Pages, 5 Figures, and 1 Table

MSC Class: 92-11 ACM Class: I.2.7

Journal ref: Journal of Medical Internet Research (JMIR) 2022;24(10):e40408

Probing the Low-mass End of the Black Hole Mass Function via a Study of Faint Local Spiral Galaxies

Authors: Michael S. Fusco, Benjamin L. Davis, Julia Kennefick, Daniel Kennefick, Marc S. Seigar

Abstract: We present an analysis of the pitch angle distribution function (PADF) for nearby galaxies and its resulting black hole mass function (BHMF) via the well-known relationship between pitch angle and black hole mass. Our sample consists of a subset of 74 spiral galaxies from the Carnegie-Irvine Galaxy Survey with absolute $B$-band magnitude $\mathfrak{M}_{B}>-19.12$ mag and luminosity distance… ▽ More We present an analysis of the pitch angle distribution function (PADF) for nearby galaxies and its resulting black hole mass function (BHMF) via the well-known relationship between pitch angle and black hole mass. Our sample consists of a subset of 74 spiral galaxies from the Carnegie-Irvine Galaxy Survey with absolute $B$-band magnitude $\mathfrak{M}_{B}>-19.12$ mag and luminosity distance $D_{\mathrm{L}} \leq 25.4$ Mpc, which is an extension of a complementary set of 140 more luminous ($\mathfrak{M}_{B}\leq-19.12$ mag) late-type galaxies. We find the PADFs of the two samples are, somewhat surprisingly, not strongly dissimilar; a result that may hold important implications for spiral formation theories. Our data show a distinct bimodal population manifest in the pitch angles of the Sa-Sc types and separately the Scd-Sm types, with Sa-Sc types having tighter spiral arms on average. Importantly, we uncover a distinct bifurcation of the BHMF, such that the Sa-Sc galaxies typically host so-called "supermassive" black holes ($M_{\bullet}\gtrsim10^6\,\mathrm{M_{\odot}}$), whereas Scd-Sm galaxies accordingly harbor black holes that are "less-than-supermassive" ($M_{\bullet}\lesssim10^6\,\mathrm{M_{\odot}}$). It is amongst this latter population of galaxies where we expect fruitful bounties of elusive intermediate-mass black holes (IMBHs), through which a better understanding will help form more precise benchmarks for future generations of gravitational wave detectors. △ Less

Submitted 15 November, 2022; originally announced November 2022.

Comments: 39 pages, 17 figures, to be published in Universe

Journal ref: Universe 2022, 8(12), 649

Elevator Optimization: Application of Spatial Process and Gibbs Random Field Approaches for Dumbwaiter Modeling and Multi-Dumbwaiter Systems

Authors: Zheng Cao, Benjamin Lu Davis, Wanchaloem Wunkaew, Xinyu Chang

Abstract: This research investigates analytical and quantitative methods for simulating elevator optimizations. To maximize overall elevator usage, we concentrate on creating a multiple-user positive-sum system that is inspired by agent-based game theory. We define and create basic "Dumbwaiter" models by attempting both the Spatial Process Approach and the Gibbs Random Field Approach. These two mathematical… ▽ More This research investigates analytical and quantitative methods for simulating elevator optimizations. To maximize overall elevator usage, we concentrate on creating a multiple-user positive-sum system that is inspired by agent-based game theory. We define and create basic "Dumbwaiter" models by attempting both the Spatial Process Approach and the Gibbs Random Field Approach. These two mathematical techniques approach the problem from different points of view: the spatial process can give an analytical solution in continuous space and the Gibbs Random Field provides a discrete framework to flexibly model the problem on a computer. Starting from the simplest case, we target the assumptions to provide concrete solutions to the models and develop a "Multi-Dumbwaiter System". This paper examines, evaluates, and proves the ultimate success of such implemented strategies to design the basic elevator's optimal policy; consequently, not only do we believe in the results' practicality for industry, but also their potential for application. △ Less

Submitted 23 December, 2022; v1 submitted 25 September, 2022; originally announced September 2022.

Comments: 14 pages

MSC Class: 93-10; 60J05; 90B36 ACM Class: G.1.6; G.3; I.6.5

TAG: Boosting Text-VQA via Text-aware Visual Question-answer Generation

Authors: Jun Wang, Mingfei Gao, Yuqian Hu, Ramprasaath R. Selvaraju, Chetan Ramaiah, Ran Xu, Joseph F. JaJa, Larry S. Davis

Abstract: Text-VQA aims at answering questions that require understanding the textual cues in an image. Despite the great progress of existing Text-VQA methods, their performance suffers from insufficient human-labeled question-answer (QA) pairs. However, we observe that, in general, the scene text is not fully exploited in the existing datasets -- only a small portion of the text in each image participates… ▽ More Text-VQA aims at answering questions that require understanding the textual cues in an image. Despite the great progress of existing Text-VQA methods, their performance suffers from insufficient human-labeled question-answer (QA) pairs. However, we observe that, in general, the scene text is not fully exploited in the existing datasets -- only a small portion of the text in each image participates in the annotated QA activities. This results in a huge waste of useful information. To address this deficiency, we develop a new method to generate high-quality and diverse QA pairs by explicitly utilizing the existing rich text available in the scene context of each image. Specifically, we propose, TAG, a text-aware visual question-answer generation architecture that learns to produce meaningful, and accurate QA samples using a multimodal transformer. The architecture exploits underexplored scene text information and enhances scene understanding of Text-VQA models by combining the generated QA pairs with the initial training data. Extensive experimental results on two well-known Text-VQA benchmarks (TextVQA and ST-VQA) demonstrate that our proposed TAG effectively enlarges the training data that helps improve the Text-VQA performance without extra labeling effort. Moreover, our model outperforms state-of-the-art approaches that are pre-trained with extra large-scale data. Code is available at https://github.com/HenryJunW/TAG. △ Less

Submitted 7 October, 2022; v1 submitted 2 August, 2022; originally announced August 2022.

Comments: BMVC 2022

Smallset Timelines: A Visual Representation of Data Preprocessing Decisions

Authors: Lydia R. Lucchesi, Petra M. Kuhnert, Jenny L. Davis, Lexing Xie

Abstract: Data preprocessing is a crucial stage in the data analysis pipeline, with both technical and social aspects to consider. Yet, the attention it receives is often lacking in research practice and dissemination. We present the Smallset Timeline, a visualisation to help reflect on and communicate data preprocessing decisions. A "Smallset" is a small selection of rows from the original dataset containi… ▽ More Data preprocessing is a crucial stage in the data analysis pipeline, with both technical and social aspects to consider. Yet, the attention it receives is often lacking in research practice and dissemination. We present the Smallset Timeline, a visualisation to help reflect on and communicate data preprocessing decisions. A "Smallset" is a small selection of rows from the original dataset containing instances of dataset alterations. The Timeline is comprised of Smallset snapshots representing different points in the preprocessing stage and captions to describe the alterations visualised at each point. Edits, additions, and deletions to the dataset are highlighted with colour. We develop the R software package, smallsets, that can create Smallset Timelines from R and Python data preprocessing scripts. Constructing the figure asks practitioners to reflect on and revise decisions as necessary, while sharing it aims to make the process accessible to a diverse range of audiences. We present two case studies to illustrate use of the Smallset Timeline for visualising preprocessing decisions. Case studies include software defect data and income survey benchmark data, in which preprocessing affects levels of data loss and group fairness in prediction tasks, respectively. We envision Smallset Timelines as a go-to data provenance tool, enabling better documentation and communication of preprocessing tasks at large. △ Less

Submitted 10 June, 2022; originally announced June 2022.

Comments: In 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22), June 21-24, 2022, Seoul, Republic of Korea

On the faintest solar coronal hard X-rays observed with FOXSI

Authors: Juan Camilo Buitrago-Casas, Lindsay Glesener, Steven Christe, Säm Krucker, Juliana Vievering, P. S. Athiray, Sophie Musset, Lance Davis, Sasha Courtade, Gregory Dalton, Paul Turin, Zoe Turin, Brian Ramsey, Stephen Bongiorno, Daniel Ryan, Tadayuki Takahashi, Kento Furukawa, Shin Watanabe, Noriyuki Narukage, Shin-nosuke Ishikawa, Ikuyuki Mitsuishi, Kouichi Hagino, Van Shourt, Jessie Duncan, Yixian Zhang , et al. (1 additional authors not shown)

Abstract: Solar nanoflares are small eruptive events releasing magnetic energy in the quiet corona. If nanoflares follow the same physics as their larger counterparts, they should emit hard X-rays (HXRs) but with a rather faint intensity. A copious and continuous presence of nanoflares would deliver enormous amounts of energy into the solar corona, possibly accounting for its high temperatures. To date, the… ▽ More Solar nanoflares are small eruptive events releasing magnetic energy in the quiet corona. If nanoflares follow the same physics as their larger counterparts, they should emit hard X-rays (HXRs) but with a rather faint intensity. A copious and continuous presence of nanoflares would deliver enormous amounts of energy into the solar corona, possibly accounting for its high temperatures. To date, there has not been any direct observation of such sustained and persistent HXRs from the quiescent Sun. However, Hannah et al. in 2010 constrained the quiet Sun HXR emission using almost 12 days of quiescent solar-off-pointing observations by RHESSI. These observations set upper limits at $3.4\times 10^{-2}$ photons$^{-1}$ s$^{-1}$ cm$^{-2}$ keV$^{-1}$ and $9.5\times 10^{-4}$ photons$^{-1}$ s$^{-1}$ cm$^{-2}$ keV$^{-1}$ for the 3-6 keV and 6-12 keV energy ranges, respectively. Observing feeble HXRs is challenging because it demands high sensitivity and dynamic range instruments in HXRs. The Focusing Optics X-ray Solar Imager (FOXSI) sounding rocket experiment excels in these two attributes. Particularly, FOXSI completed its third successful flight (FOXSI-3) on September 7th, 2018. During FOXSI-3's flight, the Sun exhibited a fairly quiet configuration, displaying only one aged non-flaring active region. Using the entire $\sim$6.5 minutes of FOXSI-3 data, we constrained the quiet Sun emission in HXRs. We found $2σ$ upper limits in the order of $\sim 10^{-3}$ photons$^{-1}$ s$^{-1}$ cm$^{-2}$ keV$^{-1}$ for the 5-10 keV energy range. FOXSI-3's upper limit is consistent with what was reported by Hannah et al., 2010, but FOXSI-3 achieved this result using $\sim$1/2640 less time than RHESSI. A possible future spacecraft using FOXSI's concept would allow enough observation time to constrain the current HXR quiet Sun limits further or perhaps even make direct detections. △ Less

Submitted 9 May, 2022; originally announced May 2022.

Journal ref: A&A 665, A103 (2022)

Disc cloaking: Establishing a lower limit to the number density of local compact massive spheroids/bulges and the potential fate of some high-z red nuggets

Authors: Dexter S. -H. Hon, Alister W. Graham, Benjamin L. Davis, Alessandro Marconi

Abstract: The near-absence of compact massive quiescent galaxies in the local Universe implies a size evolution since $z\sim2.5$. It is often theorised that such `red nuggets' have evolved into today's elliptical (E) galaxies via an E-to-E transformation. We examine an alternative scenario in which a red nugget develops a rotational disc through mergers and accretion, say, at $1\lesssim z\lesssim2$, thereby… ▽ More The near-absence of compact massive quiescent galaxies in the local Universe implies a size evolution since $z\sim2.5$. It is often theorised that such `red nuggets' have evolved into today's elliptical (E) galaxies via an E-to-E transformation. We examine an alternative scenario in which a red nugget develops a rotational disc through mergers and accretion, say, at $1\lesssim z\lesssim2$, thereby cloaking the nugget as the extant bulge/spheroid component of a larger, now old, galaxy. We have performed detailed, physically-motivated, multi-component decompositions of a volume-limited sample of 103 massive ($M_*/\rm M_{\odot} \gtrsim 1\times 10^{11}$) galaxies within 110\,Mpc. Among our 28 galaxies with existing elliptical classifications, we found that 18 have large-scale discs, and two have intermediate-scale discs, and are reclassified here as lenticulars (S0) and elliculars (ES). The local spheroid stellar mass function, size-mass diagram and bulge-to-total ($B/T$) flux ratio are presented. We report lower-limits for the volume number density of compact massive spheroids, $n_\mathrm{c,Sph}\sim (0.17$-$1.2) \times 10^{-4}\,\rm Mpc^{-3}$, based on different definitions of `red nuggets' in the literature. Similar number densities of local compact massive bulges were reported by de la Rosa et al. using automated two-component decompositions and their existence is now abundantly clear with our multi-component decompositions. We find disc-cloaking to be a salient alternative for galaxy evolution. In particular, instead of an E-to-E process, disc growth is the dominant evolutionary pathway for at least low-mass ($1\times10^{10}<M_*/\rm M_{\odot} \lessapprox 4 \times 10^{10}$) red nuggets, while our current lower-limits are within an alluring factor of a few of the peak abundance of high-mass red nuggets at $1\lesssim z\lesssim2$. △ Less

Submitted 14 December, 2022; v1 submitted 28 April, 2022; originally announced April 2022.

Comments: Published in MNRAS. 44 pages, 22 figures

Journal ref: MNRAS, 514, 3410 (2022)

Neural Space-filling Curves

Authors: Hanyu Wang, Kamal Gupta, Larry Davis, Abhinav Shrivastava

Abstract: We present Neural Space-filling Curves (SFCs), a data-driven approach to infer a context-based scan order for a set of images. Linear ordering of pixels forms the basis for many applications such as video scrambling, compression, and auto-regressive models that are used in generative modeling for images. Existing algorithms resort to a fixed scanning algorithm such as Raster scan or Hilbert scan.… ▽ More We present Neural Space-filling Curves (SFCs), a data-driven approach to infer a context-based scan order for a set of images. Linear ordering of pixels forms the basis for many applications such as video scrambling, compression, and auto-regressive models that are used in generative modeling for images. Existing algorithms resort to a fixed scanning algorithm such as Raster scan or Hilbert scan. Instead, our work learns a spatially coherent linear ordering of pixels from the dataset of images using a graph-based neural network. The resulting Neural SFC is optimized for an objective suitable for the downstream task when the image is traversed along with the scan line order. We show the advantage of using Neural SFCs in downstream applications such as image compression. Code and additional results will be made available at https://hywang66.github.io/publication/neuralsfc. △ Less

Submitted 30 July, 2022; v1 submitted 18 April, 2022; originally announced April 2022.

Comments: ECCV 2022. Project page: https://hywang66.github.io/publication/neuralsfc/

Scanning gradiometry with a single spin quantum magnetometer

Authors: William S. Huxter, Marius L. Palm, Miranda L. Davis, Pol Welter, Charles-Henri Lambert, Morgan Trassin, Christian L. Degen

Abstract: Here, we demonstrate a gradiometry technique that significantly enhances the measurement sensitivity of such static fields, leading to new opportunities in the imaging of weakly magnetic systems. Our method relies on the mechanical oscillation of a single nitrogen-vacancy center at the tip of a scanning diamond probe, which up-converts the local spatial gradients into ac magnetic fields enabling t… ▽ More Here, we demonstrate a gradiometry technique that significantly enhances the measurement sensitivity of such static fields, leading to new opportunities in the imaging of weakly magnetic systems. Our method relies on the mechanical oscillation of a single nitrogen-vacancy center at the tip of a scanning diamond probe, which up-converts the local spatial gradients into ac magnetic fields enabling the use of sensitive ac quantum protocols. We show that gradiometry provides important advantages over static field imaging: (i) an order-of-magnitude better sensitivity, (ii) a more localized and sharper image, and (iii) a strong suppression of field drifts. We demonstrate the capabilities of gradiometry by imaging the nanotesla fields appearing above topographic defects and atomic steps in an antiferromagnet, direct currents in a graphene device, and para- and diamagnetic metals. △ Less

Submitted 18 February, 2022; originally announced February 2022.

Comments: 11 pages, 5 figures

Journal ref: Nat Commun 13, 3761 (2022)

Leveraging Bitstream Metadata for Fast, Accurate, Generalized Compressed Video Quality Enhancement

Authors: Max Ehrlich, Jon Barker, Namitha Padmanabhan, Larry Davis, Andrew Tao, Bryan Catanzaro, Abhinav Shrivastava

Abstract: Video compression is a central feature of the modern internet powering technologies from social media to video conferencing. While video compression continues to mature, for many compression settings, quality loss is still noticeable. These settings nevertheless have important applications to the efficient transmission of videos over bandwidth constrained or otherwise unstable connections. In this… ▽ More Video compression is a central feature of the modern internet powering technologies from social media to video conferencing. While video compression continues to mature, for many compression settings, quality loss is still noticeable. These settings nevertheless have important applications to the efficient transmission of videos over bandwidth constrained or otherwise unstable connections. In this work, we develop a deep learning architecture capable of restoring detail to compressed videos which leverages the underlying structure and motion information embedded in the video bitstream. We show that this improves restoration accuracy compared to prior compression correction methods and is competitive when compared with recent deep-learning-based video compression methods on rate-distortion while achieving higher throughput. Furthermore, we condition our model on quantization data which is readily available in the bitstream. This allows our single model to handle a variety of different compression quality settings which required an ensemble of models in prior work. △ Less

Submitted 30 October, 2023; v1 submitted 31 January, 2022; originally announced February 2022.

Comments: WACV 2024

Central X-ray point-sources found to be abundant in low-mass, late-type galaxies predicted to contain an intermediate-mass black hole

Authors: Alister W. Graham, Roberto Soria, Benjamin L. Davis, Mari Kolehmainen, Thomas Maccarone, James Miller-Jones, Christian Motch, Douglas A. Swartz

Abstract: Building upon three late-type galaxies in the Virgo cluster with both a predicted black hole mass of less than $\sim$10$^5$ M$_{\odot}$ and a centrally-located X-ray point-source, we reveal 11 more such galaxies, more than tripling the number of active intermediate-mass black hole candidates among this population. Moreover, this amounts to a 36$\pm$8% X-ray detection rate (despite the sometimes hi… ▽ More Building upon three late-type galaxies in the Virgo cluster with both a predicted black hole mass of less than $\sim$10$^5$ M$_{\odot}$ and a centrally-located X-ray point-source, we reveal 11 more such galaxies, more than tripling the number of active intermediate-mass black hole candidates among this population. Moreover, this amounts to a 36$\pm$8% X-ray detection rate (despite the sometimes high, X-ray-absorbing, HI column densities), compared to just 10$\pm$5% for (the largely HI-free) dwarf early-type galaxies in the Virgo cluster. The expected contribution of X-ray binaries from the galaxies' inner field stars is negligible. Moreover, given that both the spiral and dwarf galaxies contain nuclear star clusters, the above inequality appears to disfavor X-ray binaries in nuclear star clusters. The higher occupation, or rather detection, fraction among the spiral galaxies may instead reflect an enhanced cool gas/fuel supply and Eddington ratio. Indeed, four of the 11 new X-ray detections are associated with known LINERs or LINER/HII composites. For all (four) of the new detections for which the X-ray flux was strong enough to establish the spectral energy distribution in the Chandra band, it is consistent with power-law spectra. Furthermore, the X-ray emission from the source with the highest flux (NGC 4197: $L_X \approx 10^{40}$ erg s$^{-1}$) suggests a non-stellar-mass black hole if the X-ray spectrum corresponds to the `low/hard state'. Follow-up observations to further probe the black hole masses, and prospects for spatially resolving the gravitational spheres-of-influence around intermediate-mass black holes, are reviewed in some detail. △ Less

Submitted 15 December, 2021; originally announced December 2021.

Comments: To appear in ApJ (accepted Sep. 2021)

Potential Black Hole Seeding of the Spiral Galaxy NGC 4424 via an Infalling Star Cluster

Authors: Alister W. Graham, Roberto Soria, Bogdan C. Ciambur, Benjamin L. Davis, Douglas A. Swartz

Abstract: Galaxies can grow through their mutual gravitational attraction and subsequent union. While orbiting a regular high-surface-brightness galaxy, the body of a low-mass galaxy can be stripped away. However, the stellar heart of the infalling galaxy, if represented by a tightly-bound nuclear star cluster, is more resilient. From archival Hubble Space Telescope images, we have discovered a red, tidally… ▽ More Galaxies can grow through their mutual gravitational attraction and subsequent union. While orbiting a regular high-surface-brightness galaxy, the body of a low-mass galaxy can be stripped away. However, the stellar heart of the infalling galaxy, if represented by a tightly-bound nuclear star cluster, is more resilient. From archival Hubble Space Telescope images, we have discovered a red, tidally-stretched star cluster positioned ~5 arcseconds (~400 pc in projection) from, and pointing toward the center of, the post-merger spiral galaxy NGC 4424. The star cluster, which we refer to as `Nikhuli', has a near-infrared luminosity of (6.88+/-1.85)x10^6 L_{solar,F160W} and likely represents the nucleus of a captured/wedded galaxy. Moreover, from our Chandra X-ray Observatory image, Nikhuli is seen to contain a high-energy X-ray point source, with L_{0.5-8 keV} = 6.31^{+7.50}_{-3.77}x10^{38} erg/s (90% confidence). We argue that this is more likely to be an active massive black hole than an X-ray binary. Lacking an outward-pointing comet-like appearance, the stellar structure of Nikhuli favors infall rather than the ejection from a gravitational-wave recoil event. A minor merger with a low-mass early-type galaxy may have sown a massive black hole, aided an X-shaped pseudobulge, and be sewing a small bulge. The stellar mass and the velocity dispersion of NGC 4424 predict a central black hole of (0.6-1.0)x10^5 M_solar, similar to the expected intermediate-mass black hole in Nikhuli, and suggestive of a black hole supply mechanism for bulgeless late-type galaxies. We may potentially be witnessing black hole seeding by capture and sinking, with a nuclear star cluster the delivery vehicle. △ Less

Submitted 9 December, 2021; originally announced December 2021.

Comments: To appear in ApJ (accepted 21 September 2021)

Journal ref: ApJ, v.923, p.146 (2021)

InvGAN: Invertible GANs

Authors: Partha Ghosh, Dominik Zietlow, Michael J. Black, Larry S. Davis, Xiaochen Hu

Abstract: Generation of photo-realistic images, semantic editing and representation learning are a few of many potential applications of high resolution generative models. Recent progress in GANs have established them as an excellent choice for such tasks. However, since they do not provide an inference model, image editing or downstream tasks such as classification can not be done on real images using the… ▽ More Generation of photo-realistic images, semantic editing and representation learning are a few of many potential applications of high resolution generative models. Recent progress in GANs have established them as an excellent choice for such tasks. However, since they do not provide an inference model, image editing or downstream tasks such as classification can not be done on real images using the GAN latent space. Despite numerous efforts to train an inference model or design an iterative method to invert a pre-trained generator, previous methods are dataset (e.g. human face images) and architecture (e.g. StyleGAN) specific. These methods are nontrivial to extend to novel datasets or architectures. We propose a general framework that is agnostic to architecture and datasets. Our key insight is that, by training the inference and the generative model together, we allow them to adapt to each other and to converge to a better quality model. Our \textbf{InvGAN}, short for Invertible GAN, successfully embeds real images to the latent space of a high quality generative model. This allows us to perform image inpainting, merging, interpolation and online data augmentation. We demonstrate this with extensive qualitative and quantitative experiments. △ Less

Submitted 10 December, 2021; v1 submitted 8 December, 2021; originally announced December 2021.

Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistency

Authors: Soubhik Sanyal, Alex Vorobiov, Timo Bolkart, Matthew Loper, Betty Mohler, Larry Davis, Javier Romero, Michael J. Black

Abstract: Synthesizing images of a person in novel poses from a single image is a highly ambiguous task. Most existing approaches require paired training images; i.e. images of the same person with the same clothing in different poses. However, obtaining sufficiently large datasets with paired data is challenging and costly. Previous methods that forego paired supervision lack realism. We propose a self-sup… ▽ More Synthesizing images of a person in novel poses from a single image is a highly ambiguous task. Most existing approaches require paired training images; i.e. images of the same person with the same clothing in different poses. However, obtaining sufficiently large datasets with paired data is challenging and costly. Previous methods that forego paired supervision lack realism. We propose a self-supervised framework named SPICE (Self-supervised Person Image CrEation) that closes the image quality gap with supervised methods. The key insight enabling self-supervision is to exploit 3D information about the human body in several ways. First, the 3D body shape must remain unchanged when reposing. Second, representing body pose in 3D enables reasoning about self occlusions. Third, 3D body parts that are visible before and after reposing, should have similar appearance features. Once trained, SPICE takes an image of a person and generates a new image of that person in a new target pose. SPICE achieves state-of-the-art performance on the DeepFashion dataset, improving the FID score from 29.9 to 7.8 compared with previous unsupervised methods, and with performance similar to the state-of-the-art supervised method (6.4). SPICE also generates temporally coherent videos given an input image and a sequence of poses, despite being trained on static images only. △ Less

Submitted 11 October, 2021; originally announced October 2021.

Comments: International Conference on Computer Vision (ICCV)

The (Black Hole Mass)-(Spheroid Stellar Density) Relations: $M_{\rm BH}$--$μ$ (and $M_{\rm BH}$--$Σ$) and $M_{\rm BH}$--$ρ$

Authors: Nandini Sahu, Alister W. Graham, Benjamin L. Davis

Abstract: This paper is the fourth in a series presenting (galaxy morphology, and thus galaxy formation)-dependent black hole mass, $M_{\rm BH}$, scaling relations. We have used a sample of 119 galaxies with directly-measured $M_{\rm BH}$ and host spheroid parameters obtained from multi-component decomposition of, primarily, $3.6\,μ$m Spitzer images. Here, we investigate the correlations between… ▽ More This paper is the fourth in a series presenting (galaxy morphology, and thus galaxy formation)-dependent black hole mass, $M_{\rm BH}$, scaling relations. We have used a sample of 119 galaxies with directly-measured $M_{\rm BH}$ and host spheroid parameters obtained from multi-component decomposition of, primarily, $3.6\,μ$m Spitzer images. Here, we investigate the correlations between $M_{\rm BH}$ and the projected luminosity density $μ$, the projected stellar mass density $Σ$, and the deprojected (internal) stellar mass density $ρ$, for various spheroid radii. We discover the predicted $M_{\rm BH}$--$μ_{\rm 0,sph}$ relation and present the first $M_{\rm BH}$--$μ_{\rm e, sph}$ and $M_{\rm BH}$--$ρ_{\rm e,int, sph}$ diagrams displaying slightly different (possibly curved) trends for early- and late-type galaxies (ETGs and LTGs) and an offset between ETGs with (fast-rotators, ES/S0) and without (slow-rotators, E) a disk. The scatter about various $M_{\rm BH}$--$\langleΣ\rangle_{\rm R,sph}$ (and $\langleρ\rangle_{\rm r,sph}$) relations is shown to systematically decrease as the enclosing aperture (and volume) increases, dropping from 0.69~dex when using the spheroid \enquote{compactness}, $\langleΣ\rangle_{\rm 1kpc,sph}$, to 0.59~dex when using $\langleΣ\rangle_{\rm 5kpc,sph}$. We also reveal that $M_{\rm BH}$ correlates with the internal density, $ρ_{\rm soi,sph}$, at the BH's sphere-of-influence radius, such that core-Sérsic (high Sérsic index, $n$) and (low-$n$) Sérsic galaxies define different relations with total rms scatters 0.21~dex and 0.77~dex, respectively.(Abridged) △ Less

Submitted 11 October, 2021; originally announced October 2021.

Comments: 27 pages, 12 figures, 2 tables, submitted to ApJ

Journal ref: ApJ, 927, 67 (2022)

Modeling Item Response Theory with Stochastic Variational Inference

Authors: Mike Wu, Richard L. Davis, Benjamin W. Domingue, Chris Piech, Noah Goodman

Abstract: Item Response Theory (IRT) is a ubiquitous model for understanding human behaviors and attitudes based on their responses to questions. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving psychometric modeling leading to improved scientific understanding and public policy. However, while larger datasets allow for more flexible approaches, many… ▽ More Item Response Theory (IRT) is a ubiquitous model for understanding human behaviors and attitudes based on their responses to questions. Large modern datasets offer opportunities to capture more nuances in human behavior, potentially improving psychometric modeling leading to improved scientific understanding and public policy. However, while larger datasets allow for more flexible approaches, many contemporary algorithms for fitting IRT models may also have massive computational demands that forbid real-world application. To address this bottleneck, we introduce a variational Bayesian inference algorithm for IRT, and show that it is fast and scalable without sacrificing accuracy. Applying this method to five large-scale item response datasets from cognitive science and education yields higher log likelihoods and higher accuracy in imputing missing data than alternative inference algorithms. Using this new inference approach we then generalize IRT with expressive Bayesian models of responses, leveraging recent advances in deep learning to capture nonlinear item characteristic curves (ICC) with neural networks. Using an eigth-grade mathematics test from TIMSS, we show our nonlinear IRT models can capture interesting asymmetric ICCs. The algorithm implementation is open-source, and easily usable. △ Less

Submitted 28 July, 2022; v1 submitted 26 August, 2021; originally announced August 2021.

Comments: version two includes added experiments; 33 pages of content; 6 pages appendix; figures at the bottom. arXiv admin note: text overlap with arXiv:2002.00276

Partitioned K-nearest neighbor local depth for scalable comparison-based learning

Authors: Jacob D. Baron, R. W. R. Darling, J. Laylon Davis, R. Pettit

Abstract: A triplet comparison oracle on a set $S$ takes an object $x \in S$ and for any pair $\{y, z\} \subset S \setminus \{x\}$ declares which of $y$ and $z$ is more similar to $x$. Partitioned Local Depth (PaLD) supplies a principled non-parametric partitioning of $S$ under such triplet comparisons but needs $O(n^2 \log{n})$ oracle calls and $O(n^3)$ post-processing steps. We introduce Partitioned Nea… ▽ More A triplet comparison oracle on a set $S$ takes an object $x \in S$ and for any pair $\{y, z\} \subset S \setminus \{x\}$ declares which of $y$ and $z$ is more similar to $x$. Partitioned Local Depth (PaLD) supplies a principled non-parametric partitioning of $S$ under such triplet comparisons but needs $O(n^2 \log{n})$ oracle calls and $O(n^3)$ post-processing steps. We introduce Partitioned Nearest Neighbors Local Depth (PaNNLD), a computationally tractable variant of PaLD leveraging the $K$-nearest neighbors digraph on $S$. PaNNLD needs only $O(n K \log{n})$ oracle calls, by replacing an oracle call by a coin flip when neither $y$ nor $z$ is adjacent to $x$ in the undirected version of the $K$-nearest neighbors digraph. By averaging over randomizations, PaNNLD subsequently requires (at best) only $O(n K^2)$ post-processing steps. Concentration of measure shows that the probability of randomization-induced error $δ$ in PaNNLD is no more than $2 e^{-δ^2 K^2}$. △ Less

Submitted 2 December, 2021; v1 submitted 19 August, 2021; originally announced August 2021.

Comments: 27 pages, 2 figures

MSC Class: 90C35 ACM Class: F.2.2

The Organization of Chaos into a Molecular Trap that Supervises Ligand-Interaction, Selection and Steric Guidance Similar to Events in Black Holes

Authors: Leroy K. Davis

Abstract: In the current study, we demonstrated that allostery transpires by entropy transfers across time-spatial scales that actualize the conception of a molecular trap that supervises ligand interaction, selection, and migration into the amphipathic groove of the 14-3-3 ζ docking protein. Ligand binding transpires by steric guidance down a multi-dimensional trap constituted of superimposed chaotic, harm… ▽ More In the current study, we demonstrated that allostery transpires by entropy transfers across time-spatial scales that actualize the conception of a molecular trap that supervises ligand interaction, selection, and migration into the amphipathic groove of the 14-3-3 ζ docking protein. Ligand binding transpires by steric guidance down a multi-dimensional trap constituted of superimposed chaotic, harmonic, and electromagnetic field gradients. The individual traps exist in discrete domains governed by disparate physics interconnected by their resonance states and are subjective to damping. Notably, the highly structured molecular entanglement was genesis by the organization of white noise emitted by the anarchic motion of residues that comprised many of the features of black holes. △ Less

Submitted 26 August, 2021; v1 submitted 2 August, 2021; originally announced August 2021.

Comments: 23 pages, 17 figures

Wordcraft: a Human-AI Collaborative Editor for Story Writing

Authors: Andy Coenen, Luke Davis, Daphne Ippolito, Emily Reif, Ann Yuan

Abstract: As neural language models grow in effectiveness, they are increasingly being applied in real-world settings. However these applications tend to be limited in the modes of interaction they support. In this extended abstract, we propose Wordcraft, an AI-assisted editor for story writing in which a writer and a dialog system collaborate to write a story. Our novel interface uses few-shot learning and… ▽ More As neural language models grow in effectiveness, they are increasingly being applied in real-world settings. However these applications tend to be limited in the modes of interaction they support. In this extended abstract, we propose Wordcraft, an AI-assisted editor for story writing in which a writer and a dialog system collaborate to write a story. Our novel interface uses few-shot learning and the natural affordances of conversation to support a variety of interactions. Our editor provides a sandbox for writers to probe the boundaries of transformer-based language models and paves the way for future human-in-the-loop training pipelines and novel evaluation methods. △ Less

Submitted 15 July, 2021; originally announced July 2021.

Journal ref: First Workshop on Bridging Human-Computer Interaction and Natural Language Processing at EACL 2021

Rethinking Pseudo Labels for Semi-Supervised Object Detection

Authors: Hengduo Li, Zuxuan Wu, Abhinav Shrivastava, Larry S. Davis

Abstract: Recent advances in semi-supervised object detection (SSOD) are largely driven by consistency-based pseudo-labeling methods for image classification tasks, producing pseudo labels as supervisory signals. However, when using pseudo labels, there is a lack of consideration in localization precision and amplified class imbalance, both of which are critical for detection tasks. In this paper, we introd… ▽ More Recent advances in semi-supervised object detection (SSOD) are largely driven by consistency-based pseudo-labeling methods for image classification tasks, producing pseudo labels as supervisory signals. However, when using pseudo labels, there is a lack of consideration in localization precision and amplified class imbalance, both of which are critical for detection tasks. In this paper, we introduce certainty-aware pseudo labels tailored for object detection, which can effectively estimate the classification and localization quality of derived pseudo labels. This is achieved by converting conventional localization as a classification task followed by refinement. Conditioned on classification and localization quality scores, we dynamically adjust the thresholds used to generate pseudo labels and reweight loss functions for each category to alleviate the class imbalance problem. Extensive experiments demonstrate that our method improves state-of-the-art SSOD performance by 1-2% AP on COCO and PASCAL VOC while being orthogonal and complementary to most existing methods. In the limited-annotation regime, our approach improves supervised baselines by up to 10% AP using only 1-10% labeled data from COCO. △ Less

Submitted 29 December, 2021; v1 submitted 31 May, 2021; originally announced June 2021.

Comments: AAAI 2022

More Than Just Attention: Improving Cross-Modal Attentions with Contrastive Constraints for Image-Text Matching

Authors: Yuxiao Chen, Jianbo Yuan, Long Zhao, Tianlang Chen, Rui Luo, Larry Davis, Dimitris N. Metaxas

Abstract: Cross-modal attention mechanisms have been widely applied to the image-text matching task and have achieved remarkable improvements thanks to its capability of learning fine-grained relevance across different modalities. However, the cross-modal attention models of existing methods could be sub-optimal and inaccurate because there is no direct supervision provided during the training process. In t… ▽ More Cross-modal attention mechanisms have been widely applied to the image-text matching task and have achieved remarkable improvements thanks to its capability of learning fine-grained relevance across different modalities. However, the cross-modal attention models of existing methods could be sub-optimal and inaccurate because there is no direct supervision provided during the training process. In this work, we propose two novel training strategies, namely Contrastive Content Re-sourcing (CCR) and Contrastive Content Swapping (CCS) constraints, to address such limitations. These constraints supervise the training of cross-modal attention models in a contrastive learning manner without requiring explicit attention annotations. They are plug-in training strategies and can be easily integrated into existing cross-modal attention models. Additionally, we introduce three metrics including Attention Precision, Recall, and F1-Score to quantitatively measure the quality of learned attention models. We evaluate the proposed constraints by incorporating them into four state-of-the-art cross-modal attention-based image-text matching models. Experimental results on both Flickr30k and MS-COCO datasets demonstrate that integrating these constraints improves the model performance in terms of both retrieval performance and attention metrics. △ Less

Submitted 3 October, 2022; v1 submitted 20 May, 2021; originally announced May 2021.

Comments: Accepted to WACV 2023

Unsupervised Super-Resolution of Satellite Imagery for High Fidelity Material Label Transfer

Authors: Arthita Ghosh, Max Ehrlich, Larry Davis, Rama Chellappa

Abstract: Urban material recognition in remote sensing imagery is a highly relevant, yet extremely challenging problem due to the difficulty of obtaining human annotations, especially on low resolution satellite images. To this end, we propose an unsupervised domain adaptation based approach using adversarial learning. We aim to harvest information from smaller quantities of high resolution data (source dom… ▽ More Urban material recognition in remote sensing imagery is a highly relevant, yet extremely challenging problem due to the difficulty of obtaining human annotations, especially on low resolution satellite images. To this end, we propose an unsupervised domain adaptation based approach using adversarial learning. We aim to harvest information from smaller quantities of high resolution data (source domain) and utilize the same to super-resolve low resolution imagery (target domain). This can potentially aid in semantic as well as material label transfer from a richly annotated source to a target domain. △ Less

Submitted 15 May, 2021; originally announced May 2021.

Comments: Published in the proceedings of the 2019 IEEE International Geoscience and Remote Sensing Symposium

Journal ref: IGARSS (2019), 5144-5147

DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Authors: Shiyi Lan, Zhiding Yu, Christopher Choy, Subhashree Radhakrishnan, Guilin Liu, Yuke Zhu, Larry S. Davis, Anima Anandkumar

Abstract: We introduce DiscoBox, a novel framework that jointly learns instance segmentation and semantic correspondence using bounding box supervision. Specifically, we propose a self-ensembling framework where instance segmentation and semantic correspondence are jointly guided by a structured teacher in addition to the bounding box supervision. The teacher is a structured energy model incorporating a pai… ▽ More We introduce DiscoBox, a novel framework that jointly learns instance segmentation and semantic correspondence using bounding box supervision. Specifically, we propose a self-ensembling framework where instance segmentation and semantic correspondence are jointly guided by a structured teacher in addition to the bounding box supervision. The teacher is a structured energy model incorporating a pairwise potential and a cross-image potential to model the pairwise pixel relationships both within and across the boxes. Minimizing the teacher energy simultaneously yields refined object masks and dense correspondences between intra-class objects, which are taken as pseudo-labels to supervise the task network and provide positive/negative correspondence pairs for dense constrastive learning. We show a symbiotic relationship where the two tasks mutually benefit from each other. Our best model achieves 37.9% AP on COCO instance segmentation, surpassing prior weakly supervised methods and is competitive to supervised methods. We also obtain state of the art weakly supervised results on PASCAL VOC12 and PF-PASCAL with real-time inference. △ Less

Submitted 5 June, 2021; v1 submitted 13 May, 2021; originally announced May 2021.

Comments: Tech Report

Refining the mass estimate for the intermediate-mass black hole candidate in NGC 3319

Authors: Benjamin L. Davis, Alister W. Graham

Abstract: Recent X-ray observations by Jiang et al. have identified an active galactic nucleus (AGN) in the bulgeless spiral galaxy NGC 3319, located just $14.3\pm1.1\,$Mpc away, and suggest the presence of an intermediate-mass black hole (IMBH; $10^2\leq M_\bullet/\mathrm{M_{\odot}}\leq10^5$) if the Eddington ratios are as high as 3 to $3\times10^{-3}$. In an effort to refine the black hole mass for this (… ▽ More Recent X-ray observations by Jiang et al. have identified an active galactic nucleus (AGN) in the bulgeless spiral galaxy NGC 3319, located just $14.3\pm1.1\,$Mpc away, and suggest the presence of an intermediate-mass black hole (IMBH; $10^2\leq M_\bullet/\mathrm{M_{\odot}}\leq10^5$) if the Eddington ratios are as high as 3 to $3\times10^{-3}$. In an effort to refine the black hole mass for this (currently) rare class of object, we have explored multiple black hole mass scaling relations, such as those involving the (not previously used) velocity dispersion, logarithmic spiral-arm pitch angle, total galaxy stellar mass, nuclear star cluster mass, rotational velocity, and colour of NGC 3319, to obtain ten mass estimates, of differing accuracy. We have calculated a mass of $3.14_{-2.20}^{+7.02}\times10^4\,\mathrm{M_\odot}$, with a confidence of 84% that it is $\leq$$10^5\,\mathrm{M_\odot}$, based on the combined probability density function from seven of these individual estimates. Our conservative approach excluded two black hole mass estimates (via the nuclear star cluster mass, and the fundamental plane of black hole activity $\unicode{x2014}$ which only applies to black holes with low accretion rates) that were upper limits of $\sim$$10^5\,{\rm M}_{\odot}$, and it did not use the $M_\bullet\unicode{x2013}L_{\rm 2-10\,keV}$ relation's prediction of $\sim$$10^5\,{\rm M}_{\odot}$. This target provides an exceptional opportunity to study an IMBH in AGN mode and advance our demographic knowledge of black holes. Furthermore, we introduce our novel method of meta-analysis as a beneficial technique for identifying new IMBH candidates by quantifying the probability that a galaxy possesses an IMBH. △ Less

Submitted 10 May, 2021; originally announced May 2021.

Comments: Original unedited manuscript (19 pages & 7 figures), accepted for publication by the Publications of the Astronomical Society of Australia, May 7, 2021

Journal ref: PASA, 38, e030 (2021)

VideoLT: Large-scale Long-tailed Video Recognition

Authors: Xing Zhang, Zuxuan Wu, Zejia Weng, Huazhu Fu, Jingjing Chen, Yu-Gang Jiang, Larry Davis

Abstract: Label distributions in real-world are oftentimes long-tailed and imbalanced, resulting in biased models towards dominant labels. While long-tailed recognition has been extensively studied for image classification tasks, limited effort has been made for video domain. In this paper, we introduce VideoLT, a large-scale long-tailed video recognition dataset, as a step toward real-world video recogniti… ▽ More Label distributions in real-world are oftentimes long-tailed and imbalanced, resulting in biased models towards dominant labels. While long-tailed recognition has been extensively studied for image classification tasks, limited effort has been made for video domain. In this paper, we introduce VideoLT, a large-scale long-tailed video recognition dataset, as a step toward real-world video recognition. Our VideoLT contains 256,218 untrimmed videos, annotated into 1,004 classes with a long-tailed distribution. Through extensive studies, we demonstrate that state-of-the-art methods used for long-tailed image recognition do not perform well in the video domain due to the additional temporal dimension in video data. This motivates us to propose FrameStack, a simple yet effective method for long-tailed video recognition task. In particular, FrameStack performs sampling at the frame-level in order to balance class distributions, and the sampling ratio is dynamically determined using knowledge derived from the network during training. Experimental results demonstrate that FrameStack can improve classification performance without sacrificing overall accuracy. Code and dataset are available at: https://github.com/17Skye17/VideoLT. △ Less

Submitted 18 August, 2021; v1 submitted 6 May, 2021; originally announced May 2021.

Comments: To appear in ICCV 2021

Learned Spatial Representations for Few-shot Talking-Head Synthesis

Authors: Moustafa Meshry, Saksham Suri, Larry S. Davis, Abhinav Shrivastava

Abstract: We propose a novel approach for few-shot talking-head synthesis. While recent works in neural talking heads have produced promising results, they can still produce images that do not preserve the identity of the subject in source images. We posit this is a result of the entangled representation of each subject in a single latent code that models 3D shape information, identity cues, colors, lightin… ▽ More We propose a novel approach for few-shot talking-head synthesis. While recent works in neural talking heads have produced promising results, they can still produce images that do not preserve the identity of the subject in source images. We posit this is a result of the entangled representation of each subject in a single latent code that models 3D shape information, identity cues, colors, lighting and even background details. In contrast, we propose to factorize the representation of a subject into its spatial and style components. Our method generates a target frame in two steps. First, it predicts a dense spatial layout for the target image. Second, an image generator utilizes the predicted layout for spatial denormalization and synthesizes the target frame. We experimentally show that this disentangled representation leads to a significant improvement over previous methods, both quantitatively and qualitatively. △ Less

Submitted 29 April, 2021; originally announced April 2021.

Comments: http://www.cs.umd.edu/~mmeshry/projects/lsr/