Showing 101–150 of 1,808 results for author: Lee, T

On the quantum time complexity of divide and conquer

Authors: Jonathan Allcock, Jinge Bao, Aleksandrs Belovs, Troy Lee, Miklos Santha

Abstract: We initiate a systematic study of the time complexity of quantum divide and conquer algorithms for classical problems. We establish generic conditions under which search and minimization problems with classical divide and conquer algorithms are amenable to quantum speedup and apply these theorems to an array of problems involving strings, integers, and geometric objects. They include LONGEST DISTI… ▽ More We initiate a systematic study of the time complexity of quantum divide and conquer algorithms for classical problems. We establish generic conditions under which search and minimization problems with classical divide and conquer algorithms are amenable to quantum speedup and apply these theorems to an array of problems involving strings, integers, and geometric objects. They include LONGEST DISTINCT SUBSTRING, KLEE'S COVERAGE, several optimization problems on stock transactions, and k-INCREASING SUBSEQUENCE. For most of these results, our quantum time upper bound matches the quantum query lower bound for the problem, up to polylogarithmic factors. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Comments: 48 pages, accepted to QIP 2024

Noise Analysis for Performance Evaluation of Biopotential Recording Front-Ends

Authors: Taeju Lee

Abstract: Noise efficiency factor (NEF) and power efficiency factor (PEF) are widely used as the figure of merit to quantify the performance of biopotential recording front-ends. NEF and PEF are discussed from the noise analysis to the trend survey. To provide a comprehensive performance comparison of the front-ends, the performance mapping is developed using the design parameters of the technology node, NE… ▽ More Noise efficiency factor (NEF) and power efficiency factor (PEF) are widely used as the figure of merit to quantify the performance of biopotential recording front-ends. NEF and PEF are discussed from the noise analysis to the trend survey. To provide a comprehensive performance comparison of the front-ends, the performance mapping is developed using the design parameters of the technology node, NEF, PEF, |PEF - NEF|, and supply voltage. Using |PEF - NEF| provides how well a front-end balances between current-noise efficiency and power-noise efficiency, in other words, how biased a front-end is between current- and power-noise efficiencies. Also, the performance mappings of different front-end architectures are presented. △ Less

Submitted 27 November, 2023; originally announced November 2023.

Comments: 16 pages, 8 figures

Positional Description Matters for Transformers Arithmetic

Authors: Ruoqi Shen, Sébastien Bubeck, Ronen Eldan, Yin Tat Lee, Yuanzhi Li, Yi Zhang

Abstract: Transformers, central to the successes in modern Natural Language Processing, often falter on arithmetic tasks despite their vast capabilities --which paradoxically include remarkable coding abilities. We observe that a crucial challenge is their naive reliance on positional information to solve arithmetic problems with a small number of digits, leading to poor performance on larger numbers. Herei… ▽ More Transformers, central to the successes in modern Natural Language Processing, often falter on arithmetic tasks despite their vast capabilities --which paradoxically include remarkable coding abilities. We observe that a crucial challenge is their naive reliance on positional information to solve arithmetic problems with a small number of digits, leading to poor performance on larger numbers. Herein, we delve deeper into the role of positional encoding, and propose several ways to fix the issue, either by modifying the positional encoding directly, or by modifying the representation of the arithmetic task to leverage standard positional encoding differently. We investigate the value of these modifications for three tasks: (i) classical multiplication, (ii) length extrapolation in addition, and (iii) addition in natural language context. For (i) we train a small model on a small dataset (100M parameters and 300k samples) with remarkable aptitude in (direct, no scratchpad) 15 digits multiplication and essentially perfect up to 12 digits, while usual training in this context would give a model failing at 4 digits multiplication. In the experiments on addition, we use a mere 120k samples to demonstrate: for (ii) extrapolation from 10 digits to testing on 12 digits numbers while usual training would have no extrapolation, and for (iii) almost perfect accuracy up to 5 digits while usual training would be correct only up to 3 digits (which is essentially memorization with a training set of 120k samples). △ Less

Submitted 21 November, 2023; originally announced November 2023.

Comments: 18 pages

RAISE -- Radiology AI Safety, an End-to-end lifecycle approach

Authors: M. Jorge Cardoso, Julia Moosbauer, Tessa S. Cook, B. Selnur Erdal, Brad Genereaux, Vikash Gupta, Bennett A. Landman, Tiarna Lee, Parashkev Nachev, Elanchezhian Somasundaram, Ronald M. Summers, Khaled Younis, Sebastien Ourselin, Franz MJ Pfister

Abstract: The integration of AI into radiology introduces opportunities for improved clinical care provision and efficiency but it demands a meticulous approach to mitigate potential risks as with any other new technology. Beginning with rigorous pre-deployment evaluation and validation, the focus should be on ensuring models meet the highest standards of safety, effectiveness and efficacy for their intende… ▽ More The integration of AI into radiology introduces opportunities for improved clinical care provision and efficiency but it demands a meticulous approach to mitigate potential risks as with any other new technology. Beginning with rigorous pre-deployment evaluation and validation, the focus should be on ensuring models meet the highest standards of safety, effectiveness and efficacy for their intended applications. Input and output guardrails implemented during production usage act as an additional layer of protection, identifying and addressing individual failures as they occur. Continuous post-deployment monitoring allows for tracking population-level performance (data drift), fairness, and value delivery over time. Scheduling reviews of post-deployment model performance and educating radiologists about new algorithmic-driven findings is critical for AI to be effective in clinical practice. Recognizing that no single AI solution can provide absolute assurance even when limited to its intended use, the synergistic application of quality assurance at multiple levels - regulatory, clinical, technical, and ethical - is emphasized. Collaborative efforts between stakeholders spanning healthcare systems, industry, academia, and government are imperative to address the multifaceted challenges involved. Trust in AI is an earned privilege, contingent on a broad set of goals, among them transparently demonstrating that the AI adheres to the same rigorous safety, effectiveness and efficacy standards as other established medical technologies. By doing so, developers can instil confidence among providers and patients alike, enabling the responsible scaling of AI and the realization of its potential benefits. The roadmap presented herein aims to expedite the achievement of deployable, reliable, and safe AI in radiology. △ Less

Submitted 24 November, 2023; originally announced November 2023.

Comments: 14 pages, 3 figures

High-Quality Face Caricature via Style Translation

Authors: Lamyanba Laishram, Muhammad Shaheryar, Jong Taek Lee, Soon Ki Jung

Abstract: Caricature is an exaggerated form of artistic portraiture that accentuates unique yet subtle characteristics of human faces. Recently, advancements in deep end-to-end techniques have yielded encouraging outcomes in capturing both style and elevated exaggerations in creating face caricatures. Most of these approaches tend to produce cartoon-like results that could be more practical for real-world a… ▽ More Caricature is an exaggerated form of artistic portraiture that accentuates unique yet subtle characteristics of human faces. Recently, advancements in deep end-to-end techniques have yielded encouraging outcomes in capturing both style and elevated exaggerations in creating face caricatures. Most of these approaches tend to produce cartoon-like results that could be more practical for real-world applications. In this study, we proposed a high-quality, unpaired face caricature method that is appropriate for use in the real world and uses computer vision techniques and GAN models. We attain the exaggeration of facial features and the stylization of appearance through a two-step process: Face caricature generation and face caricature projection. The face caricature generation step creates new caricature face datasets from real images and trains a generative model using the real and newly created caricature datasets. The Face caricature projection employs an encoder trained with real and caricature faces with the pretrained generator to project real and caricature faces. We perform an incremental facial exaggeration from the real image to the caricature faces using the encoder and generator's latent space. Our projection preserves the facial identity, attributes, and expressions from the input image. Also, it accounts for facial occlusions, such as reading glasses or sunglasses, to enhance the robustness of our model. Furthermore, we conducted a comprehensive comparison of our approach with various state-of-the-art face caricature methods, highlighting our process's distinctiveness and exceptional realism. △ Less

Submitted 22 November, 2023; originally announced November 2023.

Comments: 14 pages, 21 figures

Rotational spectroscopic characterisation of the [D2,C,S] system: an update from the laboratory and theory

Authors: Natalia Inostroza-Pino, Valerio Lattanzi, C. Zachary Palmer, Ryan C. Fortenberry, Diego Mardones, Paola Caselli, Oko E. Godwin, Timothy J. Lee

Abstract: The synergy between high-resolution rotational spectroscopy and quantum-chemical calculations is essential for exploring future detection of molecules, especially when spectroscopy parameters are not available yet. By using highly correlated ab initio quartic force fields (QFFs) from explicitly correlated coupled cluster theory, a complete set of rotational constants and centrifugal distortion con… ▽ More The synergy between high-resolution rotational spectroscopy and quantum-chemical calculations is essential for exploring future detection of molecules, especially when spectroscopy parameters are not available yet. By using highly correlated ab initio quartic force fields (QFFs) from explicitly correlated coupled cluster theory, a complete set of rotational constants and centrifugal distortion constants for D$_2$CS and cis/trans-DCSD isomers have been produced. Comparing our new ab initio results for D$_2$CS with new rotational spectroscopy laboratory data for the same species, the accuracy of the computed B and C rotational constants is within 0.1% while the A constant is only slightly higher. Additionally, quantum chemical vibrational frequencies are also provided, and these spectral reference data and new experimental rotational lines will provide additional references for potential observation of these deuterated sulfur species with either ground-based radio telescopes or space-based infrared observatories. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: 15 pages, 2 figures and 1 appendix table; published in Molecular Physics (Tim Lee Memorial Issue)

Galaxy Clusters Discovered via the Thermal Sunyaev-Zel'dovich Effect in the 500-square-degree SPTpol Survey

Authors: L. E. Bleem, M. Klein, T. M. C. Abbott, P. A. R. Ade, M. Aguena, O. Alves, A. J. Anderson, F. Andrade-Oliveira, B. Ansarinejad, M. Archipley, M. L. N. Ashby, J. E. Austermann, D. Bacon, J. A. Beall, A. N. Bender, B. A. Benson, F. Bianchini, S. Bocquet, D. Brooks, D. L. Burke, M. Calzadilla, J. E. Carlstrom, A. Carnero Rosell, J. Carretero, C. L. Chang , et al. (103 additional authors not shown)

Abstract: We present a catalog of 689 galaxy cluster candidates detected at significance $ξ>4$ via their thermal Sunyaev-Zel'dovich (SZ) effect signature in 95 and 150 GHz data from the 500-square-degree SPTpol survey. We use optical and infrared data from the Dark Energy Camera and the Wide-field Infrared Survey Explorer (WISE) and \spitzer \ satellites, to confirm 544 of these candidates as clusters with… ▽ More We present a catalog of 689 galaxy cluster candidates detected at significance $ξ>4$ via their thermal Sunyaev-Zel'dovich (SZ) effect signature in 95 and 150 GHz data from the 500-square-degree SPTpol survey. We use optical and infrared data from the Dark Energy Camera and the Wide-field Infrared Survey Explorer (WISE) and \spitzer \ satellites, to confirm 544 of these candidates as clusters with $\sim94\%$ purity. The sample has an approximately redshift-independent mass threshold at redshift $z>0.25$ and spans $1.5 \times 10^{14} < M_{500c} < 9.1 \times 10^{14}$ $M_\odot/h_{70}$ \ and $0.03<z\lesssim1.6$ in mass and redshift, respectively; 21\% of the confirmed clusters are at $z>1$. We use external radio data from the Sydney University Molonglo Sky Survey (SUMSS) to estimate contamination to the SZ signal from synchrotron sources. The contamination reduces the recovered $ξ$ by a median value of 0.032, or $\sim0.8\%$ of the $ξ=4$ threshold value, and $\sim7\%$ of candidates have a predicted contamination greater than $Δξ= 1$. With the exception of a small number of systems $(<1\%)$, an analysis of clusters detected in single-frequency 95 and 150 GHz data shows no significant contamination of the SZ signal by emission from dusty or synchrotron sources. This cluster sample will be a key component in upcoming astrophysical and cosmological analyses of clusters. The SPTpol millimeter-wave maps and associated data products used to produce this sample are available at https://pole.uchicago.edu/public/data/sptpol_500d_clusters/index.html, and the NASA LAMBDA website. An interactive sky server with the SPTpol maps and Dark Energy Survey data release 2 images is also available at NCSA https://skyviewer.ncsa.illinois.edu. △ Less

Submitted 8 February, 2024; v1 submitted 13 November, 2023; originally announced November 2023.

Comments: Matches version accepted by OJA. 19 pages + references, 14 figures, cluster candidate table provided in Appendix. Data products available at https://pole.uchicago.edu/public/data/sptpol_500d_clusters/index.html and an interactive sky server at https://skyviewer.ncsa.illinois.edu

Journal ref: Open Journal of Astrophysics, Volume 7, 2024

Regenerating Arbitrary Video Sequences with Distillation Path-Finding

Authors: Thi-Ngoc-Hanh Le, Sheng-Yi Yao, Chun-Te Wu, Tong-Yee Lee

Abstract: If the video has long been mentioned as a widespread visualization form, the animation sequence in the video is mentioned as storytelling for people. Producing an animation requires intensive human labor from skilled professional artists to obtain plausible animation in both content and motion direction, incredibly for animations with complex content, multiple moving objects, and dense movement. T… ▽ More If the video has long been mentioned as a widespread visualization form, the animation sequence in the video is mentioned as storytelling for people. Producing an animation requires intensive human labor from skilled professional artists to obtain plausible animation in both content and motion direction, incredibly for animations with complex content, multiple moving objects, and dense movement. This paper presents an interactive framework to generate new sequences according to the users' preference on the starting frame. The critical contrast of our approach versus prior work and existing commercial applications is that novel sequences with arbitrary starting frame are produced by our system with a consistent degree in both content and motion direction. To achieve this effectively, we first learn the feature correlation on the frameset of the given video through a proposed network called RSFNet. Then, we develop a novel path-finding algorithm, SDPF, which formulates the knowledge of motion directions of the source video to estimate the smooth and plausible sequences. The extensive experiments show that our framework can produce new animations on the cartoon and natural scenes and advance prior works and commercial applications to enable users to obtain more predictable results. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Comments: This paper has been accepted for publication on IEEE Transactions on Visualization and Computer Graphics (TVCG), January 2023. Project website: http://graphics.csie.ncku.edu.tw/SDPF

Multi-Agent Reinforcement Learning for the Low-Level Control of a Quadrotor UAV

Authors: Beomyeol Yu, Taeyoung Lee

Abstract: By leveraging the underlying structures of the quadrotor dynamics, we propose multi-agent reinforcement learning frameworks to innovate the low-level control of a quadrotor, where independent agents operate cooperatively to achieve a common goal. While single-agent reinforcement learning has been successfully applied in quadrotor controls, training a large monolithic network is often data-intensiv… ▽ More By leveraging the underlying structures of the quadrotor dynamics, we propose multi-agent reinforcement learning frameworks to innovate the low-level control of a quadrotor, where independent agents operate cooperatively to achieve a common goal. While single-agent reinforcement learning has been successfully applied in quadrotor controls, training a large monolithic network is often data-intensive and time-consuming. Moreover, achieving agile yawing control remains a significant challenge due to the strongly coupled nature of the quadrotor dynamics. To address this, we decompose the quadrotor dynamics into translational and yawing components and assign collaborative reinforcement learning agents to each part to facilitate more efficient training. Additionally, we introduce regularization terms to mitigate steady-state errors and prevent excessive maneuvers. Benchmark studies, including sim-to-sim transfer verification, demonstrate that our proposed training schemes substantially improve the convergence rate of training, while enhancing flight control performance and stability compared to traditional single-agent approaches. △ Less

Submitted 26 February, 2024; v1 submitted 10 November, 2023; originally announced November 2023.

Comments: 8 pages, 5 figures, 3 tables

Multi Higgs Boson Signals of a Modified Muon Yukawa Coupling at a Muon Collider

Authors: Radovan Dermisek, Keith Hermanek, Taegyu Lee, Navin McGinnis, Sangsik Yoon

Abstract: We study di-Higgs and tri-Higgs boson productions at a muon collider as functions of the modification of the muon Yukawa coupling resulting from new physics parameterized by the dimension 6 mass operator. We show that the di-Higgs signal can be used to observe a deviation in the muon Yukawa coupling at the 10 % level for $\sqrt{s} = 10$ TeV and at the 3.5 % level for $\sqrt{s} = 30$ TeV. The tri-H… ▽ More We study di-Higgs and tri-Higgs boson productions at a muon collider as functions of the modification of the muon Yukawa coupling resulting from new physics parameterized by the dimension 6 mass operator. We show that the di-Higgs signal can be used to observe a deviation in the muon Yukawa coupling at the 10 % level for $\sqrt{s} = 10$ TeV and at the 3.5 % level for $\sqrt{s} = 30$ TeV. The tri-Higgs signal improves the sensitivity dramatically with increasing $\sqrt{s}$, reaching 0.8 % at $\sqrt{s} = 30$ TeV. We also study all processes involving Goldstone bosons originating from the same operator, discuss possible model dependence resulting from other operators of dimension 6 and higher, and identify multi-Higgs productions and one additional process as golden channels. We further extend the study to the two Higgs doublet model type-II and show that di-Higgs and tri-Higgs signals involving heavy Higgs bosons can be enhanced by a factor of $(\tan β)^6$, which results in the potential sensitivity to a modified muon Yukawa coupling at the $10^{-6}$ level already at a $\sqrt{s} = 10 $ TeV muon collider. The results can be easily customized for other extensions of the Higgs sector. △ Less

Submitted 24 May, 2024; v1 submitted 8 November, 2023; originally announced November 2023.

Comments: 38 pages, 9 figures and 14 tables. V2 matches published version

Retargeting video with an end-to-end framework

Authors: Thi-Ngoc-Hanh Le, HuiGuang Huang, Yi-Ru Chen, Tong-Yee Lee

Abstract: Video holds significance in computer graphics applications. Because of the heterogeneous of digital devices, retargeting videos becomes an essential function to enhance user viewing experience in such applications. In the research of video retargeting, preserving the relevant visual content in videos, avoiding flicking, and processing time are the vital challenges. Extending image retargeting tech… ▽ More Video holds significance in computer graphics applications. Because of the heterogeneous of digital devices, retargeting videos becomes an essential function to enhance user viewing experience in such applications. In the research of video retargeting, preserving the relevant visual content in videos, avoiding flicking, and processing time are the vital challenges. Extending image retargeting techniques to the video domain is challenging due to the high running time. Prior work of video retargeting mainly utilizes time-consuming preprocessing to analyze frames. Plus, being tolerant of different video content, avoiding important objects from shrinking, and the ability to play with arbitrary ratios are the limitations that need to be resolved in these systems requiring investigation. In this paper, we present an end-to-end RETVI method to retarget videos to arbitrary aspect ratios. We eliminate the computational bottleneck in the conventional approaches by designing RETVI with two modules, content feature analyzer (CFA) and adaptive deforming estimator (ADE). The extensive experiments and evaluations show that our system outperforms previous work in quality and running time. Visit our project website for more results at http://graphics.csie.ncku.edu.tw/RETVI. △ Less

Submitted 8 November, 2023; v1 submitted 7 November, 2023; originally announced November 2023.

Comments: This paper has been accepted for publication on IEEE Transactions on Visualization and Computer Graphics, October 2023

Holistic Evaluation of Text-To-Image Models

Authors: Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Benita Teufel, Marco Bellagente, Minguk Kang, Taesung Park, Jure Leskovec, Jun-Yan Zhu, Li Fei-Fei, Jiajun Wu, Stefano Ermon, Percy Liang

Abstract: The stunning qualitative improvement of recent text-to-image models has led to their widespread attention and adoption. However, we lack a comprehensive quantitative understanding of their capabilities and risks. To fill this gap, we introduce a new benchmark, Holistic Evaluation of Text-to-Image Models (HEIM). Whereas previous evaluations focus mostly on text-image alignment and image quality, we… ▽ More The stunning qualitative improvement of recent text-to-image models has led to their widespread attention and adoption. However, we lack a comprehensive quantitative understanding of their capabilities and risks. To fill this gap, we introduce a new benchmark, Holistic Evaluation of Text-to-Image Models (HEIM). Whereas previous evaluations focus mostly on text-image alignment and image quality, we identify 12 aspects, including text-image alignment, image quality, aesthetics, originality, reasoning, knowledge, bias, toxicity, fairness, robustness, multilinguality, and efficiency. We curate 62 scenarios encompassing these aspects and evaluate 26 state-of-the-art text-to-image models on this benchmark. Our results reveal that no single model excels in all aspects, with different models demonstrating different strengths. We release the generated images and human evaluation results for full transparency at https://crfm.stanford.edu/heim/v1.1.0 and the code at https://github.com/stanford-crfm/helm, which is integrated with the HELM codebase. △ Less

Submitted 7 November, 2023; originally announced November 2023.

Comments: NeurIPS 2023. First three authors contributed equally

AI-Enabled Unmanned Vehicle-Assisted Reconfigurable Intelligent Surfaces: Deployment, Prototyping, Experiments, and Opportunities

Authors: Li-Hsiang Shen, Kai-Ten Feng, Ta-Sung Lee, Yuan-Chun Lin, Shih-Cheng Lin, Chia-Chan Chang, Sheng-Fuh Chang

Abstract: The requirement of wireless data demands is increasingly high as the sixth-generation (6G) technology evolves. Reconfigurable intelligent surface (RIS) is promisingly deemed to be one of 6G techniques for extending service coverage, reducing power consumption, and enhancing spectral efficiency. In this article, we have provided some fundamentals of RIS deployment in theory and hardware perspective… ▽ More The requirement of wireless data demands is increasingly high as the sixth-generation (6G) technology evolves. Reconfigurable intelligent surface (RIS) is promisingly deemed to be one of 6G techniques for extending service coverage, reducing power consumption, and enhancing spectral efficiency. In this article, we have provided some fundamentals of RIS deployment in theory and hardware perspectives as well as utilization of artificial intelligence (AI) and machine learning. We conducted an intelligent deployment of RIS (i-Dris) prototype, including dual-band auto-guided vehicle (AGV) assisted RISs associated with an mmWave base station (BS) and a receiver. The RISs are deployed on the AGV with configured incident/reflection angles. While, both the mmWave BS and receiver are associated with an edge server monitoring downlink packets for obtaining system throughput. We have designed a federated multi-agent reinforcement learning scheme associated with several AGV-RIS agents and sub-agents per AGV-RIS consisting of the deployment of position, height, orientation and elevation angles. The experimental results presented the stationary measurement in different aspects and scenarios. The i-Dris can reach up to 980 Mbps transmission throughput under a bandwidth of 100 MHz with comparably low complexity as well as rapid deployment, which outperforms the other existing works. At last, we highlight some opportunities and future issues in leveraging RIS-empowered wireless communication networks. △ Less

Submitted 6 November, 2023; originally announced November 2023.

Semantic Scene Graph Generation Based on an Edge Dual Scene Graph and Message Passing Neural Network

Authors: Hyeongjin Kim, Sangwon Kim, Jong Taek Lee, Byoung Chul Ko

Abstract: Along with generative AI, interest in scene graph generation (SGG), which comprehensively captures the relationships and interactions between objects in an image and creates a structured graph-based representation, has significantly increased in recent years. However, relying on object-centric and dichotomous relationships, existing SGG methods have a limited ability to accurately predict detailed… ▽ More Along with generative AI, interest in scene graph generation (SGG), which comprehensively captures the relationships and interactions between objects in an image and creates a structured graph-based representation, has significantly increased in recent years. However, relying on object-centric and dichotomous relationships, existing SGG methods have a limited ability to accurately predict detailed relationships. To solve these problems, a new approach to the modeling multiobject relationships, called edge dual scene graph generation (EdgeSGG), is proposed herein. EdgeSGG is based on a edge dual scene graph and Dual Message Passing Neural Network (DualMPNN), which can capture rich contextual interactions between unconstrained objects. To facilitate the learning of edge dual scene graphs with a symmetric graph structure, the proposed DualMPNN learns both object- and relation-centric features for more accurately predicting relation-aware contexts and allows fine-grained relational updates between objects. A comparative experiment with state-of-the-art (SoTA) methods was conducted using two public datasets for SGG operations and six metrics for three subtasks. Compared with SoTA approaches, the proposed model exhibited substantial performance improvements across all SGG subtasks. Furthermore, experiment on long-tail distributions revealed that incorporating the relationships between objects effectively mitigates existing long-tail problems. △ Less

Submitted 2 November, 2023; originally announced November 2023.

$p$-Poisson surface reconstruction in curl-free flow from point clouds

Authors: Yesom Park, Taekyung Lee, Jooyoung Hahn, Myungjoo Kang

Abstract: The aim of this paper is the reconstruction of a smooth surface from an unorganized point cloud sampled by a closed surface, with the preservation of geometric shapes, without any further information other than the point cloud. Implicit neural representations (INRs) have recently emerged as a promising approach to surface reconstruction. However, the reconstruction quality of existing methods reli… ▽ More The aim of this paper is the reconstruction of a smooth surface from an unorganized point cloud sampled by a closed surface, with the preservation of geometric shapes, without any further information other than the point cloud. Implicit neural representations (INRs) have recently emerged as a promising approach to surface reconstruction. However, the reconstruction quality of existing methods relies on ground truth implicit function values or surface normal vectors. In this paper, we show that proper supervision of partial differential equations and fundamental properties of differential vector fields are sufficient to robustly reconstruct high-quality surfaces. We cast the $p$-Poisson equation to learn a signed distance function (SDF) and the reconstructed surface is implicitly represented by the zero-level set of the SDF. For efficient training, we develop a variable splitting structure by introducing a gradient of the SDF as an auxiliary variable and impose the $p$-Poisson equation directly on the auxiliary variable as a hard constraint. Based on the curl-free property of the gradient field, we impose a curl-free constraint on the auxiliary variable, which leads to a more faithful reconstruction. Experiments on standard benchmark datasets show that the proposed INR provides a superior and robust reconstruction. The code is available at \url{https://github.com/Yebbi/PINC}. △ Less

Submitted 30 October, 2023; originally announced October 2023.

Comments: 21 pages, accepted for Advances in Neural Information Processing Systems, 2023

Dynamics of the Reynolds Shear Stress in Adverse Pressure-Gradient Flows, from the Lagrangian Transport Formalism

Authors: T. W. Lee, J. E. Park

Abstract: Using the Lagrangian transport analysis for the turbulence momentum, the Reynolds stress gradient can be expressed as a function of the local momentum flux and force terms. From this perspective of an observer moving at the local mean velocity, the Reynolds stress gradient represents the lateral transport of streamwise momentum, balanced by the u'2 transport, pressure and viscous force terms. Data… ▽ More Using the Lagrangian transport analysis for the turbulence momentum, the Reynolds stress gradient can be expressed as a function of the local momentum flux and force terms. From this perspective of an observer moving at the local mean velocity, the Reynolds stress gradient represents the lateral transport of streamwise momentum, balanced by the u'2 transport, pressure and viscous force terms. Data from direct numerical simulations (DNS) have been used to validate this approach for adverse pressure-gradient boundary layer flows at Clauser parameter of 1.4 and 39 (Kitsios et al., 2017), with a good degree of consistency and agreements. Minute fluctuations and attributes in the Reynolds shear stress profile are replicated through the Lagrangian momentum equation. Gradient analysis also leads to scaling at the first- and second-derivative levels, for u'2, v'2 and u'v'. △ Less

Submitted 7 July, 2024; v1 submitted 30 October, 2023; originally announced October 2023.

Self Attention with Temporal Prior: Can We Learn More from Arrow of Time?

Authors: Kyung Geun Kim, Byeong Tak Lee

Abstract: Many diverse phenomena in nature often inherently encode both short- and long-term temporal dependencies, which especially result from the direction of the flow of time. In this respect, we discovered experimental evidence suggesting that interrelations of these events are higher for closer time stamps. However, to be able for attention-based models to learn these regularities in short-term depend… ▽ More Many diverse phenomena in nature often inherently encode both short- and long-term temporal dependencies, which especially result from the direction of the flow of time. In this respect, we discovered experimental evidence suggesting that interrelations of these events are higher for closer time stamps. However, to be able for attention-based models to learn these regularities in short-term dependencies, it requires large amounts of data, which are often infeasible. This is because, while they are good at learning piece-wise temporal dependencies, attention-based models lack structures that encode biases in time series. As a resolution, we propose a simple and efficient method that enables attention layers to better encode the short-term temporal bias of these data sets by applying learnable, adaptive kernels directly to the attention matrices. We chose various prediction tasks for the experiments using Electronic Health Records (EHR) data sets since they are great examples with underlying long- and short-term temporal dependencies. Our experiments show exceptional classification results compared to best-performing models on most tasks and data sets. △ Less

Submitted 26 April, 2024; v1 submitted 29 October, 2023; originally announced October 2023.

Emergence of Shape Bias in Convolutional Neural Networks through Activation Sparsity

Authors: Tianqin Li, Ziqi Wen, Yangfan Li, Tai Sing Lee

Abstract: Current deep-learning models for object recognition are known to be heavily biased toward texture. In contrast, human visual systems are known to be biased toward shape and structure. What could be the design principles in human visual systems that led to this difference? How could we introduce more shape bias into the deep learning models? In this paper, we report that sparse coding, a ubiquitous… ▽ More Current deep-learning models for object recognition are known to be heavily biased toward texture. In contrast, human visual systems are known to be biased toward shape and structure. What could be the design principles in human visual systems that led to this difference? How could we introduce more shape bias into the deep learning models? In this paper, we report that sparse coding, a ubiquitous principle in the brain, can in itself introduce shape bias into the network. We found that enforcing the sparse coding constraint using a non-differential Top-K operation can lead to the emergence of structural encoding in neurons in convolutional neural networks, resulting in a smooth decomposition of objects into parts and subparts and endowing the networks with shape bias. We demonstrated this emergence of shape bias and its functional benefits for different network structures with various datasets. For object recognition convolutional neural networks, the shape bias leads to greater robustness against style and pattern change distraction. For the image synthesis generative adversary networks, the emerged shape bias leads to more coherent and decomposable structures in the synthesized images. Ablation studies suggest that sparse codes tend to encode structures, whereas the more distributed codes tend to favor texture. Our code is host at the github repository: \url{https://github.com/Crazy-Jack/nips2023_shape_vs_texture} △ Less

Submitted 29 October, 2023; originally announced October 2023.

Comments: Published as NeurIPS 2023 (Oral)

The European Low Frequency Survey

Authors: Aniello Mennella, Kam Arnold, Susanna Azzoni, Carlo Baccigalupi, Anthony Banday, R. Belen Barreiro, Darcy Barron, Marco Bersanelli, Sean Casey, Loris Colombo, Elena de la Hoz, Cristian Franceschet, Michael E. Jones, Ricardo T. Genova-Santos, Roger J. Hoyland, Adrian T. Lee, Enrique Martinez-Gonzalez, Filippo Montonati, Jose-Alberto Rubino-Martin, Angela Taylor, Patricio Vielva

Abstract: In this paper we present the European Low Frequency Survey (ELFS), a project that will enable foregrounds-free measurements of primordial $B$-mode polarization to a level 10$^{-3}$ by measuring the Galactic and extra-Galactic emissions in the 5--120\,GHz frequency window. Indeed, the main difficulty in measuring the B-mode polarization comes not just from its sheer faintness, but from the fact tha… ▽ More In this paper we present the European Low Frequency Survey (ELFS), a project that will enable foregrounds-free measurements of primordial $B$-mode polarization to a level 10$^{-3}$ by measuring the Galactic and extra-Galactic emissions in the 5--120\,GHz frequency window. Indeed, the main difficulty in measuring the B-mode polarization comes not just from its sheer faintness, but from the fact that many other objects in the Universe also emit polarized microwaves, which mask the faint CMB signal. The first stage of this project will be carried out in synergy with the Simons Array (SA) collaboration, installing a 5.5--11 GHz coherent receiver at the focus of one of the three 3.5\,m SA telescopes in Atacama, Chile ("ELFS on SA"). The receiver will be equipped with a fully digital back-end based on the latest Xilinx RF System-on-Chip devices that will provide frequency resolution of 1\,MHz across the whole observing band, allowing us to clean the scientific signal from unwanted radio frequency interference, particularly from low-Earth orbit satellite mega-constellations. This paper reviews the scientific motivation for ELFS and its instrumental characteristics, and provides an update on the development of ELFS on SA. △ Less

Submitted 22 November, 2023; v1 submitted 25 October, 2023; originally announced October 2023.

Comments: to appear in Proc. of the mm Universe 2023 conference, Grenoble (France), June 2023, published by F. Mayet et al. (Eds), EPJ Web of conferences, EDP Sciences

Neural Network with Local Converging Input (NNLCI) for Supersonic Flow Problems with Unstructured Grids

Authors: Weiming Ding, Haoxiang Huang, Tzu Jung Lee, Yingjie Liu, Vigor Yang

Abstract: In recent years, surrogate models based on deep neural networks (DNN) have been widely used to solve partial differential equations, which were traditionally handled by means of numerical simulations. This kind of surrogate models, however, focuses on global interpolation of the training dataset, and thus requires a large network structure. The process is both time consuming and computationally co… ▽ More In recent years, surrogate models based on deep neural networks (DNN) have been widely used to solve partial differential equations, which were traditionally handled by means of numerical simulations. This kind of surrogate models, however, focuses on global interpolation of the training dataset, and thus requires a large network structure. The process is both time consuming and computationally costly, thereby restricting their use for high-fidelity prediction of complex physical problems. In the present study, we develop a neural network with local converging input (NNLCI) for high-fidelity prediction using unstructured data. The framework utilizes the local domain of dependence with converging coarse solutions as input, which greatly reduces computational resource and training time. As a validation case, the NNLCI method is applied to study inviscid supersonic flows in channels with bumps. Different bump geometries and locations are considered to benchmark the effectiveness and versability of the proposed approach. Detailed flow structures, including shock-wave interactions, are examined systematically. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Comments: 23 pages, 21 figures

MSC Class: 35Q31

A Study on Prosodic Entrainment in Relation to Therapist Empathy in Counseling Conversation

Authors: Dehua Tao, Tan Lee, Harold Chui, Sarah Luk

Abstract: Counseling is carried out as spoken conversation between a therapist and a client. The empathy level expressed by the therapist is considered an important index of the quality of counseling and often assessed by an observer or the client. This research investigates the entrainment of speech prosody in relation to subjectively rated empathy. Experimental results show that the entrainment of intensi… ▽ More Counseling is carried out as spoken conversation between a therapist and a client. The empathy level expressed by the therapist is considered an important index of the quality of counseling and often assessed by an observer or the client. This research investigates the entrainment of speech prosody in relation to subjectively rated empathy. Experimental results show that the entrainment of intensity is more influential to empathy observation than that of pitch or speech rate in client-therapist interaction. The observer and the client have different perceptions of therapist empathy with the same entrained phenomena in pitch and intensity. The client's intention to make adjustment on pitch variation and intensity of speech is considered an indicator of the client's perception of counseling quality. △ Less

Submitted 22 October, 2023; originally announced October 2023.

Comments: Accepted by INTERSPEECH 2023

Modeling Intrapersonal and Interpersonal Influences for Automatic Estimation of Therapist Empathy in Counseling Conversation

Authors: Dehua Tao, Tan Lee, Harold Chui, Sarah Luk

Abstract: Counseling is usually conducted through spoken conversation between a therapist and a client. The empathy level of therapist is a key indicator of outcomes. Presuming that therapist's empathy expression is shaped by their past behavior and their perception of the client's behavior, we propose a model to estimate the therapist empathy by considering both intrapersonal and interpersonal influences.… ▽ More Counseling is usually conducted through spoken conversation between a therapist and a client. The empathy level of therapist is a key indicator of outcomes. Presuming that therapist's empathy expression is shaped by their past behavior and their perception of the client's behavior, we propose a model to estimate the therapist empathy by considering both intrapersonal and interpersonal influences. These dynamic influences are captured by applying an attention mechanism to the therapist turn and the historical turns of both therapist and client. Our findings suggest that the integration of dynamic influences enhances empathy level estimation. The influence-derived embedding should constitute a minor portion in the target turn representation for optimal empathy estimation. The client's turns (interpersonal influence) appear to slightly surpass the therapist's own turns (intrapersonal influence) in empathy estimation effectiveness. It is noted that concentrating exclusively on recent historical turns can significantly impact the estimation of therapist empathy. △ Less

Submitted 22 October, 2023; originally announced October 2023.

A model for time-evolution of coupling constants

Authors: Taekoon Lee

Abstract: A general model is proposed for time-varying coupling constants in field theory, assuming the ultraviolet cutoff is a varying entity in the expanding universe. It is assumed that the cutoff depends on the scale factor of the universe and all bare couplings remain constant. This leads to varying renormalized coupling constants that evolve in proportion to the Hubble parameter. The evolution of the… ▽ More A general model is proposed for time-varying coupling constants in field theory, assuming the ultraviolet cutoff is a varying entity in the expanding universe. It is assumed that the cutoff depends on the scale factor of the universe and all bare couplings remain constant. This leads to varying renormalized coupling constants that evolve in proportion to the Hubble parameter. The evolution of the standard model constants is discussed. △ Less

Submitted 23 December, 2023; v1 submitted 20 October, 2023; originally announced October 2023.

Comments: 7 pages, minor changes, version to appear in Phys Lett B

Journal ref: Phys.Lett.B 849 (2024) 138424

Low-energy electronic interactions in ferrimagnetic Sr2CrReO6 thin films

Authors: Guillaume Marcaud, Alex Taekyung Lee, Adam J. Hauser, F. Y. Yang, Sangjae Lee, Diego Casa, Mary Upton, Thomas Gog, Kayahan Saritas, Yilin Wang, Mark P. M. Dean, Hua Zhou, Zhan Zhang, F. J. Walker, Ignace Jarrige, Sohrab Ismail-Beigi, Charles Ahn

Abstract: We reveal in this study the fundamental low-energy landscape in the ferrimagnetic Sr2CrReO6 double perovskite and describe the underlying mechanisms responsible for the three low-energy excitations below 1.4 eV. Based on resonant inelastic x-ray scattering and magnetic dynamics calculations, and experiments collected from both Sr2CrReO6 powders and epitaxially strained thin films, we reveal a stro… ▽ More We reveal in this study the fundamental low-energy landscape in the ferrimagnetic Sr2CrReO6 double perovskite and describe the underlying mechanisms responsible for the three low-energy excitations below 1.4 eV. Based on resonant inelastic x-ray scattering and magnetic dynamics calculations, and experiments collected from both Sr2CrReO6 powders and epitaxially strained thin films, we reveal a strong competition between spin-orbit coupling, Hund's coupling, and the strain-induced tetragonal crystal field. We also demonstrate that a spin-flip process is at the origin of the lowest excitation at 200 meV, and we bring insights into the predicted presence of orbital ordering in this material. We study the nature of the magnons through a combination of ab initio and spin-wave theory calculations, and show that two nondegenerate magnon bands exist and are dominated either by rhenium or chromium spins. The rhenium band is found to be flat at about 200 meV ($\pm$25 meV) through X-L-W-U high-symmetry points and is dispersive toward $Γ$ △ Less

Submitted 17 October, 2023; originally announced October 2023.

Comments: 6 figures

Journal ref: Phys. Rev. B 108, 075132 (2023)

Hypernova signatures of the first stars in dwarf galaxies in the Local Group

Authors: Teayong Lee, Myoungwon Jeon, Volker Bromm

Abstract: Observing the first generation of stars, Population III (Pop III), is still a challenge even with the James Webb Space Telescope (JWST) due to their faintness. Instead, searching for fossil records of Pop III stars in nearby dwarf galaxies provides an alternative method for studying their physical properties. It is intriguing that a star recently discovered in the Sculptor dwarf galaxy, named AS00… ▽ More Observing the first generation of stars, Population III (Pop III), is still a challenge even with the James Webb Space Telescope (JWST) due to their faintness. Instead, searching for fossil records of Pop III stars in nearby dwarf galaxies provides an alternative method for studying their physical properties. It is intriguing that a star recently discovered in the Sculptor dwarf galaxy, named AS0039, is considered to show the unique signature of a Pop~III star. The detailed abundance patterns of AS0039 are well-matched with those predicted by nucleosynthesis models for Pop~III exploding as an energetic hypernova (HN), confirming its potential to provide insight into the properties of the first stars. This study aims to explore the environmental conditions required for the formation of such a unique star using cosmological hydrodynamic zoom-in simulations on dwarf galaxies with a mass of M_vir~10^8 solar mass at z=0 while varying the fraction of Pop~III stars that undergo HNe. Our simulations identify rapid gas inflow (~0.08 solar mass/yr) as a possible factor in facilitating the formation of stars similar to AS0039. Alternatively, the delayed formation of subsequent Pop~II stars in the gas-enriched environment may lead to low-metallicity stars like AS0039. Additionally, using the A-SLOTH code, we investigate the probability of finding remnants of Pop II stars with HN signatures in nearby dwarf satellite galaxies. We suggest that the most likely dwarf galaxies to contain HN signatures are massive satellites with a probability of 40% in the range of M_peak~10^{10}-10^{11} solar mass and M_star~10^7-10^8 solar mass, considering observational limitations. △ Less

Submitted 17 October, 2023; originally announced October 2023.

Comments: 18 pages, 17 figures, Accepted for publication in MNRAS

SoTTA: Robust Test-Time Adaptation on Noisy Data Streams

Authors: Taesik Gong, Yewon Kim, Taeckyung Lee, Sorn Chottananurak, Sung-Ju Lee

Abstract: Test-time adaptation (TTA) aims to address distributional shifts between training and testing data using only unlabeled test data streams for continual model adaptation. However, most TTA methods assume benign test streams, while test samples could be unexpectedly diverse in the wild. For instance, an unseen object or noise could appear in autonomous driving. This leads to a new threat to existing… ▽ More Test-time adaptation (TTA) aims to address distributional shifts between training and testing data using only unlabeled test data streams for continual model adaptation. However, most TTA methods assume benign test streams, while test samples could be unexpectedly diverse in the wild. For instance, an unseen object or noise could appear in autonomous driving. This leads to a new threat to existing TTA algorithms; we found that prior TTA algorithms suffer from those noisy test samples as they blindly adapt to incoming samples. To address this problem, we present Screening-out Test-Time Adaptation (SoTTA), a novel TTA algorithm that is robust to noisy samples. The key enabler of SoTTA is two-fold: (i) input-wise robustness via high-confidence uniform-class sampling that effectively filters out the impact of noisy samples and (ii) parameter-wise robustness via entropy-sharpness minimization that improves the robustness of model parameters against large gradients from noisy samples. Our evaluation with standard TTA benchmarks with various noisy scenarios shows that our method outperforms state-of-the-art TTA methods under the presence of noisy samples and achieves comparable accuracy to those methods without noisy samples. The source code is available at https://github.com/taeckyung/SoTTA . △ Less

Submitted 16 October, 2023; originally announced October 2023.

Comments: Accepted to NeurIPS 2023

PDRs4All III: JWST's NIR spectroscopic view of the Orion Bar

Authors: Els Peeters, Emilie Habart, Olivier Berne, Ameek Sidhu, Ryan Chown, Dries Van De Putte, Boris Trahin, Ilane Schroetter, Amelie Canin, Felipe Alarcon, Bethany Schefter, Baria Khan, Sofia Pasquini, Alexander G. G. M. Tielens, Mark G. Wolfire, Emmanuel Dartois, Javier R. Goicoechea, Alexandros Maragkoudakis, Takashi Onaka, Marc W. Pound, Silvia Vicente, Alain Abergel, Edwin A. Bergin, Jeronimo Bernard-Salas, Christiaan Boersma , et al. (113 additional authors not shown)

Abstract: (Abridged) We investigate the impact of radiative feedback from massive stars on their natal cloud and focus on the transition from the HII region to the atomic PDR (crossing the ionisation front (IF)), and the subsequent transition to the molecular PDR (crossing the dissociation front (DF)). We use high-resolution near-IR integral field spectroscopic data from NIRSpec on JWST to observe the Orion… ▽ More (Abridged) We investigate the impact of radiative feedback from massive stars on their natal cloud and focus on the transition from the HII region to the atomic PDR (crossing the ionisation front (IF)), and the subsequent transition to the molecular PDR (crossing the dissociation front (DF)). We use high-resolution near-IR integral field spectroscopic data from NIRSpec on JWST to observe the Orion Bar PDR as part of the PDRs4All JWST Early Release Science Program. The NIRSpec data reveal a forest of lines including, but not limited to, HeI, HI, and CI recombination lines, ionic lines, OI and NI fluorescence lines, Aromatic Infrared Bands (AIBs including aromatic CH, aliphatic CH, and their CD counterparts), CO2 ice, pure rotational and ro-vibrational lines from H2, and ro-vibrational lines HD, CO, and CH+, most of them detected for the first time towards a PDR. Their spatial distribution resolves the H and He ionisation structure in the Huygens region, gives insight into the geometry of the Bar, and confirms the large-scale stratification of PDRs. We observe numerous smaller scale structures whose typical size decreases with distance from Ori C and IR lines from CI, if solely arising from radiative recombination and cascade, reveal very high gas temperatures consistent with the hot irradiated surface of small-scale dense clumps deep inside the PDR. The H2 lines reveal multiple, prominent filaments which exhibit different characteristics. This leaves the impression of a "terraced" transition from the predominantly atomic surface region to the CO-rich molecular zone deeper in. This study showcases the discovery space created by JWST to further our understanding of the impact radiation from young stars has on their natal molecular cloud and proto-planetary disk, which touches on star- and planet formation as well as galaxy evolution. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Comments: 52 pages, 30 figures, submitted to A&A

Journal ref: A&A 685, A74 (2024)

Does resistance to style-transfer equal Global Shape Bias? Measuring network sensitivity to global shape configuration

Authors: Ziqi Wen, Tianqin Li, Zhi Jing, Tai Sing Lee

Abstract: Deep learning models are known to exhibit a strong texture bias, while human tends to rely heavily on global shape structure for object recognition. The current benchmark for evaluating a model's global shape bias is a set of style-transferred images with the assumption that resistance to the attack of style transfer is related to the development of global structure sensitivity in the model. In th… ▽ More Deep learning models are known to exhibit a strong texture bias, while human tends to rely heavily on global shape structure for object recognition. The current benchmark for evaluating a model's global shape bias is a set of style-transferred images with the assumption that resistance to the attack of style transfer is related to the development of global structure sensitivity in the model. In this work, we show that networks trained with style-transfer images indeed learn to ignore style, but its shape bias arises primarily from local detail. We provide a \textbf{Disrupted Structure Testbench (DiST)} as a direct measurement of global structure sensitivity. Our test includes 2400 original images from ImageNet-1K, each of which is accompanied by two images with the global shapes of the original image disrupted while preserving its texture via the texture synthesis program. We found that \textcolor{black}{(1) models that performed well on the previous cue-conflict dataset do not fare well in the proposed DiST; (2) the supervised trained Vision Transformer (ViT) lose its global spatial information from positional embedding, leading to no significant advantages over Convolutional Neural Networks (CNNs) on DiST. While self-supervised learning methods, especially mask autoencoder significantly improves the global structure sensitivity of ViT. (3) Improving the global structure sensitivity is orthogonal to resistance to style-transfer, indicating that the relationship between global shape structure and local texture detail is not an either/or relationship. Training with DiST images and style-transferred images are complementary, and can be combined to train network together to enhance the global shape sensitivity and robustness of local features.} Our code will be hosted in github: https://github.com/leelabcnbc/DiST △ Less

Submitted 29 February, 2024; v1 submitted 11 October, 2023; originally announced October 2023.

JWST: Deuterated PAHs, PAH-nitriles, and PAH Overtone and Combination Bands I: Program Description and First Look

Authors: C. Boersma, L. J. Allamandola, V. J. Esposito, A. Maragkoudakis, J. D. Bregman, P. Temi, T. J. Lee, R. C. Fortenberry, E. Peeters

Abstract: A first look is taken at the NIRSpec 1-5 $μ$m observations from JWST program 1591 that targets 7 objects along the low-mass stellar life cycle with PAH emission. Spectra extracted from a 1.5$^{\prime\prime}$ radius sized circular aperture are explored, showing a wealth of features, including the 3 $μ$m PAH complex, the PAH-continuum, and atomic and molecular emission lines from HI, He, H… ▽ More A first look is taken at the NIRSpec 1-5 $μ$m observations from JWST program 1591 that targets 7 objects along the low-mass stellar life cycle with PAH emission. Spectra extracted from a 1.5$^{\prime\prime}$ radius sized circular aperture are explored, showing a wealth of features, including the 3 $μ$m PAH complex, the PAH-continuum, and atomic and molecular emission lines from HI, He, H$_{\rm 2}$, and other species. CO$_{\rm 2}$- and H$_{\rm 2}$O-ice absorption and CO emission is also seen. Focusing on the bright-PDR position in M17, the PAH CH stretch falls at 3.29 $μ$m (FWHM=0.04 $μ$m). Signs of its 1.68 $μ$m overtone are confused by line emission in all targets. Multi-component decomposition reveals a possible aliphatic deuterated PAH feature centered at 4.65 $μ$m (FWHM=0.02 $μ$m), giving [D/H]$_{\rm alip.}$=31$\pm$12.7%. However, there is little sign of its aromatic counterpart between 4.36-4.43 $μ$m. There is also little sign of PAH-nitrile emission between 4.34-4.39 $μ$m. A PAH continuum rises from $\sim$1 to 3.2 $μ$m, after which it jumps by about a factor of 2.5 at 3.6 $μ$m, with bumps at 3.8, 4.04, and 4.34 $μ$m adding structure. The CO$_{\rm 2}$ absorption band in M17 is matched with 10:1 H$_{\rm 2}$O:CO$_{\rm 2}$ ice at 10 K. The $v$=0 pure rotational molecular hydrogen population diagram reveals $>$2200 K UV-pumped gas. The hydrogen Pfund series runs from levels 10 to $>$30. Considering Br$α$/Br$β$=0.381$\pm$0.01966 and Case B recombination results in A$_{\rm V}{\simeq}$8. CO emission in IRAS21282+5050 originates from 258 K gas. In-depth spectral-spatial analysis of all features and targets are planned for a series of forthcoming papers. △ Less

Submitted 9 October, 2023; originally announced October 2023.

Comments: 23 pages, 20 figures, to be published in ApJ

FABRIC: Automated Scoring and Feedback Generation for Essays

Authors: Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Hyunseung Lim, Yoonsu Kim, Tak Yeon Lee, Hwajung Hong, Juho Kim, So-Yeon Ahn, Alice Oh

Abstract: Automated essay scoring (AES) provides a useful tool for students and instructors in writing classes by generating essay scores in real-time. However, previous AES models do not provide more specific rubric-based scores nor feedback on how to improve the essays, which can be even more important than the overall scores for learning. We present FABRIC, a pipeline to help students and instructors in… ▽ More Automated essay scoring (AES) provides a useful tool for students and instructors in writing classes by generating essay scores in real-time. However, previous AES models do not provide more specific rubric-based scores nor feedback on how to improve the essays, which can be even more important than the overall scores for learning. We present FABRIC, a pipeline to help students and instructors in English writing classes by automatically generating 1) the overall scores, 2) specific rubric-based scores, and 3) detailed feedback on how to improve the essays. Under the guidance of English education experts, we chose the rubrics for the specific scores as content, organization, and language. The first component of the FABRIC pipeline is DREsS, a real-world Dataset for Rubric-based Essay Scoring (DREsS). The second component is CASE, a Corruption-based Augmentation Strategy for Essays, with which we can improve the accuracy of the baseline model by 45.44%. The third component is EssayCoT, the Essay Chain-of-Thought prompting strategy which uses scores predicted from the AES model to generate better feedback. We evaluate the effectiveness of the new dataset DREsS and the augmentation strategy CASE quantitatively and show significant improvements over the models trained with existing datasets. We evaluate the feedback generated by EssayCoT with English education experts to show significant improvements in the helpfulness of the feedback across all rubrics. Lastly, we evaluate the FABRIC pipeline with students in a college English writing class who rated the generated scores and feedback with an average of 6 on the Likert scale from 1 to 7. △ Less

Submitted 8 October, 2023; originally announced October 2023.

Gauss curvature flow with shrinking obstacle

Authors: Ki-Ahm Lee, Taehun Lee

Abstract: We consider a flow by powers of Gauss curvature under the obstruction that the flow cannot penetrate a prescribed region, so called an obstacle. For all dimensions and positive powers, we prove the optimal curvature bounds of solutions and all time existence with its long time behavior. We also prove the $C^1$ regularity of free boundaries under a uniform thickness assumption. We consider a flow by powers of Gauss curvature under the obstruction that the flow cannot penetrate a prescribed region, so called an obstacle. For all dimensions and positive powers, we prove the optimal curvature bounds of solutions and all time existence with its long time behavior. We also prove the $C^1$ regularity of free boundaries under a uniform thickness assumption. △ Less

Submitted 4 October, 2023; originally announced October 2023.

Comments: To appear in Math. Ann

MSC Class: 53E10 (Primary) 35R35; 35K96 (Secondary)

An Integer Clustering Approach for Modeling Large-Scale EV Fleets with Guaranteed Performance

Authors: Sijia Geng, Thomas Lee, Dharik Mallapragada, Audun Botterud

Abstract: Large-scale integration of electric vehicles (EVs) leads to a tighter integration between transportation and electric energy systems. In this paper, we develop a novel integer-clustering approach to model a large number of EVs that manages vehicle charging and energy at the fleet level yet maintain individual trip dispatch. The model is then used to develop a spatially and temporally-resolved deci… ▽ More Large-scale integration of electric vehicles (EVs) leads to a tighter integration between transportation and electric energy systems. In this paper, we develop a novel integer-clustering approach to model a large number of EVs that manages vehicle charging and energy at the fleet level yet maintain individual trip dispatch. The model is then used to develop a spatially and temporally-resolved decision-making tool for optimally planning and/or operating EV fleets and charging infrastructure. The tool comprises a two-stage framework where a tractable disaggregation step follows the integer-clustering problem to recover an individually feasible solution. Mathematical relationships between the integer clustering, disaggregation, and individual formulations are analyzed. We establish theoretical lower and upper bounds on the true individual formulation which underpins a guaranteed performance of the proposed method. The optimality accuracy and computational efficiency of the integer-clustering formulation are also numerically validated on a real-world case study of Boston's public transit network under extensive test instances. Substantial speedups with minimal loss in solution quality are demonstrated. △ Less

Submitted 3 October, 2023; originally announced October 2023.

Comments: 8 pages, 4 figures

Chunking: Continual Learning is not just about Distribution Shift

Authors: Thomas L. Lee, Amos Storkey

Abstract: Work on continual learning (CL) has thus far largely focused on the problems arising from shifts in the data distribution. However, CL can be decomposed into two sub-problems: (a) shifts in the data distribution, and (b) dealing with the fact that the data is split into chunks and so only a part of the data is available to be trained on at any point in time. In this work, we look at the latter sub… ▽ More Work on continual learning (CL) has thus far largely focused on the problems arising from shifts in the data distribution. However, CL can be decomposed into two sub-problems: (a) shifts in the data distribution, and (b) dealing with the fact that the data is split into chunks and so only a part of the data is available to be trained on at any point in time. In this work, we look at the latter sub-problem, the chunking of data. We show that chunking is an important part of CL, accounting for around half of the performance drop from offline learning in our experiments. Furthermore, our results reveal that current CL algorithms do not address the chunking sub-problem, only performing as well as plain SGD training when there is no shift in the data distribution. Therefore, we show that chunking is both an important and currently unaddressed sub-problem and until it is addressed CL methods will be capped in performance. Additionally, we analyse why performance drops when learning occurs on identically distributed chunks of data, and find that forgetting, which is often seen to be a problem due to distribution shift, still arises and is a significant problem. We also show that performance on the chunking sub-problem can be increased and that this performance transfers to the full CL setting, where there is distribution shift. Hence, we argue that work on chunking can help advance CL in general. △ Less

Submitted 11 July, 2024; v1 submitted 3 October, 2023; originally announced October 2023.

Comments: Published at the 3rd Conference on Lifelong Learning Agents (CoLLAs), 2024

A sublinear time quantum algorithm for longest common substring problem between run-length encoded strings

Authors: Tzu-Ching Lee, Han-Hsuan Lin

Abstract: We give a sublinear quantum algorithm for the longest common substring (LCS) problem on the run-length encoded (RLE) inputs, under the assumption that the prefix-sums of the runs are given. Our algorithm costs $\tilde{O}(n^{5/6})\cdot O(\mathrm{polylog}(\tilde{n}))$ time, where $n$ and $\tilde{n}$ are the encoded and decoded length of the inputs, respectively. We justify the use of the prefix-sum… ▽ More We give a sublinear quantum algorithm for the longest common substring (LCS) problem on the run-length encoded (RLE) inputs, under the assumption that the prefix-sums of the runs are given. Our algorithm costs $\tilde{O}(n^{5/6})\cdot O(\mathrm{polylog}(\tilde{n}))$ time, where $n$ and $\tilde{n}$ are the encoded and decoded length of the inputs, respectively. We justify the use of the prefix-sum oracles by showing that, without the oracles, there is a $Ω(n/\log^2n)$ lower-bound on the quantum query complexity of finding LCS given two RLE strings due to a reduction of $\mathsf{PARITY}$ to the problem. △ Less

Submitted 2 October, 2023; originally announced October 2023.

How Helpful do Novice Programmers Find the Feedback of an Automated Repair Tool?

Authors: Oka Kurniawan, Christopher M. Poskitt, Ismam Al Hoque, Norman Tiong Seng Lee, Cyrille Jégourel, Nachamma Sockalingam

Abstract: Immediate feedback has been shown to improve student learning. In programming courses, immediate, automated feedback is typically provided in the form of pre-defined test cases run by a submission platform. While these are excellent for highlighting the presence of logical errors, they do not provide novice programmers enough scaffolding to help them identify where an error is or how to fix it. To… ▽ More Immediate feedback has been shown to improve student learning. In programming courses, immediate, automated feedback is typically provided in the form of pre-defined test cases run by a submission platform. While these are excellent for highlighting the presence of logical errors, they do not provide novice programmers enough scaffolding to help them identify where an error is or how to fix it. To address this, several tools have been developed that provide richer feedback in the form of program repairs. Studies of such tools, however, tend to focus more on whether correct repairs can be generated, rather than how novices are using them. In this paper, we describe our experience of using CLARA, an automated repair tool, to provide feedback to novices. First, we extended CLARA to support a larger subset of the Python language, before integrating it with the Jupyter Notebooks used for our programming exercises. Second, we devised a preliminary study in which students tackled programming problems with and without support of the tool using the 'think aloud' protocol. We found that novices often struggled to understand the proposed repairs, echoing the well-known challenge to understand compiler/interpreter messages. Furthermore, we found that students valued being told where a fix was needed - without necessarily the fix itself - suggesting that 'less may be more' from a pedagogical perspective. △ Less

Submitted 7 October, 2023; v1 submitted 2 October, 2023; originally announced October 2023.

Comments: Experience report accepted by the International Conference on Teaching, Assessment, and Learning for Engineering (TALE'23)

Journal ref: Proc. TALE'23. IEEE, 2023

{SSH coupled-spring systems

Authors: Jie-Ying Kuo, Tsung-Yen Lee, Yi-Chia Chiu, Sheng-Rong Liao, Hsien-chung Kao

Abstract: It is known that there is also a topological phase in the SSH coupled-spring system with the fixed-end boundary conditions. When this is the case, there would exist edge modes on its boundaries. In contrast, if the system satisfies the free-end boundary conditions, there is no edge mode, even if it is the topological phase. We show that by varying the force constant of the spring by the boundary i… ▽ More It is known that there is also a topological phase in the SSH coupled-spring system with the fixed-end boundary conditions. When this is the case, there would exist edge modes on its boundaries. In contrast, if the system satisfies the free-end boundary conditions, there is no edge mode, even if it is the topological phase. We show that by varying the force constant of the spring by the boundary in such a system, edge modes would generally appear independent of whether the bulk of the system is in the topological or trivial phases. Moreover, edge modes could exist even if the system satisfies the free-end boundary conditions. △ Less

Submitted 30 September, 2023; originally announced October 2023.

Comments: 19 pages, 9 figures

An Investigation Into Race Bias in Random Forest Models Based on Breast DCE-MRI Derived Radiomics Features

Authors: Mohamed Huti, Tiarna Lee, Elinor Sawyer, Andrew P. King

Abstract: Recent research has shown that artificial intelligence (AI) models can exhibit bias in performance when trained using data that are imbalanced by protected attribute(s). Most work to date has focused on deep learning models, but classical AI techniques that make use of hand-crafted features may also be susceptible to such bias. In this paper we investigate the potential for race bias in random for… ▽ More Recent research has shown that artificial intelligence (AI) models can exhibit bias in performance when trained using data that are imbalanced by protected attribute(s). Most work to date has focused on deep learning models, but classical AI techniques that make use of hand-crafted features may also be susceptible to such bias. In this paper we investigate the potential for race bias in random forest (RF) models trained using radiomics features. Our application is prediction of tumour molecular subtype from dynamic contrast enhanced magnetic resonance imaging (DCE-MRI) of breast cancer patients. Our results show that radiomics features derived from DCE-MRI data do contain race-identifiable information, and that RF models can be trained to predict White and Black race from these data with 60-70% accuracy, depending on the subset of features used. Furthermore, RF models trained to predict tumour molecular subtype using race-imbalanced data seem to produce biased behaviour, exhibiting better performance on test data from the race on which they were trained. △ Less

Submitted 29 September, 2023; originally announced September 2023.

Comments: Accepted for publication at the MICCAI Workshop on Fairness of AI in Medical Imaging (FAIMI) 2023

The Simons Observatory: Cryogenic Half Wave Plate Rotation Mechanism for the Small Aperture Telescopes

Authors: K. Yamada, B. Bixler, Y. Sakurai, P. C. Ashton, J. Sugiyama, K. Arnold, J. Begin, L. Corbett, S. Day-Weiss, N. Galitzki, C. A. Hill, B. R. Johnson, B. Jost, A. Kusaka, B. J. Koopman, J. Lashner, A. T. Lee, A. Mangu, H. Nishino, L. A. Page, M. J. Randall, D. Sasaki, X. Song, J. Spisak, T. Tsan , et al. (2 additional authors not shown)

Abstract: We present the requirements, design and evaluation of the cryogenic continuously rotating half-wave plate (CHWP) for the Simons Observatory (SO). SO is a cosmic microwave background (CMB) polarization experiment at Parque Astronómico Atacama in northern Chile that covers a wide range of angular scales using both small (0.42 m) and large (6 m) aperture telescopes. In particular, the small aperture… ▽ More We present the requirements, design and evaluation of the cryogenic continuously rotating half-wave plate (CHWP) for the Simons Observatory (SO). SO is a cosmic microwave background (CMB) polarization experiment at Parque Astronómico Atacama in northern Chile that covers a wide range of angular scales using both small (0.42 m) and large (6 m) aperture telescopes. In particular, the small aperture telescopes (SATs) focus on large angular scales for primordial B-mode polarization. To this end, the SATs employ a CHWP to modulate the polarization of the incident light at 8~Hz, suppressing atmospheric $1/f$ noise and mitigating systematic uncertainties that would otherwise arise due to the differential response of detectors sensitive to orthogonal polarizations. The CHWP consists of a 505 mm diameter achromatic sapphire HWP and a cryogenic rotation mechanism, both of which are cooled down to $\sim$50 K to reduce detector thermal loading. Under normal operation the HWP is suspended by a superconducting magnetic bearing and rotates with a constant 2 Hz frequency, controlled by an electromagnetic synchronous motor. The rotation angle is detected through an angular encoder with a noise level of 0.07$μ\mathrm{rad}\sqrt{\mathrm{s}}$. During a cooldown, the rotor is held in place by a grip-and-release mechanism that serves as both an alignment device and a thermal path. In this paper we provide an overview of the SO SAT CHWP: its requirements, hardware design, and laboratory performance. △ Less

Submitted 26 September, 2023; originally announced September 2023.

Comments: 19 pages, 21 figures, submitted to RSI

Journal ref: Rev. Sci. Instrum. 95, 024504 (2024)

Efficient Black-Box Speaker Verification Model Adaptation with Reprogramming and Backend Learning

Authors: Jingyu Li, Tan Lee

Abstract: The development of deep neural networks (DNN) has significantly enhanced the performance of speaker verification (SV) systems in recent years. However, a critical issue that persists when applying DNN-based SV systems in practical applications is domain mismatch. To mitigate the performance degradation caused by the mismatch, domain adaptation becomes necessary. This paper introduces an approach t… ▽ More The development of deep neural networks (DNN) has significantly enhanced the performance of speaker verification (SV) systems in recent years. However, a critical issue that persists when applying DNN-based SV systems in practical applications is domain mismatch. To mitigate the performance degradation caused by the mismatch, domain adaptation becomes necessary. This paper introduces an approach to adapt DNN-based SV models by manipulating the learnable model inputs, inspired by the concept of adversarial reprogramming. The pre-trained SV model remains fixed and functions solely in the forward process, resembling a black-box model. A lightweight network is utilized to estimate the gradients for the learnable parameters at the input, which bypasses the gradient backpropagation through the black-box model. The reprogrammed output is processed by a two-layer backend learning module as the final adapted speaker embedding. The number of parameters involved in the gradient calculation is small in our design. With few additional parameters, the proposed method achieves both memory and parameter efficiency. The experiments are conducted in language mismatch scenarios. Using much less computation cost, the proposed method obtains close or superior performance to the fully finetuned models in our experiments, which demonstrates its effectiveness. △ Less

Submitted 24 September, 2023; originally announced September 2023.

ChEDDAR: Student-ChatGPT Dialogue in EFL Writing Education

Authors: Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Tak Yeon Lee, So-Yeon Ahn, Alice Oh

Abstract: The integration of generative AI in education is expanding, yet empirical analyses of large-scale, real-world interactions between students and AI systems still remain limited. In this study, we present ChEDDAR, ChatGPT & EFL Learner's Dialogue Dataset As Revising an essay, which is collected from a semester-long longitudinal experiment involving 212 college students enrolled in English as Foreign… ▽ More The integration of generative AI in education is expanding, yet empirical analyses of large-scale, real-world interactions between students and AI systems still remain limited. In this study, we present ChEDDAR, ChatGPT & EFL Learner's Dialogue Dataset As Revising an essay, which is collected from a semester-long longitudinal experiment involving 212 college students enrolled in English as Foreign Langauge (EFL) writing courses. The students were asked to revise their essays through dialogues with ChatGPT. ChEDDAR includes a conversation log, utterance-level essay edit history, self-rated satisfaction, and students' intent, in addition to session-level pre-and-post surveys documenting their objectives and overall experiences. We analyze students' usage patterns and perceptions regarding generative AI with respect to their intent and satisfaction. As a foundational step, we establish baseline results for two pivotal tasks in task-oriented dialogue systems within educational contexts: intent detection and satisfaction estimation. We finally suggest further research to refine the integration of generative AI into education settings, outlining potential scenarios utilizing ChEDDAR. ChEDDAR is publicly available at https://github.com/zeunie/ChEDDAR. △ Less

Submitted 20 March, 2024; v1 submitted 22 September, 2023; originally announced September 2023.

Comments: The new version of this paper is on arXiv as arXiv:2403.08272

CoMFLP: Correlation Measure based Fast Search on ASR Layer Pruning

Authors: Wei Liu, Zhiyuan Peng, Tan Lee

Abstract: Transformer-based speech recognition (ASR) model with deep layers exhibited significant performance improvement. However, the model is inefficient for deployment on resource-constrained devices. Layer pruning (LP) is a commonly used compression method to remove redundant layers. Previous studies on LP usually identify the redundant layers according to a task-specific evaluation metric. They are ti… ▽ More Transformer-based speech recognition (ASR) model with deep layers exhibited significant performance improvement. However, the model is inefficient for deployment on resource-constrained devices. Layer pruning (LP) is a commonly used compression method to remove redundant layers. Previous studies on LP usually identify the redundant layers according to a task-specific evaluation metric. They are time-consuming for models with a large number of layers, even in a greedy search manner. To address this problem, we propose CoMFLP, a fast search LP algorithm based on correlation measure. The correlation between layers is computed to generate a correlation matrix, which identifies the redundancy among layers. The search process is carried out in two steps: (1) coarse search: to determine top $K$ candidates by pruning the most redundant layers based on the correlation matrix; (2) fine search: to select the best pruning proposal among $K$ candidates using a task-specific evaluation metric. Experiments on an ASR task show that the pruning proposal determined by CoMFLP outperforms existing LP methods while only requiring constant time complexity. The code is publicly available at https://github.com/louislau1129/CoMFLP. △ Less

Submitted 21 September, 2023; originally announced September 2023.

Comments: Accepted by Interspeech 2023

Sparsely Shared LoRA on Whisper for Child Speech Recognition

Authors: Wei Liu, Ying Qin, Zhiyuan Peng, Tan Lee

Abstract: Whisper is a powerful automatic speech recognition (ASR) model. Nevertheless, its zero-shot performance on low-resource speech requires further improvement. Child speech, as a representative type of low-resource speech, is leveraged for adaptation. Recently, parameter-efficient fine-tuning (PEFT) in NLP was shown to be comparable and even better than full fine-tuning, while only needing to tune a… ▽ More Whisper is a powerful automatic speech recognition (ASR) model. Nevertheless, its zero-shot performance on low-resource speech requires further improvement. Child speech, as a representative type of low-resource speech, is leveraged for adaptation. Recently, parameter-efficient fine-tuning (PEFT) in NLP was shown to be comparable and even better than full fine-tuning, while only needing to tune a small set of trainable parameters. However, current PEFT methods have not been well examined for their effectiveness on Whisper. In this paper, only parameter composition types of PEFT approaches such as LoRA and Bitfit are investigated as they do not bring extra inference costs. Different popular PEFT methods are examined. Particularly, we compare LoRA and AdaLoRA and figure out the learnable rank coefficient is a good design. Inspired by the sparse rank distribution allocated by AdaLoRA, a novel PEFT approach Sparsely Shared LoRA (S2-LoRA) is proposed. The two low-rank decomposed matrices are globally shared. Each weight matrix only has to maintain its specific rank coefficients that are constrained to be sparse. Experiments on low-resource Chinese child speech show that with much fewer trainable parameters, S2-LoRA can achieve comparable in-domain adaptation performance to AdaLoRA and exhibit better generalization ability on out-of-domain data. In addition, the rank distribution automatically learned by S2-LoRA is found to have similar patterns to AdaLoRA's allocation. △ Less

Submitted 7 January, 2024; v1 submitted 20 September, 2023; originally announced September 2023.

Comments: Accepted by ICASSP 2024

Brief Architectural Survey of Biopotential Recording Front-Ends since the 1970s

Authors: Taeju Lee, Minkyu Je

Abstract: Measuring the bioelectric signals is one of the key functions in wearable healthcare devices and implantable medical devices. The use of wearable healthcare devices has made continuous and immediate monitoring of personal health status possible. Implantable medical devices have played an important role throughout the fields of neuroscience, brain-machine (or brain-computer) interface, and rehabili… ▽ More Measuring the bioelectric signals is one of the key functions in wearable healthcare devices and implantable medical devices. The use of wearable healthcare devices has made continuous and immediate monitoring of personal health status possible. Implantable medical devices have played an important role throughout the fields of neuroscience, brain-machine (or brain-computer) interface, and rehabilitation technology. Over the last five decades, the bioelectric signals have been observed through a variety of biopotential recording front-ends, along with advances in semiconductor technology scaling and circuit techniques. Also, for reliable and continuous signal acquisition, the front-end architectures have evolved while maintaining low power and low noise performance. In this article, the architecture history of the biopotential recording front-ends developed since the 1970s is surveyed, and overall key circuit techniques are discussed. Depending on the bioelectric signals being measured, appropriate front-end architecture needs to be chosen, and the characteristics and challenges of each architecture are also covered in this article. △ Less

Submitted 20 September, 2023; originally announced September 2023.

Comments: 5 pages, 3 figures

KoBigBird-large: Transformation of Transformer for Korean Language Understanding

Authors: Kisu Yang, Yoonna Jang, Taewoo Lee, Jinwoo Seong, Hyungjin Lee, Hwanseok Jang, Heuiseok Lim

Abstract: This work presents KoBigBird-large, a large size of Korean BigBird that achieves state-of-the-art performance and allows long sequence processing for Korean language understanding. Without further pretraining, we only transform the architecture and extend the positional encoding with our proposed Tapered Absolute Positional Encoding Representations (TAPER). In experiments, KoBigBird-large shows st… ▽ More This work presents KoBigBird-large, a large size of Korean BigBird that achieves state-of-the-art performance and allows long sequence processing for Korean language understanding. Without further pretraining, we only transform the architecture and extend the positional encoding with our proposed Tapered Absolute Positional Encoding Representations (TAPER). In experiments, KoBigBird-large shows state-of-the-art overall performance on Korean language understanding benchmarks and the best performance on document classification and question answering tasks for longer sequences against the competitive baseline models. We publicly release our model here. △ Less

Submitted 19 September, 2023; originally announced September 2023.

Comments: Accepted at IJCNLP-AACL 2023

SPT-SZ MCMF: An extension of the SPT-SZ catalog over the DES region

Authors: M. Klein, J. J. Mohr, S. Bocquet, M. Aguena, S. W. Allen, O. Alves, B. Ansarinejad, M. L. N. Ashby, D. Bacon, M. Bayliss, B. A. Benson, L. E. Bleem, M. Brodwin, D. Brooks, E. Bulbul, D. L. Burke, R. E. A. Canning, J. E. Carlstrom, A. Carnero Rosell, J. Carretero, C. L. Chang, C. Conselice, M. Costanzi, A. T. Crites, L. N. da Costa , et al. (82 additional authors not shown)

Abstract: We present an extension to a Sunyaev-Zel'dovich Effect (SZE) selected cluster catalog based on observations from the South Pole Telescope (SPT); this catalog extends to lower signal-to-noise than the previous SPT-SZ catalog and therefore includes lower mass clusters. Optically derived redshifts, centers, richnesses and morphological parameters together with catalog contamination and completeness s… ▽ More We present an extension to a Sunyaev-Zel'dovich Effect (SZE) selected cluster catalog based on observations from the South Pole Telescope (SPT); this catalog extends to lower signal-to-noise than the previous SPT-SZ catalog and therefore includes lower mass clusters. Optically derived redshifts, centers, richnesses and morphological parameters together with catalog contamination and completeness statistics are extracted using the multi-component matched filter algorithm (MCMF) applied to the S/N>4 SPT-SZ candidate list and the Dark Energy Survey (DES) photometric galaxy catalog. The main catalog contains 811 sources above S/N=4, has 91% purity and is 95% complete with respect to the original SZE selection. It contains 50% more total clusters and twice as many clusters above z=0.8 in comparison to the original SPT-SZ sample. The MCMF algorithm allows us to define subsamples of the desired purity with traceable impact on catalog completeness. As an example, we provide two subsamples with S/N>4.25 and S/N>4.5 for which the sample contamination and cleaning-induced incompleteness are both as low as the expected Poisson noise for samples of their size. The subsample with S/N>4.5 has 98% purity and 96% completeness, and will be included in a combined SPT cluster and DES weak-lensing cosmological analysis. We measure the number of false detections in the SPT-SZ candidate list as function of S/N, finding that it follows that expected from assuming Gaussian noise, but with a lower amplitude compared to previous estimates from simulations. △ Less

Submitted 4 October, 2023; v1 submitted 18 September, 2023; originally announced September 2023.

Comments: 16 pages, 17 figures, submitted to MNRAS

Estimation and Testing of Forecast Rationality with Many Moments

Authors: Tae-Hwy Lee, Tao Wang

Abstract: We in this paper utilize P-GMM (Cheng and Liao, 2015) moment selection procedure to select valid and relevant moments for estimating and testing forecast rationality under the flexible loss proposed by Elliott et al. (2005). We motivate the moment selection in a large dimensional setting, explain the fundamental mechanism of P-GMM moment selection procedure, and elucidate how to implement it in th… ▽ More We in this paper utilize P-GMM (Cheng and Liao, 2015) moment selection procedure to select valid and relevant moments for estimating and testing forecast rationality under the flexible loss proposed by Elliott et al. (2005). We motivate the moment selection in a large dimensional setting, explain the fundamental mechanism of P-GMM moment selection procedure, and elucidate how to implement it in the context of forecast rationality by allowing the existence of potentially invalid moment conditions. A set of Monte Carlo simulations is conducted to examine the finite sample performance of P-GMM estimation in integrating the information available in instruments into both the estimation and testing, and a real data analysis using data from the Survey of Professional Forecasters issued by the Federal Reserve Bank of Philadelphia is presented to further illustrate the practical value of the suggested methodology. The results indicate that the P-GMM post-selection estimator of forecaster's attitude is comparable to the oracle estimator by using the available information efficiently. The accompanying power of rationality and symmetry tests utilizing P-GMM estimation would be substantially increased through reducing the influence of uninformative instruments. When a forecast user estimates and tests for rationality of forecasts that have been produced by others such as Greenbook, P-GMM moment selection procedure can assist in achieving consistent and more efficient outcomes. △ Less

Submitted 18 September, 2023; originally announced September 2023.

FDCNet: Feature Drift Compensation Network for Class-Incremental Weakly Supervised Object Localization

Authors: Sejin Park, Taehyung Lee, Yeejin Lee, Byeongkeun Kang

Abstract: This work addresses the task of class-incremental weakly supervised object localization (CI-WSOL). The goal is to incrementally learn object localization for novel classes using only image-level annotations while retaining the ability to localize previously learned classes. This task is important because annotating bounding boxes for every new incoming data is expensive, although object localizati… ▽ More This work addresses the task of class-incremental weakly supervised object localization (CI-WSOL). The goal is to incrementally learn object localization for novel classes using only image-level annotations while retaining the ability to localize previously learned classes. This task is important because annotating bounding boxes for every new incoming data is expensive, although object localization is crucial in various applications. To the best of our knowledge, we are the first to address this task. Thus, we first present a strong baseline method for CI-WSOL by adapting the strategies of class-incremental classifiers to mitigate catastrophic forgetting. These strategies include applying knowledge distillation, maintaining a small data set from previous tasks, and using cosine normalization. We then propose the feature drift compensation network to compensate for the effects of feature drifts on class scores and localization maps. Since updating network parameters to learn new tasks causes feature drifts, compensating for the final outputs is necessary. Finally, we evaluate our proposed method by conducting experiments on two publicly available datasets (ImageNet-100 and CUB-200). The experimental results demonstrate that the proposed method outperforms other baseline methods. △ Less

Submitted 16 September, 2023; originally announced September 2023.

Comments: ACM Multimedia 2023

A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale

Authors: Hao-Jun Michael Shi, Tsung-Hsien Lee, Shintaro Iwasaki, Jose Gallego-Posada, Zhijing Li, Kaushik Rangadurai, Dheevatsa Mudigere, Michael Rabbat

Abstract: Shampoo is an online and stochastic optimization algorithm belonging to the AdaGrad family of methods for training neural networks. It constructs a block-diagonal preconditioner where each block consists of a coarse Kronecker product approximation to full-matrix AdaGrad for each parameter of the neural network. In this work, we provide a complete description of the algorithm as well as the perform… ▽ More Shampoo is an online and stochastic optimization algorithm belonging to the AdaGrad family of methods for training neural networks. It constructs a block-diagonal preconditioner where each block consists of a coarse Kronecker product approximation to full-matrix AdaGrad for each parameter of the neural network. In this work, we provide a complete description of the algorithm as well as the performance optimizations that our implementation leverages to train deep networks at-scale in PyTorch. Our implementation enables fast multi-GPU distributed data-parallel training by distributing the memory and computation associated with blocks of each parameter via PyTorch's DTensor data structure and performing an AllGather primitive on the computed search directions at each iteration. This major performance enhancement enables us to achieve at most a 10% performance reduction in per-step wall-clock time compared against standard diagonal-scaling-based adaptive gradient methods. We validate our implementation by performing an ablation study on training ImageNet ResNet50, demonstrating Shampoo's superiority over standard training recipes with minimal hyperparameter tuning. △ Less

Submitted 12 September, 2023; originally announced September 2023.

Comments: 38 pages, 8 figures, 5 tables

Textbooks Are All You Need II: phi-1.5 technical report

Authors: Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, Yin Tat Lee

Abstract: We continue the investigation into the power of smaller Transformer-based language models as initiated by \textbf{TinyStories} -- a 10 million parameter model that can produce coherent English -- and the follow-up work on \textbf{phi-1}, a 1.3 billion parameter model with Python coding performance close to the state-of-the-art. The latter work proposed to use existing Large Language Models (LLMs)… ▽ More We continue the investigation into the power of smaller Transformer-based language models as initiated by \textbf{TinyStories} -- a 10 million parameter model that can produce coherent English -- and the follow-up work on \textbf{phi-1}, a 1.3 billion parameter model with Python coding performance close to the state-of-the-art. The latter work proposed to use existing Large Language Models (LLMs) to generate ``textbook quality" data as a way to enhance the learning process compared to traditional web data. We follow the ``Textbooks Are All You Need" approach, focusing this time on common sense reasoning in natural language, and create a new 1.3 billion parameter model named \textbf{phi-1.5}, with performance on natural language tasks comparable to models 5x larger, and surpassing most non-frontier LLMs on more complex reasoning tasks such as grade-school mathematics and basic coding. More generally, \textbf{phi-1.5} exhibits many of the traits of much larger LLMs, both good -- such as the ability to ``think step by step" or perform some rudimentary in-context learning -- and bad, including hallucinations and the potential for toxic and biased generations -- encouragingly though, we are seeing improvement on that front thanks to the absence of web data. We open-source \textbf{phi-1.5} to promote further research on these urgent topics. △ Less

Submitted 11 September, 2023; originally announced September 2023.

First-principle Study of Multiple Metastable Charge Ordering States in La$_{1/3}$Sr$_{2/3}$FeO$_{3}$

Authors: Nam Nguyen, Alex Taekyung Lee, Vijay Singh, Anh T. Ngo, Hyowon Park

Abstract: La doped SrFeO$_{3}$, La$_{1/3}$Sr$_{2/3}$FeO$_{3}$, exhibits a metal-to-insulator transition accompanied by both antiferromagnetic and charge ordering states along with the Fe-O bond disproportionation below a critical temperature near 200K. Unconventionally slow charge dynamics measured in this material near the critical temperature shows that its excited charge ordering states can exhibit novel… ▽ More La doped SrFeO$_{3}$, La$_{1/3}$Sr$_{2/3}$FeO$_{3}$, exhibits a metal-to-insulator transition accompanied by both antiferromagnetic and charge ordering states along with the Fe-O bond disproportionation below a critical temperature near 200K. Unconventionally slow charge dynamics measured in this material near the critical temperature shows that its excited charge ordering states can exhibit novel electronic structures with nontrivial energy profiles. Here, we reveal possible metastable states of charge ordering structures in La$_{1/3}$Sr$_{2/3}$FeO$_{3}$ using the first-principle and climbing image nudged elastic band methods. In the strong correlation regime, La$_{1/3}$Sr$_{2/3}$FeO$_{3}$ is an antiferromagnetic insulator with a charge ordering state of the big-small-big pattern, consistent with the experimental measurement of this material at the low temperature. As the correlation effect becomes weak, we find at least two possible metastable charge ordering states with the distinct Fe-O bond disproportionation. Remarkably, a ferroelectric metallic state emerges with the small energy barrier of $\sim$7 meV, driven by a metastable CO state of the small-medium-big pattern. The electronic structures of these metastable charge ordering states are noticeably different from those of the ground-state. Our results can provide an insightful explanation to multiple metastable charge ordering states and the slow charge dynamics of this and related oxide materials. △ Less

Submitted 7 September, 2023; originally announced September 2023.

Comments: The paper has 8 pages and 6 figures