Showing 1–50 of 1,064 results for author: Tang, S

Rich and diverse molecular gas environments of closely-separated dual quasars viewed by ALMA

Authors: Shenli Tang, John D. Silverman, Zhaoxuan Liu, Manda Banerji, Tomoko Suzuki, Seiji Fujimoto, Andy Goulding, Masatoshi Imanishi, Toshihiro Kawaguchi, Connor Bottrell, Tilman Hartwig, Knud Jahnke, Masafusa Onoue, Malte Schramm, Yoshihiro Ueda

Abstract: We present a study of the molecular gas in five closely-spaced ($R_{\perp}<20$ kpc) dual quasars ($L_{\rm bol}\gtrsim10^{44}~\mathrm{erg~s}^{-1}$) at redshifts $0.4<z<0.8$ with the Atacama Large Millimeter/submillimeter Array. The dual quasar phase represents a distinctive stage during the interaction between two galaxies for investigating quasar fueling and feedback effects on the gas reservoir.… ▽ More We present a study of the molecular gas in five closely-spaced ($R_{\perp}<20$ kpc) dual quasars ($L_{\rm bol}\gtrsim10^{44}~\mathrm{erg~s}^{-1}$) at redshifts $0.4<z<0.8$ with the Atacama Large Millimeter/submillimeter Array. The dual quasar phase represents a distinctive stage during the interaction between two galaxies for investigating quasar fueling and feedback effects on the gas reservoir. The dual quasars were selected from the Sloan Digital Sky Survey and Subaru/Hyper Suprime-Cam Subaru Strategic Program, with confirmatory spectroscopic validation. Based on the detection of the CO J=2--1 emission line with Band 4, we derived key properties including CO luminosities, line widths, and molecular gas masses for these systems. Among the ten quasars of the five pairs, eight have line detections exceeding $5σ$. The detected sources prominently harbor substantial molecular gas reservoirs, with molecular gas masses ($M_{\text{molgas}}$) between $10^{9.6-10.5}~\mathrm{M_{\odot}}$, and molecular gas-to-stellar mass ratios ($μ_{\text{molgas}}$) spanning $18-97\%$. The overall $μ_{\text{molgas}}$ of these dual quasars agrees with that of inactive star-forming main-sequence galaxies at comparable redshifts, indicating no clear evidence of quenching. However, intriguing features in each individual system show possible evidence of AGN feedback, matter transfer, and compaction processes. △ Less

Submitted 12 July, 2024; originally announced July 2024.

Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models

Authors: Wanling Gao, Yunyou Huang, Dandan Cui, Zhuoming Yu, Wenjing Liu, Xiaoshuang Liang, Jiahui Zhao, Jiyue Xie, Hao Li, Li Ma, Ning Ye, Yumiao Kang, Dingfeng Luo, Peng Pan, Wei Huang, Zhongmou Liu, Jizhong Hu, Gangyuan Zhao, Chongrong Jiang, Fan Huang, Tianyi Wei, Suqin Tang, Bingjie Xia, Zhifei Zhang, Jianfeng Zhan

Abstract: A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of cl… ▽ More A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of clinicians in collaborating with AI, pivotal for determining its impact on clinical practice, is often overlooked. For the first time, we emphasize the critical necessity for rigorous and cost-effective evaluation methodologies for AI models in clinical practice, featuring patient/clinician-centered (dual-centered) AI randomized controlled trials (DC-AI RCTs) and virtual clinician-based in-silico trials (VC-MedAI) as an effective proxy for DC-AI RCTs. Leveraging 7500 diagnosis records from two-phase inaugural DC-AI RCTs across 14 medical centers with 125 clinicians, our results demonstrate the necessity of DC-AI RCTs and the effectiveness of VC-MedAI. Notably, VC-MedAI performs comparably to human clinicians, replicating insights and conclusions from prospective DC-AI RCTs. We envision DC-AI RCTs and VC-MedAI as pivotal advancements, presenting innovative and transformative evaluation methodologies for AI models in clinical practice, offering a preclinical-like setting mirroring conventional medicine, and reshaping development paradigms in a cost-effective and fast-iterative manner. Chinese Clinical Trial Registration: ChiCTR2400086816. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: 23 pages

CamFreeDiff: Camera-free Image to Panorama Generation with Diffusion Model

Authors: Xiaoding Yuan, Shitao Tang, Kejie Li, Alan Yuille, Peng Wang

Abstract: This paper introduces Camera-free Diffusion (CamFreeDiff) model for 360-degree image outpainting from a single camera-free image and text description. This method distinguishes itself from existing strategies, such as MVDiffusion, by eliminating the requirement for predefined camera poses. Instead, our model incorporates a mechanism for predicting homography directly within the multi-view diffusio… ▽ More This paper introduces Camera-free Diffusion (CamFreeDiff) model for 360-degree image outpainting from a single camera-free image and text description. This method distinguishes itself from existing strategies, such as MVDiffusion, by eliminating the requirement for predefined camera poses. Instead, our model incorporates a mechanism for predicting homography directly within the multi-view diffusion framework. The core of our approach is to formulate camera estimation by predicting the homography transformation from the input view to a predefined canonical view. The homography provides point-level correspondences between the input image and targeting panoramic images, allowing connections enforced by correspondence-aware attention in a fully differentiable manner. Qualitative and quantitative experimental results demonstrate our model's strong robustness and generalization ability for 360-degree image outpainting in the challenging context of camera-free inputs. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Learn and Don't Forget: Adding a New Language to ASR Foundation Models

Authors: Mengjie Qian, Siyuan Tang, Rao Ma, Kate M. Knill, Mark J. F. Gales

Abstract: Foundation ASR models often support many languages, e.g. 100 languages in Whisper. However, there has been limited work on integrating an additional, typically low-resource, language, while maintaining performance on the original language set. Fine-tuning, while simple, may degrade the accuracy of the original set. We compare three approaches that exploit adaptation parameters: soft language code… ▽ More Foundation ASR models often support many languages, e.g. 100 languages in Whisper. However, there has been limited work on integrating an additional, typically low-resource, language, while maintaining performance on the original language set. Fine-tuning, while simple, may degrade the accuracy of the original set. We compare three approaches that exploit adaptation parameters: soft language code tuning, train only the language code; soft prompt tuning, train prepended tokens; and LoRA where a small set of additional parameters are optimised. Elastic Weight Consolidation (EWC) offers an alternative compromise with the potential to maintain performance in specific target languages. Results show that direct fine-tuning yields the best performance for the new language but degrades existing language capabilities. EWC can address this issue for specific languages. If only adaptation parameters are used, the language capabilities are maintained but at the cost of performance in the new language. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Ask Questions with Double Hints: Visual Question Generation with Answer-awareness and Region-reference

Authors: Kai Shen, Lingfei Wu, Siliang Tang, Fangli Xu, Bo Long, Yueting Zhuang, Jian Pei

Abstract: The visual question generation (VQG) task aims to generate human-like questions from an image and potentially other side information (e.g. answer type). Previous works on VQG fall in two aspects: i) They suffer from one image to many questions mapping problem, which leads to the failure of generating referential and meaningful questions from an image. ii) They fail to model complex implicit relati… ▽ More The visual question generation (VQG) task aims to generate human-like questions from an image and potentially other side information (e.g. answer type). Previous works on VQG fall in two aspects: i) They suffer from one image to many questions mapping problem, which leads to the failure of generating referential and meaningful questions from an image. ii) They fail to model complex implicit relations among the visual objects in an image and also overlook potential interactions between the side information and image. To address these limitations, we first propose a novel learning paradigm to generate visual questions with answer-awareness and region-reference. Concretely, we aim to ask the right visual questions with Double Hints - textual answers and visual regions of interests, which could effectively mitigate the existing one-to-many mapping issue. Particularly, we develop a simple methodology to self-learn the visual hints without introducing any additional human annotations. Furthermore, to capture these sophisticated relationships, we propose a new double-hints guided Graph-to-Sequence learning framework, which first models them as a dynamic graph and learns the implicit topology end-to-end, and then utilizes a graph-to-sequence model to generate the questions with double hints. Experimental results demonstrate the priority of our proposed method. △ Less

Submitted 6 July, 2024; originally announced July 2024.

Journal ref: IEEE Transactions on Pattern Analysis and Machine Intelligence 2024

LaRa: Efficient Large-Baseline Radiance Fields

Authors: Anpei Chen, Haofei Xu, Stefano Esposito, Siyu Tang, Andreas Geiger

Abstract: Radiance field methods have achieved photorealistic novel view synthesis and geometry reconstruction. But they are mostly applied in per-scene optimization or small-baseline settings. While several recent works investigate feed-forward reconstruction with large baselines by utilizing transformers, they all operate with a standard global attention mechanism and hence ignore the local nature of 3D r… ▽ More Radiance field methods have achieved photorealistic novel view synthesis and geometry reconstruction. But they are mostly applied in per-scene optimization or small-baseline settings. While several recent works investigate feed-forward reconstruction with large baselines by utilizing transformers, they all operate with a standard global attention mechanism and hence ignore the local nature of 3D reconstruction. We propose a method that unifies local and global reasoning in transformer layers, resulting in improved quality and faster convergence. Our model represents scenes as Gaussian Volumes and combines this with an image encoder and Group Attention Layers for efficient feed-forward reconstruction. Experimental results demonstrate that our model, trained for two days on four GPUs, demonstrates high fidelity in reconstructing 360&deg radiance fields, and robustness to zero-shot and out-of-domain testing. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Micro-gesture Online Recognition using Learnable Query Points

Authors: Pengyu Liu, Fei Wang, Kun Li, Guoliang Chen, Yanyan Wei, Shengeng Tang, Zhiliang Wu, Dan Guo

Abstract: In this paper, we briefly introduce the solution developed by our team, HFUT-VUT, for the Micro-gesture Online Recognition track in the MiGA challenge at IJCAI 2024. The Micro-gesture Online Recognition task involves identifying the category and locating the start and end times of micro-gestures in video clips. Compared to the typical Temporal Action Detection task, the Micro-gesture Online Recogn… ▽ More In this paper, we briefly introduce the solution developed by our team, HFUT-VUT, for the Micro-gesture Online Recognition track in the MiGA challenge at IJCAI 2024. The Micro-gesture Online Recognition task involves identifying the category and locating the start and end times of micro-gestures in video clips. Compared to the typical Temporal Action Detection task, the Micro-gesture Online Recognition task focuses more on distinguishing between micro-gestures and pinpointing the start and end times of actions. Our solution ranks 2nd in the Micro-gesture Online Recognition track. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Comments: Technical Report of HFUT-VUT for the MiGA challenge at IJCAI 2024

Supporters and Skeptics: LLM-based Analysis of Engagement with Mental Health (Mis)Information Content on Video-sharing Platforms

Authors: Viet Cuong Nguyen, Mini Jain, Abhijat Chauhan, Heather Jaime Soled, Santiago Alvarez Lesmes, Zihang Li, Michael L. Birnbaum, Sunny X. Tang, Srijan Kumar, Munmun De Choudhury

Abstract: Over one in five adults in the US lives with a mental illness. In the face of a shortage of mental health professionals and offline resources, online short-form video content has grown to serve as a crucial conduit for disseminating mental health help and resources. However, the ease of content creation and access also contributes to the spread of misinformation, posing risks to accurate diagnosis… ▽ More Over one in five adults in the US lives with a mental illness. In the face of a shortage of mental health professionals and offline resources, online short-form video content has grown to serve as a crucial conduit for disseminating mental health help and resources. However, the ease of content creation and access also contributes to the spread of misinformation, posing risks to accurate diagnosis and treatment. Detecting and understanding engagement with such content is crucial to mitigating their harmful effects on public health. We perform the first quantitative study of the phenomenon using YouTube Shorts and Bitchute as the sites of study. We contribute MentalMisinfo, a novel labeled mental health misinformation (MHMisinfo) dataset of 739 videos (639 from Youtube and 100 from Bitchute) and 135372 comments in total, using an expert-driven annotation schema. We first found that few-shot in-context learning with large language models (LLMs) are effective in detecting MHMisinfo videos. Next, we discover distinct and potentially alarming linguistic patterns in how audiences engage with MHMisinfo videos through commentary on both video-sharing platforms. Across the two platforms, comments could exacerbate prevailing stigma with some groups showing heightened susceptibility to and alignment with MHMisinfo. We discuss technical and public health-driven adaptive solutions to tackling the "epidemic" of mental health misinformation online. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: 12 pages, in submission to ICWSM

Unleash the Power of Local Representations for Few-Shot Classification

Authors: Shi Tang, Guiming Luo, Xinchen Ye, Zhiyi Xia

Abstract: Generalizing to novel classes unseen during training is a key challenge of few-shot classification. Recent metric-based methods try to address this by local representations. However, they are unable to take full advantage of them due to (i) improper supervision for pretraining the feature extractor, and (ii) lack of adaptability in the metric for handling various possible compositions of local fea… ▽ More Generalizing to novel classes unseen during training is a key challenge of few-shot classification. Recent metric-based methods try to address this by local representations. However, they are unable to take full advantage of them due to (i) improper supervision for pretraining the feature extractor, and (ii) lack of adaptability in the metric for handling various possible compositions of local feature sets. In this work, we unleash the power of local representations in improving novel-class generalization. For the feature extractor, we design a novel pretraining paradigm that learns randomly cropped patches by soft labels. It utilizes the class-level diversity of patches while diminishing the impact of their semantic misalignments to hard labels. To align network output with soft labels, we also propose a UniCon KL-Divergence that emphasizes the equal contribution of each base class in describing "non-base" patches. For the metric, we formulate measuring local feature sets as an entropy-regularized optimal transport problem to introduce the ability to handle sets consisting of homogeneous elements. Furthermore, we design a Modulate Module to endow the metric with the necessary adaptability. Our method achieves new state-of-the-art performance on three popular benchmarks. Moreover, it exceeds state-of-the-art transductive and cross-modal methods in the fine-grained scenario. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Cross-Lingual Transfer Learning for Speech Translation

Authors: Rao Ma, Yassir Fathullah, Mengjie Qian, Siyuan Tang, Mark Gales, Kate Knill

Abstract: There has been increasing interest in building multilingual foundation models for NLP and speech research. Zero-shot cross-lingual transfer has been demonstrated on a range of NLP tasks where a model fine-tuned on task-specific data in one language yields performance gains in other languages. Here, we explore whether speech-based models exhibit the same transfer capability. Using Whisper as an exa… ▽ More There has been increasing interest in building multilingual foundation models for NLP and speech research. Zero-shot cross-lingual transfer has been demonstrated on a range of NLP tasks where a model fine-tuned on task-specific data in one language yields performance gains in other languages. Here, we explore whether speech-based models exhibit the same transfer capability. Using Whisper as an example of a multilingual speech foundation model, we examine the utterance representation generated by the speech encoder. Despite some language-sensitive information being preserved in the audio embedding, words from different languages are mapped to a similar semantic space, as evidenced by a high recall rate in a speech-to-speech retrieval task. Leveraging this shared embedding space, zero-shot cross-lingual transfer is demonstrated in speech translation. When the Whisper model is fine-tuned solely on English-to-Chinese translation data, performance improvements are observed for input utterances in other languages. Additionally, experiments on low-resource languages show that Whisper can perform speech translation for utterances from languages unseen during pre-training by utilizing cross-lingual representations. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Orbital origin of magnetic moment enhancement induced by charge density wave in kagome FeGe

Authors: Shulun Han, Linyang Li, Chi Sin Tang, Qi Wang, Lingfeng Zhang, Caozheng Diao, Mingwen Zhao, Shuo Sun, Lijun Tian, Mark B. H. Breese, Chuanbing Cai, Milorad V. Milosevic, Yanpeng Qi, Andrew T. S. Wee, Xinmao Yin

Abstract: Interactions among various electronic states such as CDW, magnetism, and superconductivity are of high significance in strongly correlated systems. While significant progress has been made in understanding the relationship between CDW and superconductivity, the interplay between CDW and magnetic order remains largely elusive. Kagome lattices, which intertwine nontrivial topology, charge order, and… ▽ More Interactions among various electronic states such as CDW, magnetism, and superconductivity are of high significance in strongly correlated systems. While significant progress has been made in understanding the relationship between CDW and superconductivity, the interplay between CDW and magnetic order remains largely elusive. Kagome lattices, which intertwine nontrivial topology, charge order, and magnetism, offer an ideal platform for such studies. The kagome magnet FeGe, hosting the unique coupling between CDW and magnetism, has recently garnered considerable attention in that respect. Here we reveal the significant role of the orbital coupling effect during the CDW phase transition, highlighting the orbital origin of the magnetic moment enhancement in FeGe. Our X ray absorption experiments and first principles calculations illuminate the temperature dependent behavior of Fe3d_Ge4p orbital hybridization and corroborate its pivotal impact on the magnetic properties of FeGe. These findings introduce an orbital dimension to the correlation between charge and magnetic degrees of freedom, advancing our understanding of the intriguing quantum phases resulting from this interplay. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Decoupling General and Personalized Knowledge in Federated Learning via Additive and Low-Rank Decomposition

Authors: Xinghao Wu, Xuefeng Liu, Jianwei Niu, Haolin Wang, Shaojie Tang, Guogang Zhu, Hao Su

Abstract: To address data heterogeneity, the key strategy of Personalized Federated Learning (PFL) is to decouple general knowledge (shared among clients) and client-specific knowledge, as the latter can have a negative impact on collaboration if not removed. Existing PFL methods primarily adopt a parameter partitioning approach, where the parameters of a model are designated as one of two types: parameters… ▽ More To address data heterogeneity, the key strategy of Personalized Federated Learning (PFL) is to decouple general knowledge (shared among clients) and client-specific knowledge, as the latter can have a negative impact on collaboration if not removed. Existing PFL methods primarily adopt a parameter partitioning approach, where the parameters of a model are designated as one of two types: parameters shared with other clients to extract general knowledge and parameters retained locally to learn client-specific knowledge. However, as these two types of parameters are put together like a jigsaw puzzle into a single model during the training process, each parameter may simultaneously absorb both general and client-specific knowledge, thus struggling to separate the two types of knowledge effectively. In this paper, we introduce FedDecomp, a simple but effective PFL paradigm that employs parameter additive decomposition to address this issue. Instead of assigning each parameter of a model as either a shared or personalized one, FedDecomp decomposes each parameter into the sum of two parameters: a shared one and a personalized one, thus achieving a more thorough decoupling of shared and personalized knowledge compared to the parameter partitioning method. In addition, as we find that retaining local knowledge of specific clients requires much lower model capacity compared with general knowledge across all clients, we let the matrix containing personalized parameters be low rank during the training process. Moreover, a new alternating training strategy is proposed to further improve the performance. Experimental results across multiple datasets and varying degrees of data heterogeneity demonstrate that FedDecomp outperforms state-of-the-art methods up to 4.9\%. △ Less

Submitted 28 June, 2024; originally announced June 2024.

Comments: 12 pages, 8 figures

AFBench: A Large-scale Benchmark for Airfoil Design

Authors: Jian Liu, Jianyu Wu, Hairun Xie, Guoqing Zhang, Jing Wang, Wei Liu, Wanli Ouyang, Junjun Jiang, Xianming Liu, Shixiang Tang, Miao Zhang

Abstract: Data-driven generative models have emerged as promising approaches towards achieving efficient mechanical inverse design. However, due to prohibitively high cost in time and money, there is still lack of open-source and large-scale benchmarks in this field. It is mainly the case for airfoil inverse design, which requires to generate and edit diverse geometric-qualified and aerodynamic-qualified ai… ▽ More Data-driven generative models have emerged as promising approaches towards achieving efficient mechanical inverse design. However, due to prohibitively high cost in time and money, there is still lack of open-source and large-scale benchmarks in this field. It is mainly the case for airfoil inverse design, which requires to generate and edit diverse geometric-qualified and aerodynamic-qualified airfoils following the multimodal instructions, \emph{i.e.,} dragging points and physical parameters. This paper presents the open-source endeavors in airfoil inverse design, \emph{AFBench}, including a large-scale dataset with 200 thousand airfoils and high-quality aerodynamic and geometric labels, two novel and practical airfoil inverse design tasks, \emph{i.e.,} conditional generation on multimodal physical parameters, controllable editing, and comprehensive metrics to evaluate various existing airfoil inverse design methods. Our aim is to establish \emph{AFBench} as an ecosystem for training and evaluating airfoil inverse design methods, with a specific focus on data-driven controllable inverse design models by multimodal instructions capable of bridging the gap between ideas and execution, the academic research and industrial applications. We have provided baseline models, comprehensive experimental observations, and analysis to accelerate future research. Our baseline model is trained on an RTX 3090 GPU within 16 hours. The codebase, datasets and benchmarks will be available at \url{https://hitcslj.github.io/afbench/}. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: Submitted to NeurIPS 2024 Dataset & Benchmark Track

Few-Shot Medical Image Segmentation with High-Fidelity Prototypes

Authors: Song Tang, Shaxu Yan, Xiaozhi Qi, Jianxin Gao, Mao Ye, Jianwei Zhang, Xiatian Zhu

Abstract: Few-shot Semantic Segmentation (FSS) aims to adapt a pretrained model to new classes with as few as a single labelled training sample per class. Despite the prototype based approaches have achieved substantial success, existing models are limited to the imaging scenarios with considerably distinct objects and not highly complex background, e.g., natural images. This makes such models suboptimal fo… ▽ More Few-shot Semantic Segmentation (FSS) aims to adapt a pretrained model to new classes with as few as a single labelled training sample per class. Despite the prototype based approaches have achieved substantial success, existing models are limited to the imaging scenarios with considerably distinct objects and not highly complex background, e.g., natural images. This makes such models suboptimal for medical imaging with both conditions invalid. To address this problem, we propose a novel Detail Self-refined Prototype Network (DSPNet) to constructing high-fidelity prototypes representing the object foreground and the background more comprehensively. Specifically, to construct global semantics while maintaining the captured detail semantics, we learn the foreground prototypes by modelling the multi-modal structures with clustering and then fusing each in a channel-wise manner. Considering that the background often has no apparent semantic relation in the spatial dimensions, we integrate channel-specific structural information under sparse channel-aware regulation. Extensive experiments on three challenging medical image benchmarks show the superiority of DSPNet over previous state-of-the-art methods. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Improving Large Models with Small models: Lower Costs and Better Performance

Authors: Dong Chen, Shuo Zhang, Yueting Zhuang, Siliang Tang, Qidong Liu, Hua Wang, Mingliang Xu

Abstract: Pretrained large models (PLMs), such as ChatGPT, have demonstrated remarkable performance across diverse tasks. However, the significant computational requirements of PLMs have discouraged most product teams from running or fine-tuning them. In such cases, to harness the exceptional performance of PLMs, one must rely on expensive APIs, thereby exacerbating the economic burden. Despite the overall… ▽ More Pretrained large models (PLMs), such as ChatGPT, have demonstrated remarkable performance across diverse tasks. However, the significant computational requirements of PLMs have discouraged most product teams from running or fine-tuning them. In such cases, to harness the exceptional performance of PLMs, one must rely on expensive APIs, thereby exacerbating the economic burden. Despite the overall inferior performance of small models, in specific distributions, they can achieve comparable or even superior results. Consequently, some input can be processed exclusively by small models. On the other hand, certain tasks can be broken down into multiple subtasks, some of which can be completed without powerful capabilities. Under these circumstances, small models can handle the simple subtasks, allowing large models to focus on challenging subtasks, thus improving the performance. We propose Data Shunt$^+$ (DS$^+$), a general paradigm for collaboration of small and large models. DS$^+$ not only substantially reduces the cost associated with querying large models but also effectively improves large models' performance. For instance, ChatGPT achieves an accuracy of $94.43\%$ on Amazon Product sentiment analysis, and DS$^+$ achieves an accuracy of $95.64\%$, while the cost has been reduced to only $31.18\%$. Besides, experiments also prove that the proposed collaborative-based paradigm can better inject specific task knowledge into PLMs compared to fine-tuning. △ Less

Submitted 15 June, 2024; originally announced June 2024.

Comments: 11 pages

Convolutional dynamical sampling and some new results

Authors: Longxiu Huang, A. Martina Neuman, Sui Tang, Yuying Xie

Abstract: In this work, we explore the dynamical sampling problem on $\ell^2(\mathbb{Z})$ driven by a convolution operator defined by a convolution kernel. This problem is inspired by the need to recover a bandlimited heat diffusion field from space-time samples and its discrete analogue. In this book chapter, we review recent results in the finite-dimensional case and extend these findings to the infinite-… ▽ More In this work, we explore the dynamical sampling problem on $\ell^2(\mathbb{Z})$ driven by a convolution operator defined by a convolution kernel. This problem is inspired by the need to recover a bandlimited heat diffusion field from space-time samples and its discrete analogue. In this book chapter, we review recent results in the finite-dimensional case and extend these findings to the infinite-dimensional case, focusing on the study of the density of space-time sampling sets. △ Less

Submitted 4 July, 2024; v1 submitted 21 June, 2024; originally announced June 2024.

Submodular Participatory Budgeting

Authors: Jing Yuan, Shaojie Tang

Abstract: Participatory budgeting refers to the practice of allocating public resources by collecting and aggregating individual preferences. Most existing studies in this field often assume an additive utility function, where each individual holds a private utility for each candidate project, and the total utility of a set of funded projects is simply the sum of the utilities of all projects. We argue that… ▽ More Participatory budgeting refers to the practice of allocating public resources by collecting and aggregating individual preferences. Most existing studies in this field often assume an additive utility function, where each individual holds a private utility for each candidate project, and the total utility of a set of funded projects is simply the sum of the utilities of all projects. We argue that this assumption does not always hold in reality. For example, building two playgrounds in the same neighborhood does not necessarily lead to twice the utility of building a single playground. To address this, we extend the existing study by proposing a submodular participatory budgeting problem, assuming that the utility function of each individual is a monotone and submodular function over funded projects. We propose and examine three preference elicitation methods, including \emph{ranking-by-marginal-values}, \emph{ranking-by-values} and \emph{threshold approval votes}, and analyze their performances in terms of distortion. Notably, if the utility function is addicative, our aggregation rule designed for threshold approval votes achieves a better distortion than the state-of-the-art approach. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Single-photon triggered quantum entanglement between two qubits or at least 2000 identical qubits

Authors: Wangjun Lu, Cuilu Zhai, Hong Tao, Yaju Song, Shiqing Tang, Lan Xu

Abstract: This paper studies the effect of single-photon light fields on quantum entanglement between two qubits and multiple identical qubits initially in a direct state. For two qubits, we first analyze the impact of the excited state's weight on single-photon-triggered entanglement, finding that excessive weight disrupts this process. We then explore how initial coherence affects entanglement, discoverin… ▽ More This paper studies the effect of single-photon light fields on quantum entanglement between two qubits and multiple identical qubits initially in a direct state. For two qubits, we first analyze the impact of the excited state's weight on single-photon-triggered entanglement, finding that excessive weight disrupts this process. We then explore how initial coherence affects entanglement, discovering that maximum initial coherence enables the single photon to achieve maximal entanglement. For multiple qubits, we similarly investigate the effects of the excited state's weight and initial coherence on entanglement control. In large qubit systems, we find that single photons cannot trigger entanglement when excited-state weights exceed ground-state weights or when all qubits are initially in the ground state. Interestingly, single photons can still trigger entanglement between any two qubits in systems with at least 2000 qubits, with the entanglement depending on initial state parameters rather than the number of qubits. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 19 pages, 11 figures

Measuring the Spot Variability of T Tauri Stars Using Near-IR Atomic Fe and Molecular OH Lines

Authors: Shih-Yun Tang, Christopher M. Johns-Krull, L. Prato, Asa G. Stahl

Abstract: As part of the Young Exoplanets Spectroscopic Survey (YESS), this study explores the spot variability of 13 T Tauri Stars (TTSs) in the near-infrared $H$ band, using spectra from the Immersion GRating INfrared Spectrometer (IGRINS). By analyzing effective temperature ($T_{\rm eff}$) sensitive lines of atomic FeI at ~1.56259 um and ~1.56362 um, and molecular OH at ~1.56310 um and ~1.56317 um, we de… ▽ More As part of the Young Exoplanets Spectroscopic Survey (YESS), this study explores the spot variability of 13 T Tauri Stars (TTSs) in the near-infrared $H$ band, using spectra from the Immersion GRating INfrared Spectrometer (IGRINS). By analyzing effective temperature ($T_{\rm eff}$) sensitive lines of atomic FeI at ~1.56259 um and ~1.56362 um, and molecular OH at ~1.56310 um and ~1.56317 um, we develop an empirical equivalent width ratio (EWR) relationship for $T_{\rm eff}$ in the range of 3400-5000 K. This relationship allows for precise relative $T_{\rm eff}$ estimates to within tens of Kelvin and demonstrates compatibility with solar metallicity target models. However, discrepancies between observational data and model predictions limit the extension of the $T_{\rm eff}$-EWR relationship to a broader parameter space. Our study reveals that both classical and weak-line TTSs can exhibit $T_{\rm eff}$ variations exceeding 150 K over a span of two years. The detection of a quarter-phase delay between the EWR and radial velocity phase curves in TTSs indicates spot-driven signals. A phase delay of 0.06 $\pm$ 0.13 for CI Tau, however, suggests additional dynamics, potentially caused by planetary interaction, inferred from a posited 1:1 commensurability between the rotation period and orbital period. Moreover, a positive correlation between $T_{\rm eff}$ variation amplitude and stellar inclination angle support the existence of high-latitude spots on TTSs, further enriching our understanding of stellar surface activity in young stars. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 31 pages, 19 figures, and 5 tables. Accepted to ApJ

Bridging Local Details and Global Context in Text-Attributed Graphs

Authors: Yaoke Wang, Yun Zhu, Wenqiao Zhang, Yueting Zhuang, Yunfei Li, Siliang Tang

Abstract: Representation learning on text-attributed graphs (TAGs) is vital for real-world applications, as they combine semantic textual and contextual structural information. Research in this field generally consist of two main perspectives: local-level encoding and global-level aggregating, respectively refer to textual node information unification (e.g., using Language Models) and structure-augmented mo… ▽ More Representation learning on text-attributed graphs (TAGs) is vital for real-world applications, as they combine semantic textual and contextual structural information. Research in this field generally consist of two main perspectives: local-level encoding and global-level aggregating, respectively refer to textual node information unification (e.g., using Language Models) and structure-augmented modeling (e.g., using Graph Neural Networks). Most existing works focus on combining different information levels but overlook the interconnections, i.e., the contextual textual information among nodes, which provides semantic insights to bridge local and global levels. In this paper, we propose GraphBridge, a multi-granularity integration framework that bridges local and global perspectives by leveraging contextual textual information, enhancing fine-grained understanding of TAGs. Besides, to tackle scalability and efficiency challenges, we introduce a graphaware token reduction module. Extensive experiments across various models and datasets show that our method achieves state-of-theart performance, while our graph-aware token reduction module significantly enhances efficiency and solves scalability issues. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Multi-spectral sirens: Gravitational-wave cosmology with (multi-) sub-populations of binary black holes

Authors: Yin-Jie Li, Shao-Peng Tang, Yuan-Zhu Wang, Yi-Zhong Fan

Abstract: The cosmic expansion rate can be directly measured with gravitational waves (GWs) of the compact binary mergers, by jointly constraining the mass function of the population and the cosmological model via the so called spectral sirens. Such a method relies on the features in the mass functions, which may originate from some individual sub-populations, and hence become blurred/indistinct due to the… ▽ More The cosmic expansion rate can be directly measured with gravitational waves (GWs) of the compact binary mergers, by jointly constraining the mass function of the population and the cosmological model via the so called spectral sirens. Such a method relies on the features in the mass functions, which may originate from some individual sub-populations, and hence become blurred/indistinct due to the superposition of different sub-populations. In this work we propose a novel approach to constrain the cosmic expansion rate with sub-populations of GW events, named multi-spectral sirens. We illustrate the advantage of the multi-spectral sirens compared to the traditional spectral sirens by simulation with mock data. The application of this approach to the GWTC-3 data yields $H_0=73.25^{+29.87}_{-25.55}~{\rm Mpc}^{-1}~{\rm km}~{\rm s}^{-1}$ (median and symmetric 68.3\% credible level). The incorporation of the bright standard siren GW170817 with a uniform prior in $ [10,200] ~{\rm Mpc}^{-1}~{\rm km}~{\rm s}^{-1}$ gives $H_0=72.38^{+15.03}_{-9.13}~{\rm Mpc}^{-1}~{\rm km}~{\rm s}^{-1}$ (68.3\% confidence level), corresponding to an improvement of $\sim28\%$ with respect to the measurement from sole GW170817. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: 14 pages, 8 figures, 1 table; comments are welcome

Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space

Authors: Yuan Wang, Zhao Wang, Junhao Gong, Di Huang, Tong He, Wanli Ouyang, Jile Jiao, Xuetao Feng, Qi Dou, Shixiang Tang, Dan Xu

Abstract: In this paper, we introduce a novel path to $\textit{general}$ human motion generation by focusing on 2D space. Traditional methods have primarily generated human motions in 3D, which, while detailed and realistic, are often limited by the scope of available 3D motion data in terms of both the size and the diversity. To address these limitations, we exploit extensive availability of 2D motion data… ▽ More In this paper, we introduce a novel path to $\textit{general}$ human motion generation by focusing on 2D space. Traditional methods have primarily generated human motions in 3D, which, while detailed and realistic, are often limited by the scope of available 3D motion data in terms of both the size and the diversity. To address these limitations, we exploit extensive availability of 2D motion data. We present $\textbf{Holistic-Motion2D}$, the first comprehensive and large-scale benchmark for 2D whole-body motion generation, which includes over 1M in-the-wild motion sequences, each paired with high-quality whole-body/partial pose annotations and textual descriptions. Notably, Holistic-Motion2D is ten times larger than the previously largest 3D motion dataset. We also introduce a baseline method, featuring innovative $\textit{whole-body part-aware attention}$ and $\textit{confidence-aware modeling}$ techniques, tailored for 2D $\underline{\text T}$ext-driv$\underline{\text{EN}}$ whole-bo$\underline{\text D}$y motion gen$\underline{\text{ER}}$ation, namely $\textbf{Tender}$. Extensive experiments demonstrate the effectiveness of $\textbf{Holistic-Motion2D}$ and $\textbf{Tender}$ in generating expressive, diverse, and realistic human motions. We also highlight the utility of 2D motion for various downstream applications and its potential for lifting to 3D motion. The page link is: https://holistic-motion2d.github.io. △ Less

Submitted 17 June, 2024; originally announced June 2024.

Comments: 22 pages, 11figures, 17 tables

Contextual Distillation Model for Diversified Recommendation

Authors: Fan Li, Xu Si, Shisong Tang, Dingmin Wang, Kunyan Han, Bing Han, Guorui Zhou, Yang Song, Hechang Chen

Abstract: The diversity of recommendation is equally crucial as accuracy in improving user experience. Existing studies, e.g., Determinantal Point Process (DPP) and Maximal Marginal Relevance (MMR), employ a greedy paradigm to iteratively select items that optimize both accuracy and diversity. However, prior methods typically exhibit quadratic complexity, limiting their applications to the re-ranking stage… ▽ More The diversity of recommendation is equally crucial as accuracy in improving user experience. Existing studies, e.g., Determinantal Point Process (DPP) and Maximal Marginal Relevance (MMR), employ a greedy paradigm to iteratively select items that optimize both accuracy and diversity. However, prior methods typically exhibit quadratic complexity, limiting their applications to the re-ranking stage and are not applicable to other recommendation stages with a larger pool of candidate items, such as the pre-ranking and ranking stages. In this paper, we propose Contextual Distillation Model (CDM), an efficient recommendation model that addresses diversification, suitable for the deployment in all stages of industrial recommendation pipelines. Specifically, CDM utilizes the candidate items in the same user request as context to enhance the diversification of the results. We propose a contrastive context encoder that employs attention mechanisms to model both positive and negative contexts. For the training of CDM, we compare each target item with its context embedding and utilize the knowledge distillation framework to learn the win probability of each target item under the MMR algorithm, where the teacher is derived from MMR outputs. During inference, ranking is performed through a linear combination of the recommendation and student model scores, ensuring both diversity and efficiency. We perform offline evaluations on two industrial datasets and conduct online A/B test of CDM on the short-video platform KuaiShou. The considerable enhancements observed in both recommendation quality and diversity, as shown by metrics, provide strong superiority for the effectiveness of CDM. △ Less

Submitted 13 June, 2024; originally announced June 2024.

Comments: accepted by KDD 2024

Incremental Learning and Self-Attention Mechanisms Improve Neural System Identification

Authors: Isaac Lin, Tianye Wang, Shang Gao, Shiming Tang, Tai Sing Lee

Abstract: Convolutional neural networks (CNNs) have been shown to be the state-of-the-art approach for modeling the transfer functions of visual cortical neurons. Cortical neurons in the primary visual cortex are are sensitive to contextual information mediated by extensive horizontal and feedback connections. Standard CNNs can integrate global spatial image information to model such contextual modulation v… ▽ More Convolutional neural networks (CNNs) have been shown to be the state-of-the-art approach for modeling the transfer functions of visual cortical neurons. Cortical neurons in the primary visual cortex are are sensitive to contextual information mediated by extensive horizontal and feedback connections. Standard CNNs can integrate global spatial image information to model such contextual modulation via two mechanisms: successive rounds of convolutions and a fully connected readout layer. In this paper, we find that non-local networks or self-attention (SA) mechanisms, theoretically related to context-dependent flexible gating mechanisms observed in the primary visual cortex, improve neural response predictions over parameter-matched CNNs in two key metrics: tuning curve correlation and tuning peak. We factorize networks to determine the relative contribution of each context mechanism. This reveals that information in the local receptive field is most important for modeling the overall tuning curve, but surround information is critically necessary for characterizing the tuning peak. We find that self-attention can replace subsequent spatial-integration convolutions when learned in an incremental manner, and is further enhanced in the presence of a fully connected readout layer, suggesting that the two context mechanisms are complementary. Finally, we find that learning a receptive-field-centric model with self-attention, before incrementally learning a fully connected readout, yields a more biologically realistic model in terms of center-surround contributions. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: Preprint NeurIPS 2024

T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text

Authors: Aoxiong Yin, Haoyuan Li, Kai Shen, Siliang Tang, Yueting Zhuang

Abstract: In this work, we propose a two-stage sign language production (SLP) paradigm that first encodes sign language sequences into discrete codes and then autoregressively generates sign language from text based on the learned codebook. However, existing vector quantization (VQ) methods are fixed-length encodings, overlooking the uneven information density in sign language, which leads to under-encoding… ▽ More In this work, we propose a two-stage sign language production (SLP) paradigm that first encodes sign language sequences into discrete codes and then autoregressively generates sign language from text based on the learned codebook. However, existing vector quantization (VQ) methods are fixed-length encodings, overlooking the uneven information density in sign language, which leads to under-encoding of important regions and over-encoding of unimportant regions. To address this issue, we propose a novel dynamic vector quantization (DVA-VAE) model that can dynamically adjust the encoding length based on the information density in sign language to achieve accurate and compact encoding. Then, a GPT-like model learns to generate code sequences and their corresponding durations from spoken language text. Extensive experiments conducted on the PHOENIX14T dataset demonstrate the effectiveness of our proposed method. To promote sign language research, we propose a new large German sign language dataset, PHOENIX-News, which contains 486 hours of sign language videos, audio, and transcription texts.Experimental analysis on PHOENIX-News shows that the performance of our model can be further improved by increasing the size of the training data. Our project homepage is https://t2sgpt-demo.yinaoxiong.cn. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: Accepted by ACL 2024

MLCM: Multistep Consistency Distillation of Latent Diffusion Model

Authors: Qingsong Xie, Zhenyi Liao, Chen chen, Zhijie Deng, Shixiang Tang, Haonan Lu

Abstract: Distilling large latent diffusion models (LDMs) into ones that are fast to sample from is attracting growing research interest. However, the majority of existing methods face a dilemma where they either (i) depend on multiple individual distilled models for different sampling budgets, or (ii) sacrifice generation quality with limited (e.g., 2-4) and/or moderate (e.g., 5-8) sampling steps. To addre… ▽ More Distilling large latent diffusion models (LDMs) into ones that are fast to sample from is attracting growing research interest. However, the majority of existing methods face a dilemma where they either (i) depend on multiple individual distilled models for different sampling budgets, or (ii) sacrifice generation quality with limited (e.g., 2-4) and/or moderate (e.g., 5-8) sampling steps. To address these, we extend the recent multistep consistency distillation (MCD) strategy to representative LDMs, establishing the Multistep Latent Consistency Models (MLCMs) approach for low-cost high-quality image synthesis. MLCM serves as a unified model for various sampling steps due to the promise of MCD. We further augment MCD with a progressive training strategy to strengthen inter-segment consistency to boost the quality of few-step generations. We take the states from the sampling trajectories of the teacher model as training data for MLCMs to lift the requirements for high-quality training datasets and to bridge the gap between the training and inference of the distilled model. MLCM is compatible with preference learning strategies for further improvement of visual quality and aesthetic appeal. Empirically, MLCM can generate high-quality, delightful images with only 2-8 sampling steps. On the MSCOCO-2017 5K benchmark, MLCM distilled from SDXL gets a CLIP Score of 33.30, Aesthetic Score of 6.19, and Image Reward of 1.20 with only 4 steps, substantially surpassing 4-step LCM [23], 8-step SDXL-Lightning [17], and 8-step HyperSD [33]. We also demonstrate the versatility of MLCMs in applications including controllable generation, image style transfer, and Chinese-to-image generation. △ Less

Submitted 11 June, 2024; v1 submitted 9 June, 2024; originally announced June 2024.

A Two-Stage Adverse Weather Semantic Segmentation Method for WeatherProof Challenge CVPR 2024 Workshop UG2+

Authors: Jianzhao Wang, Yanyan Wei, Dehua Hu, Yilin Zhang, Shengeng Tang, Kun Li, Zhao Zhang

Abstract: This technical report presents our team's solution for the WeatherProof Dataset Challenge: Semantic Segmentation in Adverse Weather at CVPR'24 UG2+. We propose a two-stage deep learning framework for this task. In the first stage, we preprocess the provided dataset by concatenating images into video sequences. Subsequently, we leverage a low-rank video deraining method to generate high-fidelity ps… ▽ More This technical report presents our team's solution for the WeatherProof Dataset Challenge: Semantic Segmentation in Adverse Weather at CVPR'24 UG2+. We propose a two-stage deep learning framework for this task. In the first stage, we preprocess the provided dataset by concatenating images into video sequences. Subsequently, we leverage a low-rank video deraining method to generate high-fidelity pseudo ground truths. These pseudo ground truths offer superior alignment compared to the original ground truths, facilitating model convergence during training. In the second stage, we employ the InternImage network to train for the semantic segmentation task using the generated pseudo ground truths. Notably, our meticulously designed framework demonstrates robustness to degraded data captured under adverse weather conditions. In the challenge, our solution achieved a competitive score of 0.43 on the Mean Intersection over Union (mIoU) metric, securing a respectable rank of 4th. △ Less

Submitted 10 July, 2024; v1 submitted 8 June, 2024; originally announced June 2024.

Improving Logits-based Detector without Logits from Black-box LLMs

Authors: Cong Zeng, Shengkun Tang, Xianjun Yang, Yuanzhou Chen, Yiyou Sun, zhiqiang xu, Yao Li, Haifeng Chen, Wei Cheng, Dongkuan Xu

Abstract: The advent of Large Language Models (LLMs) has revolutionized text generation, producing outputs that closely mimic human writing. This blurring of lines between machine- and human-written text presents new challenges in distinguishing one from the other a task further complicated by the frequent updates and closed nature of leading proprietary LLMs. Traditional logits-based detection methods leve… ▽ More The advent of Large Language Models (LLMs) has revolutionized text generation, producing outputs that closely mimic human writing. This blurring of lines between machine- and human-written text presents new challenges in distinguishing one from the other a task further complicated by the frequent updates and closed nature of leading proprietary LLMs. Traditional logits-based detection methods leverage surrogate models for identifying LLM-generated content when the exact logits are unavailable from black-box LLMs. However, these methods grapple with the misalignment between the distributions of the surrogate and the often undisclosed target models, leading to performance degradation, particularly with the introduction of new, closed-source models. Furthermore, while current methodologies are generally effective when the source model is identified, they falter in scenarios where the model version remains unknown, or the test set comprises outputs from various source models. To address these limitations, we present Distribution-Aligned LLMs Detection (DALD), an innovative framework that redefines the state-of-the-art performance in black-box text detection even without logits from source LLMs. DALD is designed to align the surrogate model's distribution with that of unknown target LLMs, ensuring enhanced detection capability and resilience against rapid model iterations with minimal training investment. By leveraging corpus samples from publicly accessible outputs of advanced models such as ChatGPT, GPT-4 and Claude-3, DALD fine-tunes surrogate models to synchronize with unknown source model distributions effectively. △ Less

Submitted 11 June, 2024; v1 submitted 7 June, 2024; originally announced June 2024.

Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories

Authors: Yan Zhang, Sergey Prokudin, Marko Mihajlovic, Qianli Ma, Siyu Tang

Abstract: Understanding the dynamics of generic 3D scenes is fundamentally challenging in computer vision, essential in enhancing applications related to scene reconstruction, motion tracking, and avatar creation. In this work, we address the task as the problem of inferring dense, long-range motion of 3D points. By observing a set of point trajectories, we aim to learn an implicit motion field parameterize… ▽ More Understanding the dynamics of generic 3D scenes is fundamentally challenging in computer vision, essential in enhancing applications related to scene reconstruction, motion tracking, and avatar creation. In this work, we address the task as the problem of inferring dense, long-range motion of 3D points. By observing a set of point trajectories, we aim to learn an implicit motion field parameterized by a neural network to predict the movement of novel points within the same domain, without relying on any data-driven or scene-specific priors. To achieve this, our approach builds upon the recently introduced dynamic point field model that learns smooth deformation fields between the canonical frame and individual observation frames. However, temporal consistency between consecutive frames is neglected, and the number of required parameters increases linearly with the sequence length due to per-frame modeling. To address these shortcomings, we exploit the intrinsic regularization provided by SIREN, and modify the input layer to produce a spatiotemporally smooth motion field. Additionally, we analyze the motion field Jacobian matrix, and discover that the motion degrees of freedom (DOFs) in an infinitesimal area around a point and the network hidden variables have different behaviors to affect the model's representational power. This enables us to improve the model representation capability while retaining the model compactness. Furthermore, to reduce the risk of overfitting, we introduce a regularization term based on the assumption of piece-wise motion smoothness. Our experiments assess the model's performance in predicting unseen point trajectories and its application in temporal mesh alignment with guidance. The results demonstrate its superiority and effectiveness. The code and data for the project are publicly available: \url{https://yz-cnsdqz.github.io/eigenmotion/DOMA/} △ Less

Submitted 5 June, 2024; originally announced June 2024.

Comments: cvpr24 post camera ready

AD-H: Autonomous Driving with Hierarchical Agents

Authors: Zaibin Zhang, Shiyu Tang, Yuanhang Zhang, Talas Fu, Yifan Wang, Yang Liu, Dong Wang, Jing Shao, Lijun Wang, Huchuan Lu

Abstract: Due to the impressive capabilities of multimodal large language models (MLLMs), recent works have focused on employing MLLM-based agents for autonomous driving in large-scale and dynamic environments. However, prevalent approaches often directly translate high-level instructions into low-level vehicle control signals, which deviates from the inherent language generation paradigm of MLLMs and fails… ▽ More Due to the impressive capabilities of multimodal large language models (MLLMs), recent works have focused on employing MLLM-based agents for autonomous driving in large-scale and dynamic environments. However, prevalent approaches often directly translate high-level instructions into low-level vehicle control signals, which deviates from the inherent language generation paradigm of MLLMs and fails to fully harness their emergent powers. As a result, the generalizability of these methods is highly restricted by autonomous driving datasets used during fine-tuning. To tackle this challenge, we propose to connect high-level instructions and low-level control signals with mid-level language-driven commands, which are more fine-grained than high-level instructions but more universal and explainable than control signals, and thus can effectively bridge the gap in between. We implement this idea through a hierarchical multi-agent driving system named AD-H, including a MLLM planner for high-level reasoning and a lightweight controller for low-level execution. The hierarchical design liberates the MLLM from low-level control signal decoding and therefore fully releases their emergent capability in high-level perception, reasoning, and planning. We build a new dataset with action hierarchy annotations. Comprehensive closed-loop evaluations demonstrate several key advantages of our proposed AD-H system. First, AD-H can notably outperform state-of-the-art methods in achieving exceptional driving performance, even exhibiting self-correction capabilities during vehicle operation, a scenario not encountered in the training dataset. Second, AD-H demonstrates superior generalization under long-horizon instructions and novel environmental conditions, significantly surpassing current state-of-the-art methods. We will make our data and code publicly accessible at https://github.com/zhangzaibin/AD-H △ Less

Submitted 5 June, 2024; originally announced June 2024.

A Shared-Aperture Dual-Band sub-6 GHz and mmWave Reconfigurable Intelligent Surface With Independent Operation

Authors: Junhui Rao, Yujie Zhang, Shiwen Tang, Zan Li, Zhaoyang Ming, Jichen Zhang, Chi Yuk Chiu, Ross Murch

Abstract: A novel dual-band reconfigurable intelligent surface (DBI-RIS) design that combines the functionalities of millimeter-wave (mmWave) and sub-6 GHz bands within a single aperture is proposed. This design aims to bridge the gap between current single-band reconfigurable intelligent surfaces (RISs) and wireless systems utilizing sub-6 GHz and mmWave bands that require RIS with independently reconfigur… ▽ More A novel dual-band reconfigurable intelligent surface (DBI-RIS) design that combines the functionalities of millimeter-wave (mmWave) and sub-6 GHz bands within a single aperture is proposed. This design aims to bridge the gap between current single-band reconfigurable intelligent surfaces (RISs) and wireless systems utilizing sub-6 GHz and mmWave bands that require RIS with independently reconfigurable dual-band operation. The mmWave element is realized by a double-layer patch antenna loaded with 1-bit phase shifters, providing two reconfigurable states. An 8x8 mmWave element array is selectively interconnected using three RF switches to form a reconfigurable sub-6 GHz element at 3.5 GHz. A suspended electromagnetic band gap (EBG) structure is proposed to suppress surface waves and ensure sufficient geometric space for the phase shifter and control networks in the mmWave element. A low-cost planar spiral inductor (PSI) is carefully optimized to connect mmWave elements, enabling the sub-6 GHz function without affecting mmWave operation. Finally, prototypes of the DBI-RIS are fabricated, and experimental verification is conducted using two separate measurement testbeds. The fabricated sub-6 GHz RIS successfully achieves beam steering within the range of -35 to 35 degrees for DBI-RIS with 4x4 sub-6 GHz elements, while the mmWave RIS demonstrates beam steering between -30 to 30 degrees for DBI-RIS with 8x8 mmWave elements, and have good agreement with simulation results. △ Less

Submitted 5 June, 2024; originally announced June 2024.

Proxy Denoising for Source-Free Domain Adaptation

Authors: Song Tang, Wenxin Su, Mao Ye, Jianwei Zhang, Xiatian Zhu

Abstract: Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to an unlabeled target domain with no access to the source data. Inspired by the success of pre-trained large vision-language (ViL) models in many other applications, the latest SFDA methods have also validated the benefit of ViL models by leveraging their predictions as pseudo supervision. However, we observe that ViL's… ▽ More Source-free Domain Adaptation (SFDA) aims to adapt a pre-trained source model to an unlabeled target domain with no access to the source data. Inspired by the success of pre-trained large vision-language (ViL) models in many other applications, the latest SFDA methods have also validated the benefit of ViL models by leveraging their predictions as pseudo supervision. However, we observe that ViL's predictions could be noisy and inaccurate at an unknown rate, potentially introducing additional negative effects during adaption. To address this thus-far ignored challenge, in this paper, we introduce a novel Proxy Denoising (ProDe) approach. Specifically, we leverage the ViL model as a proxy to facilitate the adaptation process towards the latent domain-invariant space. Critically, we design a proxy denoising mechanism for correcting ViL's predictions. This is grounded on a novel proxy confidence theory by modeling elegantly the domain adaption effect of the proxy's divergence against the domain-invariant space. To capitalize the corrected proxy, we further derive a mutual knowledge distilling regularization. Extensive experiments show that our ProDe significantly outperforms the current state-of-the-art alternatives under both conventional closed-set setting and the more challenging open-set, partial-set and generalized SFDA settings. The code will release soon. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Low threshold optical bistability based on MoS2 in asymmetric Fabry-Perot cavity structure in visible light band

Authors: Songqing Tang, Mengjiao Ren, Zhiheng Li, Zhiwei Zheng, Leyong Jiang

Abstract: This article theoretically proposes a multi-layer Fabry-Perot cavity structure based on nonlinear MoS2, whose cavity is composed of asymmetric photonic crystals. In this structure, we observed a low threshold optical bistability phenomenon on the order of a in the visible light band, which is caused by the large third-order nonlinear conductivity of the bilayer MoS2 and the Fabry-Perot cavity reso… ▽ More This article theoretically proposes a multi-layer Fabry-Perot cavity structure based on nonlinear MoS2, whose cavity is composed of asymmetric photonic crystals. In this structure, we observed a low threshold optical bistability phenomenon on the order of a in the visible light band, which is caused by the large third-order nonlinear conductivity of the bilayer MoS2 and the Fabry-Perot cavity resonance. Research has found that when light is incident from two different directions in an asymmetric Fabry-Perot cavity, the optical bistability exhibits not exactly the same behavior. In addition, we further investigated and found that the optical bistability behavior in this simple multi-layer structure is closely related to parameters such as incident wavelength, Fabry-Perot cavity length, and refractive index of the photonic crystal dielectric. This work provides a new approach for the implementation of low threshold optical bistable devices in the visible light band, which is expected to be applied in nonlinear optical fields such as all optical switches and all optical logic devices. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: 22 pages, 6 figures

MoS2-based optical bistability in silver-Bragg reflector multilayer structure at visible light band

Authors: Songqing Tang, Mengjiao Ren, Zhiheng Li, Zhiwei Zheng, Leyong Jiang

Abstract: In this paper, we present a theoretical analysis of the optical bistability in a metallic silver-Bragg reflector structure by embedding bilayer MoS2 at the visible band. The nonlinear OB is achieved due to the nonlinear conductivity of the bilayer MoS2 and the excitation of the optical Tamm state at the interface between the silver and the Bragg reflector. It is found that the hysteresis behaviour… ▽ More In this paper, we present a theoretical analysis of the optical bistability in a metallic silver-Bragg reflector structure by embedding bilayer MoS2 at the visible band. The nonlinear OB is achieved due to the nonlinear conductivity of the bilayer MoS2 and the excitation of the optical Tamm state at the interface between the silver and the Bragg reflector. It is found that the hysteresis behaviour and the threshold width of the OB can be effectively tuned by varying the incident light wavelength. In addition, the optical bistable behaviour of the structure can be adjusted by varying the position of the MoS2 inset in the defect layer, incident angle and the structural parameters of the spacer layer. Although the current threshold cannot be commercialized, we believe that this solution will provide a meaningful path reference for low threshold bistability in the visible light band. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: 23 pages, 6 figures

Mott insulating phase and coherent-incoherent crossover across magnetic phase transition in 2D antiferromagnetic CrSBr

Authors: Fan Wu, Xuefeng Zhang, Yi Chen, Ding Pei, Mengwen Zhan, Zicheng Tao, Cheng Chen, Shipeng Lu, Jingzhi Chen, Shujie Tang, Xia Wang, Yanfeng Guo, Lexian Yang, Yan Zhang, Yulin Chen, Qixi Mi, Gang Li, Zhongkai Liu

Abstract: In two-dimensional van der Waals magnetic materials, the interplay between magnetism and electron correlation can give rise to new ground states and lead to novel transport and optical properties. A fundamental question in these materials is how the electron correlation manifests and interacts with the magnetic orders. In this study, we demonstrate that the recently discovered 2D antiferromagnetic… ▽ More In two-dimensional van der Waals magnetic materials, the interplay between magnetism and electron correlation can give rise to new ground states and lead to novel transport and optical properties. A fundamental question in these materials is how the electron correlation manifests and interacts with the magnetic orders. In this study, we demonstrate that the recently discovered 2D antiferromagnetic material, CrSBr, is a Mott insulator, through the combined use of resonant and temperature-dependent angle-resolved photoemission spectroscopy techiniques, supplemented by dynamical mean-field theory analysis. Intriguingly, we found that as the system transitions from the antiferromagnetic to the paramagnetic phases, its Mott bands undergo a reconfiguration, and a coherent-incoherent crossover, driven by the dissolution of the magnetic order. Our findings reveal a distinctive evolution of band structure associated with magnetic phase transitions, shedding light on the investigation of intricate interplay between correlation and magnetic orders in strongly correlated van der Waals magnetic materials. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable

Authors: Martin Bertran, Shuai Tang, Michael Kearns, Jamie Morgenstern, Aaron Roth, Zhiwei Steven Wu

Abstract: Machine unlearning is motivated by desire for data autonomy: a person can request to have their data's influence removed from deployed models, and those models should be updated as if they were retrained without the person's data. We show that, counter-intuitively, these updates expose individuals to high-accuracy reconstruction attacks which allow the attacker to recover their data in its entiret… ▽ More Machine unlearning is motivated by desire for data autonomy: a person can request to have their data's influence removed from deployed models, and those models should be updated as if they were retrained without the person's data. We show that, counter-intuitively, these updates expose individuals to high-accuracy reconstruction attacks which allow the attacker to recover their data in its entirety, even when the original models are so simple that privacy risk might not otherwise have been a concern. We show how to mount a near-perfect attack on the deleted data point from linear regression models. We then generalize our attack to other loss functions and architectures, and empirically demonstrate the effectiveness of our attacks across a wide range of datasets (capturing both tabular and image data). Our work highlights that privacy risk is significant even for extremely simple model classes when individuals can request deletion of their data from the model. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Estimating before Debiasing: A Bayesian Approach to Detaching Prior Bias in Federated Semi-Supervised Learning

Authors: Guogang Zhu, Xuefeng Liu, Xinghao Wu, Shaojie Tang, Chao Tang, Jianwei Niu, Hao Su

Abstract: Federated Semi-Supervised Learning (FSSL) leverages both labeled and unlabeled data on clients to collaboratively train a model.In FSSL, the heterogeneous data can introduce prediction bias into the model, causing the model's prediction to skew towards some certain classes. Existing FSSL methods primarily tackle this issue by enhancing consistency in model parameters or outputs. However, as the mo… ▽ More Federated Semi-Supervised Learning (FSSL) leverages both labeled and unlabeled data on clients to collaboratively train a model.In FSSL, the heterogeneous data can introduce prediction bias into the model, causing the model's prediction to skew towards some certain classes. Existing FSSL methods primarily tackle this issue by enhancing consistency in model parameters or outputs. However, as the models themselves are biased, merely constraining their consistency is not sufficient to alleviate prediction bias. In this paper, we explore this bias from a Bayesian perspective and demonstrate that it principally originates from label prior bias within the training data. Building upon this insight, we propose a debiasing method for FSSL named FedDB. FedDB utilizes the Average Prediction Probability of Unlabeled Data (APP-U) to approximate the biased prior.During local training, FedDB employs APP-U to refine pseudo-labeling through Bayes' theorem, thereby significantly reducing the label prior bias. Concurrently, during the model aggregation, FedDB uses APP-U from participating clients to formulate unbiased aggregate weights, thereby effectively diminishing bias in the global model. Experimental results show that FedDB can surpass existing FSSL methods. The code is available at https://github.com/GuogangZhu/FedDB. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: Accepted by IJCAI 2024

Continuously Optimizing Radar Placement with Model Predictive Path Integrals

Authors: Michael Potter, Shuo Tang, Paul Ghanem, Milica Stojanovic, Pau Closas, Murat Akcakaya, Ben Wright, Marius Necsoiu, Deniz Erdogmus, Michael Everett, Tales Imbiriba

Abstract: Continuously optimizing sensor placement is essential for precise target localization in various military and civilian applications. While information theory has shown promise in optimizing sensor placement, many studies oversimplify sensor measurement models or neglect dynamic constraints of mobile sensors. To address these challenges, we employ a range measurement model that incorporates radar p… ▽ More Continuously optimizing sensor placement is essential for precise target localization in various military and civilian applications. While information theory has shown promise in optimizing sensor placement, many studies oversimplify sensor measurement models or neglect dynamic constraints of mobile sensors. To address these challenges, we employ a range measurement model that incorporates radar parameters and radar-target distance, coupled with Model Predictive Path Integral (MPPI) control to manage complex environmental obstacles and dynamic constraints. We compare the proposed approach against stationary radars or simplified range measurement models based on the root mean squared error (RMSE) of the Cubature Kalman Filter (CKF) estimator for the targets' state. Additionally, we visualize the evolving geometry of radars and targets over time, highlighting areas of highest measurement information gain, demonstrating the strengths of the approach. The proposed strategy outperforms stationary radars and simplified range measurement models in target localization, achieving a 38-74% reduction in mean RMSE and a 33-79% reduction in the upper tail of the 90% Highest Density Interval (HDI) over 500 Monte Carl (MC) trials across all time steps. Code will be made publicly available upon acceptance. △ Less

Submitted 29 May, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

Comments: Submitted to IEEE Aerospace and Electronic Systems

Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification

Authors: Weizhen He, Yiheng Deng, Yunfeng Yan, Feng Zhu, Yizhou Wang, Lei Bai, Qingsong Xie, Donglian Qi, Wanli Ouyang, Shixiang Tang

Abstract: Human intelligence can retrieve any person according to both visual and language descriptions. However, the current computer vision community studies specific person re-identification (ReID) tasks in different scenarios separately, which limits the applications in the real world. This paper strives to resolve this problem by proposing a novel instruct-ReID task that requires the model to retrieve… ▽ More Human intelligence can retrieve any person according to both visual and language descriptions. However, the current computer vision community studies specific person re-identification (ReID) tasks in different scenarios separately, which limits the applications in the real world. This paper strives to resolve this problem by proposing a novel instruct-ReID task that requires the model to retrieve images according to the given image or language instructions. Instruct-ReID is the first exploration of a general ReID setting, where existing 6 ReID tasks can be viewed as special cases by assigning different instructions. To facilitate research in this new instruct-ReID task, we propose a large-scale OmniReID++ benchmark equipped with diverse data and comprehensive evaluation methods e.g., task specific and task-free evaluation settings. In the task-specific evaluation setting, gallery sets are categorized according to specific ReID tasks. We propose a novel baseline model, IRM, with an adaptive triplet loss to handle various retrieval tasks within a unified framework. For task-free evaluation setting, where target person images are retrieved from task-agnostic gallery sets, we further propose a new method called IRM++ with novel memory bank-assisted learning. Extensive evaluations of IRM and IRM++ on OmniReID++ benchmark demonstrate the superiority of our proposed methods, achieving state-of-the-art performance on 10 test sets. The datasets, the model, and the code will be available at https://github.com/hwz-zju/Instruct-ReID △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2306.07520

Electron form factors in Basis Light-front Quantization

Authors: Lingdi Meng, Shuo Tang, Zhi Hu, Guo-Li Wang, Yang Li, Xingbo Zhao, James P. Vary

Abstract: In this paper, we evaluate the electromagnetic and gravitational form factors as well as the corresponding generalized parton distributions of the electron using the Basis Light-front Quantization approach to QED. We compare our results with those from light-front perturbation theory. We adopt a novel basis with its scale depending on the constituents' longitudinal momentum fraction. We show that… ▽ More In this paper, we evaluate the electromagnetic and gravitational form factors as well as the corresponding generalized parton distributions of the electron using the Basis Light-front Quantization approach to QED. We compare our results with those from light-front perturbation theory. We adopt a novel basis with its scale depending on the constituents' longitudinal momentum fraction. We show that this basis improves convergence of the form factors with increasing basis dimension, compared to that calculated in the original basis with fixed scale. These results both validate the BLFQ approach and provide guidance for its efficient implementation in solving light-front Hamiltonian mass eigenstates for more complex systems in QED and QCD. △ Less

Submitted 27 May, 2024; originally announced May 2024.

High-field magnetoelectric coupling and successive magnetic transitions in Mn-doped polar antiferromagnet Ni3TeO6

Authors: J. H. Zhang, L. Lin, C. Dong, Y. T. Chang, J. F. Wang, C. L. Lu, P. Z. Chen, W. J. Zhai, G. Z. Zhou, L. Huang, Y. S. Tang, S. H. Zheng, M. F. Liu, X. H. Zhou, Z. B. Yan, J. -M. Liu

Abstract: Among the 3d transition metal ions doped polar Ni3TeO6, Mn-doped Ni3TeO6 has stimulated great interest due to its high magnetic ordering temperature and complex magnetic phases, but the mechanism of magnetoelectric (ME) coupling is far from understood. Herein we report our systematic investigation of the chemical control of magnetism, metamagnetic transition, and ME properties of Ni3-xMnxTeO6 sing… ▽ More Among the 3d transition metal ions doped polar Ni3TeO6, Mn-doped Ni3TeO6 has stimulated great interest due to its high magnetic ordering temperature and complex magnetic phases, but the mechanism of magnetoelectric (ME) coupling is far from understood. Herein we report our systematic investigation of the chemical control of magnetism, metamagnetic transition, and ME properties of Ni3-xMnxTeO6 single crystals in high magnetic field (H) up to 52 T. We present a previously unreported weak ferromagnetic behavior appeared in the ab plane below 9.5 K in addition to the incommensurate helical and commensurate collinear antiferromagnetic states. In the low-field region, a spin-flop type metamagnetic transition without any hysteresis occurs at Hc1 for H // c, while another metamagnetic transition accompanied with a change in electric polarization is observed at Hc2 in the high-field region both for H // c and H // ab above 30 K, which can be attributed to the sudden rotation of magnetic moments at Ni2 sites. The ME measurements reveal that a first-order ME effect is observed in the low-T and low-H regions, while a second-order ME coupling term appears above 30 K in the magnetic field range of Hc1 < H < Hc2 for H // c and H < Hc2 for H // ab, both becoming significant with increasing temperature. Eventually, they are dominated by the second-order ME effect near the antiferromagnetic transition temperature. The present work demonstrates that Ni3-xMnxTeO6 is an exotic magnetoelectric material compared with Ni3TeO6 and its derivatives, thereby providing insights to better understand the magnetism and ME coupling in Ni3TeO6 and its derivatives. △ Less

Submitted 29 May, 2024; v1 submitted 24 May, 2024; originally announced May 2024.

Comments: 30 pages with 8 figures

Journal ref: Phys. Rev. B 109, 184112 (2024)

Constraints on Einstein-dilation-Gauss-Bonnet gravity and electric charge of compact binary systems from GW230529

Authors: Bo Gao, Shao-Peng Tang, Hai-Tian Wang, Jingzhi Yan, Yi-Zhong Fan

Abstract: In this work, we study the implications of GW230529 on gravity theories and the charge of black holes. The GW230529, which was initially released in O4a, is most likely neutron star-black hole (NSBH) mergers. We reanalyze the data from the GW230529 event to obtain bounds on the Einstein-dilation-Gauss-Bonnet (EdGB) gravity parameter $\sqrt{α_{\rm EdGB}}$ and the electric charge of compact binary s… ▽ More In this work, we study the implications of GW230529 on gravity theories and the charge of black holes. The GW230529, which was initially released in O4a, is most likely neutron star-black hole (NSBH) mergers. We reanalyze the data from the GW230529 event to obtain bounds on the Einstein-dilation-Gauss-Bonnet (EdGB) gravity parameter $\sqrt{α_{\rm EdGB}}$ and the electric charge of compact binary systems. The event places a $90\%$ credible upper bounds on $\sqrt{α_{\rm EdGB}}$ of $\lesssim 0.298$ km. After including high order corrections of EdGB gravity, the bounds improve to $\sqrt{α_{\rm EdGB}} \lesssim 0.260$ km. Analyses of GW230529 also yield a $90\%$ credible upper bounds on the combination of charge-to-mass ratio of the binary components $ζ\lesssim 0.024$. The constraints are more stringent than those derived from previously observed single gravitational wave merger event. △ Less

Submitted 12 July, 2024; v1 submitted 21 May, 2024; originally announced May 2024.

Comments: 7 pages, 4 figures, PRD accepted

DuetRAG: Collaborative Retrieval-Augmented Generation

Authors: Dian Jiao, Li Cai, Jingsheng Huang, Wenqiao Zhang, Siliang Tang, Yueting Zhuang

Abstract: Retrieval-Augmented Generation (RAG) methods augment the input of Large Language Models (LLMs) with relevant retrieved passages, reducing factual errors in knowledge-intensive tasks. However, contemporary RAG approaches suffer from irrelevant knowledge retrieval issues in complex domain questions (e.g., HotPot QA) due to the lack of corresponding domain knowledge, leading to low-quality generation… ▽ More Retrieval-Augmented Generation (RAG) methods augment the input of Large Language Models (LLMs) with relevant retrieved passages, reducing factual errors in knowledge-intensive tasks. However, contemporary RAG approaches suffer from irrelevant knowledge retrieval issues in complex domain questions (e.g., HotPot QA) due to the lack of corresponding domain knowledge, leading to low-quality generations. To address this issue, we propose a novel Collaborative Retrieval-Augmented Generation framework, DuetRAG. Our bootstrapping philosophy is to simultaneously integrate the domain fintuning and RAG models to improve the knowledge retrieval quality, thereby enhancing generation quality. Finally, we demonstrate DuetRAG' s matches with expert human researchers on HotPot QA. △ Less

Submitted 12 May, 2024; originally announced May 2024.

Comments: 5 pages

DisenStudio: Customized Multi-subject Text-to-Video Generation with Disentangled Spatial Control

Authors: Hong Chen, Xin Wang, Yipeng Zhang, Yuwei Zhou, Zeyang Zhang, Siao Tang, Wenwu Zhu

Abstract: Generating customized content in videos has received increasing attention recently. However, existing works primarily focus on customized text-to-video generation for single subject, suffering from subject-missing and attribute-binding problems when the video is expected to contain multiple subjects. Furthermore, existing models struggle to assign the desired actions to the corresponding subjects… ▽ More Generating customized content in videos has received increasing attention recently. However, existing works primarily focus on customized text-to-video generation for single subject, suffering from subject-missing and attribute-binding problems when the video is expected to contain multiple subjects. Furthermore, existing models struggle to assign the desired actions to the corresponding subjects (action-binding problem), failing to achieve satisfactory multi-subject generation performance. To tackle the problems, in this paper, we propose DisenStudio, a novel framework that can generate text-guided videos for customized multiple subjects, given few images for each subject. Specifically, DisenStudio enhances a pretrained diffusion-based text-to-video model with our proposed spatial-disentangled cross-attention mechanism to associate each subject with the desired action. Then the model is customized for the multiple subjects with the proposed motion-preserved disentangled finetuning, which involves three tuning strategies: multi-subject co-occurrence tuning, masked single-subject tuning, and multi-subject motion-preserved tuning. The first two strategies guarantee the subject occurrence and preserve their visual attributes, and the third strategy helps the model maintain the temporal motion-generation ability when finetuning on static images. We conduct extensive experiments to demonstrate our proposed DisenStudio significantly outperforms existing methods in various metrics. Additionally, we show that DisenStudio can be used as a powerful tool for various controllable generation applications. △ Less

Submitted 21 May, 2024; originally announced May 2024.

Revealing the Trade-off in ISAC Systems: The KL Divergence Perspective

Authors: Zesong Fei, Shuntian Tang, Xinyi Wang, Fanghao Xia, Fan Liu, J. Andrew Zhang

Abstract: Integrated sensing and communication (ISAC) is regarded as a promising technique for 6G communication network. In this letter, we investigate the Pareto bound of the ISAC system in terms of a unified Kullback-Leibler (KL) divergence performance metric. We firstly present the relationship between KL divergence and explicit ISAC performance metric, i.e., demodulation error and probability of detecti… ▽ More Integrated sensing and communication (ISAC) is regarded as a promising technique for 6G communication network. In this letter, we investigate the Pareto bound of the ISAC system in terms of a unified Kullback-Leibler (KL) divergence performance metric. We firstly present the relationship between KL divergence and explicit ISAC performance metric, i.e., demodulation error and probability of detection. Thereafter, we investigate the impact of constellation and beamforming design on the Pareto bound via deep learning and semi-definite relaxation (SDR) techniques. Simulation results show the trade-off between sensing and communication performance in terms of bit error rate (BER) and probability of detection under different parameter set-ups. △ Less

Submitted 17 May, 2024; originally announced May 2024.

Comments: 5 pages, 5 figures; submitted to IEEE journals for possible publication

Magnetic structure and magnetoelectric coupling in antiferromagnet Co5(TeO3)4Cl2

Authors: B. Yu, L. Huang, J. S. Li, L. Lin, V. Ovidiu Garlea, Q. Zhang, T. Zou, J. C. Zhang, J. Peng, Y. S. Tang, G. Z. Zhou, J. H. Zhang, S. H. Zheng, M. F. Liu, Z. B. Yan, X. H. Zhou, S. Dong, J. G. Wan, J. -M. Liu

Abstract: The van der Waals (vdW) layered multiferroics, which host simultaneous ferroelectric and magnetic orders, have attracted attention not only for their potentials to be utilized in nanoelectric devices and spintronics, but also offer alternative opportunities for emergent physical phenomena. To date, the vdW layered multiferroic materials are still very rare. In this work, we have investigated the m… ▽ More The van der Waals (vdW) layered multiferroics, which host simultaneous ferroelectric and magnetic orders, have attracted attention not only for their potentials to be utilized in nanoelectric devices and spintronics, but also offer alternative opportunities for emergent physical phenomena. To date, the vdW layered multiferroic materials are still very rare. In this work, we have investigated the magnetic structure and magnetoelectric effects in Co5(TeO3)4Cl2, a promising new multiferroic compound with antiferromagnetic (AFM) Neel point TN = 18 K. The neutron powder diffraction reveals the non-coplanar AFM state with preferred Neel vector along the c-axis, while a spin re-orientation occurring between 8 K and 15 K is identified, which results from the distinct temperature dependence of the non-equivalent Co sites moment in Co5(TeO3)4Cl2. What is more, it is found that Co5(TeO3)4Cl2 is one of the best vdW multiferroics studied so far in terms of the multiferroic performance. The measured linear ME coefficient exhibits the emergent oscillation dependence of the angle between magnetic field and electric field, and the maximal value is as big as 45 ps/m. It is suggested that Co5(TeO3)4Cl2 is an appreciated platform for exploring the emergent multiferroicity in vdW layered compounds. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Comments: 31 pages, 9 figures

Journal ref: Phys. Rev. B 109, 184106(2024)

Digging into the ultraviolet luminosity functions of galaxies at high redshifts: galaxies evolution, reionization, and cosmological parameters

Authors: Yi-Ying Wang, Lei Lei, Shao-Peng Tang, Guan-Wen Yuan, Yi-Zhong Fan

Abstract: Thanks to the successful performance of the James Webb Space Telescope, our understanding of the epoch of reionization of the Universe has been advanced. The ultraviolet luminosity functions (UV LFs) of galaxies span a wide range of redshift, not only revealing the connection between galaxies and dark matter (DM) halos but also providing the information during reionization. In this work, we develo… ▽ More Thanks to the successful performance of the James Webb Space Telescope, our understanding of the epoch of reionization of the Universe has been advanced. The ultraviolet luminosity functions (UV LFs) of galaxies span a wide range of redshift, not only revealing the connection between galaxies and dark matter (DM) halos but also providing the information during reionization. In this work, we develop a model connecting galaxy counts and apparent magnitude based on UV LFs, which incorporates redshift-dependent star formation efficiency (SFE) and corrections for dust attenuation. By synthesizing some observations across the redshift range $4\le z \le 10$ from various galaxy surveys, we discern the evolving SFE with increasing redshift and DM halo mass through model fitting. Subsequent analyses indicate that the Thomson scattering optical depth was $τ_{\rm e} = 0.052^{+0.003}_{-0.002}$ and the epoch of reionization started (ended) at $z=20.58^{+6.25}_{-6.75}$ ($z=5.38^{+0.65}_{-0.70}$) which is insensitive to the choice of the truncated magnitude of the UV LFs. Incorporating additional dataset and some reasonable constraints, the amplitude of matter perturbation is found to be $σ_8=0.79\pm0.05$, which is consistent with the standard $Λ$CDM model. Future galaxy surveys and the dynamical simulations of galaxy evolution will break the degeneracy between SFE and cosmological parameters, improving the accuracy and the precision of the UV LF model further. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Comments: 15 pages, 9 figures and 3 tables

Enhancing Image Privacy in Semantic Communication over Wiretap Channels leveraging Differential Privacy

Authors: Weixuan Chen, Shunpu Tang, Qianqian Yang

Abstract: Semantic communication (SemCom) enhances transmission efficiency by sending only task-relevant information compared to traditional methods. However, transmitting semantic-rich data over insecure or public channels poses security and privacy risks. This paper addresses the privacy problem of transmitting images over wiretap channels and proposes a novel SemCom approach ensuring privacy through a di… ▽ More Semantic communication (SemCom) enhances transmission efficiency by sending only task-relevant information compared to traditional methods. However, transmitting semantic-rich data over insecure or public channels poses security and privacy risks. This paper addresses the privacy problem of transmitting images over wiretap channels and proposes a novel SemCom approach ensuring privacy through a differential privacy (DP)-based image protection and deprotection mechanism. The method utilizes the GAN inversion technique to extract disentangled semantic features and applies a DP mechanism to protect sensitive features within the extracted semantic information. To address the non-invertibility of DP, we introduce two neural networks to approximate the DP application and removal processes, offering a privacy protection level close to that by the original DP process. Simulation results validate the effectiveness of our method in preventing eavesdroppers from obtaining sensitive information while maintaining high-fidelity image reconstruction at the legitimate receiver. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Rene: A Pre-trained Multi-modal Architecture for Auscultation of Respiratory Diseases

Authors: Pengfei Zhang, Zhihang Zheng, Shichen Zhang, Minghao Yang, Shaojun Tang

Abstract: Compared with invasive examinations that require tissue sampling, respiratory sound testing is a non-invasive examination method that is safer and easier for patients to accept. In this study, we introduce Rene, a pioneering large-scale model tailored for respiratory sound recognition. Rene has been rigorously fine-tuned with an extensive dataset featuring a broad array of respiratory audio sample… ▽ More Compared with invasive examinations that require tissue sampling, respiratory sound testing is a non-invasive examination method that is safer and easier for patients to accept. In this study, we introduce Rene, a pioneering large-scale model tailored for respiratory sound recognition. Rene has been rigorously fine-tuned with an extensive dataset featuring a broad array of respiratory audio samples, targeting disease detection, sound pattern classification, and event identification. Our innovative approach applies a pre-trained speech recognition model to process respiratory sounds, augmented with patient medical records. The resulting multi-modal deep-learning framework addresses interpretability and real-time diagnostic challenges that have hindered previous respiratory-focused models. Benchmark comparisons reveal that Rene significantly outperforms existing models, achieving improvements of 10.27%, 16.15%, 15.29%, and 18.90% in respiratory event detection and audio classification on the SPRSound database. Disease prediction accuracy on the ICBHI database improved by 23% over the baseline in both mean average and harmonic scores. Moreover, we have developed a real-time respiratory sound discrimination system utilizing the Rene architecture. Employing state-of-the-art Edge AI technology, this system enables rapid and accurate responses for respiratory sound auscultation(https://github.com/zpforlove/Rene). △ Less

Submitted 6 June, 2024; v1 submitted 12 May, 2024; originally announced May 2024.

Unraveling Anisotropic Hybridizations of Solid-state Electrolyte Nano-films in Li-ion Batteries

Authors: Yuanjie Ning, Wenjun Wu, Liang Dai, Shuo Sun, Zhigang Zeng, Dengsong Zhang, Mark B. H. Breese, Chuanbing Cai, Chi Sin Tang, Xinmao Yin

Abstract: Li2WO4 (LWO) is recognized for its potential as a solid-state electrolyte and it has demonstrated the ability to enhance the electrochemical performance of LiCoO2 (LCO) cathodes in Li-ion batteries. However, prior investigations into LWO have predominantly involved polycrystalline structures, thereby lacking a comprehensive understanding of its behavior when interfaced with single crystal systems,… ▽ More Li2WO4 (LWO) is recognized for its potential as a solid-state electrolyte and it has demonstrated the ability to enhance the electrochemical performance of LiCoO2 (LCO) cathodes in Li-ion batteries. However, prior investigations into LWO have predominantly involved polycrystalline structures, thereby lacking a comprehensive understanding of its behavior when interfaced with single crystal systems, particularly those intricately connected to LCO. In this study, we employ pulsed laser deposition (PLD) to epitaxially synthesize LWO nano-films on LCO layers with different orientations. Based on a series of high-resolution synchrotron-based techniques including X-ray absorption spectroscopy (XAS) and X-ray photoemission spectroscopy (XPS), the electronic structure of LWO is carefully scrutinized where a higher main energy level of W5d(eg)-O2p orbitals hybridization in LWO/LCO(104) as compared to LWO/LCO(003) has been observed. This experimental finding is further validated by a comprehensive set of density of states calculations. Furthermore, detailed polarized XAS characterization unveils distinct anisotropy between the two oriented LWO configurations. This comprehensive scientific investigation, harnessing the capabilities of synchrotron-based techniques, provides invaluable insights for future studies, offering guidance for the optimized utilization of LWO as a solid-state electrolyte or modification layer for LCO cathodes in high-powered Li-ion batteries. △ Less

Submitted 12 May, 2024; originally announced May 2024.

Comments: 8 pages, 5 figures