Showing 1–50 of 5,816 results for author: Liu, S

LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Models

Authors: Hai Jiang, Ao Luo, Xiaohong Liu, Songchen Han, Shuaicheng Liu

Abstract: In this paper, we propose a diffusion-based unsupervised framework that incorporates physically explainable Retinex theory with diffusion models for low-light image enhancement, named LightenDiffusion. Specifically, we present a content-transfer decomposition network that performs Retinex decomposition within the latent space instead of image space as in previous approaches, enabling the encoded f… ▽ More In this paper, we propose a diffusion-based unsupervised framework that incorporates physically explainable Retinex theory with diffusion models for low-light image enhancement, named LightenDiffusion. Specifically, we present a content-transfer decomposition network that performs Retinex decomposition within the latent space instead of image space as in previous approaches, enabling the encoded features of unpaired low-light and normal-light images to be decomposed into content-rich reflectance maps and content-free illumination maps. Subsequently, the reflectance map of the low-light image and the illumination map of the normal-light image are taken as input to the diffusion model for unsupervised restoration with the guidance of the low-light feature, where a self-constrained consistency loss is further proposed to eliminate the interference of normal-light content on the restored results to improve overall visual quality. Extensive experiments on publicly available real-world benchmarks show that the proposed LightenDiffusion outperforms state-of-the-art unsupervised competitors and is comparable to supervised methods while being more generalizable to various scenes. Our code is available at https://github.com/JianghaiSCU/LightenDiffusion. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: Accepted by ECCV 2024

Global-Local Collaborative Inference with LLM for Lidar-Based Open-Vocabulary Detection

Authors: Xingyu Peng, Yan Bai, Chen Gao, Lirong Yang, Fei Xia, Beipeng Mu, Xiaofei Wang, Si Liu

Abstract: Open-Vocabulary Detection (OVD) is the task of detecting all interesting objects in a given scene without predefined object classes. Extensive work has been done to deal with the OVD for 2D RGB images, but the exploration of 3D OVD is still limited. Intuitively, lidar point clouds provide 3D information, both object level and scene level, to generate trustful detection results. However, previous l… ▽ More Open-Vocabulary Detection (OVD) is the task of detecting all interesting objects in a given scene without predefined object classes. Extensive work has been done to deal with the OVD for 2D RGB images, but the exploration of 3D OVD is still limited. Intuitively, lidar point clouds provide 3D information, both object level and scene level, to generate trustful detection results. However, previous lidar-based OVD methods only focus on the usage of object-level features, ignoring the essence of scene-level information. In this paper, we propose a Global-Local Collaborative Scheme (GLIS) for the lidar-based OVD task, which contains a local branch to generate object-level detection result and a global branch to obtain scene-level global feature. With the global-local information, a Large Language Model (LLM) is applied for chain-of-thought inference, and the detection result can be refined accordingly. We further propose Reflected Pseudo Labels Generation (RPLG) to generate high-quality pseudo labels for supervision and Background-Aware Object Localization (BAOL) to select precise object proposals. Extensive experiments on ScanNetV2 and SUN RGB-D demonstrate the superiority of our methods. Code is released at https://github.com/GradiusTwinbee/GLIS. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: accepted by ECCV 2024

Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklist

Authors: Zihao Zhou, Shudong Liu, Maizhen Ning, Wei Liu, Jindong Wang, Derek F. Wong, Xiaowei Huang, Qiufeng Wang, Kaizhu Huang

Abstract: Exceptional mathematical reasoning ability is one of the key features that demonstrate the power of large language models (LLMs). How to comprehensively define and evaluate the mathematical abilities of LLMs, and even reflect the user experience in real-world scenarios, has emerged as a critical issue. Current benchmarks predominantly concentrate on problem-solving capabilities, which presents a s… ▽ More Exceptional mathematical reasoning ability is one of the key features that demonstrate the power of large language models (LLMs). How to comprehensively define and evaluate the mathematical abilities of LLMs, and even reflect the user experience in real-world scenarios, has emerged as a critical issue. Current benchmarks predominantly concentrate on problem-solving capabilities, which presents a substantial risk of model overfitting and fails to accurately represent genuine mathematical reasoning abilities. In this paper, we argue that if a model really understands a problem, it should be robustly and readily applied across a diverse array of tasks. Motivated by this, we introduce MATHCHECK, a well-designed checklist for testing task generalization and reasoning robustness, as well as an automatic tool to generate checklists efficiently. MATHCHECK includes multiple mathematical reasoning tasks and robustness test types to facilitate a comprehensive evaluation of both mathematical reasoning ability and behavior testing. Utilizing MATHCHECK, we develop MATHCHECK-GSM and MATHCHECK-GEO to assess mathematical textual reasoning and multi-modal reasoning capabilities, respectively, serving as upgraded versions of benchmarks including GSM8k, GeoQA, UniGeo, and Geometry3K. We adopt MATHCHECK-GSM and MATHCHECK-GEO to evaluate over 20 LLMs and 11 MLLMs, assessing their comprehensive mathematical reasoning abilities. Our results demonstrate that while frontier LLMs like GPT-4o continue to excel in various abilities on the checklist, many other model families exhibit a significant decline. Further experiments indicate that, compared to traditional math benchmarks, MATHCHECK better reflects true mathematical abilities and represents mathematical intelligence more linearly, thereby supporting our design. On our MATHCHECK, we can easily conduct detailed behavior analysis to deeply investigate models. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: 35 pages, 10 figures, preprint

Autoregressive Speech Synthesis without Vector Quantization

Authors: Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen Meng, Furu Wei

Abstract: We present MELLE, a novel continuous-valued tokens based language modeling approach for text to speech synthesis (TTS). MELLE autoregressively generates continuous mel-spectrogram frames directly from text condition, bypassing the need for vector quantization, which are originally designed for audio compression and sacrifice fidelity compared to mel-spectrograms. Specifically, (i) instead of cross… ▽ More We present MELLE, a novel continuous-valued tokens based language modeling approach for text to speech synthesis (TTS). MELLE autoregressively generates continuous mel-spectrogram frames directly from text condition, bypassing the need for vector quantization, which are originally designed for audio compression and sacrifice fidelity compared to mel-spectrograms. Specifically, (i) instead of cross-entropy loss, we apply regression loss with a proposed spectrogram flux loss function to model the probability distribution of the continuous-valued tokens. (ii) we have incorporated variational inference into MELLE to facilitate sampling mechanisms, thereby enhancing the output diversity and model robustness. Experiments demonstrate that, compared to the two-stage codec language models VALL-E and its variants, the single-stage MELLE mitigates robustness issues by avoiding the inherent flaws of sampling discrete codes, achieves superior performance across multiple metrics, and, most importantly, offers a more streamlined paradigm. See https://aka.ms/melle for demos of our work. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients

Authors: Zhenyu Zhang, Ajay Jaiswal, Lu Yin, Shiwei Liu, Jiawei Zhao, Yuandong Tian, Zhangyang Wang

Abstract: Training Large Language Models (LLMs) is memory-intensive due to the large number of parameters and associated optimization states. GaLore, a recent method, reduces memory usage by projecting weight gradients into a low-rank subspace without compromising performance. However, GaLore relies on time-consuming Singular Value Decomposition (SVD) operations to identify the subspace, and the frequent su… ▽ More Training Large Language Models (LLMs) is memory-intensive due to the large number of parameters and associated optimization states. GaLore, a recent method, reduces memory usage by projecting weight gradients into a low-rank subspace without compromising performance. However, GaLore relies on time-consuming Singular Value Decomposition (SVD) operations to identify the subspace, and the frequent subspace updates lead to significant training time overhead. Moreover, GaLore offers minimal improvements in accuracy and efficiency compared to LoRA in more accessible fine-tuning scenarios. To address these limitations, we introduce Q-Galore, a novel approach that substantially reduces memory usage by combining quantization and low-rank projection, surpassing the benefits of GaLore. Our method is based on two key observations: (i) the gradient subspace exhibits diverse properties, with some layers converging early in training while others are subject to frequent changes; (ii) the projection matrices are highly resilient to low-bit quantization. Leveraging these insights, Q-GaLore adaptively updates the gradient subspace based on its convergence statistics, achieving comparable performance while significantly reducing the number of SVD operations. We maintain the projection matrices in INT4 format and weights in INT8 format, incorporating stochastic rounding to capture accumulated gradient information. This approach enables a high-precision training trajectory using only low-precision weights. We demonstrate that Q-GaLore achieves highly competitive performance with exceptional memory efficiency. At pre-training, Q-GaLore facilitates training a LLaMA-7B model from scratch on a single NVIDIA RTX 4060 Ti with only 16 GB memory. At fine-tuning, it reduces memory consumption by up to 50% compared to LoRA and GaLore, while consistently outperforming QLoRA at the same memory cost. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Uncovering Emergent Spacetime Supersymmetry with Rydberg Atom Arrays

Authors: Chengshu Li, Shang Liu, Hanteng Wang, Wenjun Zhang, Zi-Xiang Li, Hui Zhai, Yingfei Gu

Abstract: In the zoo of emergent symmetries in quantum many-body physics, the previously unrealized emergent spacetime supersymmetry (SUSY) is particularly intriguing. Although it was known that spacetime SUSY could emerge at the (1+1)d tricritical Ising transition, an experimental realization is still absent. In this letter, we propose to realize the tricritical Ising transition with Rydberg atom arrays, t… ▽ More In the zoo of emergent symmetries in quantum many-body physics, the previously unrealized emergent spacetime supersymmetry (SUSY) is particularly intriguing. Although it was known that spacetime SUSY could emerge at the (1+1)d tricritical Ising transition, an experimental realization is still absent. In this letter, we propose to realize the tricritical Ising transition with Rydberg atom arrays, taking advantage of the reconfigurability of these systems. In such systems, the spacetime SUSY manifests itself in the respective correlation functions of a bosonic mode and its fermionic partner. However, the correlation function of the fermionic mode inevitably involves a string operator, making direct measurement challenging in the conventional setting. Here, we utilize the analog--digital hybrid nature of the Rydberg atom arrays, which can simulate a physical Hamiltonian and perform a digital quantum circuit on the same platform, to measure the correlation function of the fermionic mode. This hybridized protocol provides an experimentally feasible way to reveal the hidden structure of the spacetime SUSY that emerges at the tricritical Ising transition. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: 7 pages, 3 figures

RoLoRA: Fine-tuning Rotated Outlier-free LLMs for Effective Weight-Activation Quantization

Authors: Xijie Huang, Zechun Liu, Shih-Yang Liu, Kwang-Ting Cheng

Abstract: Low-Rank Adaptation (LoRA), as a representative Parameter-Efficient Fine-Tuning (PEFT)method, significantly enhances the training efficiency by updating only a small portion of the weights in Large Language Models (LLMs). Recently, weight-only quantization techniques have also been applied to LoRA methods to reduce the memory footprint of fine-tuning. However, applying weight-activation quantizati… ▽ More Low-Rank Adaptation (LoRA), as a representative Parameter-Efficient Fine-Tuning (PEFT)method, significantly enhances the training efficiency by updating only a small portion of the weights in Large Language Models (LLMs). Recently, weight-only quantization techniques have also been applied to LoRA methods to reduce the memory footprint of fine-tuning. However, applying weight-activation quantization to the LoRA pipeline is under-explored, and we observe substantial performance degradation primarily due to the presence of activation outliers. In this work, we propose RoLoRA, the first LoRA-based scheme for effective weight-activation quantization. RoLoRA utilizes rotation for outlier elimination and proposes rotation-aware fine-tuning to preserve the outlier-free characteristics in rotated LLMs. Experimental results show RoLoRA consistently improves low-bit LoRA convergence and post-training quantization robustness in weight-activation settings. We evaluate RoLoRA across LLaMA2-7B/13B, LLaMA3-8B models, achieving up to 29.5% absolute accuracy gain of 4-bit weight-activation quantized LLaMA2- 13B on commonsense reasoning tasks compared to LoRA baseline. We further demonstrate its effectiveness on Large Multimodal Models (LLaVA-1.5-7B). Codes are available at https://github.com/HuangOwen/RoLoRA △ Less

Submitted 10 July, 2024; originally announced July 2024.

Study of the decay and production properties of $D_{s1}(2536)$ and $D_{s2}^*(2573)$

Authors: M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere, A. Brueggemann , et al. (645 additional authors not shown)

Abstract: The $e^+e^-\rightarrow D_s^+D_{s1}(2536)^-$ and $e^+e^-\rightarrow D_s^+D^*_{s2}(2573)^-$ processes are studied using data samples collected with the BESIII detector at center-of-mass energies from 4.530 to 4.946~GeV. The absolute branching fractions of $D_{s1}(2536)^- \rightarrow \bar{D}^{*0}K^-$ and $D_{s2}^*(2573)^- \rightarrow \bar{D}^0K^-$ are measured for the first time to be… ▽ More The $e^+e^-\rightarrow D_s^+D_{s1}(2536)^-$ and $e^+e^-\rightarrow D_s^+D^*_{s2}(2573)^-$ processes are studied using data samples collected with the BESIII detector at center-of-mass energies from 4.530 to 4.946~GeV. The absolute branching fractions of $D_{s1}(2536)^- \rightarrow \bar{D}^{*0}K^-$ and $D_{s2}^*(2573)^- \rightarrow \bar{D}^0K^-$ are measured for the first time to be $(35.9\pm 4.8\pm 3.5)\%$ and $(37.4\pm 3.1\pm 4.6)\%$, respectively. The measurements are in tension with predictions based on the assumption that the $D_{s1}(2536)$ and $D_{s2}^*(2573)$ are dominated by a bare $c\bar{s}$ component. The $e^+e^-\rightarrow D_s^+D_{s1}(2536)^-$ and $e^+e^-\rightarrow D_s^+D^*_{s2}(2573)^-$ cross sections are measured, and a resonant structure at around 4.6~GeV with a width of 50~MeV is observed for the first time with a statistical significance of $15σ$ in the $e^+e^-\rightarrow D_s^+D^*_{s2}(2573)^-$ process. It could be the $Y(4626)$ found by the Belle collaboration in the $D_s^+D_{s1}(2536)^{-}$ final state, since they have similar masses and widths. There is also evidence for a structure at around 4.75~GeV in both processes. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Controllable Navigation Instruction Generation with Chain of Thought Prompting

Authors: Xianghao Kong, Jinyu Chen, Wenguan Wang, Hang Su, Xiaolin Hu, Yi Yang, Si Liu

Abstract: Instruction generation is a vital and multidisciplinary research area with broad applications. Existing instruction generation models are limited to generating instructions in a single style from a particular dataset, and the style and content of generated instructions cannot be controlled. Moreover, most existing instruction generation methods also disregard the spatial modeling of the navigation… ▽ More Instruction generation is a vital and multidisciplinary research area with broad applications. Existing instruction generation models are limited to generating instructions in a single style from a particular dataset, and the style and content of generated instructions cannot be controlled. Moreover, most existing instruction generation methods also disregard the spatial modeling of the navigation environment. Leveraging the capabilities of Large Language Models (LLMs), we propose C-Instructor, which utilizes the chain-of-thought-style prompt for style-controllable and content-controllable instruction generation. Firstly, we propose a Chain of Thought with Landmarks (CoTL) mechanism, which guides the LLM to identify key landmarks and then generate complete instructions. CoTL renders generated instructions more accessible to follow and offers greater controllability over the manipulation of landmark objects. Furthermore, we present a Spatial Topology Modeling Task to facilitate the understanding of the spatial structure of the environment. Finally, we introduce a Style-Mixed Training policy, harnessing the prior knowledge of LLMs to enable style control for instruction generation based on different prompts within a single model instance. Extensive experiments demonstrate that instructions generated by C-Instructor outperform those generated by previous methods in text metrics, navigation guidance evaluation, and user studies. △ Less

Submitted 10 July, 2024; originally announced July 2024.

Comments: ECCV 2024

Convergence and Error Estimates of A Semi-Lagrangian scheme for the Minimum Time Problem

Authors: Marianne Akian, Shanqing Liu

Abstract: We consider a semi-Lagrangian scheme for solving the minimum time problem, with a given target, and the associated eikonal type equation. We first use a discrete time deterministic optimal control problem interpretation of the time discretization scheme, and show that the discrete time value function is semiconcave under regularity assumptions on the dynamics and the boundary of target set. We est… ▽ More We consider a semi-Lagrangian scheme for solving the minimum time problem, with a given target, and the associated eikonal type equation. We first use a discrete time deterministic optimal control problem interpretation of the time discretization scheme, and show that the discrete time value function is semiconcave under regularity assumptions on the dynamics and the boundary of target set. We establish a convergence rate of order $1$ in terms of time step based on this semiconcavity property. Then, we use a discrete time stochastic optimal control interpretation of the full discretization scheme, and we establish a convergence rate of order $1$ in terms of both time and spatial steps using certain interpolation operators, under further regularity assumptions. We extend our convergence results to problems with particular state constraints. We apply our results to analyze the convergence rate and computational complexity of the fast-marching method. We also consider the multi-level fast-marching method recently introduced by the authors. △ Less

Submitted 9 July, 2024; originally announced July 2024.

A Back-End Electronics Based on Fiber Communication for Small to Medium-Scale Physics Experiments

Authors: Jianguo Liu, Yu Wang, Changqing Feng, Shubin Liu, Qian Chen

Abstract: Many small and medium-sized physics experiments are being conducted worldwide. These experiments have similar requirements for readout electronics, especially the back-end electronics. Some experiments need a trigger logic unit(TLU) to provide timing and synchronous control signals. This paper introduces a back-end electronics design for small and medium-sized physics experiments; it adopts a daug… ▽ More Many small and medium-sized physics experiments are being conducted worldwide. These experiments have similar requirements for readout electronics, especially the back-end electronics. Some experiments need a trigger logic unit(TLU) to provide timing and synchronous control signals. This paper introduces a back-end electronics design for small and medium-sized physics experiments; it adopts a daughter-motherboard structure integrated TLU function to provide greater flexibility. Different interfaces and protocols can be flexibly selected according to data bandwidth requirements. It supports 32 optical fiber interfaces based on a field-programmable gate array (FPGA) of normal IOs with 400Mbps of data bandwidth for each channel. At the same time, it supports 16 high-speed communication interfaces based on GTX port with several Gbps data bandwidth of each channel. For the TLU function, this design has 8 HDMI interfaces and one RJ45 interface to provide synchronous triggers and other control signals, and it has six analog LEMOs and four digital LEMOs to accept asynchronous signals from an external source. These design specifications can meet the needs of most small and medium-sized experiments. This set of back-end electronics has been successfully used in experiments such as PandaX-III, VLAST, and moungraphy. Moreover, it has successfully conducted beam tests with a readout of the data of VLAST detectors at CERN. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Ricci-DeTurck Flow from Initial Metric with Morrey-type Integrability Condition

Authors: Man-Chun Lee, Stephen Shang Yi Liu

Abstract: In this work, we study the short-time existence theory of Ricci-DeTurck flow starting from rough metrics which satisfy a Morrey-type integrability condition. Using the rough existence theory, we show the preservation and improvement of distributional scalar curvature lower bounds provided the singular set for such metrics is not too large. As an application, we use the Ricci flow smoothing to stud… ▽ More In this work, we study the short-time existence theory of Ricci-DeTurck flow starting from rough metrics which satisfy a Morrey-type integrability condition. Using the rough existence theory, we show the preservation and improvement of distributional scalar curvature lower bounds provided the singular set for such metrics is not too large. As an application, we use the Ricci flow smoothing to study the removable singularity in positive mass theorem under Morrey regularity conditions. Our result supplements those of Jiang-Sheng-Zhang. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Comments: 23 pages

MSC Class: 53E20

Composable Interventions for Language Models

Authors: Arinbjorn Kolbeinsson, Kyle O'Brien, Tianjin Huang, Shanghua Gao, Shiwei Liu, Jonathan Richard Schwarz, Anurag Vaidya, Faisal Mahmood, Marinka Zitnik, Tianlong Chen, Thomas Hartvigsen

Abstract: Test-time interventions for language models can enhance factual accuracy, mitigate harmful outputs, and improve model efficiency without costly retraining. But despite a flood of new methods, different types of interventions are largely developing independently. In practice, multiple interventions must be applied sequentially to the same model, yet we lack standardized ways to study how interventi… ▽ More Test-time interventions for language models can enhance factual accuracy, mitigate harmful outputs, and improve model efficiency without costly retraining. But despite a flood of new methods, different types of interventions are largely developing independently. In practice, multiple interventions must be applied sequentially to the same model, yet we lack standardized ways to study how interventions interact. We fill this gap by introducing composable interventions, a framework to study the effects of using multiple interventions on the same language models, featuring new metrics and a unified codebase. Using our framework, we conduct extensive experiments and compose popular methods from three emerging intervention categories -- Knowledge Editing, Model Compression, and Machine Unlearning. Our results from 310 different compositions uncover meaningful interactions: compression hinders editing and unlearning, composing interventions hinges on their order of application, and popular general-purpose metrics are inadequate for assessing composability. Taken together, our findings showcase clear gaps in composability, suggesting a need for new multi-objective interventions. All of our code is public: https://github.com/hartvigsen-group/composable-interventions. △ Less

Submitted 8 July, 2024; originally announced July 2024.

DFedSat: Communication-Efficient and Robust Decentralized Federated Learning for LEO Satellite Constellations

Authors: Minghao Yang, Jingjing Zhang, Shengyun Liu

Abstract: Low Earth Orbit (LEO) satellites play a crucial role in the development of 6G mobile networks and space-air-ground integrated systems. Recent advancements in space technology have empowered LEO satellites with the capability to run AI applications. However, centralized approaches, where ground stations (GSs) act as servers and satellites as clients, often encounter slow convergence and inefficienc… ▽ More Low Earth Orbit (LEO) satellites play a crucial role in the development of 6G mobile networks and space-air-ground integrated systems. Recent advancements in space technology have empowered LEO satellites with the capability to run AI applications. However, centralized approaches, where ground stations (GSs) act as servers and satellites as clients, often encounter slow convergence and inefficiencies due to intermittent connectivity between satellites and GSs. In contrast, decentralized federated learning (DFL) offers a promising alternative by facilitating direct communication between satellites (clients) via inter-satellite links (ISLs). However, inter-plane ISLs connecting satellites from different orbital planes are dynamic due to Doppler shifts and pointing limitations. This could impact model propagation and lead to slower convergence. To mitigate these issues, we propose DFedSat, a fully decentralized federated learning framework tailored for LEO satellites. DFedSat accelerates the training process by employing two adaptive mechanisms for intra-plane and inter-plane model aggregation, respectively. Furthermore, a self-compensation mechanism is integrated to enhance the robustness of inter-plane ISLs against transmission failure. Additionally, we derive the sublinear convergence rate for the non-convex case of DFedSat. Extensive experimental results demonstrate DFedSat's superiority over other DFL baselines regarding convergence rate, communication efficiency, and resilience to unreliable links. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: 13 pages, 10 figures

InverseCoder: Unleashing the Power of Instruction-Tuned Code LLMs with Inverse-Instruct

Authors: Yutong Wu, Di Huang, Wenxuan Shi, Wei Wang, Lingzhe Gao, Shihao Liu, Ziyuan Nan, Kaizhao Yuan, Rui Zhang, Xishan Zhang, Zidong Du, Qi Guo, Yewen Pu, Dawei Yin, Xing Hu, Yunji Chen

Abstract: Recent advancements in open-source code large language models (LLMs) have demonstrated remarkable coding abilities by fine-tuning on the data generated from powerful closed-source LLMs such as GPT-3.5 and GPT-4 for instruction tuning. This paper explores how to further improve an instruction-tuned code LLM by generating data from itself rather than querying closed-source LLMs. Our key observation… ▽ More Recent advancements in open-source code large language models (LLMs) have demonstrated remarkable coding abilities by fine-tuning on the data generated from powerful closed-source LLMs such as GPT-3.5 and GPT-4 for instruction tuning. This paper explores how to further improve an instruction-tuned code LLM by generating data from itself rather than querying closed-source LLMs. Our key observation is the misalignment between the translation of formal and informal languages: translating formal language (i.e., code) to informal language (i.e., natural language) is more straightforward than the reverse. Based on this observation, we propose INVERSE-INSTRUCT, which summarizes instructions from code snippets instead of the reverse. Specifically, given an instruction tuning corpus for code and the resulting instruction-tuned code LLM, we ask the code LLM to generate additional high-quality instructions for the original corpus through code summarization and self-evaluation. Then, we fine-tune the base LLM on the combination of the original corpus and the self-generated one, which yields a stronger instruction-tuned LLM. We present a series of code LLMs named InverseCoder, which surpasses the performance of the original code LLMs on a wide range of benchmarks, including Python text-to-code generation, multilingual coding, and data-science code generation. △ Less

Submitted 8 July, 2024; originally announced July 2024.

LLM-Based Open-Domain Integrated Task and Knowledge Assistants with Programmable Policies

Authors: Harshit Joshi, Shicheng Liu, James Chen, Robert Weigle, Monica S. Lam

Abstract: Programming LLM-based knowledge and task assistants that faithfully conform to developer-provided policies is challenging. These agents must retrieve and provide consistent, accurate, and relevant information to address user's queries and needs. Yet such agents generate unfounded responses ("hallucinate"). Traditional dialogue trees can only handle a limited number of conversation flows, making th… ▽ More Programming LLM-based knowledge and task assistants that faithfully conform to developer-provided policies is challenging. These agents must retrieve and provide consistent, accurate, and relevant information to address user's queries and needs. Yet such agents generate unfounded responses ("hallucinate"). Traditional dialogue trees can only handle a limited number of conversation flows, making them inherently brittle. To this end, we present KITA - a programmable framework for creating task-oriented conversational agents that are designed to handle complex user interactions. Unlike LLMs, KITA provides reliable grounded responses, with controllable agent policies through its expressive specification, KITA Worksheet. In contrast to dialog trees, it is resilient to diverse user queries, helpful with knowledge sources, and offers ease of programming policies through its declarative paradigm. Through a real-user study involving 62 participants, we show that KITA beats the GPT-4 with function calling baseline by 26.1, 22.5, and 52.4 points on execution accuracy, dialogue act accuracy, and goal completion rate, respectively. We also release 22 real-user conversations with KITA manually corrected to ensure accuracy. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: preprint

Ada-adapter:Fast Few-shot Style Personlization of Diffusion Model with Pre-trained Image Encoder

Authors: Jia Liu, Changlin Li, Qirui Sun, Jiahui Ming, Chen Fang, Jue Wang, Bing Zeng, Shuaicheng Liu

Abstract: Fine-tuning advanced diffusion models for high-quality image stylization usually requires large training datasets and substantial computational resources, hindering their practical applicability. We propose Ada-Adapter, a novel framework for few-shot style personalization of diffusion models. Ada-Adapter leverages off-the-shelf diffusion models and pre-trained image feature encoders to learn a com… ▽ More Fine-tuning advanced diffusion models for high-quality image stylization usually requires large training datasets and substantial computational resources, hindering their practical applicability. We propose Ada-Adapter, a novel framework for few-shot style personalization of diffusion models. Ada-Adapter leverages off-the-shelf diffusion models and pre-trained image feature encoders to learn a compact style representation from a limited set of source images. Our method enables efficient zero-shot style transfer utilizing a single reference image. Furthermore, with a small number of source images (three to five are sufficient) and a few minutes of fine-tuning, our method can capture intricate style details and conceptual characteristics, generating high-fidelity stylized images that align well with the provided text prompts. We demonstrate the effectiveness of our approach on various artistic styles, including flat art, 3D rendering, and logo design. Our experimental results show that Ada-Adapter outperforms existing zero-shot and few-shot stylization methods in terms of output quality, diversity, and training efficiency. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Comments: 16 pages, 11 figures

MSC Class: 68T07 ACM Class: I.4.0

16-channel Photonic Solver for Optimization Problems on a Silicon Chip

Authors: Jiayi Ouyang, Shengping Liu, Ziyue Yang, Wei Wang, Xue Feng, Yongzhuo Li, Yidong Huang

Abstract: In this article, we proposed a programmable 16-channel photonic solver for quadratic unconstrained binary optimization (QUBO) problems. The solver is based on a hybrid optoelectronic scheme including a photonic chip and the corresponding electronic driving circuit. The photonic chip is fabricated on silicon on insulator (SOI) substrate and integrates high-speed electro-optic modulators, thermo-opt… ▽ More In this article, we proposed a programmable 16-channel photonic solver for quadratic unconstrained binary optimization (QUBO) problems. The solver is based on a hybrid optoelectronic scheme including a photonic chip and the corresponding electronic driving circuit. The photonic chip is fabricated on silicon on insulator (SOI) substrate and integrates high-speed electro-optic modulators, thermo-optic phase shifters and photodetectors to conduct the 16-dimensional optical vector-matrix multiplication (OVMM). Due to the parallel and low latency propagation of lightwave, the calculation of the QUBO cost function can be accelerated. Besides, the electronic processor is employed to run the heuristic algorithm to search the optimal solution. In the experiment, two 16-dimensional randomly generated QUBO problems are solved with high successful probabilities. To our knowledge, it is the largest scale of programmable and on-chip photonic solver ever reported. Moreover, the computing speed of the OVMM on photonic chip is ~2 TFLOP/s. It shows the potential of fast solving such optimization problems with integrated photonic systems. △ Less

Submitted 5 June, 2024; originally announced July 2024.

Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition

Authors: Ye Bai, Jingping Chen, Jitong Chen, Wei Chen, Zhuo Chen, Chuang Ding, Linhao Dong, Qianqian Dong, Yujiao Du, Kepan Gao, Lu Gao, Yi Guo, Minglun Han, Ting Han, Wenchao Hu, Xinying Hu, Yuxiang Hu, Deyu Hua, Lu Huang, Mingkun Huang, Youjia Huang, Jishuo Jin, Fanliu Kong, Zongwei Lan, Tianyu Li , et al. (30 additional authors not shown)

Abstract: Modern automatic speech recognition (ASR) model is required to accurately transcribe diverse speech signals (from different domains, languages, accents, etc) given the specific contextual information in various application scenarios. Classic end-to-end models fused with extra language models perform well, but mainly in data matching scenarios and are gradually approaching a bottleneck. In this wor… ▽ More Modern automatic speech recognition (ASR) model is required to accurately transcribe diverse speech signals (from different domains, languages, accents, etc) given the specific contextual information in various application scenarios. Classic end-to-end models fused with extra language models perform well, but mainly in data matching scenarios and are gradually approaching a bottleneck. In this work, we introduce Seed-ASR, a large language model (LLM) based speech recognition model. Seed-ASR is developed based on the framework of audio conditioned LLM (AcLLM), leveraging the capabilities of LLMs by inputting continuous speech representations together with contextual information into the LLM. Through stage-wise large-scale training and the elicitation of context-aware capabilities in LLM, Seed-ASR demonstrates significant improvement over end-to-end models on comprehensive evaluation sets, including multiple domains, accents/dialects and languages. Additionally, Seed-ASR can be further deployed to support specific needs in various scenarios without requiring extra language models. Compared to recently released large ASR models, Seed-ASR achieves 10%-40% reduction in word (or character, for Chinese) error rates on Chinese and English public test sets, further demonstrating its powerful performance. △ Less

Submitted 10 July, 2024; v1 submitted 5 July, 2024; originally announced July 2024.

Encoding of algebraic geometry codes with quasi-linear complexity $O(N\log N)$

Authors: Songsong Li, Shu Liu, Liming Ma, Yunqi Wan, Chaoping Xing

Abstract: Fast encoding and decoding of codes have been always an important topic in code theory as well as complexity theory. Although encoding is easier than decoding in general, designing an encoding algorithm of codes of length $N$ with quasi-linear complexity $O(N\log N)$ is not an easy task. Despite the fact that algebraic geometry codes were discovered in the early of 1980s, encoding algorithms of al… ▽ More Fast encoding and decoding of codes have been always an important topic in code theory as well as complexity theory. Although encoding is easier than decoding in general, designing an encoding algorithm of codes of length $N$ with quasi-linear complexity $O(N\log N)$ is not an easy task. Despite the fact that algebraic geometry codes were discovered in the early of 1980s, encoding algorithms of algebraic geometry codes with quasi-linear complexity $O(N\log N)$ have not been found except for the simplest algebraic geometry codes--Reed-Solomon codes. The best-known encoding algorithm of algebraic geometry codes based on a class of plane curves has quasi-linear complexity at least $O(N\log^2 N)$. In this paper, we design an encoding algorithm of algebraic geometry codes with quasi-linear complexity $O(N\log N)$. Our algorithm works well for a large class of algebraic geometry codes based on both plane and non-plane curves. The main idea of this paper is to generalize the divide-and-conquer method from the fast Fourier Transform over finite fields to algebraic curves. Suppose we consider encoding of algebraic geometry codes based on an algebraic curve ${\mathcal X}$ over $\mathbb{F}_q$. We first consider a tower of Galois coverings ${\mathcal X}={\mathcal X}_0\rightarrow{\mathcal X}_1\rightarrow\cdots\rightarrow{\mathcal X}_r$ over a finite field $\mathbb{F}_q$, i.e., their function field tower $\mathbb{F}_q({\mathcal X}_0)\supsetneq\mathbb{F}_q({\mathcal X}_{1})\supsetneq\cdots \supsetneq\mathbb{F}_q({\mathcal X}_r)$ satisfies that each of extension $\mathbb{F}_q({\mathcal X}_{i-1})/\mathbb{F}_q({\mathcal X}_i)$ is a Galois extension and the extension degree $[\mathbb{F}_q({\mathcal X}_{i-1}):\mathbb{F}_q({\mathcal X}_i)]$ {is a constant}. Then encoding of an algebraic geometry code based on ${\mathcal X}$ is reduced to the encoding of an algebraic geometry code based on ${\mathcal X}_r$. △ Less

Submitted 5 July, 2024; originally announced July 2024.

VCD-Texture: Variance Alignment based 3D-2D Co-Denoising for Text-Guided Texturing

Authors: Shang Liu, Chaohui Yu, Chenjie Cao, Wen Qian, Fan Wang

Abstract: Recent research on texture synthesis for 3D shapes benefits a lot from dramatically developed 2D text-to-image diffusion models, including inpainting-based and optimization-based approaches. However, these methods ignore the modal gap between the 2D diffusion model and 3D objects, which primarily render 3D objects into 2D images and texture each image separately. In this paper, we revisit the text… ▽ More Recent research on texture synthesis for 3D shapes benefits a lot from dramatically developed 2D text-to-image diffusion models, including inpainting-based and optimization-based approaches. However, these methods ignore the modal gap between the 2D diffusion model and 3D objects, which primarily render 3D objects into 2D images and texture each image separately. In this paper, we revisit the texture synthesis and propose a Variance alignment based 3D-2D Collaborative Denoising framework, dubbed VCD-Texture, to address these issues. Formally, we first unify both 2D and 3D latent feature learning in diffusion self-attention modules with re-projected 3D attention receptive fields. Subsequently, the denoised multi-view 2D latent features are aggregated into 3D space and then rasterized back to formulate more consistent 2D predictions. However, the rasterization process suffers from an intractable variance bias, which is theoretically addressed by the proposed variance alignment, achieving high-fidelity texture synthesis. Moreover, we present an inpainting refinement to further improve the details with conflicting regions. Notably, there is not a publicly available benchmark to evaluate texture synthesis, which hinders its development. Thus we construct a new evaluation set built upon three open-source 3D datasets and propose to use four metrics to thoroughly validate the texturing performance. Comprehensive experiments demonstrate that VCD-Texture achieves superior performance against other counterparts. △ Less

Submitted 5 July, 2024; originally announced July 2024.

Comments: ECCV 2024

Corki: Enabling Real-time Embodied AI Robots via Algorithm-Architecture Co-Design

Authors: Yiyang Huang, Yuhui Hao, Bo Yu, Feng Yan, Yuxin Yang, Feng Min, Yinhe Han, Lin Ma, Shaoshan Liu, Qiang Liu, Yiming Gan

Abstract: Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture. Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate. In particular, today's computing systems for embodied AI robots are designed purely based on the interest of algorithm developers, where robot act… ▽ More Embodied AI robots have the potential to fundamentally improve the way human beings live and manufacture. Continued progress in the burgeoning field of using large language models to control robots depends critically on an efficient computing substrate. In particular, today's computing systems for embodied AI robots are designed purely based on the interest of algorithm developers, where robot actions are divided into a discrete frame-basis. Such an execution pipeline creates high latency and energy consumption. This paper proposes Corki, an algorithm-architecture co-design framework for real-time embodied AI robot control. Our idea is to decouple LLM inference, robotic control and data communication in the embodied AI robots compute pipeline. Instead of predicting action for one single frame, Corki predicts the trajectory for the near future to reduce the frequency of LLM inference. The algorithm is coupled with a hardware that accelerates transforming trajectory into actual torque signals used to control robots and an execution pipeline that parallels data communication with computation. Corki largely reduces LLM inference frequency by up to 8.0x, resulting in up to 3.6x speed up. The success rate improvement can be up to 17.3%. Code is provided for re-implementation. https://github.com/hyy0613/Corki △ Less

Submitted 5 July, 2024; originally announced July 2024.

Surviving in the Hot Neptune Desert: The Discovery of the Ultra-Hot Neptune TOI-3261b

Authors: Emma Nabbie, Chelsea X. Huang, Jennifer A. Burt, David J. Armstrong, Eric E. Mamajek, Vardan Adibekyan, Sérgio G. Sousa, Eric D. Lopez, Daniel P. Thorngren, Jorge Fernández, Gongjie Li, James S. Jenkins, Jose I. Vines, João Gomes da Silva, Robert A. Wittenmyer, Daniel Bayliss, César Briceño, Karen A. Collins, Xavier Dumusque, Keith D. Horne, Marcelo F. Keniger, Nicholas Law, Jorge Lillo-Box, Shang-Fei Liu, Andrew W. Mann , et al. (23 additional authors not shown)

Abstract: The recent discoveries of Neptune-sized ultra-short period planets (USPs) challenge existing planet formation theories. It is unclear whether these residents of the Hot Neptune Desert have similar origins to smaller, rocky USPs, or if this discrete population is evidence of a different formation pathway altogether. We report the discovery of TOI-3261b, an ultra-hot Neptune with an orbital period… ▽ More The recent discoveries of Neptune-sized ultra-short period planets (USPs) challenge existing planet formation theories. It is unclear whether these residents of the Hot Neptune Desert have similar origins to smaller, rocky USPs, or if this discrete population is evidence of a different formation pathway altogether. We report the discovery of TOI-3261b, an ultra-hot Neptune with an orbital period $P$ = 0.88 days. The host star is a $V = 13.2$ magnitude, slightly super-solar metallicity ([Fe/H] $\simeq$ 0.15), inactive K1.5 main sequence star at $d = 300$ pc. Using data from the Transiting Exoplanet Survey Satellite and the Las Cumbres Observatory Global Telescope, we find that TOI-3261b has a radius of $3.82_{-0.35}^{+0.42}$ $R_{\oplus}$. Moreover, radial velocities from ESPRESSO and HARPS reveal a mass of $30.3_{-2.4}^{+2.2}$ $M_{\oplus}$, more than twice the median mass of Neptune-sized planets on longer orbits. We investigate multiple mechanisms of mass loss that can reproduce the current-day properties of TOI-3261b, simulating the evolution of the planet via tidal stripping and photoevaporation. Thermal evolution models suggest that TOI-3261b should retain an envelope potentially enriched with volatiles constituting $\sim$5% of its total mass. This is the second highest envelope mass fraction among ultra-hot Neptunes discovered to date, making TOI-3261b an ideal candidate for atmospheric follow-up observations. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Comments: 20 pages, 11 figures, accepted to AJ

HAF-RM: A Hybrid Alignment Framework for Reward Model Training

Authors: Shujun Liu, Xiaoyu Shen, Yuhang Lai, Siyuan Wang, Shengbin Yue, Zengfeng Huang, Xuanjing Huang, Zhongyu Wei

Abstract: The reward model has become increasingly important in alignment, assessment, and data construction for large language models (LLMs). Most existing researchers focus on enhancing reward models through data improvements, following the conventional training framework for reward models that directly optimizes the predicted rewards. In this paper, we propose a hybrid alignment framework HaF-RM for rewa… ▽ More The reward model has become increasingly important in alignment, assessment, and data construction for large language models (LLMs). Most existing researchers focus on enhancing reward models through data improvements, following the conventional training framework for reward models that directly optimizes the predicted rewards. In this paper, we propose a hybrid alignment framework HaF-RM for reward model training by introducing an additional constraint on token-level policy probabilities in addition to the reward score. It can simultaneously supervise the internal preference model at the token level and optimize the mapping layer of the reward model at the sequence level. Theoretical justifications and experiment results on five datasets show the validity and effectiveness of our proposed hybrid framework for training a high-quality reward model. By decoupling the reward modeling procedure and incorporating hybrid supervision, our HaF-RM framework offers a principled and effective approach to enhancing the performance and alignment of reward models, a critical component in the responsible development of powerful language models. We release our code at https://haf-rm.github.io. △ Less

Submitted 11 July, 2024; v1 submitted 4 July, 2024; originally announced July 2024.

TALENT: A Tabular Analytics and Learning Toolbox

Authors: Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, Han-Jia Ye

Abstract: Tabular data is one of the most common data sources in machine learning. Although a wide range of classical methods demonstrate practical utilities in this field, deep learning methods on tabular data are becoming promising alternatives due to their flexibility and ability to capture complex interactions within the data. Considering that deep tabular methods have diverse design philosophies, inclu… ▽ More Tabular data is one of the most common data sources in machine learning. Although a wide range of classical methods demonstrate practical utilities in this field, deep learning methods on tabular data are becoming promising alternatives due to their flexibility and ability to capture complex interactions within the data. Considering that deep tabular methods have diverse design philosophies, including the ways they handle features, design learning objectives, and construct model architectures, we introduce a versatile deep-learning toolbox called TALENT (Tabular Analytics and LEarNing Toolbox) to utilize, analyze, and compare tabular methods. TALENT encompasses an extensive collection of more than 20 deep tabular prediction methods, associated with various encoding and normalization modules, and provides a unified interface that is easily integrable with new methods as they emerge. In this paper, we present the design and functionality of the toolbox, illustrate its practical application through several case studies, and investigate the performance of various methods fairly based on our toolbox. Code is available at https://github.com/qile2000/LAMDA-TALENT. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Perception-Guided Quality Metric of 3D Point Clouds Using Hybrid Strategy

Authors: Yujie Zhang, Qi Yang, Yiling Xu, Shan Liu

Abstract: Full-reference point cloud quality assessment (FR-PCQA) aims to infer the quality of distorted point clouds with available references. Most of the existing FR-PCQA metrics ignore the fact that the human visual system (HVS) dynamically tackles visual information according to different distortion levels (i.e., distortion detection for high-quality samples and appearance perception for low-quality sa… ▽ More Full-reference point cloud quality assessment (FR-PCQA) aims to infer the quality of distorted point clouds with available references. Most of the existing FR-PCQA metrics ignore the fact that the human visual system (HVS) dynamically tackles visual information according to different distortion levels (i.e., distortion detection for high-quality samples and appearance perception for low-quality samples) and measure point cloud quality using unified features. To bridge the gap, in this paper, we propose a perception-guided hybrid metric (PHM) that adaptively leverages two visual strategies with respect to distortion degree to predict point cloud quality: to measure visible difference in high-quality samples, PHM takes into account the masking effect and employs texture complexity as an effective compensatory factor for absolute difference; on the other hand, PHM leverages spectral graph theory to evaluate appearance degradation in low-quality samples. Variations in geometric signals on graphs and changes in the spectral graph wavelet coefficients are utilized to characterize geometry and texture appearance degradation, respectively. Finally, the results obtained from the two components are combined in a non-linear method to produce an overall quality score of the tested point cloud. The results of the experiment on five independent databases show that PHM achieves state-of-the-art (SOTA) performance and offers significant performance improvement in multiple distortion environments. The code is publicly available at https://github.com/zhangyujie-1998/PHM. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Charmless decays of the spin-2 partner of $X(3872)$

Authors: Zu-Xin Cai, Zhao-Sai Jia, Gang Li, Shi-Dong Liu, Ju-Jun Xie

Abstract: The Belle collaboration recently reported a promising candidate for the spin-2 $D^*\bar{D}^*$ partner of the $X(3872)$, called the $X_2$ for short, having a mass of $(4014.3 \pm 4.0 \pm 1.5)~\mathrm{MeV}$ and a width of $(4 \pm 11 \pm 6)~\mathrm{MeV} $. In present work, we assume the $X_2$ as a pure molecule of the $D^*\bar{D}^*$ under three cases, i.e., pure neutral components ($θ= 0$), isospin s… ▽ More The Belle collaboration recently reported a promising candidate for the spin-2 $D^*\bar{D}^*$ partner of the $X(3872)$, called the $X_2$ for short, having a mass of $(4014.3 \pm 4.0 \pm 1.5)~\mathrm{MeV}$ and a width of $(4 \pm 11 \pm 6)~\mathrm{MeV} $. In present work, we assume the $X_2$ as a pure molecule of the $D^*\bar{D}^*$ under three cases, i.e., pure neutral components ($θ= 0$), isospin singlet ($θ= π/4$) and neutral components dominant ($θ= π/6$), where $θ$ is a phase angle describing the proportion of neutral and charged constituents. Using an effective Lagrangian approach, we calculated the partial widths of $X_2\to VV$ and $X_2 \to PP$ ($V$ and $P$ stand for light vector and pseudoscalar mesons, respectively). The predicted decay widths of $X_2 \to VV$ can reach a few hundreds of $\mathrm{keV}$, while the decay widths of $X_2 \to PP$ are about several tens of $\mathrm{keV}$. In addition, the effects from the proportion of neutral and charged constituent on the decay widths of $X_2\to VV$ and $PP$ are also investigated. We hope that the present calculations will be checked experimentally in the future. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Comments: 8 pages, 7figure, comments welcome

Submillimeter and Mid-Infrared Variability of Young Stellar Objects in the M17SWex Intermediate-Mass Star-Forming Region

Authors: Geumsook Park, Doug Johnstone, Carlos Contreras Pena, Jeong-Eun Lee, Sheng-Yuan Liu, Gregory Herczeg, Steve Mairs, Zhiwei Chen, Jennifer Hatchell, Kee-Tae Kim, Mi-Ryang Kim, Keping Qiu, Yao-Te Wang, Xu Zhang, The JCMT Transient Team

Abstract: We present a comprehensive analysis of young stellar object (YSO) variability within the M17 Southwest Extension (M17 SWex), using 3.5 years of monitoring data from the JCMT Transient Survey at sub-millimeter (sub-mm) and 9 years from the NEOWISE mission at mid-infrared (mid-IR). Our study encompasses observations of 147 bright sub-mm peaks identified within our deep JCMT co-added map as well as 1… ▽ More We present a comprehensive analysis of young stellar object (YSO) variability within the M17 Southwest Extension (M17 SWex), using 3.5 years of monitoring data from the JCMT Transient Survey at sub-millimeter (sub-mm) and 9 years from the NEOWISE mission at mid-infrared (mid-IR). Our study encompasses observations of 147 bright sub-mm peaks identified within our deep JCMT co-added map as well as 156 YSOs in NEOWISE W1 and 179 in W2 that were previously identified in Spitzer surveys. We find three robust sub-mm variables: two are candidate YSOs and one is a likely extragalactic source. At mid-IR wavelengths, our analysis reveals secular and stochastic variability in 47 YSOs, with the highest fraction of secular variability occurring at the earliest evolutionary stage. This is similar to what has previously been observed for low-mass YSO variability within the Gould Belt. However, we observe less overall variability in M17SWex at both the sub-mm and mid-IR. We suspect that this lower fraction is due to the greater distance to M17 SWex. Our findings showcase the utility of multi-wavelength observations to better capture the complex variability phenomena inherent to star formation processes and demonstrate the importance of years-long monitoring of a diverse selection of star-forming environments. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Comments: Accepted for Publication in The Astronomical Journal

JailbreakHunter: A Visual Analytics Approach for Jailbreak Prompts Discovery from Large-Scale Human-LLM Conversational Datasets

Authors: Zhihua Jin, Shiyi Liu, Haotian Li, Xun Zhao, Huamin Qu

Abstract: Large Language Models (LLMs) have gained significant attention but also raised concerns due to the risk of misuse. Jailbreak prompts, a popular type of adversarial attack towards LLMs, have appeared and constantly evolved to breach the safety protocols of LLMs. To address this issue, LLMs are regularly updated with safety patches based on reported jailbreak prompts. However, malicious users often… ▽ More Large Language Models (LLMs) have gained significant attention but also raised concerns due to the risk of misuse. Jailbreak prompts, a popular type of adversarial attack towards LLMs, have appeared and constantly evolved to breach the safety protocols of LLMs. To address this issue, LLMs are regularly updated with safety patches based on reported jailbreak prompts. However, malicious users often keep their successful jailbreak prompts private to exploit LLMs. To uncover these private jailbreak prompts, extensive analysis of large-scale conversational datasets is necessary to identify prompts that still manage to bypass the system's defenses. This task is highly challenging due to the immense volume of conversation data, diverse characteristics of jailbreak prompts, and their presence in complex multi-turn conversations. To tackle these challenges, we introduce JailbreakHunter, a visual analytics approach for identifying jailbreak prompts in large-scale human-LLM conversational datasets. We have designed a workflow with three analysis levels: group-level, conversation-level, and turn-level. Group-level analysis enables users to grasp the distribution of conversations and identify suspicious conversations using multiple criteria, such as similarity with reported jailbreak prompts in previous research and attack success rates. Conversation-level analysis facilitates the understanding of the progress of conversations and helps discover jailbreak prompts within their conversation contexts. Turn-level analysis allows users to explore the semantic similarity and token overlap between a singleturn prompt and the reported jailbreak prompts, aiding in the identification of new jailbreak strategies. The effectiveness and usability of the system were verified through multiple case studies and expert interviews. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Comments: 18 pages, 9 figures

Single Image Rolling Shutter Removal with Diffusion Models

Authors: Zhanglei Yang, Haipeng Li, Mingbo Hong, Bing Zeng, Shuaicheng Liu

Abstract: We present RS-Diffusion, the first Diffusion Models-based method for single-frame Rolling Shutter (RS) correction. RS artifacts compromise visual quality of frames due to the row wise exposure of CMOS sensors. Most previous methods have focused on multi-frame approaches, using temporal information from consecutive frames for the motion rectification. However, few approaches address the more challe… ▽ More We present RS-Diffusion, the first Diffusion Models-based method for single-frame Rolling Shutter (RS) correction. RS artifacts compromise visual quality of frames due to the row wise exposure of CMOS sensors. Most previous methods have focused on multi-frame approaches, using temporal information from consecutive frames for the motion rectification. However, few approaches address the more challenging but important single frame RS correction. In this work, we present an ``image-to-motion'' framework via diffusion techniques, with a designed patch-attention module. In addition, we present the RS-Real dataset, comprised of captured RS frames alongside their corresponding Global Shutter (GS) ground-truth pairs. The GS frames are corrected from the RS ones, guided by the corresponding Inertial Measurement Unit (IMU) gyroscope data acquired during capture. Experiments show that our RS-Diffusion surpasses previous single RS correction methods. Our method and proposed RS-Real dataset lay a solid foundation for advancing the field of RS correction. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Measurement of the branching fraction of the decay $J/ψ\to p \bar{p} η$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (639 additional authors not shown)

Abstract: A high precision measurement of the branching fraction of the decay $J/ψ\to p \bar{p} η$ is performed using $(10 087 \pm 44) \times 10^6$ $J/ψ$ events recorded by the {BESIII} detector at the {BEPCII} storage ring. The branching fractions of the two decays $J/ψ\to p \bar{p} η(η\to γγ)$ and $J/ψ\to p \bar{p} η(η\to π^+ π^- π^0)$ are measured individually to be… ▽ More A high precision measurement of the branching fraction of the decay $J/ψ\to p \bar{p} η$ is performed using $(10 087 \pm 44) \times 10^6$ $J/ψ$ events recorded by the {BESIII} detector at the {BEPCII} storage ring. The branching fractions of the two decays $J/ψ\to p \bar{p} η(η\to γγ)$ and $J/ψ\to p \bar{p} η(η\to π^+ π^- π^0)$ are measured individually to be $\mathcal{B}(J/ψ\to p \bar{p} η(η\to γγ)) = (1.480 \pm 0.001 \pm 0.024)\times\,10^{-3}$ and $\mathcal{B}(J/ψ\to p \bar{p} η(η\to π^+ π^- π^0)) = (1.557 \pm 0.003 \pm 0.038)\times\,10^{-3}$, where the first uncertainties are statistical and the second systematic. Both results are compatible within their uncorrelated systematic uncertainties. The combined result is $\mathcal{B}(J/ψ\to p \bar{p} η)=(1.495 \pm 0.001 \pm 0.023)\times\,10^{-3}$ where the first uncertainty is the combined statistical uncertainty and the second one the combined systematic uncertainty of both analyses, incorporating correlations between them. In addition, the $p \bar{p}$ threshold region is investigated for a potential threshold enhancement, and no evidence for one is observed. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Origin of Interstitial Doping Induced Coercive Field Reduction in Ferroelectric Hafnia

Authors: Tianyuan Zhu, Liyang Ma, Xu Duan, Shi Liu

Abstract: Hafnia-based ferroelectrics hold promise for nonvolatile ferroelectric memory devices. However, the high coercive field required for polarization switching remains a prime obstacle to their practical applications. A notable reduction in coercive field has been achieved in ferroelectric Hf(Zr)$_{1+x}$O$_2$ films with interstitial Hf(Zr) dopants [Science 381, 558 (2023)], suggesting a less-explored… ▽ More Hafnia-based ferroelectrics hold promise for nonvolatile ferroelectric memory devices. However, the high coercive field required for polarization switching remains a prime obstacle to their practical applications. A notable reduction in coercive field has been achieved in ferroelectric Hf(Zr)$_{1+x}$O$_2$ films with interstitial Hf(Zr) dopants [Science 381, 558 (2023)], suggesting a less-explored strategy for coercive field optimization. Supported by density functional theory calculations, we demonstrate the $Pca2_1$ phase, with a moderate concentration of interstitial Hf dopants, serves as a minimal model to explain the experimental observations, rather than the originally assumed rhombohedral phase. Large-scale deep potential molecular dynamics simulations suggest that interstitial defects promote the polarization reversal by facilitating $Pbcn$-like mobile 180$^\circ$ domain walls. A simple pre-poling treatment could reduce the switching field to less than 1 MV/cm and enable switching on a subnanosecond timescale. High-throughput calculations reveal a negative correlation between the switching barrier and dopant size and identify a few promising interstitial dopants for coercive field reduction. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Comparison of Short-Range Order in GeSn Grown by Molecular Beam Epitaxy and Chemical Vapor Deposition

Authors: Shang Liu, Yunfan Liang, Haochen Zhao, Nirosh M. Eldose, Jin-Hee Bae, Omar Concepcion, Xiaochen Jin, Shunda Chen, Ilias Bikmukhametov, Austin Akey, Cory T. Cline, Alejandra Cuervo Covian, Xiaoxin Wang, Tianshu Li, Yuping Zeng, Dan Buca, Shui-Qing Yu, Gregory J. Salamo, Shengbai Zhang, Jifeng Liu

Abstract: Atomic short-range order (SRO) in direct-bandgap GeSn for infrared photonics has recently attracted attention due to its notable impact on band structures. However, the SRO in GeSn thin films grown by different methods have hardly been compared. This paper compares SRO in GeSn thin films of similar compositions grown by molecular beam epitaxy (MBE) and chemical vapor deposition (CVD) using atom pr… ▽ More Atomic short-range order (SRO) in direct-bandgap GeSn for infrared photonics has recently attracted attention due to its notable impact on band structures. However, the SRO in GeSn thin films grown by different methods have hardly been compared. This paper compares SRO in GeSn thin films of similar compositions grown by molecular beam epitaxy (MBE) and chemical vapor deposition (CVD) using atom probe tomography. An $\sim$15% stronger preference for Sn-Sn 1$^{st}$ nearest neighbor (1NN) is observed in MBE GeSn than CVD GeSn for both thin film and quantum well samples with Sn composition ranging from 7 to 20%. Interestingly, samples grown by different deposition tools under the same method (either MBE or CVD) showed remarkable consistency in Sn-Sn 1NN SRO, while MBE vs. CVD showed clear differences. Supported by theoretical modeling, we consider that this difference in SRO originates from the impact of surface termination, where MBE surfaces are exposed to ultrahigh vacuum while CVD surfaces are terminated by H to a good extent. This finding not only suggests engineering surface termination or surfactants during the growth as a potential approach to control SRO in GeSn, but also provides insight into the underlying reasons for very different growth temperature between MBE and CVD that directly impact the strain relaxation behavior. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Multi-Scenario Combination Based on Multi-Agent Reinforcement Learning to Optimize the Advertising Recommendation System

Authors: Yang Zhao, Chang Zhou, Jin Cao, Yi Zhao, Shaobo Liu, Chiyu Cheng, Xingchen Li

Abstract: This paper explores multi-scenario optimization on large platforms using multi-agent reinforcement learning (MARL). We address this by treating scenarios like search, recommendation, and advertising as a cooperative, partially observable multi-agent decision problem. We introduce the Multi-Agent Recurrent Deterministic Policy Gradient (MARDPG) algorithm, which aligns different scenarios under a sh… ▽ More This paper explores multi-scenario optimization on large platforms using multi-agent reinforcement learning (MARL). We address this by treating scenarios like search, recommendation, and advertising as a cooperative, partially observable multi-agent decision problem. We introduce the Multi-Agent Recurrent Deterministic Policy Gradient (MARDPG) algorithm, which aligns different scenarios under a shared objective and allows for strategy communication to boost overall performance. Our results show marked improvements in metrics such as click-through rate (CTR), conversion rate, and total sales, confirming our method's efficacy in practical settings. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: Accepted by 2024 5th International Conference on Artificial Intelligence and Electromechanical Automation IEEE (ISBN: 979-8-3503-6617-4)

Evolution of High-energy Electron Distribution in Pulsar Wind Nebulae

Authors: Yi-Ming Liu, Hou-Dun Zeng, Yu-Liang Xin, Si-Ming Liu, Yi Zhang

Abstract: In this paper, we analyze the spectral energy distributions (SEDs) of 17 powerful (with a spin-down luminosity greater than $10^{35}$ erg s$^{-1}$) young (with an age less than 15000 yrs) pulsar wind nebulae (PWNe) using a simple time-independent one-zone emission model. Our aim is to investigate correlations between model parameters and the ages of the corresponding PWNe, thereby revealing the ev… ▽ More In this paper, we analyze the spectral energy distributions (SEDs) of 17 powerful (with a spin-down luminosity greater than $10^{35}$ erg s$^{-1}$) young (with an age less than 15000 yrs) pulsar wind nebulae (PWNe) using a simple time-independent one-zone emission model. Our aim is to investigate correlations between model parameters and the ages of the corresponding PWNe, thereby revealing the evolution of high-energy electron distributions within PWNe. Our findings are as follows: (1) The electron distributions in PWNe can be characterized by a double power-law with a superexponential cutoff; (2) As PWNe evolve, the high-energy end of the electron distribution spectrum becomes harder with the index decreasing from approximately 3.5 to 2.5, while the low-energy end spectrum index remains constant near 1.5; (3) There is no apparent correlation between the break energy or cutoff energy and the age of PWNe. (4) The average magnetic field within PWNe decreases with age, leading to a positive correlation between the energy loss timescale of electrons at the break energy or the high-energy cutoff, and the age of the PWN. (5) The total electron energy within PWNe remains constant near $2 \times 10^{48}$ erg, while the total magnetic energy decreases with age. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: 19 pages, 20 figures, 1 table accepted for publication in RAA

MMedAgent: Learning to Use Medical Tools with Multi-modal Agent

Authors: Binxu Li, Tiankai Yan, Yuanting Pan, Zhe Xu, Jie Luo, Ruiyang Ji, Shilong Liu, Haoyu Dong, Zihao Lin, Yixin Wang

Abstract: Multi-Modal Large Language Models (MLLMs), despite being successful, exhibit limited generality and often fall short when compared to specialized models. Recently, LLM-based agents have been developed to address these challenges by selecting appropriate specialized models as tools based on user inputs. However, such advancements have not been extensively explored within the medical domain. To brid… ▽ More Multi-Modal Large Language Models (MLLMs), despite being successful, exhibit limited generality and often fall short when compared to specialized models. Recently, LLM-based agents have been developed to address these challenges by selecting appropriate specialized models as tools based on user inputs. However, such advancements have not been extensively explored within the medical domain. To bridge this gap, this paper introduces the first agent explicitly designed for the medical field, named \textbf{M}ulti-modal \textbf{Med}ical \textbf{Agent} (MMedAgent). We curate an instruction-tuning dataset comprising six medical tools solving seven tasks, enabling the agent to choose the most suitable tools for a given task. Comprehensive experiments demonstrate that MMedAgent achieves superior performance across a variety of medical tasks compared to state-of-the-art open-source methods and even the closed-source model, GPT-4o. Furthermore, MMedAgent exhibits efficiency in updating and integrating new medical tools. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models

Authors: Fei Shen, Hu Ye, Sibo Liu, Jun Zhang, Cong Wang, Xiao Han, Wei Yang

Abstract: Recent research showcases the considerable potential of conditional diffusion models for generating consistent stories. However, current methods, which predominantly generate stories in an autoregressive and excessively caption-dependent manner, often underrate the contextual consistency and relevance of frames during sequential generation. To address this, we propose a novel Rich-contextual Condi… ▽ More Recent research showcases the considerable potential of conditional diffusion models for generating consistent stories. However, current methods, which predominantly generate stories in an autoregressive and excessively caption-dependent manner, often underrate the contextual consistency and relevance of frames during sequential generation. To address this, we propose a novel Rich-contextual Conditional Diffusion Models (RCDMs), a two-stage approach designed to enhance story generation's semantic consistency and temporal consistency. Specifically, in the first stage, the frame-prior transformer diffusion model is presented to predict the frame semantic embedding of the unknown clip by aligning the semantic correlations between the captions and frames of the known clip. The second stage establishes a robust model with rich contextual conditions, including reference images of the known clip, the predicted frame semantic embedding of the unknown clip, and text embeddings of all captions. By jointly injecting these rich contextual conditions at the image and feature levels, RCDMs can generate semantic and temporal consistency stories. Moreover, RCDMs can generate consistent stories with a single forward inference compared to autoregressive models. Our qualitative and quantitative results demonstrate that our proposed RCDMs outperform in challenging scenarios. The code and model will be available at https://github.com/muzishen/RCDMs. △ Less

Submitted 3 July, 2024; v1 submitted 2 July, 2024; originally announced July 2024.

Uncertainty-Aware Decarbonization for Datacenters

Authors: Amy Li, Sihang Liu, Yi Ding

Abstract: This paper represents the first effort to quantify uncertainty in carbon intensity forecasting for datacenter decarbonization. We identify and analyze two types of uncertainty -- temporal and spatial -- and discuss their system implications. To address the temporal dynamics in quantifying uncertainty for carbon intensity forecasting, we introduce a conformal prediction-based framework. Evaluation… ▽ More This paper represents the first effort to quantify uncertainty in carbon intensity forecasting for datacenter decarbonization. We identify and analyze two types of uncertainty -- temporal and spatial -- and discuss their system implications. To address the temporal dynamics in quantifying uncertainty for carbon intensity forecasting, we introduce a conformal prediction-based framework. Evaluation results show that our technique robustly achieves target coverages in uncertainty quantification across various significance levels. We conduct two case studies using production power traces, focusing on temporal and spatial load shifting respectively. The results show that incorporating uncertainty into scheduling decisions can prevent a 5% and 14% increase in carbon emissions, respectively. These percentages translate to an absolute reduction of 2.1 and 10.4 tons of carbon emissions in a 20 MW datacenter cluster. △ Less

Submitted 2 July, 2024; originally announced July 2024.

MTMamba: Enhancing Multi-Task Dense Scene Understanding by Mamba-Based Decoders

Authors: Baijiong Lin, Weisen Jiang, Pengguang Chen, Yu Zhang, Shu Liu, Ying-Cong Chen

Abstract: Multi-task dense scene understanding, which learns a model for multiple dense prediction tasks, has a wide range of application scenarios. Modeling long-range dependency and enhancing cross-task interactions are crucial to multi-task dense prediction. In this paper, we propose MTMamba, a novel Mamba-based architecture for multi-task scene understanding. It contains two types of core blocks: self-t… ▽ More Multi-task dense scene understanding, which learns a model for multiple dense prediction tasks, has a wide range of application scenarios. Modeling long-range dependency and enhancing cross-task interactions are crucial to multi-task dense prediction. In this paper, we propose MTMamba, a novel Mamba-based architecture for multi-task scene understanding. It contains two types of core blocks: self-task Mamba (STM) block and cross-task Mamba (CTM) block. STM handles long-range dependency by leveraging Mamba, while CTM explicitly models task interactions to facilitate information exchange across tasks. Experiments on NYUDv2 and PASCAL-Context datasets demonstrate the superior performance of MTMamba over Transformer-based and CNN-based methods. Notably, on the PASCAL-Context dataset, MTMamba achieves improvements of +2.08, +5.01, and +4.90 over the previous best method in the tasks of semantic segmentation, human parsing, and object boundary detection, respectively. The code is available at \url{https://github.com/EnVision-Research/MTMamba}. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Enhanced Second-Harmonic Generation in Thin-Film Lithium Niobate Circular Bragg Nanocavity

Authors: Zengya Li, Zhuoran Hu, Xiaona Ye, Zhengyang Mao, Juan Feng, Hao Li, Shijie Liu, Bo Wang, Yuanlin Zheng, Xianfeng Chen

Abstract: Second-order nonlinearity gives rise to many distinctive physical phenomena, e.g., second-harmonic generation, which plays an important role in fundamental science and various applications. Lithium niobate, one of the most widely used nonlinear crystals, exhibits strong second-order nonlinear effects and electro-optic properties. However, its moderate refractive index and etching sidewall angle li… ▽ More Second-order nonlinearity gives rise to many distinctive physical phenomena, e.g., second-harmonic generation, which plays an important role in fundamental science and various applications. Lithium niobate, one of the most widely used nonlinear crystals, exhibits strong second-order nonlinear effects and electro-optic properties. However, its moderate refractive index and etching sidewall angle limit its capability in confining light into nanoscales, restricting its application in nanophotonics. Here, we exploit nanocavities formed by second-order circular Bragg gratings, which support resonant anapole modes to achieve highly enhanced SHG in thin film lithium niobate. The CBG nanocavity exhibits a record-high normalized conversion efficiency of $1.21\times10^{-2}\mathrm{cm^2/GW}$ under the pump intensity of $1.9$ $\mathrm{MW/cm^2}$. An SHG enhancement of $42,000$ is realized compared to TFLN. Besides, we also show s- and p-polarization independent SHG in elliptical Bragg nanocavities. This work could inspire studying nonlinear optics at the nanoscale on TFLN as well as other novel photonic platforms. △ Less

Submitted 11 July, 2024; v1 submitted 2 July, 2024; originally announced July 2024.

Comments: 19 pages, 5 figures

Unveiling Global Interactive Patterns across Graphs: Towards Interpretable Graph Neural Networks

Authors: Yuwen Wang, Shunyu Liu, Tongya Zheng, Kaixuan Chen, Mingli Song

Abstract: Graph Neural Networks (GNNs) have emerged as a prominent framework for graph mining, leading to significant advances across various domains. Stemmed from the node-wise representations of GNNs, existing explanation studies have embraced the subgraph-specific viewpoint that attributes the decision results to the salient features and local structures of nodes. However, graph-level tasks necessitate l… ▽ More Graph Neural Networks (GNNs) have emerged as a prominent framework for graph mining, leading to significant advances across various domains. Stemmed from the node-wise representations of GNNs, existing explanation studies have embraced the subgraph-specific viewpoint that attributes the decision results to the salient features and local structures of nodes. However, graph-level tasks necessitate long-range dependencies and global interactions for advanced GNNs, deviating significantly from subgraph-specific explanations. To bridge this gap, this paper proposes a novel intrinsically interpretable scheme for graph classification, termed as Global Interactive Pattern (GIP) learning, which introduces learnable global interactive patterns to explicitly interpret decisions. GIP first tackles the complexity of interpretation by clustering numerous nodes using a constrained graph clustering module. Then, it matches the coarsened global interactive instance with a batch of self-interpretable graph prototypes, thereby facilitating a transparent graph-level reasoning process. Extensive experiments conducted on both synthetic and real-world benchmarks demonstrate that the proposed GIP yields significantly superior interpretability and competitive performance to~the state-of-the-art counterparts. Our code will be made publicly available. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Comments: Accepted in KDD2024

CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents

Authors: Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Philip Torr, Bernard Ghanem, Guohao Li

Abstract: The development of autonomous agents increasingly relies on Multimodal Language Models (MLMs) to perform tasks described in natural language with GUI environments, such as websites, desktop computers, or mobile phones. Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the compl… ▽ More The development of autonomous agents increasingly relies on Multimodal Language Models (MLMs) to perform tasks described in natural language with GUI environments, such as websites, desktop computers, or mobile phones. Existing benchmarks for MLM agents in interactive environments are limited by their focus on a single environment, lack of detailed and generalized evaluation methods, and the complexities of constructing tasks and evaluators. To overcome these limitations, we introduce Crab, the first agent benchmark framework designed to support cross-environment tasks, incorporating a graph-based fine-grained evaluation method and an efficient mechanism for task and evaluator construction. Our framework supports multiple devices and can be easily extended to any environment with a Python interface. Leveraging Crab, we developed a cross-platform Crab Benchmark-v0 comprising 100 tasks in computer desktop and mobile phone environments. We evaluated four advanced MLMs using different single and multi-agent system configurations on this benchmark. The experimental results demonstrate that the single agent with GPT-4o achieves the best completion ratio of 35.26%. All framework code, agent code, and task datasets are publicly available at https://github.com/camel-ai/crab. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Mechanism design for coordinating vehicle-based mobile sensing tasks within the ride-hailing platform

Authors: Shenglin Liu, Qian Ge, Ke Han, Daisuke Fukuda, Takao Dantsuji

Abstract: This paper evaluates the benefit of integrating vehicle-based mobile crowd-sensing tasks into the ride-hailing system through the collaboration between the data user and the ride-hailing platform. In such a system, the ride-hailing platform commissions high-valued sensing tasks to idle drivers who can undertake either ride-hailing or sensing requests. Considering the different service requirements… ▽ More This paper evaluates the benefit of integrating vehicle-based mobile crowd-sensing tasks into the ride-hailing system through the collaboration between the data user and the ride-hailing platform. In such a system, the ride-hailing platform commissions high-valued sensing tasks to idle drivers who can undertake either ride-hailing or sensing requests. Considering the different service requirements and time windows between sensing and ride-hailing requests, we design a staggered operation strategy for ride-hailing order matching and the sensing task assignment. The auction-based mechanisms are employed to minimize costs while incentivizing driver participation in mobile sensing. To address the budget deficit problem of the primal VCG-based task assignment mechanism, we refine the driver selection approach and tailor the payment rule by imposing additional budget constraints. We demonstrate the benefits of our proposed mechanism through a series of numerical experiments using the NYC Taxi data. Experimental results reveal the potential of the mechanism for achieving high completion rates of sensing tasks at low social costs without degrading ride-hailing services. Furthermore, drivers who participate in both mobile sensing tasks and ride-hailing requests may gain higher income, but this advantage may diminish with an increasing number of such drivers and higher demand for ride-hailing services. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Comments: 30 pages, 9 figures

A Closer Look at Deep Learning on Tabular Data

Authors: Han-Jia Ye, Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, De-Chuan Zhan

Abstract: Tabular data is prevalent across various domains in machine learning. Although Deep Neural Network (DNN)-based methods have shown promising performance comparable to tree-based ones, in-depth evaluation of these methods is challenging due to varying performance ranks across diverse datasets. In this paper, we propose a comprehensive benchmark comprising 300 tabular datasets, covering a wide range… ▽ More Tabular data is prevalent across various domains in machine learning. Although Deep Neural Network (DNN)-based methods have shown promising performance comparable to tree-based ones, in-depth evaluation of these methods is challenging due to varying performance ranks across diverse datasets. In this paper, we propose a comprehensive benchmark comprising 300 tabular datasets, covering a wide range of task types, size distributions, and domains. We perform an extensive comparison between state-of-the-art deep tabular methods and tree-based methods, revealing the average rank of all methods and highlighting the key factors that influence the success of deep tabular methods. Next, we analyze deep tabular methods based on their training dynamics, including changes in validation metrics and other statistics. For each dataset-method pair, we learn a mapping from both the meta-features of datasets and the first part of the validation curve to the final validation set performance and even the evolution of validation curves. This mapping extracts essential meta-features that influence prediction accuracy, helping the analysis of tabular methods from novel aspects. Based on the performance of all methods on this large benchmark, we identify two subsets of 45 datasets each. The first subset contains datasets that favor either tree-based methods or DNN-based methods, serving as effective analysis tools to evaluate strategies (e.g., attribute encoding strategies) for improving deep tabular models. The second subset contains datasets where the ranks of methods are consistent with the overall benchmark, acting as a probe for tabular analysis. These ``tiny tabular benchmarks'' will facilitate further studies on tabular data. △ Less

Submitted 1 July, 2024; originally announced July 2024.

FoldGPT: Simple and Effective Large Language Model Compression Scheme

Authors: Songwei Liu, Chao Zeng, Lianqiang Li, Chenqian Yan, Lean Fu, Xing Mei, Fangmin Chen

Abstract: The demand for deploying large language models(LLMs) on mobile devices continues to increase, driven by escalating data security concerns and cloud costs. However, network bandwidth and memory limitations pose challenges for deploying billion-level models on mobile devices. In this study, we investigate the outputs of different layers across various scales of LLMs and found that the outputs of mos… ▽ More The demand for deploying large language models(LLMs) on mobile devices continues to increase, driven by escalating data security concerns and cloud costs. However, network bandwidth and memory limitations pose challenges for deploying billion-level models on mobile devices. In this study, we investigate the outputs of different layers across various scales of LLMs and found that the outputs of most layers exhibit significant similarity. Moreover, this similarity becomes more pronounced as the model size increases, indicating substantial redundancy in the depth direction of the LLMs. Based on this observation, we propose an efficient model volume compression strategy, termed FoldGPT, which combines block removal and block parameter sharing.This strategy consists of three parts: (1) Based on the learnable gating parameters, we determine the block importance ranking while modeling the coupling effect between blocks. Then we delete some redundant layers based on the given removal rate. (2) For the retained blocks, we apply a specially designed group parameter sharing strategy, where blocks within the same group share identical weights, significantly compressing the number of parameters and slightly reducing latency overhead. (3) After sharing these Blocks, we "cure" the mismatch caused by sparsity with a minor amount of fine-tuning and introduce a tail-layer distillation strategy to improve the performance. Experiments demonstrate that FoldGPT outperforms previous state-of-the-art(SOTA) methods in efficient model compression, demonstrating the feasibility of achieving model lightweighting through straightforward block removal and parameter sharing. △ Less

Submitted 30 June, 2024; originally announced July 2024.

Active Healing of Microtubule-Motor Networks

Authors: Fan Yang, Shichen Liu, Heun Jin Lee, Rob Phillips, Matt Thomson

Abstract: Cytoskeletal networks have a self-healing property where networks can repair defects to maintain structural integrity. However, both the mechanisms and dynamics of healing remain largely unknown. Here we report an unexplored healing mechanism in microtubule-motor networks by active crosslinking. We directly generate network cracks using a light-controlled microtubule-motor system, and observe that… ▽ More Cytoskeletal networks have a self-healing property where networks can repair defects to maintain structural integrity. However, both the mechanisms and dynamics of healing remain largely unknown. Here we report an unexplored healing mechanism in microtubule-motor networks by active crosslinking. We directly generate network cracks using a light-controlled microtubule-motor system, and observe that the cracks can self-heal. Combining theory and experiment, we find that the networks must overcome internal elastic resistance in order to heal cracks, giving rise to a bifurcation of dynamics dependent on the initial opening angle of the crack: the crack heals below a critical angle and opens up at larger angles. Simulation of a continuum model reproduces the bifurcation dynamics, revealing the importance of a boundary layer where free motors and microtubules can actively crosslink and thereby heal the crack. We also formulate a simple elastic-rod model that can qualitatively predict the critical angle, which is found to be tunable by two dimensionless geometric parameters, the ratio of the boundary layer and network width, and the aspect ratio of the network. Our results provide a new framework for understanding healing in cytoskeletal networks and designing self-healable biomaterials. △ Less

Submitted 30 June, 2024; originally announced July 2024.

pFLFE: Cross-silo Personalized Federated Learning via Feature Enhancement on Medical Image Segmentation

Authors: Luyuan Xie, Manqing Lin, Siyuan Liu, ChenMing Xu, Tianyu Luan, Cong Li, Yuejian Fang, Qingni Shen, Zhonghai Wu

Abstract: In medical image segmentation, personalized cross-silo federated learning (FL) is becoming popular for utilizing varied data across healthcare settings to overcome data scarcity and privacy concerns. However, existing methods often suffer from client drift, leading to inconsistent performance and delayed training. We propose a new framework, Personalized Federated Learning via Feature Enhancement… ▽ More In medical image segmentation, personalized cross-silo federated learning (FL) is becoming popular for utilizing varied data across healthcare settings to overcome data scarcity and privacy concerns. However, existing methods often suffer from client drift, leading to inconsistent performance and delayed training. We propose a new framework, Personalized Federated Learning via Feature Enhancement (pFLFE), designed to mitigate these challenges. pFLFE consists of two main stages: feature enhancement and supervised learning. The first stage improves differentiation between foreground and background features, and the second uses these enhanced features for learning from segmentation masks. We also design an alternative training approach that requires fewer communication rounds without compromising segmentation quality, even with limited communication resources. Through experiments on three medical segmentation tasks, we demonstrate that pFLFE outperforms the state-of-the-art methods. △ Less

Submitted 29 June, 2024; originally announced July 2024.

Enhancing Accuracy and Parameter-Efficiency of Neural Representations for Network Parameterization

Authors: Hongjun Choi, Jayaraman J. Thiagarajan, Ruben Glatt, Shusen Liu

Abstract: In this work, we investigate the fundamental trade-off regarding accuracy and parameter efficiency in the parameterization of neural network weights using predictor networks. We present a surprising finding that, when recovering the original model accuracy is the sole objective, it can be achieved effectively through the weight reconstruction objective alone. Additionally, we explore the underlyin… ▽ More In this work, we investigate the fundamental trade-off regarding accuracy and parameter efficiency in the parameterization of neural network weights using predictor networks. We present a surprising finding that, when recovering the original model accuracy is the sole objective, it can be achieved effectively through the weight reconstruction objective alone. Additionally, we explore the underlying factors for improving weight reconstruction under parameter-efficiency constraints, and propose a novel training scheme that decouples the reconstruction objective from auxiliary objectives such as knowledge distillation that leads to significant improvements compared to state-of-the-art approaches. Finally, these results pave way for more practical scenarios, where one needs to achieve improvements on both model accuracy and predictor network parameter-efficiency simultaneously. △ Less

Submitted 29 June, 2024; originally announced July 2024.

Observation of the Electromagnetic Dalitz Transition $h_c \rightarrow e^+e^-η_c$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, S. Ahmed, M. Albrecht, R. Aliberti, A. Amoroso, M. R. An, Q. An, X. H. Bai, Y. Bai, O. Bakina, R. Baldini Ferroli, I. Balossino, Y. Ban, K. Begzsuren, N. Berger, M. Bertani, D. Bettoni, F. Bianchi, J. Bloms, A. Bortone, I. Boyko, R. A. Briere , et al. (495 additional authors not shown)

Abstract: Using $(27.12\pm 0.14)\times10^8$ $ψ(3686)$ decays and data samples of $e^+e^-$ collisions with $\sqrt{s}$ from 4.130 to 4.780~GeV collected with the BESIII detector, we report the first observation of the electromagnetic Dalitz transition $h_c\to e^+e^-η_c$ with a statistical significance of $5.4σ$. We measure the ratio of the branching fractions… ▽ More Using $(27.12\pm 0.14)\times10^8$ $ψ(3686)$ decays and data samples of $e^+e^-$ collisions with $\sqrt{s}$ from 4.130 to 4.780~GeV collected with the BESIII detector, we report the first observation of the electromagnetic Dalitz transition $h_c\to e^+e^-η_c$ with a statistical significance of $5.4σ$. We measure the ratio of the branching fractions $\frac{\mathcal{B}(h_c\rightarrow e^+e^-η_c)}{\mathcal{B}(h_c\rightarrow γη_c)}$ separately for the $h_c$ samples produced via $ψ(3686)\toπ^0h_c$ and $e^+e^-\toπ^+π^-h_c$. The average ratio is determined to be $(0.59\pm0.10(\text{stat.})\pm0.04(\text{syst.}))\%$, where the uncertainty includes both statistical and systematic components. △ Less

Submitted 2 July, 2024; v1 submitted 28 June, 2024; originally announced July 2024.

Counterexamples to "Transitive Regret"

Authors: Yuan Chang, Shuo Li Liu

Abstract: Theorem 1 in Bikhchandani & Segal (2011; Theoretical Economics) suggests that a complete, transitive, monotonic, and continuous preference is regret based if and only if it is expected utility. Their Proposition 1 suggests that transitivity and continuity of a regret-based preference implies an equivalence condition: if random variables $X$ and $Y$ have the same distribution, then $X\sim Y$. We gi… ▽ More Theorem 1 in Bikhchandani & Segal (2011; Theoretical Economics) suggests that a complete, transitive, monotonic, and continuous preference is regret based if and only if it is expected utility. Their Proposition 1 suggests that transitivity and continuity of a regret-based preference implies an equivalence condition: if random variables $X$ and $Y$ have the same distribution, then $X\sim Y$. We give counterexamples to Proposition 1. △ Less

Submitted 14 June, 2024; originally announced July 2024.