Showing 1–50 of 270 results for author: Yin, Q

Algebraic cycles and Hitchin systems

Authors: Davesh Maulik, Junliang Shen, Qizheng Yin

Abstract: The purpose of this paper is to study motivic aspects of the Hitchin system for $\mathrm{GL}_n$. Our results include the following. (a) We prove the motivic decomposition conjecture of Corti-Hanamura for the Hitchin system; in particular, the decomposition theorem associated with the Hitchin system is induced by algebraic cycles. This yields an unconditional construction of the motivic perverse fi… ▽ More The purpose of this paper is to study motivic aspects of the Hitchin system for $\mathrm{GL}_n$. Our results include the following. (a) We prove the motivic decomposition conjecture of Corti-Hanamura for the Hitchin system; in particular, the decomposition theorem associated with the Hitchin system is induced by algebraic cycles. This yields an unconditional construction of the motivic perverse filtration for the Hitchin system, which lifts the cohomological/sheaf-theoretic perverse filtration. (b) We prove that the inverse of the relative Hard Lefschetz symmetry is induced by a relative algebraic correspondence, confirming the relative Lefschetz standard conjecture for the Hitchin system. (c) We show a strong perversity bound for the normalized Chern classes of a universal bundle with respect to the motivic perverse filtration; this specializes to the sheaf-theoretic result obtained earlier by Maulik-Shen. (d) We prove a $χ$-independence result for the relative Chow motives associated with Hitchin systems. Our methods combine Fourier transforms for compactified Jacobian fibrations associated with integral locally planar curves, nearby and vanishing cycle techniques, and a Springer-theoretic interpretation of parabolic Hitchin moduli spaces. △ Less

Submitted 6 July, 2024; originally announced July 2024.

Comments: 45 pages. Comments are welcome

M$^2$IST: Multi-Modal Interactive Side-Tuning for Memory-efficient Referring Expression Comprehension

Authors: Xuyang Liu, Ting Liu, Siteng Huang, Yue Hu, Quanjun Yin, Donglin Wang, Honggang Chen

Abstract: Referring expression comprehension (REC) is a vision-language task to locate a target object in an image based on a language expression. Fully fine-tuning general-purpose pre-trained models for REC yields impressive performance but becomes increasingly costly. Parameter-efficient transfer learning (PETL) methods have shown strong performance with fewer tunable parameters. However, applying PETL to… ▽ More Referring expression comprehension (REC) is a vision-language task to locate a target object in an image based on a language expression. Fully fine-tuning general-purpose pre-trained models for REC yields impressive performance but becomes increasingly costly. Parameter-efficient transfer learning (PETL) methods have shown strong performance with fewer tunable parameters. However, applying PETL to REC faces two challenges: (1) insufficient interaction between pre-trained vision and language encoders, and (2) high GPU memory usage due to gradients passing through both heavy encoders. To address these issues, we present M$^2$IST: Multi-Modal Interactive Side-Tuning with M$^3$ISAs: Mixture of Multi-Modal Interactive Side-Adapters. During fine-tuning, we keep the pre-trained vision and language encoders fixed and update M$^3$ISAs on side networks to establish connections between them, thereby achieving parameter- and memory-efficient tuning for REC. Empirical results on three benchmarks show M$^2$IST achieves the best performance-parameter-memory trade-off compared to full fine-tuning and other PETL methods, with only 3.14M tunable parameters (2.11% of full fine-tuning) and 15.44GB GPU memory usage (39.61% of full fine-tuning). Source code will soon be publicly available. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Hybrid atom-photon entangling gates via Gaussian soft control

Authors: Wanrang Yu, Qiuyu Yin, Yanzhao Liang, Ning Ji, Thibault Vogt

Abstract: Hybrid atom-photon gates play an important role for the realization of a quantum interface capable of mapping atomic states to photons for communication across quantum networks. Here, we propose a feasible theoretical scheme for implementing a hybrid atom-photon controlled-Z gate between an atom and a microwave photon in a superconducting coplanar waveguide resonator based on the Gaussian soft con… ▽ More Hybrid atom-photon gates play an important role for the realization of a quantum interface capable of mapping atomic states to photons for communication across quantum networks. Here, we propose a feasible theoretical scheme for implementing a hybrid atom-photon controlled-Z gate between an atom and a microwave photon in a superconducting coplanar waveguide resonator based on the Gaussian soft control technique. The gate protocol employs a classical auxiliary field that induces an atomic transition between one state of the atomic qubit and Rydberg states for obtaining strong coupling of the atom and microwave resonator. By tailoring the amplitude of this field with Gaussian temporal modulation, the gate performances are improved in various aspects. Numerical simulations demonstrate that the controlled-Z gate based on Gaussian soft control is resilient to the variation of the atom-photon coupling strength, deviation in the gate time, and less sensitive to the Rydberg level shifts caused by stray electric fields. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Constraints on Ultra Heavy Dark Matter Properties from Dwarf Spheroidal Galaxies with LHAASO Observations

Authors: Zhen Cao, F. Aharonian, Q. An, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (255 additional authors not shown)

Abstract: In this work we try to search for signals generated by ultra-heavy dark matter at the Large High Altitude Air Shower Observatory (LHAASO) data. We look for possible gamma-ray by dark matter annihilation or decay from 16 dwarf spheroidal galaxies in the field of view of LHAASO. Dwarf spheroidal galaxies are among the most promising targets for indirect detection of dark matter which have low fluxes… ▽ More In this work we try to search for signals generated by ultra-heavy dark matter at the Large High Altitude Air Shower Observatory (LHAASO) data. We look for possible gamma-ray by dark matter annihilation or decay from 16 dwarf spheroidal galaxies in the field of view of LHAASO. Dwarf spheroidal galaxies are among the most promising targets for indirect detection of dark matter which have low fluxes of astrophysical $γ$-ray background while large amount of dark matter. By analyzing more than 700 days observational data at LHAASO, no significant dark matter signal from 1 TeV to 1 EeV is detected. Accordingly we derive the most stringent constraints on the ultra-heavy dark matter annihilation cross-section up to EeV. The constraints on the lifetime of dark matter in decay mode are also derived. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 17 pages, 12 figures, accepted by PRL

Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference

Authors: Ting Liu, Xuyang Liu, Liangtao Shi, Zunnan Xu, Siteng Huang, Yi Xin, Quanjun Yin

Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as a popular approach for adapting pre-trained Vision Transformer (ViT) models to downstream applications. While current PEFT methods achieve parameter efficiency, they overlook GPU memory and time efficiency during both fine-tuning and inference, due to the repeated computation of redundant tokens in the ViT architecture. This falls short of prac… ▽ More Parameter-efficient fine-tuning (PEFT) has emerged as a popular approach for adapting pre-trained Vision Transformer (ViT) models to downstream applications. While current PEFT methods achieve parameter efficiency, they overlook GPU memory and time efficiency during both fine-tuning and inference, due to the repeated computation of redundant tokens in the ViT architecture. This falls short of practical requirements for downstream task adaptation. In this paper, we propose \textbf{Sparse-Tuning}, a novel tuning paradigm that substantially enhances both fine-tuning and inference efficiency for pre-trained ViT models. Sparse-Tuning efficiently fine-tunes the pre-trained ViT by sparsely preserving the informative tokens and merging redundant ones, enabling the ViT to focus on the foreground while reducing computational costs on background regions in the images. To accurately distinguish informative tokens from uninformative ones, we introduce a tailored Dense Adapter, which establishes dense connections across different encoder layers in the ViT, thereby enhancing the representational capacity and quality of token sparsification. Empirical results on VTAB-1K, three complete image datasets, and two complete video datasets demonstrate that Sparse-Tuning reduces the GFLOPs to \textbf{62\%-70\%} of the original ViT-B while achieving state-of-the-art performance. Source code is available at \url{https://github.com/liuting20/Sparse-Tuning}. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Optimizing Four-Wave Mixing in Rydberg Atoms for Microwave-Optical Conversion

Authors: Ning Ji, Yanzhao Liang, Wanrang Yu, Qiuyu Yin, Thibault Vogt

Abstract: We perform a numerical and analytical investigation of microwave-to-optical conversion based on four-wave mixing in Rydberg atoms. Our work demonstrates that both all-resonant and off-resonant frequency-mixing configurations achieve near-unit photon conversion efficiencies. We review the conditions that can lead to the presence of two possible dark states. We find that for both configurations, one… ▽ More We perform a numerical and analytical investigation of microwave-to-optical conversion based on four-wave mixing in Rydberg atoms. Our work demonstrates that both all-resonant and off-resonant frequency-mixing configurations achieve near-unit photon conversion efficiencies. We review the conditions that can lead to the presence of two possible dark states. We find that for both configurations, one of the dark states can be detrimental at high microwave powers, and show that an additional limitation to all-resonant frequency mixing is microwave-induced fluorescence. Finally, we confirm that the off-resonant configuration is more appropriate as it allows for efficient photon conversion on a wider range of input microwave intensities with reduced total power of the auxiliary fields. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Data quality control system and long-term performance monitor of the LHAASO-KM2A

Authors: Zhen Cao, F. Aharonian, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, W. Bian, A. V. Bukevich, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, H. X. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. Chen , et al. (263 additional authors not shown)

Abstract: The KM2A is the largest sub-array of the Large High Altitude Air Shower Observatory (LHAASO). It consists of 5216 electromagnetic particle detectors (EDs) and 1188 muon detectors (MDs). The data recorded by the EDs and MDs are used to reconstruct primary information of cosmic ray and gamma-ray showers. This information is used for physical analysis in gamma-ray astronomy and cosmic ray physics. To… ▽ More The KM2A is the largest sub-array of the Large High Altitude Air Shower Observatory (LHAASO). It consists of 5216 electromagnetic particle detectors (EDs) and 1188 muon detectors (MDs). The data recorded by the EDs and MDs are used to reconstruct primary information of cosmic ray and gamma-ray showers. This information is used for physical analysis in gamma-ray astronomy and cosmic ray physics. To ensure the reliability of the LHAASO-KM2A data, a three-level quality control system has been established. It is used to monitor the status of detector units, stability of reconstructed parameters and the performance of the array based on observations of the Crab Nebula and Moon shadow. This paper will introduce the control system and its application on the LHAASO-KM2A data collected from August 2021 to July 2023. During this period, the pointing and angular resolution of the array were stable. From the observations of the Moon shadow and Crab Nebula, the results achieved using the two methods are consistent with each other. According to the observation of the Crab Nebula at energies from 25 TeV to 100 TeV, the time averaged pointing errors are estimated to be $-0.003^{\circ} \pm 0.005^{\circ}$ and $0.001^{\circ} \pm 0.006^{\circ}$ in the R.A. and Dec directions, respectively. △ Less

Submitted 13 June, 2024; v1 submitted 20 May, 2024; originally announced May 2024.

Comments: 15 pages, 9 figures

Discovery of Very-high-energy Gamma-ray Emissions from the Low Luminosity AGN NGC 4278 by LHAASO

Authors: Zhen Cao, F. Aharonian, Q. An, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (255 additional authors not shown)

Abstract: The first source catalog of Large High Altitude Air Shower Observatory reported the detection of a very-high-energy gamma ray source, 1LHAASO J1219+2915. In this paper a further detailed study of the spectral and temporal behavior of this point-like source have been carried. The best-fit position of the TeV source ($\rm{RA}=185.05^{\circ}\pm0.04^{\circ}$, $\rm{Dec}=29.25^{\circ}\pm0.03^{\circ}$) i… ▽ More The first source catalog of Large High Altitude Air Shower Observatory reported the detection of a very-high-energy gamma ray source, 1LHAASO J1219+2915. In this paper a further detailed study of the spectral and temporal behavior of this point-like source have been carried. The best-fit position of the TeV source ($\rm{RA}=185.05^{\circ}\pm0.04^{\circ}$, $\rm{Dec}=29.25^{\circ}\pm0.03^{\circ}$) is compatible with NGC 4278 within $\sim0.03$ degree. Variation analysis shows an indication of the variability at a few months level in the TeV band, which is consistent with low frequency observations. Based on these observations, we report the detection of TeV $γ$-ray emissions from this low-luminosity AGN NGC 4278. The observations by LHAASO-WCDA during active period has a significance level of 8.8\,$σ$ with best-fit photon spectral index $\varGamma=2.56\pm0.14$ and a flux $f_{1-10\,\rm{TeV}}=(7.0\pm1.1_{\rm{sta}}\pm0.35_{\rm{syst}})\times10^{-13}\,\rm{photons\,cm^{-2}\,s^{-1}}$, or approximately $5\%$ of the Crab Nebula. The discovery of VHE from NGC 4278 indicates that the compact, weak radio jet can efficiently accelerate particles and emit TeV photons. △ Less

Submitted 13 May, 2024; originally announced May 2024.

Comments: 11 pages, 5 figures

DARA: Domain- and Relation-aware Adapters Make Parameter-efficient Tuning for Visual Grounding

Authors: Ting Liu, Xuyang Liu, Siteng Huang, Honggang Chen, Quanjun Yin, Long Qin, Donglin Wang, Yue Hu

Abstract: Visual grounding (VG) is a challenging task to localize an object in an image based on a textual description. Recent surge in the scale of VG models has substantially improved performance, but also introduced a significant burden on computational costs during fine-tuning. In this paper, we explore applying parameter-efficient transfer learning (PETL) to efficiently transfer the pre-trained vision-… ▽ More Visual grounding (VG) is a challenging task to localize an object in an image based on a textual description. Recent surge in the scale of VG models has substantially improved performance, but also introduced a significant burden on computational costs during fine-tuning. In this paper, we explore applying parameter-efficient transfer learning (PETL) to efficiently transfer the pre-trained vision-language knowledge to VG. Specifically, we propose \textbf{DARA}, a novel PETL method comprising \underline{\textbf{D}}omain-aware \underline{\textbf{A}}dapters (DA Adapters) and \underline{\textbf{R}}elation-aware \underline{\textbf{A}}dapters (RA Adapters) for VG. DA Adapters first transfer intra-modality representations to be more fine-grained for the VG domain. Then RA Adapters share weights to bridge the relation between two modalities, improving spatial reasoning. Empirical results on widely-used benchmarks demonstrate that DARA achieves the best accuracy while saving numerous updated parameters compared to the full fine-tuning and other PETL methods. Notably, with only \textbf{2.13\%} tunable backbone parameters, DARA improves average accuracy by \textbf{0.81\%} across the three benchmarks compared to the baseline model. Our code is available at \url{https://github.com/liuting20/DARA}. △ Less

Submitted 8 June, 2024; v1 submitted 9 May, 2024; originally announced May 2024.

Comments: Accepted by ICME 2024 (Oral)

Polarization Perspectives on Hercules X-1: Further Constraining the Geometry

Authors: Qingchang Zhao, Hancheng Li, Lian Tao, Hua Feng, Shuangnan Zhang, Roland Walter, Mingyu Ge, Hao Tong, Long Ji, Liang Zhang, Jinlu Qu, Yue Huang, Xiang Ma, Shu Zhang, Qianqing Yin, Hongxing Yin, Ruican Ma, Shujie Zhao, Panping Li, Zixu Yang, Hexin Liu, Wei Yu, Yiming Huang, Zexi Li, Yajun Li , et al. (2 additional authors not shown)

Abstract: We conduct a comprehensive analysis of the accreting X-ray pulsar, Hercules X-1, utilizing data from IXPE and NuSTAR. IXPE performed five observations of Her X-1, consisting of three in the Main-on state and two in the Short-on state. Our time-resolved analysis uncovers the linear correlations between the flux and polarization degree as well as the pulse fraction and polarization degree. Geometry… ▽ More We conduct a comprehensive analysis of the accreting X-ray pulsar, Hercules X-1, utilizing data from IXPE and NuSTAR. IXPE performed five observations of Her X-1, consisting of three in the Main-on state and two in the Short-on state. Our time-resolved analysis uncovers the linear correlations between the flux and polarization degree as well as the pulse fraction and polarization degree. Geometry parameters are rigorously constrained by fitting the phase-resolved modulations of Cyclotron Resonance Scattering Feature and polarization angle with a simple dipole model and Rotating Vector Model respectively, yielding roughly consistent results. The changes of $χ_{\rm p}$ (the position angle of the pulsar's spin axis on the plane of the sky) between different Main-on observations suggest the possible forced precession of the neutron star crust. Furthermore, a linear association between the energy of Cyclotron Resonance Scattering Feature and polarization angle implies the prevalence of a dominant dipole magnetic field, and their phase-resolved modulations likely arise from viewing angle effects. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: Accepted for MNRAS

Insights from the Gaussian Processes Method for the FRB-associated X-ray Burst of SGR 1935+2154

Authors: Ruijing Tang, Dahai Yan, Haiyun Zhang, Qingchang Zhao, Lian Tao, Chengkui Li, Mingyu Ge, Xiaobo Li, Qianqing Yin, Ce Cai

Abstract: Gaussian processes method is employed to analyze the light curves of bursts detected by Insight-HXMT, NICER, and GECAM from SGR 1935+2154 between 2020 to 2022. It is found that a stochastically driven damped simple harmonic oscillator (SHO) is necessary to capture the characteristics of the X-ray bursts. Variability timescale of the X-ray bursts, corresponding to the broken frequencies in the SHO… ▽ More Gaussian processes method is employed to analyze the light curves of bursts detected by Insight-HXMT, NICER, and GECAM from SGR 1935+2154 between 2020 to 2022. It is found that a stochastically driven damped simple harmonic oscillator (SHO) is necessary to capture the characteristics of the X-ray bursts. Variability timescale of the X-ray bursts, corresponding to the broken frequencies in the SHO power spectral densities (PSDs), are extracted. In particular, a high broken frequency of 35 Hz where the index of the SHO PSD changes from -4 to -2 is constrained by the HXMT-HE burst associated with FRB 200428. It is suggested that the corresponding timescale of 0.03 s could be the retarding timescale of the system driven by some energy release, and the production of the HE photon should be quasi-simultaneous with the response. The other special event is a NICER burst with a retarding timescale of 1/39 Hz (0.02 s). In the normal X-ray bursts, no retarding timescale is constrained; a long relax/equilibrium timescale (corresponding to a broken frequency of 1-10 Hz where the index of the SHO PSD changing from -4/-2 to 0 in the SHO PSD) is obtained. The results indicate that the FRB-associated HXMT-HE X-ray burst could be produced immediately when the system is responding to the energy disturbance, far before the equilibrium state. △ Less

Submitted 19 June, 2024; v1 submitted 19 April, 2024; originally announced April 2024.

Comments: 13 pages,17 figures,1 table

MSC Class: 85-02

Cramer's moderate deviations of modularity in network

Authors: Yu Miao, Qing Yin

Abstract: Complex networks play a crucial role in understanding physical, biological, social and technological systems. One of the most relevant features of graphs representing real systems is community structure. In this paper, for a specific partition of a given network, we prove the Cramer's moderate deviations of modularity for the partition when the size of the network gets large. Complex networks play a crucial role in understanding physical, biological, social and technological systems. One of the most relevant features of graphs representing real systems is community structure. In this paper, for a specific partition of a given network, we prove the Cramer's moderate deviations of modularity for the partition when the size of the network gets large. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Comments: arXiv admin note: text overlap with arXiv:2404.10325

MSC Class: 05C82; 60F05

Berry-Esseen bound of modularity in network

Authors: Yu Miao, Qing Yin

Abstract: In this paper, the model is a specific partition of a given network. Berry-Esseen bound and strong law of large numbers of modularity for the partition are proved when the size of the network gets large. In this paper, the model is a specific partition of a given network. Berry-Esseen bound and strong law of large numbers of modularity for the partition are proved when the size of the network gets large. △ Less

Submitted 16 April, 2024; originally announced April 2024.

MSC Class: 05C82; 60F05

LHAASO-KM2A detector simulation using Geant4

Authors: Zhen Cao, F. Aharonian, Q. An, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (254 additional authors not shown)

Abstract: KM2A is one of the main sub-arrays of LHAASO, working on gamma ray astronomy and cosmic ray physics at energies above 10 TeV. Detector simulation is the important foundation for estimating detector performance and data analysis. It is a big challenge to simulate the KM2A detector in the framework of Geant4 due to the need to track numerous photons from a large number of detector units (>6000) with… ▽ More KM2A is one of the main sub-arrays of LHAASO, working on gamma ray astronomy and cosmic ray physics at energies above 10 TeV. Detector simulation is the important foundation for estimating detector performance and data analysis. It is a big challenge to simulate the KM2A detector in the framework of Geant4 due to the need to track numerous photons from a large number of detector units (>6000) with large altitude difference (30 m) and huge coverage (1.3 km^2). In this paper, the design of the KM2A simulation code G4KM2A based on Geant4 is introduced. The process of G4KM2A is optimized mainly in memory consumption to avoid memory overffow. Some simpliffcations are used to signiffcantly speed up the execution of G4KM2A. The running time is reduced by at least 30 times compared to full detector simulation. The particle distributions and the core/angle resolution comparison between simulation and experimental data of the full KM2A array are also presented, which show good agreement. △ Less

Submitted 7 April, 2024; originally announced April 2024.

A Simple Yet Effective Approach for Diversified Session-Based Recommendation

Authors: Qing Yin, Hui Fang, Zhu Sun, Yew-Soon Ong

Abstract: Session-based recommender systems (SBRSs) have become extremely popular in view of the core capability of capturing short-term and dynamic user preferences. However, most SBRSs primarily maximize recommendation accuracy but ignore user minor preferences, thus leading to filter bubbles in the long run. Only a handful of works, being devoted to improving diversity, depend on unique model designs and… ▽ More Session-based recommender systems (SBRSs) have become extremely popular in view of the core capability of capturing short-term and dynamic user preferences. However, most SBRSs primarily maximize recommendation accuracy but ignore user minor preferences, thus leading to filter bubbles in the long run. Only a handful of works, being devoted to improving diversity, depend on unique model designs and calibrated loss functions, which cannot be easily adapted to existing accuracy-oriented SBRSs. It is thus worthwhile to come up with a simple yet effective design that can be used as a plugin to facilitate existing SBRSs on generating a more diversified list in the meantime preserving the recommendation accuracy. In this case, we propose an end-to-end framework applied for every existing representative (accuracy-oriented) SBRS, called diversified category-aware attentive SBRS (DCA-SBRS), to boost the performance on recommendation diversity. It consists of two novel designs: a model-agnostic diversity-oriented loss function, and a non-invasive category-aware attention mechanism. Extensive experiments on three datasets showcase that our framework helps existing SBRSs achieve extraordinary performance in terms of recommendation diversity and comprehensive performance, without significantly deteriorating recommendation accuracy compared to state-of-the-art accuracy-oriented SBRSs. △ Less

Submitted 30 March, 2024; originally announced April 2024.

The Future of Combating Rumors? Retrieval, Discrimination, and Generation

Authors: Junhao Xu, Longdi Xian, Zening Liu, Mingliang Chen, Qiuyang Yin, Fenghua Song

Abstract: Artificial Intelligence Generated Content (AIGC) technology development has facilitated the creation of rumors with misinformation, impacting societal, economic, and political ecosystems, challenging democracy. Current rumor detection efforts fall short by merely labeling potentially misinformation (classification task), inadequately addressing the issue, and it is unrealistic to have authoritativ… ▽ More Artificial Intelligence Generated Content (AIGC) technology development has facilitated the creation of rumors with misinformation, impacting societal, economic, and political ecosystems, challenging democracy. Current rumor detection efforts fall short by merely labeling potentially misinformation (classification task), inadequately addressing the issue, and it is unrealistic to have authoritative institutions debunk every piece of information on social media. Our proposed comprehensive debunking process not only detects rumors but also provides explanatory generated content to refute the authenticity of the information. The Expert-Citizen Collective Wisdom (ECCW) module we designed aensures high-precision assessment of the credibility of information and the retrieval module is responsible for retrieving relevant knowledge from a Real-time updated debunking database based on information keywords. By using prompt engineering techniques, we feed results and knowledge into a LLM (Large Language Model), achieving satisfactory discrimination and explanatory effects while eliminating the need for fine-tuning, saving computational costs, and contributing to debunking efforts. △ Less

Submitted 29 March, 2024; originally announced March 2024.

Comments: 8 pages

MSC Class: 68T99

IterAlign: Iterative Constitutional Alignment of Large Language Models

Authors: Xiusi Chen, Hongzhi Wen, Sreyashi Nag, Chen Luo, Qingyu Yin, Ruirui Li, Zheng Li, Wei Wang

Abstract: With the rapid development of large language models (LLMs), aligning LLMs with human values and societal norms to ensure their reliability and safety has become crucial. Reinforcement learning with human feedback (RLHF) and Constitutional AI (CAI) have been proposed for LLM alignment. However, these methods require either heavy human annotations or explicitly pre-defined constitutions, which are l… ▽ More With the rapid development of large language models (LLMs), aligning LLMs with human values and societal norms to ensure their reliability and safety has become crucial. Reinforcement learning with human feedback (RLHF) and Constitutional AI (CAI) have been proposed for LLM alignment. However, these methods require either heavy human annotations or explicitly pre-defined constitutions, which are labor-intensive and resource-consuming. To overcome these drawbacks, we study constitution-based LLM alignment and propose a data-driven constitution discovery and self-alignment framework called IterAlign. IterAlign leverages red teaming to unveil the weaknesses of an LLM and automatically discovers new constitutions using a stronger LLM. These constitutions are then used to guide self-correction of the base LLM. Such a constitution discovery pipeline can be run iteratively and automatically to discover new constitutions that specifically target the alignment gaps in the current LLM. Empirical results on several safety benchmark datasets and multiple base LLMs show that IterAlign successfully improves truthfulness, helpfulness, harmlessness and honesty, improving the LLM alignment by up to $13.5\%$ in harmlessness. △ Less

Submitted 27 March, 2024; originally announced March 2024.

Comments: NAACL 2024

Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond

Authors: Tianxin Wei, Bowen Jin, Ruirui Li, Hansi Zeng, Zhengyang Wang, Jianhui Sun, Qingyu Yin, Hanqing Lu, Suhang Wang, Jingrui He, Xianfeng Tang

Abstract: Developing a universal model that can effectively harness heterogeneous resources and respond to a wide range of personalized needs has been a longstanding community aspiration. Our daily choices, especially in domains like fashion and retail, are substantially shaped by multi-modal data, such as pictures and textual descriptions. These modalities not only offer intuitive guidance but also cater t… ▽ More Developing a universal model that can effectively harness heterogeneous resources and respond to a wide range of personalized needs has been a longstanding community aspiration. Our daily choices, especially in domains like fashion and retail, are substantially shaped by multi-modal data, such as pictures and textual descriptions. These modalities not only offer intuitive guidance but also cater to personalized user preferences. However, the predominant personalization approaches mainly focus on the ID or text-based recommendation problem, failing to comprehend the information spanning various tasks or modalities. In this paper, our goal is to establish a Unified paradigm for Multi-modal Personalization systems (UniMP), which effectively leverages multi-modal data while eliminating the complexities associated with task- and modality-specific customization. We argue that the advancements in foundational generative modeling have provided the flexibility and effectiveness necessary to achieve the objective. In light of this, we develop a generic and extensible personalization generative framework, that can handle a wide range of personalized needs including item recommendation, product search, preference prediction, explanation generation, and further user-guided image generation. Our methodology enhances the capabilities of foundational language models for personalized tasks by seamlessly ingesting interleaved cross-modal user history information, ensuring a more precise and customized experience for users. To train and evaluate the proposed multi-modal personalized tasks, we also introduce a novel and comprehensive benchmark covering a variety of user requirements. Our experiments on the real-world benchmark showcase the model's potential, outperforming competitive methods specialized for each task. △ Less

Submitted 27 March, 2024; v1 submitted 15 March, 2024; originally announced March 2024.

Comments: ICLR 2024

Measurements of All-Particle Energy Spectrum and Mean Logarithmic Mass of Cosmic Rays from 0.3 to 30 PeV with LHAASO-KM2A

Authors: The LHAASO Collaboration, Zhen Cao, F. Aharonian, Q. An, A. Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen , et al. (256 additional authors not shown)

Abstract: We present the measurements of all-particle energy spectrum and mean logarithmic mass of cosmic rays in the energy range of 0.3-30 PeV using data collected from LHAASO-KM2A between September 2021 and December 2022, which is based on a nearly composition-independent energy reconstruction method, achieving unprecedented accuracy. Our analysis reveals the position of the knee at… ▽ More We present the measurements of all-particle energy spectrum and mean logarithmic mass of cosmic rays in the energy range of 0.3-30 PeV using data collected from LHAASO-KM2A between September 2021 and December 2022, which is based on a nearly composition-independent energy reconstruction method, achieving unprecedented accuracy. Our analysis reveals the position of the knee at $3.67 \pm 0.05 \pm 0.15$ PeV. Below the knee, the spectral index is found to be -$2.7413 \pm 0.0004 \pm 0.0050$, while above the knee, it is -$3.128 \pm 0.005 \pm 0.027$, with the sharpness of the transition measured with a statistical error of 2%. The mean logarithmic mass of cosmic rays is almost heavier than helium in the whole measured energy range. It decreases from 1.7 at 0.3 PeV to 1.3 at 3 PeV, representing a 24% decline following a power law with an index of -$0.1200 \pm 0.0003 \pm 0.0341$. This is equivalent to an increase in abundance of light components. Above the knee, the mean logarithmic mass exhibits a power law trend towards heavier components, which is reversal to the behavior observed in the all-particle energy spectrum. Additionally, the knee position and the change in power-law index are approximately the same. These findings suggest that the knee observed in the all-particle spectrum corresponds to the knee of the light component, rather than the medium-heavy components. △ Less

Submitted 26 March, 2024; v1 submitted 15 March, 2024; originally announced March 2024.

Comments: 8 pages, 3 figures

Journal ref: Physical Review Letters 132, 131002 (2024)

Distribution-Free Fair Federated Learning with Small Samples

Authors: Qichuan Yin, Junzhou Huang, Huaxiu Yao, Linjun Zhang

Abstract: As federated learning gains increasing importance in real-world applications due to its capacity for decentralized data training, addressing fairness concerns across demographic groups becomes critically important. However, most existing machine learning algorithms for ensuring fairness are designed for centralized data environments and generally require large-sample and distributional assumptions… ▽ More As federated learning gains increasing importance in real-world applications due to its capacity for decentralized data training, addressing fairness concerns across demographic groups becomes critically important. However, most existing machine learning algorithms for ensuring fairness are designed for centralized data environments and generally require large-sample and distributional assumptions, underscoring the urgent need for fairness techniques adapted for decentralized and heterogeneous systems with finite-sample and distribution-free guarantees. To address this issue, this paper introduces FedFaiREE, a post-processing algorithm developed specifically for distribution-free fair learning in decentralized settings with small samples. Our approach accounts for unique challenges in decentralized environments, such as client heterogeneity, communication costs, and small sample sizes. We provide rigorous theoretical guarantees for both fairness and accuracy, and our experimental results further provide robust empirical validation for our proposed method. △ Less

Submitted 25 February, 2024; originally announced February 2024.

Learning with Semantics: Towards a Semantics-Aware Routing Anomaly Detection System

Authors: Yihao Chen, Qilei Yin, Qi Li, Zhuotao Liu, Ke Xu, Yi Xu, Mingwei Xu, Ziqian Liu, Jianping Wu

Abstract: BGP is the de facto inter-domain routing protocol to ensure global connectivity of the Internet. However, various reasons, such as deliberate attacks or misconfigurations, could cause BGP routing anomalies. Traditional methods for BGP routing anomaly detection require significant manual investigation of routes by network operators. Although machine learning has been applied to automate the process… ▽ More BGP is the de facto inter-domain routing protocol to ensure global connectivity of the Internet. However, various reasons, such as deliberate attacks or misconfigurations, could cause BGP routing anomalies. Traditional methods for BGP routing anomaly detection require significant manual investigation of routes by network operators. Although machine learning has been applied to automate the process, prior arts typically impose significant training overhead (such as large-scale data labeling and feature crafting), and only produce uninterpretable results. To address these limitations, this paper presents a routing anomaly detection system centering around a novel network representation learning model named BEAM. The core design of BEAM is to accurately learn the unique properties (defined as \emph{routing role}) of each Autonomous System (AS) in the Internet by incorporating BGP semantics. As a result, routing anomaly detection, given BEAM, is reduced to a matter of discovering unexpected routing role churns upon observing new route announcements. We implement a prototype of our routing anomaly detection system and extensively evaluate its performance. The experimental results, based on 18 real-world RouteViews datasets containing over 11 billion route announcement records, demonstrate that our system can detect all previously-confirmed routing anomalies, while only introducing at most five false alarms every 180 million route announcements. We also deploy our system at a large ISP to perform real-world detection for one month. During the course of deployment, our system detects 497 true anomalies in the wild with an average of only 1.65 false alarms per day. △ Less

Submitted 25 February, 2024; originally announced February 2024.

Comments: To be published in USENIX Security 2024

Simulation Studies for the First Pathfinder of the CATCH Space Mission

Authors: Yiming Huang, Juan Zhang, Lian Tao, Zhengwei Li, Donghua Zhao, Qian-Qing Yin, Xiangyang Wen, Jingyu Xiao, Chen Zhang, Shuang-Nan Zhang, Shaolin Xiong, Qingcui Bu, Jirong Cang, Dezhi Cao, Wen Chen, Siran Ding, Min Gao, Yang Gao, Shujin Hou, Liping Jia, Ge Jin, Dalin Li, Jinsong Li, Panping Li, Yajun Li , et al. (20 additional authors not shown)

Abstract: The Chasing All Transients Constellation Hunters (CATCH) space mission is an intelligent constellation consisting of 126 micro-satellites in three types (A, B, and C), designed for X-ray observation with the objective of studying the dynamic universe. Currently, we are actively developing the first Pathfinder (CATCH-1) for the CATCH mission, specifically for type-A satellites. CATCH-1 is equipped… ▽ More The Chasing All Transients Constellation Hunters (CATCH) space mission is an intelligent constellation consisting of 126 micro-satellites in three types (A, B, and C), designed for X-ray observation with the objective of studying the dynamic universe. Currently, we are actively developing the first Pathfinder (CATCH-1) for the CATCH mission, specifically for type-A satellites. CATCH-1 is equipped with Micro Pore Optics (MPO) and a 4-pixel Silicon Drift Detector (SDD) array. To assess its scientific performance, including the effective area of the optical system, on-orbit background, and telescope sensitivity, we employ the Monte Carlo software Geant4 for simulation in this study. The MPO optics exhibit an effective area of $41$ cm$^2$ at the focal spot for 1 keV X-rays, while the entire telescope system achieves an effective area of $29$ cm$^2$ at 1 keV when taking into account the SDD detector's detection efficiency. The primary contribution to the background is found to be from the Cosmic X-ray Background. Assuming a 625 km orbit with an inclination of $29^\circ$, the total background for CATCH-1 is estimated to be $8.13\times10^{-2}$ counts s$^{-1}$ in the energy range of 0.5--4 keV. Based on the background within the central detector and assuming a Crab-like source spectrum, the estimated ideal sensitivity could achieve $1.9\times10^{-12}$ erg cm$^{-2}$ s$^{-1}$ for an exposure of 10$^4$ s in the energy band of 0.5--4 keV. Furthermore, after simulating the background caused by low-energy charged particles near the geomagnetic equator, we have determined that there is no need to install a magnetic deflector. △ Less

Submitted 23 February, 2024; originally announced February 2024.

On generalized Beauville decompositions

Authors: Younghan Bae, Davesh Maulik, Junliang Shen, Qizheng Yin

Abstract: Motivated by the Beauville decomposition of an abelian scheme and the "Perverse = Chern" phenomenon for a compactified Jacobian fibration, we study in this paper splittings of the perverse filtration for compactified Jacobian fibrations. On the one hand, we prove for the Beauville-Mukai system associated with an irreducible curve class on a K3 surface the existence of a Fourier-stable multiplica… ▽ More Motivated by the Beauville decomposition of an abelian scheme and the "Perverse = Chern" phenomenon for a compactified Jacobian fibration, we study in this paper splittings of the perverse filtration for compactified Jacobian fibrations. On the one hand, we prove for the Beauville-Mukai system associated with an irreducible curve class on a K3 surface the existence of a Fourier-stable multiplicative splitting of the perverse filtration, which extends the Beauville decomposition for the nonsingular fibers. Our approach is to construct a Lefschetz decomposition associated with a Fourier-conjugate $\mathfrak{sl}_2$-triple, which relies heavily on recent work concerning the interaction between derived equivalences and LLV algebras for hyper-Kähler varieties. Motivic lifting and connections to the Beauville-Voisin conjectures are also discussed. On the other hand, we construct for any $g\geq 2$ a compactified Jacobian fibration of genus g curves such that each curve is integral with at worst simple nodes and the (multiplicative) perverse filtration does not admit a multiplicative splitting. This shows that in general an extension of the Beauville decomposition cannot exist for compactified Jacobian fibrations even when the simplest singular point appears. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Comments: 38 pages. Comments are welcome!

Conversational Crowdsensing: A Parallel Intelligence Powered Novel Sensing Approach

Authors: Zhengqiu Zhu, Yong Zhao, Bin Chen, Sihang Qiu, Kai Xu, Quanjun Yin, Jincai Huang, Zhong Liu, Fei-Yue Wang

Abstract: The transition from CPS-based Industry 4.0 to CPSS-based Industry 5.0 brings new requirements and opportunities to current sensing approaches, especially in light of recent progress in Chatbots and Large Language Models (LLMs). Therefore, the advancement of parallel intelligence-powered Crowdsensing Intelligence (CSI) is witnessed, which is currently advancing towards linguistic intelligence. In t… ▽ More The transition from CPS-based Industry 4.0 to CPSS-based Industry 5.0 brings new requirements and opportunities to current sensing approaches, especially in light of recent progress in Chatbots and Large Language Models (LLMs). Therefore, the advancement of parallel intelligence-powered Crowdsensing Intelligence (CSI) is witnessed, which is currently advancing towards linguistic intelligence. In this paper, we propose a novel sensing paradigm, namely conversational crowdsensing, for Industry 5.0. It can alleviate workload and professional requirements of individuals and promote the organization and operation of diverse workforce, thereby facilitating faster response and wider popularization of crowdsensing systems. Specifically, we design the architecture of conversational crowdsensing to effectively organize three types of participants (biological, robotic, and digital) from diverse communities. Through three levels of effective conversation (i.e., inter-human, human-AI, and inter-AI), complex interactions and service functionalities of different workers can be achieved to accomplish various tasks across three sensing phases (i.e., requesting, scheduling, and executing). Moreover, we explore the foundational technologies for realizing conversational crowdsensing, encompassing LLM-based multi-agent systems, scenarios engineering and conversational human-AI cooperation. Finally, we present potential industrial applications of conversational crowdsensing and discuss its implications. We envision that conversations in natural language will become the primary communication channel during crowdsensing process, enabling richer information exchange and cooperative problem-solving among humans, robots, and AI. △ Less

Submitted 4 February, 2024; originally announced February 2024.

StableMask: Refining Causal Masking in Decoder-only Transformer

Authors: Qingyu Yin, Xuzheng He, Xiang Zhuang, Yu Zhao, Jianhua Yao, Xiaoyu Shen, Qiang Zhang

Abstract: The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling. Despite its exceptional performance across various tasks, we have identified two limitations: First, it requires all attention scores to be non-zero and sum up to 1, even if the current embedding has sufficient self-contained information. This comp… ▽ More The decoder-only Transformer architecture with causal masking and relative position encoding (RPE) has become the de facto choice in language modeling. Despite its exceptional performance across various tasks, we have identified two limitations: First, it requires all attention scores to be non-zero and sum up to 1, even if the current embedding has sufficient self-contained information. This compels the model to assign disproportional excessive attention to specific tokens. Second, RPE-based Transformers are not universal approximators due to their limited capacity at encoding absolute positional information, which limits their application in position-critical tasks. In this work, we propose StableMask: a parameter-free method to address both limitations by refining the causal mask. It introduces pseudo-attention values to balance attention distributions and encodes absolute positional information via a progressively decreasing mask ratio. StableMask's effectiveness is validated both theoretically and empirically, showing significant enhancements in language models with parameter sizes ranging from 71M to 1.4B across diverse datasets and encoding methods. We further show that it naturally supports (1) efficient extrapolation without special tricks such as StreamingLLM and (2) easy integration with existing attention optimization techniques. △ Less

Submitted 7 February, 2024; originally announced February 2024.

Comments: Preprint

MEMORYLLM: Towards Self-Updatable Large Language Models

Authors: Yu Wang, Yifan Gao, Xiusi Chen, Haoming Jiang, Shiyang Li, Jingfeng Yang, Qingyu Yin, Zheng Li, Xian Li, Bing Yin, Jingbo Shang, Julian McAuley

Abstract: Existing Large Language Models (LLMs) usually remain static after deployment, which might make it hard to inject new knowledge into the model. We aim to build models containing a considerable portion of self-updatable parameters, enabling the model to integrate new knowledge effectively and efficiently. To this end, we introduce MEMORYLLM, a model that comprises a transformer and a fixed-size memo… ▽ More Existing Large Language Models (LLMs) usually remain static after deployment, which might make it hard to inject new knowledge into the model. We aim to build models containing a considerable portion of self-updatable parameters, enabling the model to integrate new knowledge effectively and efficiently. To this end, we introduce MEMORYLLM, a model that comprises a transformer and a fixed-size memory pool within the latent space of the transformer. MEMORYLLM can self-update with text knowledge and memorize the knowledge injected earlier. Our evaluations demonstrate the ability of MEMORYLLM to effectively incorporate new knowledge, as evidenced by its performance on model editing benchmarks. Meanwhile, the model exhibits long-term information retention capacity, which is validated through our custom-designed evaluations and long-context benchmarks. MEMORYLLM also shows operational integrity without any sign of performance degradation even after nearly a million memory updates. Our code and model are open-sourced at https://github.com/wangyu-ustc/MemoryLLM. △ Less

Submitted 26 May, 2024; v1 submitted 7 February, 2024; originally announced February 2024.

Comments: 13 pages, 9 figures

Scientific Large Language Models: A Survey on Biological & Chemical Domains

Authors: Qiang Zhang, Keyang Ding, Tianwen Lyv, Xinda Wang, Qingyu Yin, Yiwen Zhang, Jing Yu, Yuhao Wang, Xiaotong Li, Zhuoyi Xiang, Xiang Zhuang, Zeyuan Wang, Ming Qin, Mengyao Zhang, Jinlu Zhang, Jiyu Cui, Renjun Xu, Hongyang Chen, Xiaohui Fan, Huabin Xing, Huajun Chen

Abstract: Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension, representing a significant stride toward artificial general intelligence. The application of LLMs extends beyond conventional linguistic boundaries, encompassing specialized linguistic systems developed within various scientific disciplines. This growing interest has led to the advent o… ▽ More Large Language Models (LLMs) have emerged as a transformative power in enhancing natural language comprehension, representing a significant stride toward artificial general intelligence. The application of LLMs extends beyond conventional linguistic boundaries, encompassing specialized linguistic systems developed within various scientific disciplines. This growing interest has led to the advent of scientific LLMs, a novel subclass specifically engineered for facilitating scientific discovery. As a burgeoning area in the community of AI for Science, scientific LLMs warrant comprehensive exploration. However, a systematic and up-to-date survey introducing them is currently lacking. In this paper, we endeavor to methodically delineate the concept of "scientific language", whilst providing a thorough review of the latest advancements in scientific LLMs. Given the expansive realm of scientific disciplines, our analysis adopts a focused lens, concentrating on the biological and chemical domains. This includes an in-depth examination of LLMs for textual knowledge, small molecules, macromolecular proteins, genomic sequences, and their combinations, analyzing them in terms of model architectures, capabilities, datasets, and evaluation. Finally, we critically examine the prevailing challenges and point out promising research directions along with the advances of LLMs. By offering a comprehensive overview of technical developments in this field, this survey aspires to be an invaluable resource for researchers navigating the intricate landscape of scientific LLMs. △ Less

Submitted 26 January, 2024; originally announced January 2024.

The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness

Authors: Mengyao Du, Miao Zhang, Yuwen Pu, Kai Xu, Shouling Ji, Quanjun Yin

Abstract: To tackle the scarcity and privacy issues associated with domain-specific datasets, the integration of federated learning in conjunction with fine-tuning has emerged as a practical solution. However, our findings reveal that federated learning has the risk of skewing fine-tuning features and compromising the out-of-distribution robustness of the model. By introducing three robustness indicators an… ▽ More To tackle the scarcity and privacy issues associated with domain-specific datasets, the integration of federated learning in conjunction with fine-tuning has emerged as a practical solution. However, our findings reveal that federated learning has the risk of skewing fine-tuning features and compromising the out-of-distribution robustness of the model. By introducing three robustness indicators and conducting experiments across diverse robust datasets, we elucidate these phenomena by scrutinizing the diversity, transferability, and deviation within the model feature space. To mitigate the negative impact of federated learning on model robustness, we introduce GNP, a \underline{G}eneral \underline{N}oisy \underline{P}rojection-based robust algorithm, ensuring no deterioration of accuracy on the target distribution. Specifically, the key strategy for enhancing model robustness entails the transfer of robustness from the pre-trained model to the fine-tuned model, coupled with adding a small amount of Gaussian noise to augment the representative capacity of the model. Comprehensive experimental results demonstrate that our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods and confronting different levels of data heterogeneity. △ Less

Submitted 25 January, 2024; originally announced January 2024.

Comments: 12 pages, 10 figures

A photon-level broadband dual-comb interferometer for turbulent open-air trace gases detection application

Authors: Wei Zhong, Yingyu Liu, Qin Yin, Ruocan Zhao, Yiwei Ding, Chong Wang, Tindi Chen, Xiankang Dou, Xianghui Xue

Abstract: Open-path dual-comb spectroscopy (DCS) significantly enhances our understanding of regional trace gases. However, due to technical challenges, cost considerations, and eye-safety regulations, its sensing range and flexibility remain limited. The photon-counting DCS demonstrated recently heralds potential innovations over open-path DCS. Nevertheless, a major challenge in open-air applications of th… ▽ More Open-path dual-comb spectroscopy (DCS) significantly enhances our understanding of regional trace gases. However, due to technical challenges, cost considerations, and eye-safety regulations, its sensing range and flexibility remain limited. The photon-counting DCS demonstrated recently heralds potential innovations over open-path DCS. Nevertheless, a major challenge in open-air applications of this approach lies in accurately extracting information from the arrival time of photons that have traversed the turbulent atmosphere. Here, we demonstrate a photon-level dual-comb interferometer for field deployment in open-air environments, uniquely designed to counteract the impact of optical path-length variations caused by atmospheric turbulence and fiber-length wandering. Under variable optical path-length conditions, 20nm broadband absorption spectrum of H13C14N is acquired, with the power per comb line detected as low as 4 attowatt . Furthermore, this photon-level DCS achieves comb-line resolution with a quantum-noise-limited signal-to-noise (SNR). This paves the way for novel open-path DCS applications, including non-cooperative target sensing and sensing over a hundred-kilometers range, all within a portable, fieldable, eye-safety and low power consumption system. △ Less

Submitted 21 January, 2024; originally announced January 2024.

Comments: 24 pages, 10 figures

The First Polarimetric View on Quasi-Periodic Oscillations in a Black Hole X-ray Binary

Authors: Qingchang Zhao, Lian Tao, Hancheng Li, Shuangnan Zhang, Hua Feng, Mingyu Ge, Long Ji, Yanan Wang, Yue Huang, Xiang Ma, Liang Zhang, Jinlu Qu, Yanjun Xu, Shu Zhang, Qianqing Yin, Qingcang Shui, Ruican Ma, Shujie Zhao, Panping Li, Zixu Yang, Hexin Liu, WeiYu

Abstract: We present the first polarimetric analysis of Quasi-Periodic Oscillations (QPO) in a black hole binary utilizing \textit{IXPE} data. Our study focuses on Swift J1727.8--1613, which experienced a massive outburst that was observed by various telescopes across different wavelengths. The \textit{IXPE} observation we studied was conducted during the Hard-Intermediate state. The polarization degree (PD… ▽ More We present the first polarimetric analysis of Quasi-Periodic Oscillations (QPO) in a black hole binary utilizing \textit{IXPE} data. Our study focuses on Swift J1727.8--1613, which experienced a massive outburst that was observed by various telescopes across different wavelengths. The \textit{IXPE} observation we studied was conducted during the Hard-Intermediate state. The polarization degree (PD) and polarization angle (PA) were measured at 4.28$\pm$0.20\% and $1.9^{\circ}\pm1.4^{\circ}$, respectively. Remarkably, significant QPO signals were detected during this observation, with a QPO frequency of approximately 1.34 Hz and a fractional root-mean-square (RMS) amplitude of about 12.3\%. Furthermore, we conducted a phase-resolved analysis of the QPO using the Hilbert-Huang transform technique. The photon index showed a strong modulation with respect to the QPO phase. In contrast, the PD and PA exhibit no modulations in relation to the QPO phase, which is inconsistent with the expectation of the Lense-Thirring precession of the inner flow. Further theoretical studies are needed to conform with the observational results. △ Less

Submitted 16 January, 2024; originally announced January 2024.

Comments: Accepted for publication in APJL

Strong and Highly Switchable Soft Sticky Adhesives

Authors: Qianfeng Yin, Yilmaz Arin Manav, Yichen Wan, Benyamin Davaji, Ruobing Bai

Abstract: Many biological systems can form strong adhesion to various materials with complex shapes. The adhesion is further switchable between strongly adhering and completely non-adhering in a simple and fast manner. By contrast, no engineering system has yet achieved the same robust adherence and switching. This limitation severely hinders the advancement of several emerging technologies including biomim… ▽ More Many biological systems can form strong adhesion to various materials with complex shapes. The adhesion is further switchable between strongly adhering and completely non-adhering in a simple and fast manner. By contrast, no engineering system has yet achieved the same robust adherence and switching. This limitation severely hinders the advancement of several emerging technologies including biomimetic robots, assembly-based manufacturing, precision medicine, wearable and implantable devices, as well as on-demand material dismantling and recycling for sustainability. Here we present a design approach for strong and highly switchable adhesion by synergizing the surface stickiness, bulk energy dissipation, and stimuli-responsive polymer chains in a thermo-switchable soft sticky adhesive. The adhesive has a high adhesion strength of about 80 kPa with diverse materials at room temperature. The adhesion is highly switchable to near-vanishing (about 0.6 kPa) at an elevated temperature due to the thermo-responsive surface polymer chain retraction. This adhesion switching is reversible and repeatable for many cycles, enabling selective pick-and-release of objects with various materials, shapes, sizes, and weights. The switching time is around 10 s with an adhesive layer of 1 mm, governed by thermal conduction through the adhesive, faster than or comparable to most state-of-the-art methods. The adhesive is self-healing, and can be recycled, dried, stored, reswollen, and reused with nearly intact adhesion and switching properties. The synergistic design combining strong adhesion and stimuli-responsive switching can be potentially extended to various polymer systems, and further enhanced by optimized surface architectures. △ Less

Submitted 3 January, 2024; originally announced January 2024.

Comments: Submitted manuscript

WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuning

Authors: Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang, Can Xu, Yishujie Zhao, Wenxiang Hu, Qiufeng Yin

Abstract: Recent work demonstrates that, after instruction tuning, Code Large Language Models (Code LLMs) can obtain impressive capabilities to address a wide range of code-related tasks. However, current instruction tuning methods for Code LLMs mainly focus on the traditional code generation task, resulting in poor performance in complex multi-task scenarios. In this paper, we concentrate on multiple code-… ▽ More Recent work demonstrates that, after instruction tuning, Code Large Language Models (Code LLMs) can obtain impressive capabilities to address a wide range of code-related tasks. However, current instruction tuning methods for Code LLMs mainly focus on the traditional code generation task, resulting in poor performance in complex multi-task scenarios. In this paper, we concentrate on multiple code-related tasks and present WaveCoder, a series of Code LLMs trained with Widespread And Versatile Enhanced instruction data. To enable the models to tackle complex code-related tasks, we propose a method to stably generate diverse, high-quality instruction data from open source code dataset in multi-task scenarios and obtain CodeSeaXDataset, a dataset comprising 19,915 instruction instances across 4 code-related tasks, which is aimed at improving the generalization ability of Code LLM. Our experiments demonstrate that WaveCoder models significantly outperform other open-source models in terms of the generalization ability across different code-related tasks. Moreover, WaveCoder-Ultra-6.7B presents the state-of-the-art generalization abilities on a wide range of code-related tasks. △ Less

Submitted 7 June, 2024; v1 submitted 20 December, 2023; originally announced December 2023.

Understanding Inter-Session Intentions via Complex Logical Reasoning

Authors: Jiaxin Bai, Chen Luo, Zheng Li, Qingyu Yin, Yangqiu Song

Abstract: Understanding user intentions is essential for improving product recommendations, navigation suggestions, and query reformulations. However, user intentions can be intricate, involving multiple sessions and attribute requirements connected by logical operators such as And, Or, and Not. For instance, a user may search for Nike or Adidas running shoes across various sessions, with a preference for p… ▽ More Understanding user intentions is essential for improving product recommendations, navigation suggestions, and query reformulations. However, user intentions can be intricate, involving multiple sessions and attribute requirements connected by logical operators such as And, Or, and Not. For instance, a user may search for Nike or Adidas running shoes across various sessions, with a preference for purple. In another example, a user may have purchased a mattress in a previous session and is now looking for a matching bed frame without intending to buy another mattress. Existing research on session understanding has not adequately addressed making product or attribute recommendations for such complex intentions. In this paper, we present the task of logical session complex query answering (LS-CQA), where sessions are treated as hyperedges of items, and we frame the problem of complex intention understanding as an LS-CQA task on an aggregated hypergraph of sessions, items, and attributes. This is a unique complex query answering task with sessions as ordered hyperedges. We also introduce a new model, the Logical Session Graph Transformer (LSGT), which captures interactions among items across different sessions and their logical connections using a transformer structure. We analyze the expressiveness of LSGT and prove the permutation invariance of the inputs for the logical operators. By evaluating LSGT on three datasets, we demonstrate that it achieves state-of-the-art results. △ Less

Submitted 14 June, 2024; v1 submitted 21 December, 2023; originally announced December 2023.

The bright black hole X-ray binary 4U 1543--47 during 2021 outburst: a thick accretion disk inflated by high luminosity

Authors: S. J. Zhao, L. Tao, P. P. Li, R. Soria, H. Feng, Y. X. Zhang, R. C. Ma, W. D. Zhang, E. L. Qiao, Q. Q. Yin, S. N. Zhang, L. Zhang, Q. C. Bu, X. Ma, Y. Huang, M. Y. Ge, X. B. Li, Q. C. Zhao, J. Q. Peng, Y. X. Xiao

Abstract: The black hole X-ray binary source 4U 1543--47 experienced a super-Eddington outburst in 2021, reaching a peak flux of up to $\sim1.96\times10^{-7}\rm erg\ \rm cm^{-2}\ \rm s^{-1}$ ($\sim 8.2$ Crab) in the 2--10\,keV band. Soon after the outburst began, it rapidly transitioned into the soft state. Our goal is to understand how the accretion disk structure deviates from a standard thin disk when th… ▽ More The black hole X-ray binary source 4U 1543--47 experienced a super-Eddington outburst in 2021, reaching a peak flux of up to $\sim1.96\times10^{-7}\rm erg\ \rm cm^{-2}\ \rm s^{-1}$ ($\sim 8.2$ Crab) in the 2--10\,keV band. Soon after the outburst began, it rapidly transitioned into the soft state. Our goal is to understand how the accretion disk structure deviates from a standard thin disk when the accretion rate is near Eddington. To do so, we analyzed spectra obtained from quasi-simultaneous observations conducted by the Hard X-ray Modulation Telescope (Insight-HXMT), the Nuclear Spectroscopic Telescope Array (NuSTAR), and the Neil Gehrels Swift Observatory (Swift). These spectra are well-fitted by a model comprising a disk, a weak corona, and a reflection component. We suggest that the reflection component is caused by disk self-irradiation, that is by photons emitted from the inner disk which return to the accretion disk surface, as their trajectories are bent by the strong gravity field. In this scenario, the best-fitting parameters imply that the reflected flux represents more than half of the total flux. Using general relativistic ray-tracing simulations, we show that this scenario is viable when the disk becomes geometrically thick, with a funnel-like shape, as the accretion rate is near or above the Eddington limit. In the specific case of 4U 1543--47, an angle $\gtrsim$ 45 deg between the disk surface and the equatorial plane can explain the required amount of self-irradiation. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Comments: Accepted for publication in Astronomy and Astrophysics. 15 pages, 4 tables, 12 figures

Journal ref: A&A 685, A42 (2024)

DAP: Domain-aware Prompt Learning for Vision-and-Language Navigation

Authors: Ting Liu, Yue Hu, Wansen Wu, Youkai Wang, Kai Xu, Quanjun Yin

Abstract: Following language instructions to navigate in unseen environments is a challenging task for autonomous embodied agents. With strong representation capabilities, pretrained vision-and-language models are widely used in VLN. However, most of them are trained on web-crawled general-purpose datasets, which incurs a considerable domain gap when used for VLN tasks. To address the problem, we propose a… ▽ More Following language instructions to navigate in unseen environments is a challenging task for autonomous embodied agents. With strong representation capabilities, pretrained vision-and-language models are widely used in VLN. However, most of them are trained on web-crawled general-purpose datasets, which incurs a considerable domain gap when used for VLN tasks. To address the problem, we propose a novel and model-agnostic domain-aware prompt learning (DAP) framework. For equipping the pretrained models with specific object-level and scene-level cross-modal alignment in VLN tasks, DAP applies a low-cost prompt tuning paradigm to learn soft visual prompts for extracting in-domain image semantics. Specifically, we first generate a set of in-domain image-text pairs with the help of the CLIP model. Then we introduce soft visual prompts in the input space of the visual encoder in a pretrained model. DAP injects in-domain visual knowledge into the visual encoder of the pretrained model in an efficient way. Experimental results on both R2R and REVERIE show the superiority of DAP compared to existing state-of-the-art methods. △ Less

Submitted 28 December, 2023; v1 submitted 29 November, 2023; originally announced November 2023.

Comments: 4 pages. arXiv admin note: substantial text overlap with arXiv:2309.03661

Pump-induced terahertz conductivity response and peculiar bound state in Mn3Si2Te6

Authors: Qiong Wu, Qiangwei Yin, Sijie Zhang, Tianchen Hu, Dong Wu, Li Yue, Bohan Li, Shuxiang Xu, Rongsheng Li, Qiaomei Liu, Hechang Lei, Tao Dong, Nanlin Wang

Abstract: We report the significant enhancement on ultrafast terahertz optical conductivity and the unexpected formation of a polaronic-like state in semiconductor Mn3Si2Te6 at room temperature. With the absorption of pump photons, the low-frequency terahertz photoconductivity spectrum exhibits a significant rise, quickly forming a broad peak and subsequently shifting to higher energy. The short-lived natur… ▽ More We report the significant enhancement on ultrafast terahertz optical conductivity and the unexpected formation of a polaronic-like state in semiconductor Mn3Si2Te6 at room temperature. With the absorption of pump photons, the low-frequency terahertz photoconductivity spectrum exhibits a significant rise, quickly forming a broad peak and subsequently shifting to higher energy. The short-lived nature of the broad peak, as well as the distribution of optical constants, strongly points towards a transient polaron mechanism. Our study not only provides profound insights into the remarkable photoelectric response of Mn3Si2Te6 but also highlights its significant potential for future photoelectric applications. △ Less

Submitted 25 October, 2023; originally announced November 2023.

Calico Salmon Migration Algorithm: A novel meta-heuristic optimization algorithm

Authors: Chao Min, Junyi Cui, Liwen Zhou, Qian Yin, Yijia Wang

Abstract: A novel population-based optimization method is proposed in this paper, the Calico Salmon Migration Algorithm (CSMA), which is inspired by the natural behavior of calico salmon during their migration for mating. The CSMA optimization process comprises four stages: selecting the search space by swimming into the river, expanding the search space from the river into the ocean, performing precise sea… ▽ More A novel population-based optimization method is proposed in this paper, the Calico Salmon Migration Algorithm (CSMA), which is inspired by the natural behavior of calico salmon during their migration for mating. The CSMA optimization process comprises four stages: selecting the search space by swimming into the river, expanding the search space from the river into the ocean, performing precise search during the migrating process, and breeding new subspecies by the remaining calico salmon population. To evaluate the effectiveness of the new optimizer, we conducted a series of experiments using different optimization problems and compared the results with various optimization algorithms in the literature. The numerical experimental results for benchmark functions demonstrate that the proposed CSMA outperforms other competing optimization algorithms in terms of convergence speed, accuracy, and stability. Furthermore, the Friedman ranking test shows that the CSMA is ranked first among similar algorithms. △ Less

Submitted 10 November, 2023; originally announced November 2023.

Dual-comb spectroscopy over 100km open-air path

Authors: Jin-Jian Han, Wei Zhong, Ruo-Can Zhao, Ting Zeng, Min Li, Jian Lu, Xin-Xin Peng, Xi-Ping Shi, Qin Yin, Yong Wang, Ali Esamdin, Qi Shen, Jian-Yu Guan, Lei Hou, Ji-Gang Ren, Jian-Jun Jia, Yu Wang, Hai-Feng Jiang, XiangHui Xue, Qiang Zhang, Xian-Kang Dou, Jian-Wei Pan

Abstract: Satellite-based greenhouse gases (GHG) sensing technologies play a critical role in the study of global carbon emissions and climate change. However, none of the existing satellite-based GHG sensing technologies can achieve the measurement of broad bandwidth, high temporal-spatial resolution, and high sensitivity at the same time. Recently, dual-comb spectroscopy (DCS) has been proposed as a super… ▽ More Satellite-based greenhouse gases (GHG) sensing technologies play a critical role in the study of global carbon emissions and climate change. However, none of the existing satellite-based GHG sensing technologies can achieve the measurement of broad bandwidth, high temporal-spatial resolution, and high sensitivity at the same time. Recently, dual-comb spectroscopy (DCS) has been proposed as a superior candidate technology for GHG sensing because it can measure broadband spectra with high temporal-spatial resolution and high sensitivity. The main barrier to DCS's display on satellites is its short measurement distance in open air achieved thus far. Prior research has not been able to implement DCS over 20 km of open-air path. Here, by developing a bistatic setup using time-frequency dissemination and high-power optical frequency combs, we have implemented DCS over a 113 km turbulent horizontal open-air path. Our experiment successfully measured GHG with 7 nm spectral bandwidth and a 10 kHz frequency and achieved a CO2 sensing precision of <2 ppm in 5 minutes and <0.6 ppm in 36 minutes. Our results represent a significant step towards advancing the implementation of DCS as a satellite-based technology and improving technologies for GHG monitoring △ Less

Submitted 31 October, 2023; v1 submitted 30 October, 2023; originally announced October 2023.

Comments: 24 pages, 6 figures

Does or did the supernova remnant Cassiopeia A operate as a PeVatron?

Authors: Zhen Cao, F. Aharonian, Q. An, Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (255 additional authors not shown)

Abstract: For decades, supernova remnants (SNRs) have been considered the prime sources of Galactic Cosmic rays (CRs). But whether SNRs can accelerate CR protons to PeV energies and thus dominate CR flux up to the knee is currently under intensive theoretical and phenomenological debate. The direct test of the ability of SNRs to operate as CR PeVatrons can be provided by ultrahigh-energy (UHE;… ▽ More For decades, supernova remnants (SNRs) have been considered the prime sources of Galactic Cosmic rays (CRs). But whether SNRs can accelerate CR protons to PeV energies and thus dominate CR flux up to the knee is currently under intensive theoretical and phenomenological debate. The direct test of the ability of SNRs to operate as CR PeVatrons can be provided by ultrahigh-energy (UHE; $E_γ\geq 100$~TeV) $γ$-rays. In this context, the historical SNR Cassiopeia A (Cas A) is considered one of the most promising target for UHE observations. This paper presents the observation of Cas A and its vicinity by the LHAASO KM2A detector. The exceptional sensitivity of LHAASO KM2A in the UHE band, combined with the young age of Cas A, enabled us to derive stringent model-independent limits on the energy budget of UHE protons and nuclei accelerated by Cas A at any epoch after the explosion. The results challenge the prevailing paradigm that Cas A-type SNRs are major suppliers of PeV CRs in the Milky Way. △ Less

Submitted 25 October, 2023; originally announced October 2023.

Comments: 11 pages, 3 figures, Accepted by the APJL

Mediation Analysis using Semi-parametric Shape-Restricted Regression with Applications

Authors: Qing Yin, Jong-Hyeon Jeong, Xu Qin, Shyamal D Peddada, Jennifer Adibi

Abstract: Often linear regression is used to perform mediation analysis. However, in many instances, the underlying relationships may not be linear, as in the case of placental-fetal hormones and fetal development. Although, the exact functional form of the relationship may be unknown, one may hypothesize the general shape of the relationship. For these reasons, we develop a novel shape-restricted inference… ▽ More Often linear regression is used to perform mediation analysis. However, in many instances, the underlying relationships may not be linear, as in the case of placental-fetal hormones and fetal development. Although, the exact functional form of the relationship may be unknown, one may hypothesize the general shape of the relationship. For these reasons, we develop a novel shape-restricted inference-based methodology for conducting mediation analysis. This work is motivated by an application in fetal endocrinology where researchers are interested in understanding the effects of pesticide application on birth weight, with human chorionic gonadotropin (hCG) as the mediator. We assume a practically plausible set of nonlinear effects of hCG on the birth weight and a linear relationship between pesticide exposure and hCG, with both exposure-outcome and exposure-mediator models being linear in the confounding factors. Using the proposed methodology on a population-level prenatal screening program data, with hCG as the mediator, we discovered that, while the natural direct effects suggest a positive association between pesticide application and birth weight, the natural indirect effects were negative. △ Less

Submitted 13 October, 2023; originally announced October 2023.

Very high energy gamma-ray emission beyond 10 TeV from GRB 221009A

Authors: Zhen Cao, F. Aharonian, Q. An, A. Axikegu, Y. X. Bai, Y. W. Bao, D. Bastieri, X. J. Bi, Y. J. Bi, J. T. Cai, Q. Cao, W. Y. Cao, Zhe Cao, J. Chang, J. F. Chang, A. M. Chen, E. S. Chen, Liang Chen, Lin Chen, Long Chen, M. J. Chen, M. L. Chen, Q. H. Chen, S. H. Chen, S. Z. Chen , et al. (255 additional authors not shown)

Abstract: The highest energy gamma-rays from gamma-ray bursts (GRBs) have important implications for their radiation mechanism. Here we report for the first time the detection of gamma-rays up to 13 TeV from the brightest GRB 221009A by the Large High Altitude Air-shower Observatory (LHAASO). The LHAASO-KM2A detector registered more than 140 gamma-rays with energies above 3 TeV during 230$-$900s after the t… ▽ More The highest energy gamma-rays from gamma-ray bursts (GRBs) have important implications for their radiation mechanism. Here we report for the first time the detection of gamma-rays up to 13 TeV from the brightest GRB 221009A by the Large High Altitude Air-shower Observatory (LHAASO). The LHAASO-KM2A detector registered more than 140 gamma-rays with energies above 3 TeV during 230$-$900s after the trigger. The intrinsic energy spectrum of gamma-rays can be described by a power-law after correcting for extragalactic background light (EBL) absorption. Such a hard spectrum challenges the synchrotron self-Compton (SSC) scenario of relativistic electrons for the afterglow emission above several TeV. Observations of gamma-rays up to 13 TeV from a source with a measured redshift of z=0.151 hints more transparency in intergalactic space than previously expected. Alternatively, one may invoke new physics such as Lorentz Invariance Violation (LIV) or an axion origin of very high energy (VHE) signals. △ Less

Submitted 22 November, 2023; v1 submitted 13 October, 2023; originally announced October 2023.

Comments: 49pages, 11figures

Journal ref: Science Advances, 9, eadj2778 (2023) 15 November 2023

Local Structure-Preserving Relaxation Method for Charged Systems on Unstructured Meshes

Authors: Zhonghua Qiao, Zhenli Xu, Qian Yin, Shenggao Zhou

Abstract: This work considers charged systems described by the modified Poisson--Nernst--Planck (PNP) equations, which incorporate ionic steric effects and the Born solvation energy for dielectric inhomogeneity. Solving the steady-state modified PNP equations poses numerical challenges due to the emergence of sharp boundary layers caused by small Debye lengths, particularly when local ionic concentrations r… ▽ More This work considers charged systems described by the modified Poisson--Nernst--Planck (PNP) equations, which incorporate ionic steric effects and the Born solvation energy for dielectric inhomogeneity. Solving the steady-state modified PNP equations poses numerical challenges due to the emergence of sharp boundary layers caused by small Debye lengths, particularly when local ionic concentrations reach saturation. To address this, we first reformulate the steady-state problem as a constraint optimization, where the ionic concentrations on unstructured Delaunay nodes are treated as fractional particles moving along edges between nodes. The electric fields are then updated to minimize the objective free energy while satisfying the discrete Gauss's law. We develop a local relaxation method on unstructured meshes that inherently respects the discrete Gauss's law, ensuring curl-free electric fields. Numerical analysis demonstrates that the optimal mass of the moving fractional particles guarantees the positivity of both ionic and solvent concentrations. Additionally, the free energy of the charged system consistently decreases during successive updates of ionic concentrations and electric fields. We conduct numerical tests to validate the expected numerical accuracy, positivity, free-energy dissipation, and robustness of our method in simulating charged systems with sharp boundary layers. △ Less

Submitted 9 October, 2023; originally announced October 2023.

Asymmetrically Decentralized Federated Learning

Authors: Qinglun Li, Miao Zhang, Nan Yin, Quanjun Yin, Li Shen

Abstract: To address the communication burden and privacy concerns associated with the centralized server in Federated Learning (FL), Decentralized Federated Learning (DFL) has emerged, which discards the server with a peer-to-peer (P2P) communication framework. However, most existing DFL algorithms are based on symmetric topologies, such as ring and grid topologies, which can easily lead to deadlocks and a… ▽ More To address the communication burden and privacy concerns associated with the centralized server in Federated Learning (FL), Decentralized Federated Learning (DFL) has emerged, which discards the server with a peer-to-peer (P2P) communication framework. However, most existing DFL algorithms are based on symmetric topologies, such as ring and grid topologies, which can easily lead to deadlocks and are susceptible to the impact of network link quality in practice. To address these issues, this paper proposes the DFedSGPSM algorithm, which is based on asymmetric topologies and utilizes the Push-Sum protocol to effectively solve consensus optimization problems. To further improve algorithm performance and alleviate local heterogeneous overfitting in Federated Learning (FL), our algorithm combines the Sharpness Aware Minimization (SAM) optimizer and local momentum. The SAM optimizer employs gradient perturbations to generate locally flat models and searches for models with uniformly low loss values, mitigating local heterogeneous overfitting. The local momentum accelerates the optimization process of the SAM optimizer. Theoretical analysis proves that DFedSGPSM achieves a convergence rate of $\mathcal{O}(\frac{1}{\sqrt{T}})$ in a non-convex smooth setting under mild assumptions. This analysis also reveals that better topological connectivity achieves tighter upper bounds. Empirically, extensive experiments are conducted on the MNIST, CIFAR10, and CIFAR100 datasets, demonstrating the superior performance of our algorithm compared to state-of-the-art optimizers. △ Less

Submitted 8 October, 2023; originally announced October 2023.

Improve the efficiency of deep reinforcement learning through semantic exploration guided by natural language

Authors: Zhourui Guo, Meng Yao, Yang Yu, Qiyue Yin

Abstract: Reinforcement learning is a powerful technique for learning from trial and error, but it often requires a large number of interactions to achieve good performance. In some domains, such as sparse-reward tasks, an oracle that can provide useful feedback or guidance to the agent during the learning process is really of great importance. However, querying the oracle too frequently may be costly or im… ▽ More Reinforcement learning is a powerful technique for learning from trial and error, but it often requires a large number of interactions to achieve good performance. In some domains, such as sparse-reward tasks, an oracle that can provide useful feedback or guidance to the agent during the learning process is really of great importance. However, querying the oracle too frequently may be costly or impractical, and the oracle may not always have a clear answer for every situation. Therefore, we propose a novel method for interacting with the oracle in a selective and efficient way, using a retrieval-based approach. We assume that the interaction can be modeled as a sequence of templated questions and answers, and that there is a large corpus of previous interactions available. We use a neural network to encode the current state of the agent and the oracle, and retrieve the most relevant question from the corpus to ask the oracle. We then use the oracle's answer to update the agent's policy and value function. We evaluate our method on an object manipulation task. We show that our method can significantly improve the efficiency of RL by reducing the number of interactions needed to reach a certain level of performance, compared to baselines that do not use the oracle or use it in a naive way. △ Less

Submitted 20 September, 2023; originally announced September 2023.

Low-Quality Training Data Only? A Robust Framework for Detecting Encrypted Malicious Network Traffic

Authors: Yuqi Qing, Qilei Yin, Xinhao Deng, Yihao Chen, Zhuotao Liu, Kun Sun, Ke Xu, Jia Zhang, Qi Li

Abstract: Machine learning (ML) is promising in accurately detecting malicious flows in encrypted network traffic; however, it is challenging to collect a training dataset that contains a sufficient amount of encrypted malicious data with correct labels. When ML models are trained with low-quality training data, they suffer degraded performance. In this paper, we aim at addressing a real-world low-quality t… ▽ More Machine learning (ML) is promising in accurately detecting malicious flows in encrypted network traffic; however, it is challenging to collect a training dataset that contains a sufficient amount of encrypted malicious data with correct labels. When ML models are trained with low-quality training data, they suffer degraded performance. In this paper, we aim at addressing a real-world low-quality training dataset problem, namely, detecting encrypted malicious traffic generated by continuously evolving malware. We develop RAPIER that fully utilizes different distributions of normal and malicious traffic data in the feature space, where normal data is tightly distributed in a certain area and the malicious data is scattered over the entire feature space to augment training data for model training. RAPIER includes two pre-processing modules to convert traffic into feature vectors and correct label noises. We evaluate our system on two public datasets and one combined dataset. With 1000 samples and 45% noises from each dataset, our system achieves the F1 scores of 0.770, 0.776, and 0.855, respectively, achieving average improvements of 352.6%, 284.3%, and 214.9% over the existing methods, respectively. Furthermore, We evaluate RAPIER with a real-world dataset obtained from a security enterprise. RAPIER effectively achieves encrypted malicious traffic detection with the best F1 score of 0.773 and improves the F1 score of existing methods by an average of 272.5%. △ Less

Submitted 9 September, 2023; originally announced September 2023.

Prompt-based Context- and Domain-aware Pretraining for Vision and Language Navigation

Authors: Ting Liu, Yue Hu, Wansen Wu, Youkai Wang, Kai Xu, Quanjun Yin

Abstract: Pretrained visual-language models have extensive world knowledge and are widely used in visual and language navigation (VLN). However, they are not sensitive to indoor scenarios for VLN tasks. Another challenge for VLN is how the agent understands the contextual relations between actions on a path and performs cross-modal alignment sequentially. In this paper, we propose a novel Prompt-bAsed coNte… ▽ More Pretrained visual-language models have extensive world knowledge and are widely used in visual and language navigation (VLN). However, they are not sensitive to indoor scenarios for VLN tasks. Another challenge for VLN is how the agent understands the contextual relations between actions on a path and performs cross-modal alignment sequentially. In this paper, we propose a novel Prompt-bAsed coNtext- and inDoor-Aware (PANDA) pretraining framework to address these problems. It performs prompting in two stages. In the indoor-aware stage, we apply an efficient tuning paradigm to learn deep visual prompts from an indoor dataset, in order to augment pretrained models with inductive biases towards indoor environments. This can enable more sample-efficient adaptation for VLN agents. Furthermore, in the context-aware stage, we design a set of hard context prompts to capture the sequence-level semantics in the instruction. They enable further tuning of the pretrained models via contrastive learning. Experimental results on both R2R and REVERIE show the superiority of PANDA compared to existing state-of-the-art methods. △ Less

Submitted 14 December, 2023; v1 submitted 7 September, 2023; originally announced September 2023.

Comments: 12 pages

Perverse filtrations and Fourier transforms

Authors: Davesh Maulik, Junliang Shen, Qizheng Yin

Abstract: We study the interaction between Fourier-Mukai transforms and perverse filtrations for a certain class of dualizable abelian fibrations. Multiplicativity of the perverse filtration and the "Perverse $\supset$ Chern" phenomenon for these abelian fibrations are immediate consequences of our theory. We also show that our class of fibrations include families of compactified Jacobians of integral local… ▽ More We study the interaction between Fourier-Mukai transforms and perverse filtrations for a certain class of dualizable abelian fibrations. Multiplicativity of the perverse filtration and the "Perverse $\supset$ Chern" phenomenon for these abelian fibrations are immediate consequences of our theory. We also show that our class of fibrations include families of compactified Jacobians of integral locally planar curves. Applications include the following: (a) we prove the motivic decomposition conjecture for this class (including compactified Jacobian fibrations), which generalizes Deninger-Murre's theorem for abelian schemes; (b) we provide a new proof of the P=W conjecture for $\mathrm{GL}_r$; (c) we prove half of the P=C conjecture concerning refined BPS invariants for the local $\mathbb{P}^2$; (d) we show that the perverse filtration for the compactified Jacobian associated with an integral locally planar curve is multiplicative, which generalizes a result of Oblomkov-Yun for homogeneous singularities. Our techniques combine Arinkin's autoduality for coherent categories, Ngô's support theorem for the decomposition theorem, Adams operations in operational K-theory, and Corti-Hanamura's theory of relative Chow motives. △ Less

Submitted 24 August, 2023; originally announced August 2023.

Comments: 55 pages; comments are welcome

Nearly-room-temperature ferromagnetism and tunable anomalous Hall effect in atomically thin Fe4CoGeTe2

Authors: Shaohua Yan, Hui-Hui He, Yang Fu, Ning-Ning Zhao, Shangjie Tian, Qiangwei Yin, Fanyu Meng, Xinyu Cao, Le Wang, Shanshan Chen, Ki-Hoon Son, Jun Woo Choi, Hyejin Ryu, Shouguo Wang, Xiao Zhang, Kai Liu, Hechang Lei

Abstract: Itinerant ferromagnetism at room temperature is a key ingredient for spin transport and manipulation. Here, we report the realization of nearly-room-temperature itinerant ferromagnetism in Co doped Fe5GeTe2 thin flakes. The ferromagnetic transition temperature TC (323 K - 337 K) is almost unchanged when thickness is down to 12 nm and is still about 284 K at 2 nm (bilayer thickness). Theoretical ca… ▽ More Itinerant ferromagnetism at room temperature is a key ingredient for spin transport and manipulation. Here, we report the realization of nearly-room-temperature itinerant ferromagnetism in Co doped Fe5GeTe2 thin flakes. The ferromagnetic transition temperature TC (323 K - 337 K) is almost unchanged when thickness is down to 12 nm and is still about 284 K at 2 nm (bilayer thickness). Theoretical calculations further indicate that the ferromagnetism persists in monolayer Fe4CoGeTe2. In addition to the robust ferromagnetism down to the ultrathin limit, Fe4CoGeTe2 exhibits an unusual temperature- and thickness-dependent intrinsic anomalous Hall effect. We propose that it could be ascribed to the dependence of band structure on thickness that changes the Berry curvature near the Fermi energy level subtly. The nearly-room-temperature ferromagnetism and tunable anomalous Hall effect in atomically thin Fe4CoGeTe2 provide opportunities to understand the exotic transport properties of two-dimensional van der Waals magnetic materials and explore their potential applications in spintronics. △ Less

Submitted 24 August, 2023; originally announced August 2023.

Comments: 28 pages, 4 figures, 1 table

DFedADMM: Dual Constraints Controlled Model Inconsistency for Decentralized Federated Learning

Authors: Qinglun Li, Li Shen, Guanghao Li, Quanjun Yin, Dacheng Tao

Abstract: To address the communication burden issues associated with federated learning (FL), decentralized federated learning (DFL) discards the central server and establishes a decentralized communication network, where each client communicates only with neighboring clients. However, existing DFL methods still suffer from two major challenges: local inconsistency and local heterogeneous overfitting, which… ▽ More To address the communication burden issues associated with federated learning (FL), decentralized federated learning (DFL) discards the central server and establishes a decentralized communication network, where each client communicates only with neighboring clients. However, existing DFL methods still suffer from two major challenges: local inconsistency and local heterogeneous overfitting, which have not been fundamentally addressed by existing DFL methods. To tackle these issues, we propose novel DFL algorithms, DFedADMM and its enhanced version DFedADMM-SAM, to enhance the performance of DFL. The DFedADMM algorithm employs primal-dual optimization (ADMM) by utilizing dual variables to control the model inconsistency raised from the decentralized heterogeneous data distributions. The DFedADMM-SAM algorithm further improves on DFedADMM by employing a Sharpness-Aware Minimization (SAM) optimizer, which uses gradient perturbations to generate locally flat models and searches for models with uniformly low loss values to mitigate local heterogeneous overfitting. Theoretically, we derive convergence rates of $\small \mathcal{O}\Big(\frac{1}{\sqrt{KT}}+\frac{1}{KT(1-ψ)^2}\Big)$ and $\small \mathcal{O}\Big(\frac{1}{\sqrt{KT}}+\frac{1}{KT(1-ψ)^2}+ \frac{1}{T^{3/2}K^{1/2}}\Big)$ in the non-convex setting for DFedADMM and DFedADMM-SAM, respectively, where $1 - ψ$ represents the spectral gap of the gossip matrix. Empirically, extensive experiments on MNIST, CIFAR10 and CIFAR100 datesets demonstrate that our algorithms exhibit superior performance in terms of both generalization and convergence speed compared to existing state-of-the-art (SOTA) optimizers in DFL. △ Less

Submitted 16 August, 2023; originally announced August 2023.

Comments: 24 pages

Beam Detection Based on Machine Learning Algorithms

Authors: Haoyuan Li, Qing Yin

Abstract: The positions of free electron laser beams on screens are precisely determined by a sequence of machine learning models. Transfer training is conducted in a self-constructed convolutional neural network based on VGG16 model. Output of intermediate layers are passed as features to a support vector regression model. With this sequence, 85.8% correct prediction is achieved on test data. The positions of free electron laser beams on screens are precisely determined by a sequence of machine learning models. Transfer training is conducted in a self-constructed convolutional neural network based on VGG16 model. Output of intermediate layers are passed as features to a support vector regression model. With this sequence, 85.8% correct prediction is achieved on test data. △ Less

Submitted 31 July, 2023; originally announced August 2023.