Showing 1–50 of 94 results for author: Fung, C

FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuning

Authors: Jiaxing QI, Zhongzhi Luan, Shaohan Huang, Carol Fung, Hailong Yang, Depei Qian

Abstract: Large language models (LLMs) have emerged as important components across various fields, yet their training requires substantial computation resources and abundant labeled data. It poses a challenge to robustly training LLMs for individual users (clients). To tackle this challenge, the intuitive idea is to introduce federated learning (FL), which can collaboratively train models on distributed pri… ▽ More Large language models (LLMs) have emerged as important components across various fields, yet their training requires substantial computation resources and abundant labeled data. It poses a challenge to robustly training LLMs for individual users (clients). To tackle this challenge, the intuitive idea is to introduce federated learning (FL), which can collaboratively train models on distributed private data. However, existing methods suffer from the challenges of data heterogeneity, system heterogeneity, and model size, resulting in suboptimal performance and high costs. In this work, we proposed a variant of personalized federated learning (PFL) framework, namely FDLoRA, which allows the client to be a single device or a cluster and adopts low-rank adaptation (LoRA) tuning. FDLoRA sets dual LoRA modules on each client to capture personalized and global knowledge, respectively, and only the global LoRA module uploads parameters to the central server to aggregate cross-client knowledge. Finally, an adaptive fusion approach is employed to combine the parameters of the dual LoRAs. This enables FDLoRA to make effective use of private data distributed across different clients, thereby improving performance on the client without incurring high communication and computing costs. We conducted extensive experiments in two practice scenarios. The results demonstrate that FDLoRA outperforms six baselines in terms of performance, stability, robustness, computation cost, and communication cost. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Astrocytic NMDA Receptors Modulate the Dynamics of Continuous Attractors

Authors: Zihan Liu, Flavia Nathaline Chanentia, Patteera Supvithayanong, Chi Chung Alan Fung

Abstract: Neuronal networking supports complex brain functions, with neurotransmitters facilitating communication through chemical synapses. The release probability of neurotransmitters varies and is influenced by pre-synaptic neuronal activity. Recent findings suggest that blocking astrocytic N-Methyl-D-Aspartate (NMDA) receptors reduces this variation. However, the theoretical implications of this reducti… ▽ More Neuronal networking supports complex brain functions, with neurotransmitters facilitating communication through chemical synapses. The release probability of neurotransmitters varies and is influenced by pre-synaptic neuronal activity. Recent findings suggest that blocking astrocytic N-Methyl-D-Aspartate (NMDA) receptors reduces this variation. However, the theoretical implications of this reduction on neuronal dynamics have not been thoroughly investigated. Utilizing continuous attractor neural network (CANN) models with short-term synaptic depression (STD), we explore the effects of reduced release probability variation. Our results show that blocking astrocytic NMDA receptors stabilizes attractor states and diminishes their mobility. These insights enhance our understanding of NMDA receptors' role in astrocytes and their broader impact on neural computation and memory, with potential implications for neurological conditions involving NMDA receptor antagonists. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: 22 pages, 6 figures

A Revisit of the Optimal Excess-of-Loss Contract

Authors: Ernest Aboagye, Vali Asimit, Tsz Chai Fung, Liang Peng, Qiuqi Wang

Abstract: It is well-known that Excess-of-Loss reinsurance has more marketability than Stop-Loss reinsurance, though Stop-Loss reinsurance is the most prominent setting discussed in the optimal (re)insurance design literature. We point out that optimal reinsurance policy under Stop-Loss leads to a zero insolvency probability, which motivates our paper. We provide a remedy to this peculiar property of the op… ▽ More It is well-known that Excess-of-Loss reinsurance has more marketability than Stop-Loss reinsurance, though Stop-Loss reinsurance is the most prominent setting discussed in the optimal (re)insurance design literature. We point out that optimal reinsurance policy under Stop-Loss leads to a zero insolvency probability, which motivates our paper. We provide a remedy to this peculiar property of the optimal Stop-Loss reinsurance contract by investigating the optimal Excess-of-Loss reinsurance contract instead. We also provide estimators for the optimal Excess-of-Loss and Stop-Loss contracts and investigate their statistical properties under many premium principle assumptions and various risk preferences, which according to our knowledge, have never been investigated in the literature. Simulated data and real-life data are used to illustrate our main theoretical findings. △ Less

Submitted 30 April, 2024; originally announced May 2024.

Dynamic Neural Control Flow Execution: An Agent-Based Deep Equilibrium Approach for Binary Vulnerability Detection

Authors: Litao Li, Steven H. H. Ding, Andrew Walenstein, Philippe Charland, Benjamin C. M. Fung

Abstract: Software vulnerabilities are a challenge in cybersecurity. Manual security patches are often difficult and slow to be deployed, while new vulnerabilities are created. Binary code vulnerability detection is less studied and more complex compared to source code, and this has important practical implications. Deep learning has become an efficient and powerful tool in the security domain, where it pro… ▽ More Software vulnerabilities are a challenge in cybersecurity. Manual security patches are often difficult and slow to be deployed, while new vulnerabilities are created. Binary code vulnerability detection is less studied and more complex compared to source code, and this has important practical implications. Deep learning has become an efficient and powerful tool in the security domain, where it provides end-to-end and accurate prediction. Modern deep learning approaches learn the program semantics through sequence and graph neural networks, using various intermediate representation of programs, such as abstract syntax trees (AST) or control flow graphs (CFG). Due to the complex nature of program execution, the output of an execution depends on the many program states and inputs. Also, a CFG generated from static analysis can be an overestimation of the true program flow. Moreover, the size of programs often does not allow a graph neural network with fixed layers to aggregate global information. To address these issues, we propose DeepEXE, an agent-based implicit neural network that mimics the execution path of a program. We use reinforcement learning to enhance the branching decision at every program state transition and create a dynamic environment to learn the dependency between a vulnerability and certain program states. An implicitly defined neural network enables nearly infinite state transitions until convergence, which captures the structural information at a higher level. The experiments are conducted on two semi-synthetic and two real-world datasets. We show that DeepEXE is an accurate and efficient method and outperforms the state-of-the-art vulnerability detection methods. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Carthago Delenda Est: Co-opetitive Indirect Information Diffusion Model for Influence Operations on Online Social Media

Authors: Jwen Fai Low, Benjamin C. M. Fung, Farkhund Iqbal, Claude Fachkha

Abstract: For a state or non-state actor whose credibility is bankrupt, relying on bots to conduct non-attributable, non-accountable, and seemingly-grassroots-but-decentralized-in-actuality influence/information operations (info ops) on social media can help circumvent the issue of trust deficit while advancing its interests. Planning and/or defending against decentralized info ops can be aided by computati… ▽ More For a state or non-state actor whose credibility is bankrupt, relying on bots to conduct non-attributable, non-accountable, and seemingly-grassroots-but-decentralized-in-actuality influence/information operations (info ops) on social media can help circumvent the issue of trust deficit while advancing its interests. Planning and/or defending against decentralized info ops can be aided by computational simulations in lieu of ethically-fraught live experiments on social media. In this study, we introduce Diluvsion, an agent-based model for contested information propagation efforts on Twitter-like social media. The model emphasizes a user's belief in an opinion (stance) being impacted by the perception of potentially illusory popular support from constant incoming floods of indirect information, floods that can be cooperatively engineered in an uncoordinated manner by bots as they compete to spread their stances. Our model, which has been validated against real-world data, is an advancement over previous models because we account for engagement metrics in influencing stance adoption, non-social tie spreading of information, neutrality as a stance that can be spread, and themes that are analogous to media's framing effect and are symbiotic with respect to stance propagation. The strengths of the Diluvsion model are demonstrated in simulations of orthodox info ops, e.g., maximizing adoption of one stance; creating echo chambers; inducing polarization; and unorthodox info ops, e.g., simultaneous support of multiple stances as a Trojan horse tactic for the dissemination of a theme. △ Less

Submitted 6 February, 2024; v1 submitted 2 February, 2024; originally announced February 2024.

Comments: 60 pages, 9 figures, 1 table

Model Selection of Zero-shot Anomaly Detectors in the Absence of Labeled Validation Data

Authors: Clement Fung, Chen Qiu, Aodong Li, Maja Rudolph

Abstract: Anomaly detection requires detecting abnormal samples in large unlabeled datasets. While progress in deep learning and the advent of foundation models has produced powerful zero-shot anomaly detection methods, their deployment in practice is often hindered by the lack of labeled data -- without it, their detection performance cannot be evaluated reliably. In this work, we propose SWSA (Selection W… ▽ More Anomaly detection requires detecting abnormal samples in large unlabeled datasets. While progress in deep learning and the advent of foundation models has produced powerful zero-shot anomaly detection methods, their deployment in practice is often hindered by the lack of labeled data -- without it, their detection performance cannot be evaluated reliably. In this work, we propose SWSA (Selection With Synthetic Anomalies): a general-purpose framework to select image-based anomaly detectors with a generated synthetic validation set. Our proposed anomaly generation method assumes access to only a small support set of normal images and requires no training or fine-tuning. Once generated, our synthetic validation set is used to create detection tasks that compose a validation framework for model selection. In an empirical study, we find that SWSA often selects models that match selections made with a ground-truth validation set, resulting in higher AUROCs than baseline methods. We also find that SWSA selects prompts for CLIP-based anomaly detection that outperform baseline prompt selection strategies on all datasets, including the challenging MVTec-AD and VisA datasets. △ Less

Submitted 9 February, 2024; v1 submitted 16 October, 2023; originally announced October 2023.

Comments: 14 pages

Short-Term Postsynaptic Plasticity Facilitates Predictive Tracking in Continuous Attractors

Authors: Huilin Zhao, Sungchil Yang, Chi Chung Alan Fung

Abstract: The N-methyl-D-aspartate receptor (NMDAR) is a crucial component of synaptic transmission, and its dysfunction is implicated in many neurological diseases and psychiatric conditions. NMDAR-based short-term postsynaptic plasticity (STPP) is a newly discovered postsynaptic response facilitation mechanism. Our group has suggested that long-lasting glutamate binding of NMDAR allows input information t… ▽ More The N-methyl-D-aspartate receptor (NMDAR) is a crucial component of synaptic transmission, and its dysfunction is implicated in many neurological diseases and psychiatric conditions. NMDAR-based short-term postsynaptic plasticity (STPP) is a newly discovered postsynaptic response facilitation mechanism. Our group has suggested that long-lasting glutamate binding of NMDAR allows input information to be held for up to 500 ms or longer in brain slices, which contributes to response facilitation. However, the implications of STPP in the dynamics of neuronal populations remain unknown. In this study, we implemented STPP in a continuous attractor neural network (CANN) model to describe the neural information encoded in neuronal populations. Unlike short-term facilitation, which is a kind of presynaptic plasticity, the temporally enhanced synaptic efficacy induced by STPP destabilizes the network state of the CANN by increasing the mobility of the system. This nontrivial dynamical effect enables a CANN with STPP to track a moving stimulus predictively, i.e., the network state responds to the anticipated stimulus. Our findings reveal a novel STPP-based mechanism for sensory prediction that can help develop brain-inspired computational algorithms for prediction. △ Less

Submitted 27 September, 2023; originally announced September 2023.

Comments: 29 pages, 9 figures

LogGPT: Exploring ChatGPT for Log-Based Anomaly Detection

Authors: Jiaxing Qi, Shaohan Huang, Zhongzhi Luan, Carol Fung, Hailong Yang, Depei Qian

Abstract: The increasing volume of log data produced by software-intensive systems makes it impractical to analyze them manually. Many deep learning-based methods have been proposed for log-based anomaly detection. These methods face several challenges such as high-dimensional and noisy log data, class imbalance, generalization, and model interpretability. Recently, ChatGPT has shown promising results in va… ▽ More The increasing volume of log data produced by software-intensive systems makes it impractical to analyze them manually. Many deep learning-based methods have been proposed for log-based anomaly detection. These methods face several challenges such as high-dimensional and noisy log data, class imbalance, generalization, and model interpretability. Recently, ChatGPT has shown promising results in various domains. However, there is still a lack of study on the application of ChatGPT for log-based anomaly detection. In this work, we proposed LogGPT, a log-based anomaly detection framework based on ChatGPT. By leveraging the ChatGPT's language interpretation capabilities, LogGPT aims to explore the transferability of knowledge from large-scale corpora to log-based anomaly detection. We conduct experiments to evaluate the performance of LogGPT and compare it with three deep learning-based methods on BGL and Spirit datasets. LogGPT shows promising results and has good interpretability. This study provides preliminary insights into prompt-based models, such as ChatGPT, for the log-based anomaly detection task. △ Less

Submitted 3 September, 2023; originally announced September 2023.

Pluvio: Assembly Clone Search for Out-of-domain Architectures and Libraries through Transfer Learning and Conditional Variational Information Bottleneck

Authors: Zhiwei Fu, Steven H. H. Ding, Furkan Alaca, Benjamin C. M. Fung, Philippe Charland

Abstract: The practice of code reuse is crucial in software development for a faster and more efficient development lifecycle. In reality, however, code reuse practices lack proper control, resulting in issues such as vulnerability propagation and intellectual property infringements. Assembly clone search, a critical shift-right defence mechanism, has been effective in identifying vulnerable code resulting… ▽ More The practice of code reuse is crucial in software development for a faster and more efficient development lifecycle. In reality, however, code reuse practices lack proper control, resulting in issues such as vulnerability propagation and intellectual property infringements. Assembly clone search, a critical shift-right defence mechanism, has been effective in identifying vulnerable code resulting from reuse in released executables. Recent studies on assembly clone search demonstrate a trend towards using machine learning-based methods to match assembly code variants produced by different toolchains. However, these methods are limited to what they learn from a small number of toolchain variants used in training, rendering them inapplicable to unseen architectures and their corresponding compilation toolchain variants. This paper presents the first study on the problem of assembly clone search with unseen architectures and libraries. We propose incorporating human common knowledge through large-scale pre-trained natural language models, in the form of transfer learning, into current learning-based approaches for assembly clone search. Transfer learning can aid in addressing the limitations of the existing approaches, as it can bring in broader knowledge from human experts in assembly code. We further address the sequence limit issue by proposing a reinforcement learning agent to remove unnecessary and redundant tokens. Coupled with a new Variational Information Bottleneck learning strategy, the proposed system minimizes the reliance on potential indicators of architectures and optimization settings, for a better generalization of unseen architectures. We simulate the unseen architecture clone search scenarios and the experimental results show the effectiveness of the proposed approach against the state-of-the-art solutions. △ Less

Submitted 20 July, 2023; originally announced July 2023.

Comments: 13 pages and 4 figures. This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Diagnostic Tests Before Modeling Longitudinal Actuarial Data

Authors: Yinhuan Li, Tsz Chai Fung, Liang Peng, Linyi Qian

Abstract: In non-life insurance, it is essential to understand the serial dynamics and dependence structure of the longitudinal insurance data before using them. Existing actuarial literature primarily focuses on modeling, which typically assumes a lack of serial dynamics and a pre-specified dependence structure of claims across multiple years. To fill in the research gap, we develop two diagnostic tests, n… ▽ More In non-life insurance, it is essential to understand the serial dynamics and dependence structure of the longitudinal insurance data before using them. Existing actuarial literature primarily focuses on modeling, which typically assumes a lack of serial dynamics and a pre-specified dependence structure of claims across multiple years. To fill in the research gap, we develop two diagnostic tests, namely the serial dynamic test and correlation test, to assess the appropriateness of these assumptions and provide justifiable modeling directions. The tests involve the following ingredients: i) computing the change of the cross-sectional estimated parameters under a logistic regression model and the empirical residual correlations of the claim occurrence indicators across time, which serve as the indications to detect serial dynamics; ii) quantifying estimation uncertainty using the randomly weighted bootstrap approach; iii) developing asymptotic theories to construct proper test statistics. The proposed tests are examined by simulated data and applied to two non-life insurance datasets, revealing that the two datasets behave differently. △ Less

Submitted 30 April, 2023; originally announced May 2023.

LogQA: Question Answering in Unstructured Logs

Authors: Shaohan Huang, Yi Liu, Carol Fung, Jiaxing Qi, Hailong Yang, Zhongzhi Luan

Abstract: Modern systems produce a large volume of logs to record run-time status and events. System operators use these raw logs to track a system in order to obtain some useful information to diagnose system anomalies. One of the most important problems in this area is to help operators find the answers to log-based questions efficiently and user-friendly. In this work, we propose LogQA, which aims at ans… ▽ More Modern systems produce a large volume of logs to record run-time status and events. System operators use these raw logs to track a system in order to obtain some useful information to diagnose system anomalies. One of the most important problems in this area is to help operators find the answers to log-based questions efficiently and user-friendly. In this work, we propose LogQA, which aims at answering log-based questions in the form of natural language based on large-scale unstructured log corpora. Our system presents the answer to a question directly instead of returning a list of relevant snippets, thus offering better user-friendliness and efficiency. LogQA represents the first approach to solve question answering in lod domain. LogQA has two key components: Log Retriever and Log Reader. Log Retriever aims at retrieving relevant logs w.r.t. a given question, while Log Reader is responsible for inferring the final answer. Given the lack of a public dataset for log questing answering, we manually labelled a QA dataset of three open-source log corpus and will make them publicly available. We evaluated our proposed model on these datasets by comparing its performance with 6 other baseline methods. Our experimental results demonstrate that LogQA has outperformed other baseline methods. △ Less

Submitted 21 March, 2023; originally announced March 2023.

Modelling Multi-relations for Convolutional-based Knowledge Graph Embedding

Authors: Sirui Li, Kok Wai Wong, Dengya Zhu, Chun Che Fung

Abstract: Representation learning of knowledge graphs aims to embed entities and relations into low-dimensional vectors. Most existing works only consider the direct relations or paths between an entity pair. It is considered that such approaches disconnect the semantic connection of multi-relations between an entity pair, and we propose a convolutional and multi-relational representation learning model, Co… ▽ More Representation learning of knowledge graphs aims to embed entities and relations into low-dimensional vectors. Most existing works only consider the direct relations or paths between an entity pair. It is considered that such approaches disconnect the semantic connection of multi-relations between an entity pair, and we propose a convolutional and multi-relational representation learning model, ConvMR. The proposed ConvMR model addresses the multi-relation issue in two aspects: (1) Encoding the multi-relations between an entity pair into a unified vector that maintains the semantic connection. (2) Since not all relations are necessary while joining multi-relations, we propose an attention-based relation encoder to automatically assign weights to different relations based on semantic hierarchy. Experimental results on two popular datasets, FB15k-237 and WN18RR, achieved consistent improvements on the mean rank. We also found that ConvMR is efficient to deal with less frequent entities. △ Less

Submitted 20 October, 2022; originally announced October 2022.

Comments: Knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 26th International Conference KES2022

A Posteriori Risk Classification and Ratemaking with Random Effects in the Mixture-of-Experts Model

Authors: Spark C. Tseung, Ian Weng Chan, Tsz Chai Fung, Andrei L. Badescu, X. Sheldon Lin

Abstract: A well-designed framework for risk classification and ratemaking in automobile insurance is key to insurers' profitability and risk management, while also ensuring that policyholders are charged a fair premium according to their risk profile. In this paper, we propose to adapt a flexible regression model, called the Mixed LRMoE, to the problem of a posteriori risk classification and ratemaking, wh… ▽ More A well-designed framework for risk classification and ratemaking in automobile insurance is key to insurers' profitability and risk management, while also ensuring that policyholders are charged a fair premium according to their risk profile. In this paper, we propose to adapt a flexible regression model, called the Mixed LRMoE, to the problem of a posteriori risk classification and ratemaking, where policyholder-level random effects are incorporated to better infer their risk profile reflected by the claim history. We also develop a stochastic variational Expectation-Conditional-Maximization algorithm for estimating model parameters and inferring the posterior distribution of random effects, which is numerically efficient and scalable to large insurance portfolios. We then apply the Mixed LRMoE model to a real, multiyear automobile insurance dataset, where the proposed framework is shown to offer better fit to data and produce posterior premium which accurately reflects policyholders' claim history. △ Less

Submitted 29 September, 2022; originally announced September 2022.

Mixture of experts models for multilevel data: modelling framework and approximation theory

Authors: Tsz Chai Fung, Spark C. Tseung

Abstract: Multilevel data are prevalent in many real-world applications. However, it remains an open research problem to identify and justify a class of models that flexibly capture a wide range of multilevel data. Motivated by the versatility of the mixture of experts (MoE) models in fitting regression data, in this article we extend upon the MoE and study a class of mixed MoE (MMoE) models for multilevel… ▽ More Multilevel data are prevalent in many real-world applications. However, it remains an open research problem to identify and justify a class of models that flexibly capture a wide range of multilevel data. Motivated by the versatility of the mixture of experts (MoE) models in fitting regression data, in this article we extend upon the MoE and study a class of mixed MoE (MMoE) models for multilevel data. Under some regularity conditions, we prove that the MMoE is dense in the space of any continuous mixed effects models in the sense of weak convergence. As a result, the MMoE has a potential to accurately resemble almost all characteristics inherited in multilevel data, including the marginal distributions, dependence structures, regression links, random intercepts and random slopes. In a particular case where the multilevel data is hierarchical, we further show that a nested version of the MMoE universally approximates a broad range of dependence structures of the random effects among different factor levels. △ Less

Submitted 29 September, 2022; originally announced September 2022.

Robust estimation and model diagnostic of insurance loss data: a weighted likelihood approach

Authors: Tsz Chai Fung

Abstract: This paper presents a score-based weighted likelihood estimator (SWLE) for robust estimations of generalized linear model (GLM) for insurance loss data. The SWLE exhibits a limited sensitivity to the outliers, theoretically justifying its robustness against model contaminations. Also, with the specially designed weight function to effectively diminish the contributions of extreme losses to the GLM… ▽ More This paper presents a score-based weighted likelihood estimator (SWLE) for robust estimations of generalized linear model (GLM) for insurance loss data. The SWLE exhibits a limited sensitivity to the outliers, theoretically justifying its robustness against model contaminations. Also, with the specially designed weight function to effectively diminish the contributions of extreme losses to the GLM parameter estimations, most statistical quantities can still be derived analytically, minimizing the computational burden for parameter calibrations. Apart from robust estimations, the SWLE can also act as a quantitative diagnostic tool to detect outliers and systematic model misspecifications. Motivated by the coverage modifications which make insurance losses often random censored and truncated, the SWLE is extended to accommodate censored and truncated data. We exemplify the SWLE on three simulation studies and two real insurance datasets. Empirical results suggest that the SWLE produces more reliable parameter estimates than the MLE if outliers contaminate the dataset. The SWLE diagnostic tool also successfully detects any systematic model misspecifications with high power, accompanying some potential model improvements. △ Less

Submitted 21 April, 2022; originally announced April 2022.

Improving Question Answering over Knowledge Graphs Using Graph Summarization

Authors: Sirui Li, Kok Kai Wong, Dengya Zhu, Chun Che Fung

Abstract: Question Answering (QA) systems over Knowledge Graphs (KGs) (KGQA) automatically answer natural language questions using triples contained in a KG. The key idea is to represent questions and entities of a KG as low-dimensional embeddings. Previous KGQAs have attempted to represent entities using Knowledge Graph Embedding (KGE) and Deep Learning (DL) methods. However, KGEs are too shallow to captur… ▽ More Question Answering (QA) systems over Knowledge Graphs (KGs) (KGQA) automatically answer natural language questions using triples contained in a KG. The key idea is to represent questions and entities of a KG as low-dimensional embeddings. Previous KGQAs have attempted to represent entities using Knowledge Graph Embedding (KGE) and Deep Learning (DL) methods. However, KGEs are too shallow to capture the expressive features and DL methods process each triple independently. Recently, Graph Convolutional Network (GCN) has shown to be excellent in providing entity embeddings. However, using GCNs to KGQAs is inefficient because GCNs treat all relations equally when aggregating neighbourhoods. Also, a problem could occur when using previous KGQAs: in most cases, questions often have an uncertain number of answers. To address the above issues, we propose a graph summarization technique using Recurrent Convolutional Neural Network (RCNN) and GCN. The combination of GCN and RCNN ensures that the embeddings are propagated together with the relations relevant to the question, and thus better answers. The proposed graph summarization technique can be used to tackle the issue that KGQAs cannot answer questions with an uncertain number of answers. In this paper, we demonstrated the proposed technique on the most common type of questions, which is single-relation questions. Experiments have demonstrated that the proposed graph summarization technique using RCNN and GCN can provide better results when compared to the GCN. The proposed graph summarization technique significantly improves the recall of actual answers when the questions have an uncertain number of answers. △ Less

Submitted 25 March, 2022; originally announced March 2022.

Comments: The paper is accepted by ICONIP 2021

On the Ricci curvature of Kahler-Ricci Flow

Authors: Cheuk Yan Fung

Abstract: In this paper, we consider $n$-dimensional compact K$\ddot{a}$hler manifold with semi-ample canonical line bundle under the long time solution of K$\ddot{a}$hler Ricci Flow. In particular, if the Kodaira dimension is one, Ricci curvature converge to negative of generalized K$\ddot{a}$hler Einstein metric $ω_B$ locally away from singular set in $C^0_{loc}(ω(t))$ topology. In this paper, we consider $n$-dimensional compact K$\ddot{a}$hler manifold with semi-ample canonical line bundle under the long time solution of K$\ddot{a}$hler Ricci Flow. In particular, if the Kodaira dimension is one, Ricci curvature converge to negative of generalized K$\ddot{a}$hler Einstein metric $ω_B$ locally away from singular set in $C^0_{loc}(ω(t))$ topology. △ Less

Submitted 24 November, 2021; originally announced November 2021.

Comments: 17pages

MSC Class: 53E30

Privacy Guarantees of BLE Contact Tracing: A Case Study on COVIDWISE

Authors: Salman Ahmed, Ya Xiao, Taejoong, Chung, Carol Fung, Moti Yung, Danfeng, Yao

Abstract: Google and Apple jointly introduced a digital contact tracing technology and an API called "exposure notification," to help health organizations and governments with contact tracing. The technology and its interplay with security and privacy constraints require investigation. In this study, we examine and analyze the security, privacy, and reliability of the technology with actual and typical scen… ▽ More Google and Apple jointly introduced a digital contact tracing technology and an API called "exposure notification," to help health organizations and governments with contact tracing. The technology and its interplay with security and privacy constraints require investigation. In this study, we examine and analyze the security, privacy, and reliability of the technology with actual and typical scenarios (and expected typical adversary in mind), and quite realistic use cases. We do it in the context of Virginia's COVIDWISE app. This experimental analysis validates the properties of the system under the above conditions, a result that seems crucial for the peace of mind of the exposure notification technology adopting authorities, and may also help with the system's transparency and overall user trust. △ Less

Submitted 16 December, 2021; v1 submitted 16 November, 2021; originally announced November 2021.

Comments: \{copyright} 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Journal ref: IEEE Computer 2021

On the Effectiveness of Interpretable Feedforward Neural Network

Authors: Miles Q. Li, Benjamin C. M. Fung, Adel Abusitta

Abstract: Deep learning models have achieved state-of-the-art performance in many classification tasks. However, most of them cannot provide an interpretation for their classification results. Machine learning models that are interpretable are usually linear or piecewise linear and yield inferior performance. Non-linear models achieve much better classification performance, but it is hard to interpret their… ▽ More Deep learning models have achieved state-of-the-art performance in many classification tasks. However, most of them cannot provide an interpretation for their classification results. Machine learning models that are interpretable are usually linear or piecewise linear and yield inferior performance. Non-linear models achieve much better classification performance, but it is hard to interpret their classification results. This may have been changed by an interpretable feedforward neural network (IFFNN) proposed that achieves both high classification performance and interpretability for malware detection. If the IFFNN can perform well in a more flexible and general form for other classification tasks while providing meaningful interpretations, it may be of great interest to the applied machine learning community. In this paper, we propose a way to generalize the interpretable feedforward neural network to multi-class classification scenarios and any type of feedforward neural networks, and evaluate its classification performance and interpretability on intrinsic interpretable datasets. We conclude by finding that the generalized IFFNNs achieve comparable classification performance to their normal feedforward neural network counterparts and provide meaningful interpretations. Thus, this kind of neural network architecture has great practical use. △ Less

Submitted 3 November, 2021; originally announced November 2021.

Inconsistent Few-Shot Relation Classification via Cross-Attentional Prototype Networks with Contrastive Learning

Authors: Hongru Wang, Zhijing Jin, Jiarun Cao, Gabriel Pui Cheong Fung, Kam-Fai Wong

Abstract: Standard few-shot relation classification (RC) is designed to learn a robust classifier with only few labeled data for each class. However, previous works rarely investigate the effects of a different number of classes (i.e., $N$-way) and number of labeled data per class (i.e., $K$-shot) during training vs. testing. In this work, we define a new task, \textit{inconsistent few-shot RC}, where the m… ▽ More Standard few-shot relation classification (RC) is designed to learn a robust classifier with only few labeled data for each class. However, previous works rarely investigate the effects of a different number of classes (i.e., $N$-way) and number of labeled data per class (i.e., $K$-shot) during training vs. testing. In this work, we define a new task, \textit{inconsistent few-shot RC}, where the model needs to handle the inconsistency of $N$ and $K$ between training and testing. To address this new task, we propose Prototype Network-based cross-attention contrastive learning (ProtoCACL) to capture the rich mutual interactions between the support set and query set. Experimental results demonstrate that our ProtoCACL can outperform the state-of-the-art baseline model under both inconsistent $K$ and inconsistent $N$ settings, owing to its more robust and discriminate representations. Moreover, we identify that in the inconsistent few-shot learning setting, models can achieve better performance with \textit{less data} than the standard few-shot setting with carefully-selected $N$ and $K$. In the end of the paper, we provide further analyses and suggestions to systematically guide the selection of $N$ and $K$ under different scenarios. △ Less

Submitted 13 October, 2021; originally announced October 2021.

Prior Omission of Dissimilar Source Domain(s) for Cost-Effective Few-Shot Learning

Authors: Zezhong Wang, Hongru Wang, Kwan Wai Chung, Jia Zhu, Gabriel Pui Cheong Fung, Kam-Fai Wong

Abstract: Few-shot slot tagging is an emerging research topic in the field of Natural Language Understanding (NLU). With sufficient annotated data from source domains, the key challenge is how to train and adapt the model to another target domain which only has few labels. Conventional few-shot approaches use all the data from the source domains without considering inter-domain relations and implicitly assu… ▽ More Few-shot slot tagging is an emerging research topic in the field of Natural Language Understanding (NLU). With sufficient annotated data from source domains, the key challenge is how to train and adapt the model to another target domain which only has few labels. Conventional few-shot approaches use all the data from the source domains without considering inter-domain relations and implicitly assume each sample in the domain contributes equally. However, our experiments show that the data distribution bias among different domains will significantly affect the adaption performance. Moreover, transferring knowledge from dissimilar domains will even introduce some extra noises so that affect the performance of models. To tackle this problem, we propose an effective similarity-based method to select data from the source domains. In addition, we propose a Shared-Private Network (SP-Net) for the few-shot slot tagging task. The words from the same class would have some shared features. We extract those shared features from the limited annotated data on the target domain and merge them together as the label embedding to help us predict other unlabelled data on the target domain. The experiment shows that our method outperforms the state-of-the-art approaches with fewer source data. The result also proves that some training data from dissimilar sources are redundant and even negative for the adaption. △ Less

Submitted 11 September, 2021; originally announced September 2021.

TopicRefine: Joint Topic Prediction and Dialogue Response Generation for Multi-turn End-to-End Dialogue System

Authors: Hongru Wang, Mingyu Cui, Zimo Zhou, Gabriel Pui Cheong Fung, Kam-Fai Wong

Abstract: A multi-turn dialogue always follows a specific topic thread, and topic shift at the discourse level occurs naturally as the conversation progresses, necessitating the model's ability to capture different topics and generate topic-aware responses. Previous research has either predicted the topic first and then generated the relevant response, or simply applied the attention mechanism to all topics… ▽ More A multi-turn dialogue always follows a specific topic thread, and topic shift at the discourse level occurs naturally as the conversation progresses, necessitating the model's ability to capture different topics and generate topic-aware responses. Previous research has either predicted the topic first and then generated the relevant response, or simply applied the attention mechanism to all topics, ignoring the joint distribution of the topic prediction and response generation models and resulting in uncontrollable and unrelated responses. In this paper, we propose a joint framework with a topic refinement mechanism to learn these two tasks simultaneously. Specifically, we design a three-pass iteration mechanism to generate coarse response first, then predict corresponding topics, and finally generate refined response conditioned on predicted topics. Moreover, we utilize GPT2DoubleHeads and BERT for the topic prediction task respectively, aiming to investigate the effects of joint learning and the understanding ability of GPT model. Experimental results demonstrate that our proposed framework achieves new state-of-the-art performance at response generation task and the great potential understanding capability of GPT model. △ Less

Submitted 11 September, 2021; originally announced September 2021.

Maximum weighted likelihood estimator for robust heavy-tail modelling of finite mixture models

Authors: Tsz Chai Fung

Abstract: In this article, we present the maximum weighted likelihood estimator (MWLE) for robust estimations of heavy-tail finite mixture models (FMM). This is motivated by the complex distributional phenomena of insurance claim severity data, where flexible density estimation tools such as FMM are needed but MLE often produces unstable tail estimates under FMM. Under some regularity conditions, MWLE is pr… ▽ More In this article, we present the maximum weighted likelihood estimator (MWLE) for robust estimations of heavy-tail finite mixture models (FMM). This is motivated by the complex distributional phenomena of insurance claim severity data, where flexible density estimation tools such as FMM are needed but MLE often produces unstable tail estimates under FMM. Under some regularity conditions, MWLE is proved to be consistent and asymptotically normal. We further prove that the tail index obtained by MWLE is consistent even if the model is misspecified, justifying the robustness of MWLE in estimating the tail part of FMM. With a probabilistic interpretation for MWLE, Generalized Expectation-Maximization (GEM) algorithm is still applicable for efficient parameter estimations. We therefore present and compare two distinctive constructions of complete data to implement the GEM algorithm. By exemplifying our approach on two simulation studies and a real motor insurance data set, we show that comparing to MLE, MWLE produces more appropriate estimations on the tail part of FMM, without much sacrificing the flexibility of FMM in capturing the body part. △ Less

Submitted 3 August, 2021; originally announced August 2021.

The Topic Confusion Task: A Novel Scenario for Authorship Attribution

Authors: Malik H. Altakrori, Jackie Chi Kit Cheung, Benjamin C. M. Fung

Abstract: Authorship attribution is the problem of identifying the most plausible author of an anonymous text from a set of candidate authors. Researchers have investigated same-topic and cross-topic scenarios of authorship attribution, which differ according to whether new, unseen topics are used in the testing phase. However, neither scenario allows us to explain whether errors are caused by a failure to… ▽ More Authorship attribution is the problem of identifying the most plausible author of an anonymous text from a set of candidate authors. Researchers have investigated same-topic and cross-topic scenarios of authorship attribution, which differ according to whether new, unseen topics are used in the testing phase. However, neither scenario allows us to explain whether errors are caused by a failure to capture authorship writing style or by a topic shift. Motivated by this, we propose the \emph{topic confusion} task where we switch the author-topic configuration between the training and testing sets. This setup allows us to distinguish two types of errors: those caused by the topic shift and those caused by the features' inability to capture the writing styles. We show that stylometric features with part-of-speech tags are the least susceptible to topic variations. We further show that combining them with other features leads to significantly lower topic confusion and higher attribution accuracy. Finally, we show that pretrained language models such as BERT and RoBERTa perform poorly on this task and are surpassed by simple features such as word-level $n$-grams. △ Less

Submitted 9 September, 2021; v1 submitted 17 April, 2021; originally announced April 2021.

Comments: 15 pages (9 + ref./appin.), 6 figures, Accepted to Findings of EMNLP 2021

Mixture composite regression models with multi-type feature selection

Authors: Tsz Chai Fung, George Tzougas, Mario Wuthrich

Abstract: The aim of this paper is to present a mixture composite regression model for claim severity modelling. Claim severity modelling poses several challenges such as multimodality, heavy-tailedness and systematic effects in data. We tackle this modelling problem by studying a mixture composite regression model for simultaneous modeling of attritional and large claims, and for considering systematic eff… ▽ More The aim of this paper is to present a mixture composite regression model for claim severity modelling. Claim severity modelling poses several challenges such as multimodality, heavy-tailedness and systematic effects in data. We tackle this modelling problem by studying a mixture composite regression model for simultaneous modeling of attritional and large claims, and for considering systematic effects in both the mixture components as well as the mixing probabilities. For model fitting, we present a group-fused regularization approach that allows us for selecting the explanatory variables which significantly impact the mixing probabilities and the different mixture components, respectively. We develop an asymptotic theory for this regularized estimation approach, and fitting is performed using a novel Generalized Expectation-Maximization algorithm. We exemplify our approach on real motor insurance data set. △ Less

Submitted 30 July, 2021; v1 submitted 12 March, 2021; originally announced March 2021.

KddRES: A Multi-level Knowledge-driven Dialogue Dataset for Restaurant Towards Customized Dialogue System

Authors: Hongru Wang, Min Li, Zimo Zhou, Gabriel Pui Cheong Fung, Kam-Fai Wong

Abstract: Compared with CrossWOZ (Chinese) and MultiWOZ (English) dataset which have coarse-grained information, there is no dataset which handle fine-grained and hierarchical level information properly. In this paper, we publish a first Cantonese knowledge-driven Dialogue Dataset for REStaurant (KddRES) in Hong Kong, which grounds the information in multi-turn conversations to one specific restaurant. Our… ▽ More Compared with CrossWOZ (Chinese) and MultiWOZ (English) dataset which have coarse-grained information, there is no dataset which handle fine-grained and hierarchical level information properly. In this paper, we publish a first Cantonese knowledge-driven Dialogue Dataset for REStaurant (KddRES) in Hong Kong, which grounds the information in multi-turn conversations to one specific restaurant. Our corpus contains 0.8k conversations which derive from 10 restaurants with various styles in different regions. In addition to that, we designed fine-grained slots and intents to better capture semantic information. The benchmark experiments and data statistic analysis show the diversity and rich annotations of our dataset. We believe the publish of KddRES can be a necessary supplement of current dialogue datasets and more suitable and valuable for small and middle enterprises (SMEs) of society, such as build a customized dialogue system for each restaurant. The corpus and benchmark models are publicly available. △ Less

Submitted 14 December, 2021; v1 submitted 17 November, 2020; originally announced November 2020.

Comments: 8 pages,2 figures

Learning Inter-Modal Correspondence and Phenotypes from Multi-Modal Electronic Health Records

Authors: Kejing Yin, William K. Cheung, Benjamin C. M. Fung, Jonathan Poon

Abstract: Non-negative tensor factorization has been shown a practical solution to automatically discover phenotypes from the electronic health records (EHR) with minimal human supervision. Such methods generally require an input tensor describing the inter-modal interactions to be pre-established; however, the correspondence between different modalities (e.g., correspondence between medications and diagnos… ▽ More Non-negative tensor factorization has been shown a practical solution to automatically discover phenotypes from the electronic health records (EHR) with minimal human supervision. Such methods generally require an input tensor describing the inter-modal interactions to be pre-established; however, the correspondence between different modalities (e.g., correspondence between medications and diagnoses) can often be missing in practice. Although heuristic methods can be applied to estimate them, they inevitably introduce errors, and leads to sub-optimal phenotype quality. This is particularly important for patients with complex health conditions (e.g., in critical care) as multiple diagnoses and medications are simultaneously present in the records. To alleviate this problem and discover phenotypes from EHR with unobserved inter-modal correspondence, we propose the collective hidden interaction tensor factorization (cHITF) to infer the correspondence between multiple modalities jointly with the phenotype discovery. We assume that the observed matrix for each modality is marginalization of the unobserved inter-modal correspondence, which are reconstructed by maximizing the likelihood of the observed matrices. Extensive experiments conducted on the real-world MIMIC-III dataset demonstrate that cHITF effectively infers clinically meaningful inter-modal correspondence, discovers phenotypes that are more clinically relevant and diverse, and achieves better predictive performance compared with a number of state-of-the-art computational phenotyping models. △ Less

Submitted 12 November, 2020; originally announced November 2020.

Comments: Accepted by IEEE Transactions on Knowledge and Data Engineering (TKDE)

CUHK at SemEval-2020 Task 4: CommonSense Explanation, Reasoning and Prediction with Multi-task Learning

Authors: Hongru Wang, Xiangru Tang, Sunny Lai, Kwong Sak Leung, Jia Zhu, Gabriel Pui Cheong Fung, Kam-Fai Wong

Abstract: This paper describes our system submitted to task 4 of SemEval 2020: Commonsense Validation and Explanation (ComVE) which consists of three sub-tasks. The task is to directly validate the given sentence whether or not it makes sense and require the model to explain it. Based on BERTarchitecture with a multi-task setting, we propose an effective and interpretable "Explain, Reason and Predict" (ERP)… ▽ More This paper describes our system submitted to task 4 of SemEval 2020: Commonsense Validation and Explanation (ComVE) which consists of three sub-tasks. The task is to directly validate the given sentence whether or not it makes sense and require the model to explain it. Based on BERTarchitecture with a multi-task setting, we propose an effective and interpretable "Explain, Reason and Predict" (ERP) system to solve the three sub-tasks about commonsense: (a) Validation, (b)Reasoning, and (c) Explanation. Inspired by cognitive studies of common sense, our system first generates a reason or understanding of the sentences and then chooses which one statement makes sense, which is achieved by multi-task learning. During the post-evaluation, our system has reached 92.9% accuracy in subtask A (rank 11), 89.7% accuracy in subtask B (rank 9), andBLEU score of 12.9 in subtask C (rank 8) △ Less

Submitted 27 July, 2020; v1 submitted 12 June, 2020; originally announced June 2020.

Deep Learning-based Stress Determinator for Mouse Psychiatric Analysis using Hippocampus Activity

Authors: Donghan Liu, Benjamin C. M. Fung, Tak Pan Wong

Abstract: Decoding neurons to extract information from transmission and employ them into other use is the goal of neuroscientists' study. Due to that the field of neuroscience is utilizing the traditional methods presently, we hence combine the state-of-the-art deep learning techniques with the theory of neuron decoding to discuss its potential of accomplishment. Besides, the stress level that is related to… ▽ More Decoding neurons to extract information from transmission and employ them into other use is the goal of neuroscientists' study. Due to that the field of neuroscience is utilizing the traditional methods presently, we hence combine the state-of-the-art deep learning techniques with the theory of neuron decoding to discuss its potential of accomplishment. Besides, the stress level that is related to neuron activity in hippocampus is statistically examined as well. The experiments suggest that our state-of-the-art deep learning-based stress determinator provides good performance with respect to its model prediction accuracy and additionally, there is strong evidence against equivalence of mouse stress level under diverse environments. △ Less

Submitted 27 June, 2020; v1 submitted 11 June, 2020; originally announced June 2020.

Comments: The paper need re-evaluated and reviewed, may cause some significant changes

I-MAD: Interpretable Malware Detector Using Galaxy Transformer

Authors: Miles Q. Li, Benjamin C. M. Fung, Philippe Charland, Steven H. H. Ding

Abstract: Malware currently presents a number of serious threats to computer users. Signature-based malware detection methods are limited in detecting new malware samples that are significantly different from known ones. Therefore, machine learning-based methods have been proposed, but there are two challenges these methods face. The first is to model the full semantics behind the assembly code of malware.… ▽ More Malware currently presents a number of serious threats to computer users. Signature-based malware detection methods are limited in detecting new malware samples that are significantly different from known ones. Therefore, machine learning-based methods have been proposed, but there are two challenges these methods face. The first is to model the full semantics behind the assembly code of malware. The second challenge is to provide interpretable results while keeping excellent detection performance. In this paper, we propose an Interpretable MAlware Detector (I-MAD) that outperforms state-of-the-art static malware detection models regarding accuracy with excellent interpretability. To improve the detection performance, I-MAD incorporates a novel network component called the Galaxy Transformer network that can understand assembly code at the basic block, function, and executable levels. It also incorporates our proposed interpretable feed-forward neural network to provide interpretations for its detection results by quantifying the impact of each feature with respect to the prediction. Experiment results show that our model significantly outperforms existing state-of-the-art static malware detection models and presents meaningful interpretations. △ Less

Submitted 20 June, 2021; v1 submitted 15 September, 2019; originally announced September 2019.

Comments: Published by Elsevier Computers & Security

ER-AE: Differentially Private Text Generation for Authorship Anonymization

Authors: Haohan Bo, Steven H. H. Ding, Benjamin C. M. Fung, Farkhund Iqbal

Abstract: Most of privacy protection studies for textual data focus on removing explicit sensitive identifiers. However, personal writing style, as a strong indicator of the authorship, is often neglected. Recent studies, such as SynTF, have shown promising results on privacy-preserving text mining. However, their anonymization algorithm can only output numeric term vectors which are difficult for the recip… ▽ More Most of privacy protection studies for textual data focus on removing explicit sensitive identifiers. However, personal writing style, as a strong indicator of the authorship, is often neglected. Recent studies, such as SynTF, have shown promising results on privacy-preserving text mining. However, their anonymization algorithm can only output numeric term vectors which are difficult for the recipients to interpret. We propose a novel text generation model with a two-set exponential mechanism for authorship anonymization. By augmenting the semantic information through a REINFORCE training reward function, the model can generate differentially private text that has a close semantic and similar grammatical structure to the original text while removing personal traits of the writing style. It does not assume any conditioned labels or paralleled text data for training. We evaluate the performance of the proposed model on the real-life peer reviews dataset and the Yelp review dataset. The result suggests that our model outperforms the state-of-the-art on semantic preservation, authorship obfuscation, and stylometric transformation. △ Less

Submitted 13 May, 2021; v1 submitted 19 July, 2019; originally announced July 2019.

Biscotti: A Ledger for Private and Secure Peer-to-Peer Machine Learning

Authors: Muhammad Shayan, Clement Fung, Chris J. M. Yoon, Ivan Beschastnikh

Abstract: Federated Learning is the current state of the art in supporting secure multi-party machine learning (ML): data is maintained on the owner's device and the updates to the model are aggregated through a secure protocol. However, this process assumes a trusted centralized infrastructure for coordination, and clients must trust that the central service does not use the byproducts of client data. In a… ▽ More Federated Learning is the current state of the art in supporting secure multi-party machine learning (ML): data is maintained on the owner's device and the updates to the model are aggregated through a secure protocol. However, this process assumes a trusted centralized infrastructure for coordination, and clients must trust that the central service does not use the byproducts of client data. In addition to this, a group of malicious clients could also harm the performance of the model by carrying out a poisoning attack. As a response, we propose Biscotti: a fully decentralized peer to peer (P2P) approach to multi-party ML, which uses blockchain and cryptographic primitives to coordinate a privacy-preserving ML process between peering clients. Our evaluation demonstrates that Biscotti is scalable, fault tolerant, and defends against known attacks. For example, Biscotti is able to protect the privacy of an individual client's update and the performance of the global model at scale when 30% of adversaries are trying to poison the model. The implementation can be found at: https://github.com/DistributedML/Biscotti △ Less

Submitted 11 December, 2019; v1 submitted 24 November, 2018; originally announced November 2018.

Comments: 20 pages

Dancing in the Dark: Private Multi-Party Machine Learning in an Untrusted Setting

Authors: Clement Fung, Jamie Koerner, Stewart Grant, Ivan Beschastnikh

Abstract: Distributed machine learning (ML) systems today use an unsophisticated threat model: data sources must trust a central ML process. We propose a brokered learning abstraction that allows data sources to contribute towards a globally-shared model with provable privacy guarantees in an untrusted setting. We realize this abstraction by building on federated learning, the state of the art in multi-part… ▽ More Distributed machine learning (ML) systems today use an unsophisticated threat model: data sources must trust a central ML process. We propose a brokered learning abstraction that allows data sources to contribute towards a globally-shared model with provable privacy guarantees in an untrusted setting. We realize this abstraction by building on federated learning, the state of the art in multi-party ML, to construct TorMentor: an anonymous hidden service that supports private multi-party ML. We define a new threat model by characterizing, developing and evaluating new attacks in the brokered learning setting, along with new defenses for these attacks. We show that TorMentor effectively protects data providers against known ML attacks while providing them with a tunable trade-off between model accuracy and privacy. We evaluate TorMentor with local and geo-distributed deployments on Azure/Tor. In an experiment with 200 clients and 14 MB of data per client, our prototype trained a logistic regression model using stochastic gradient descent in 65s. Code is available at: https://github.com/DistributedML/TorML △ Less

Submitted 23 February, 2019; v1 submitted 23 November, 2018; originally announced November 2018.

Comments: 16 pages

Discrete-attractor-like Tracking in Continuous Attractor Neural Networks

Authors: Chi Chung Alan Fung, Tomoki Fukai

Abstract: Continuous attractor neural networks generate a set of smoothly connected attractor states. In memory systems of the brain, these attractor states may represent continuous pieces of information such as spatial locations and head directions of animals. However, during the replay of previous experiences, hippocampal neurons show a discontinuous sequence in which discrete transitions of neural state… ▽ More Continuous attractor neural networks generate a set of smoothly connected attractor states. In memory systems of the brain, these attractor states may represent continuous pieces of information such as spatial locations and head directions of animals. However, during the replay of previous experiences, hippocampal neurons show a discontinuous sequence in which discrete transitions of neural state are phase-locked with the slow-gamma (30-40 Hz) oscillation. Here, we explored the underlying mechanisms of the discontinuous sequence generation. We found that a continuous attractor neural network has several phases depending on the interactions between external input and local inhibitory feedback. The discrete-attractor-like behavior naturally emerges in one of these phases without any discreteness assumption. We propose that the dynamics of continuous attractor neural networks is the key to generate discontinuous state changes phase-locked to the brain rhythm. △ Less

Submitted 21 August, 2018; originally announced August 2018.

Comments: 5 pages, 4 figures

Journal ref: Phys. Rev. Lett. 122, 018102 (2019)

All One Needs to Know about Fog Computing and Related Edge Computing Paradigms: A Complete Survey

Authors: Ashkan Yousefpour, Caleb Fung, Tam Nguyen, Krishna Kadiyala, Fatemeh Jalali, Amirreza Niakanlahiji, Jian Kong, Jason P. Jue

Abstract: With the Internet of Things (IoT) becoming part of our daily life and our environment, we expect rapid growth in the number of connected devices. IoT is expected to connect billions of devices and humans to bring promising advantages for us. With this growth, fog computing, along with its related edge computing paradigms, such as multi-access edge computing (MEC) and cloudlet, are seen as promisin… ▽ More With the Internet of Things (IoT) becoming part of our daily life and our environment, we expect rapid growth in the number of connected devices. IoT is expected to connect billions of devices and humans to bring promising advantages for us. With this growth, fog computing, along with its related edge computing paradigms, such as multi-access edge computing (MEC) and cloudlet, are seen as promising solutions for handling the large volume of security-critical and time-sensitive data that is being produced by the IoT. In this paper, we first provide a tutorial on fog computing and its related computing paradigms, including their similarities and differences. Next, we provide a taxonomy of research topics in fog computing, and through a comprehensive survey, we summarize and categorize the efforts on fog computing and its related computing paradigms. Finally, we provide challenges and future directions for research in fog computing. △ Less

Submitted 13 February, 2019; v1 submitted 15 August, 2018; originally announced August 2018.

Comments: 48 pages, 7 tables, 11 figures, 450 references. The data (categories and features/objectives of the papers) of this survey are now available publicly. Accepted by Elsevier Journal of Systems Architecture

Mitigating Sybils in Federated Learning Poisoning

Authors: Clement Fung, Chris J. M. Yoon, Ivan Beschastnikh

Abstract: Machine learning (ML) over distributed multi-party data is required for a variety of domains. Existing approaches, such as federated learning, collect the outputs computed by a group of devices at a central aggregator and run iterative algorithms to train a globally shared model. Unfortunately, such approaches are susceptible to a variety of attacks, including model poisoning, which is made substa… ▽ More Machine learning (ML) over distributed multi-party data is required for a variety of domains. Existing approaches, such as federated learning, collect the outputs computed by a group of devices at a central aggregator and run iterative algorithms to train a globally shared model. Unfortunately, such approaches are susceptible to a variety of attacks, including model poisoning, which is made substantially worse in the presence of sybils. In this paper we first evaluate the vulnerability of federated learning to sybil-based poisoning attacks. We then describe \emph{FoolsGold}, a novel defense to this problem that identifies poisoning sybils based on the diversity of client updates in the distributed learning process. Unlike prior work, our system does not bound the expected number of attackers, requires no auxiliary information outside of the learning process, and makes fewer assumptions about clients and their data. In our evaluation we show that FoolsGold exceeds the capabilities of existing state of the art approaches to countering sybil-based label-flipping and backdoor poisoning attacks. Our results hold for different distributions of client data, varying poisoning targets, and various sybil strategies. Code can be found at: https://github.com/DistributedML/FoolsGold △ Less

Submitted 15 July, 2020; v1 submitted 14 August, 2018; originally announced August 2018.

Comments: 16 pages, Extended technical version of conference paper "The Limitations of Federated Learning in Sybil Settings" accepted at RAID 2020

Pilot-assisted intradyne reception for high-speed continuous-variable quantum key distribution with true local oscillator

Authors: Fabian Laudenbach, Bernhard Schrenk, Christoph Pacher, Michael Hentschel, Chi-Hang Fred Fung, Fotini Karinou, Andreas Poppe, Momtchil Peev, Hannes Hübel

Abstract: We present a pilot-assisted coherent intradyne reception methodology for CV-QKD with true local oscillator. An optically phase-locked reference tone, prepared using carrier-suppressed optical single-sideband modulation, is multiplexed in polarisation and frequency to the 250 Mbaud quantum signal in order to provide optical frequency- and phase matching between quantum signal and local oscillator.… ▽ More We present a pilot-assisted coherent intradyne reception methodology for CV-QKD with true local oscillator. An optically phase-locked reference tone, prepared using carrier-suppressed optical single-sideband modulation, is multiplexed in polarisation and frequency to the 250 Mbaud quantum signal in order to provide optical frequency- and phase matching between quantum signal and local oscillator. Our concept allows for high symbol rates and can be operated at an extremely low excess-noise level, as validated by experimental measurements. △ Less

Submitted 3 October, 2019; v1 submitted 29 December, 2017; originally announced December 2017.

Comments: 12 pages, 6 figures, 1 table

Journal ref: Quantum 3, 193 (2019)

Instant Accident Reporting and Crowdsensed Road Condition Analytics for Smart Cities

Authors: Ashkan Yousefpour, Caleb Fung, Tam Nguyen, David Hong, Daniel Zhang

Abstract: The following report contains information about a proposed technology by the authors, which consists of a device that sits inside of a vehicle and constantly monitors the car information. It can determine speed, g-force, and location coordinates. Using these data, the device can detect a car crash or pothole on the road. The data collected from the car is forwarded to a server to for more in-depth… ▽ More The following report contains information about a proposed technology by the authors, which consists of a device that sits inside of a vehicle and constantly monitors the car information. It can determine speed, g-force, and location coordinates. Using these data, the device can detect a car crash or pothole on the road. The data collected from the car is forwarded to a server to for more in-depth analytics. If there is an accident, the server promptly contacts the emergency services with the location of the crash. Moreover, the pothole information is used for analytics of road conditions. △ Less

Submitted 17 November, 2017; originally announced November 2017.

Comments: 8 pages, 7 figures, submitted to "Communication Technology Changing the World Competition", Sponsored by IEEE Communication Society

A note on the impact of management fees on the pricing of variable annuity guarantees

Authors: Jin Sun, Pavel V. Shevchenko, Man Chung Fung

Abstract: Variable annuities, as a class of retirement income products, allow equity market exposure for a policyholder's retirement fund with electable additional guarantees to limit the downside risk of the market. Management fees and guarantee insurance fees are charged respectively for the market exposure and for the protection from the downside risk. We investigate the impact of management fees on the… ▽ More Variable annuities, as a class of retirement income products, allow equity market exposure for a policyholder's retirement fund with electable additional guarantees to limit the downside risk of the market. Management fees and guarantee insurance fees are charged respectively for the market exposure and for the protection from the downside risk. We investigate the impact of management fees on the pricing of variable annuity guarantees under optimal withdrawal strategies. Two optimal strategies, from policyholder's and from insurer's perspectives, are respectively formulated and the corresponding pricing problems are solved using dynamic programming. Our results show that when management fees are present, the two strategies can deviate significantly from each other, leading to a substantial difference of the guarantee insurance fees. This provides a possible explanation of lower guarantee insurance fees observed in the market. Numerical experiments are conducted to illustrate our results. △ Less

Submitted 10 May, 2017; v1 submitted 10 May, 2017; originally announced May 2017.

MSC Class: 93E20

Continuous-Variable Quantum Key Distribution with Gaussian Modulation -- The Theory of Practical Implementations

Authors: Fabian Laudenbach, Christoph Pacher, Chi-Hang Fred Fung, Andreas Poppe, Momtchil Peev, Bernhard Schrenk, Michael Hentschel, Philip Walther, Hannes Hübel

Abstract: Quantum key distribution using weak coherent states and homodyne detection is a promising candidate for practical quantum-cryptographic implementations due to its compatibility with existing telecom equipment and high detection efficiencies. However, despite the actual simplicity of the protocol, the security analysis of this method is rather involved compared to discrete-variable QKD. In this art… ▽ More Quantum key distribution using weak coherent states and homodyne detection is a promising candidate for practical quantum-cryptographic implementations due to its compatibility with existing telecom equipment and high detection efficiencies. However, despite the actual simplicity of the protocol, the security analysis of this method is rather involved compared to discrete-variable QKD. In this article we review the theoretical foundations of continuous-variable quantum key distribution (CV-QKD) with Gaussian modulation and rederive the essential relations from scratch in a pedagogical way. The aim of this paper is to be as comprehensive and self-contained as possible in order to be well intelligible even for readers with little pre-knowledge on the subject. Although the present article is a theoretical discussion of CV-QKD, its focus lies on practical implementations, taking into account various kinds of hardware imperfections and suggesting practical methods to perform the security analysis subsequent to the key exchange. Apart from a review of well known results, this manuscript presents a set of new original noise models which are helpful to get an estimate of how well a given set of hardware will perform in practice. △ Less

Submitted 10 May, 2018; v1 submitted 27 March, 2017; originally announced March 2017.

Comments: 71 pages, 9 figures, 1 table

Journal ref: Adv. Quantum Technol. 1800011 (2018)

Cohort effects in mortality modelling: a Bayesian state-space approach

Authors: Man Chung Fung, Gareth W. Peters, Pavel V. Shevchenko

Abstract: Cohort effects are important factors in determining the evolution of human mortality for certain countries. Extensions of dynamic mortality models with cohort features have been proposed in the literature to account for these factors under the generalised linear modelling framework. In this paper we approach the problem of mortality modelling with cohort factors incorporated through a novel formul… ▽ More Cohort effects are important factors in determining the evolution of human mortality for certain countries. Extensions of dynamic mortality models with cohort features have been proposed in the literature to account for these factors under the generalised linear modelling framework. In this paper we approach the problem of mortality modelling with cohort factors incorporated through a novel formulation under a state-space methodology. In the process we demonstrate that cohort factors can be formulated naturally under the state-space framework, despite the fact that cohort factors are indexed according to year-of-birth rather than year. Bayesian inference for cohort models in a state-space formulation is then developed based on an efficient Markov chain Monte Carlo sampler, allowing for the quantification of parameter uncertainty in cohort models and resulting mortality forecasts that are used for life expectancy and life table constructions. The effectiveness of our approach is examined through comprehensive empirical studies involving male and female populations from various countries. Our results show that cohort patterns are present for certain countries that we studied and the inclusion of cohort factors are crucial in capturing these phenomena, thus highlighting the benefits of introducing cohort models in the state-space framework. Forecasting of cohort models is also discussed in light of the projection of cohort factors. △ Less

Submitted 24 March, 2017; originally announced March 2017.

Comments: 41 pages, 12 figures

Journal ref: Ann. actuar. sci. 13 (2019) 109-144

Learning Stylometric Representations for Authorship Analysis

Authors: Steven H. H. Ding, Benjamin C. M. Fung, Farkhund Iqbal, William K. Cheung

Abstract: Authorship analysis (AA) is the study of unveiling the hidden properties of authors from a body of exponentially exploding textual data. It extracts an author's identity and sociolinguistic characteristics based on the reflected writing styles in the text. It is an essential process for various areas, such as cybercrime investigation, psycholinguistics, political socialization, etc. However, most… ▽ More Authorship analysis (AA) is the study of unveiling the hidden properties of authors from a body of exponentially exploding textual data. It extracts an author's identity and sociolinguistic characteristics based on the reflected writing styles in the text. It is an essential process for various areas, such as cybercrime investigation, psycholinguistics, political socialization, etc. However, most of the previous techniques critically depend on the manual feature engineering process. Consequently, the choice of feature set has been shown to be scenario- or dataset-dependent. In this paper, to mimic the human sentence composition process using a neural network approach, we propose to incorporate different categories of linguistic features into distributed representation of words in order to learn simultaneously the writing style representations based on unlabeled texts for authorship analysis. In particular, the proposed models allow topical, lexical, syntactical, and character-level feature vectors of each document to be extracted as stylometrics. We evaluate the performance of our approach on the problems of authorship characterization and authorship verification with the Twitter, novel, and essay datasets. The experiments suggest that our proposed text representation outperforms the bag-of-lexical-n-grams, Latent Dirichlet Allocation, Latent Semantic Analysis, PVDM, PVDBOW, and word2vec representations. △ Less

Submitted 3 June, 2016; originally announced June 2016.

ACM Class: K.4.1; I.7.5; I.2.7

A unified approach to mortality modelling using state-space framework: characterisation, identification, estimation and forecasting

Authors: Man Chung Fung, Gareth W. Peters, Pavel V. Shevchenko

Abstract: This paper explores and develops alternative statistical representations and estimation approaches for dynamic mortality models. The framework we adopt is to reinterpret popular mortality models such as the Lee-Carter class of models in a general state-space modelling methodology, which allows modelling, estimation and forecasting of mortality under a unified framework. Furthermore, we propose an… ▽ More This paper explores and develops alternative statistical representations and estimation approaches for dynamic mortality models. The framework we adopt is to reinterpret popular mortality models such as the Lee-Carter class of models in a general state-space modelling methodology, which allows modelling, estimation and forecasting of mortality under a unified framework. Furthermore, we propose an alternative class of model identification constraints which is more suited to statistical inference in filtering and parameter estimation settings based on maximization of the marginalized likelihood or in Bayesian inference. We then develop a novel class of Bayesian state-space models which incorporate apriori beliefs about the mortality model characteristics as well as for more flexible and appropriate assumptions relating to heteroscedasticity that present in observed mortality data. We show that multiple period and cohort effect can be cast under a state-space structure. To study long term mortality dynamics, we introduce stochastic volatility to the period effect. The estimation of the resulting stochastic volatility model of mortality is performed using a recent class of Monte Carlo procedure specifically designed for state and parameter estimation in Bayesian state-space models, known as the class of particle Markov chain Monte Carlo methods. We illustrate the framework we have developed using Danish male mortality data, and show that incorporating heteroscedasticity and stochastic volatility markedly improves model fit despite an increase of model complexity. Forecasting properties of the enhanced models are examined with long term and short term calibration periods on the reconstruction of life tables. △ Less

Submitted 30 May, 2016; originally announced May 2016.

Comments: 46 pages

Journal ref: Annals of Actuarial Science 11 (2), pp. 343-389, 2017

Tight bounds of quantum speed limit for noisy dynamics via maximal rotation angles

Authors: Zihao Hu, Lingna Wang, Hongzhen Chen, Haidong Yuan, Chi-Hang Fred Fung, Jing Liu, Zibo Miao

Abstract: The laws of quantum physics place a limit on the speed of computation. In particular, the evolution time of a system from an initial state to a final state cannot be arbitrarily short. Bounds on the speed of evolution for unitary dynamics have long been studied. A few bounds on the speed of evolution for noisy dynamics have also been obtained recently, which, however, are in general not tight. In… ▽ More The laws of quantum physics place a limit on the speed of computation. In particular, the evolution time of a system from an initial state to a final state cannot be arbitrarily short. Bounds on the speed of evolution for unitary dynamics have long been studied. A few bounds on the speed of evolution for noisy dynamics have also been obtained recently, which, however, are in general not tight. In this paper, we present a new framework for quantum speed limit concerning noisy dynamics. Within this framework we obtain the exact maximal rotation angle that noisy dynamics can achieve at any given time, which gives rise to a tight bound on the evolution time for noisy dynamics. The obtained bound clearly reveals that noisy dynamics are essentially different from unitary dynamics. Furthermore, we show that the orthogonalization time, which is the minimum time needed to evolve any state to its orthogonal state, is in general not applicable to noisy dynamics. △ Less

Submitted 21 November, 2023; v1 submitted 2 January, 2016; originally announced January 2016.

Comments: 11 pages

On sign changes of cusp forms and the halting of an algorithm to construct a supersingular elliptic curve with a given endomorphism ring

Authors: King Cheong Fung, Ben Kane

Abstract: Chevyrev and Galbraith recently devised an algorithm which inputs a maximal order of the quaternion algebra ramified at one prime and infinity and constructs a supersingular elliptic curve whose endomorphism ring is precisely this maximal order. They proved that their algorithm is correct whenever it halts, but did not show that it always terminates. They did however prove that the algorithm halts… ▽ More Chevyrev and Galbraith recently devised an algorithm which inputs a maximal order of the quaternion algebra ramified at one prime and infinity and constructs a supersingular elliptic curve whose endomorphism ring is precisely this maximal order. They proved that their algorithm is correct whenever it halts, but did not show that it always terminates. They did however prove that the algorithm halts under a reasonable assumption which they conjectured to be true. It is the purpose of this paper to verify their conjecture and in turn prove that their algorithm always halts. More precisely, Chevyrev and Galbraith investigated the theta series associated with the norm maps from primitive elements of two maximal orders. They conjectured that if one of these theta series "dominated" the other in the sense that the $n$th (Fourier) coefficient of one was always larger than or equal to the $n$th coefficient of the other, then the maximal orders are actually the same. We prove that this is the case. △ Less

Submitted 10 July, 2016; v1 submitted 6 November, 2015; originally announced November 2015.

Comments: 12 pages

MSC Class: 11E20; 11E45; 11F37; 11G05; 16H05; 68W40

A State-Space Estimation of the Lee-Carter Mortality Model and Implications for Annuity Pricing

Authors: Man Chung Fung, Gareth W. Peters, Pavel V. Shevchenko

Abstract: In this article we investigate a state-space representation of the Lee-Carter model which is a benchmark stochastic mortality model for forecasting age-specific death rates. Existing relevant literature focuses mainly on mortality forecasting or pricing of longevity derivatives, while the full implications and methods of using the state-space representation of the Lee-Carter model in pricing retir… ▽ More In this article we investigate a state-space representation of the Lee-Carter model which is a benchmark stochastic mortality model for forecasting age-specific death rates. Existing relevant literature focuses mainly on mortality forecasting or pricing of longevity derivatives, while the full implications and methods of using the state-space representation of the Lee-Carter model in pricing retirement income products is yet to be examined. The main contribution of this article is twofold. First, we provide a rigorous and detailed derivation of the posterior distributions of the parameters and the latent process of the Lee-Carter model via Gibbs sampling. Our assumption for priors is slightly more general than the current literature in this area. Moreover, we suggest a new form of identification constraint not yet utilised in the actuarial literature that proves to be a more convenient approach for estimating the model under the state-space framework. Second, by exploiting the posterior distribution of the latent process and parameters, we examine the pricing range of annuities, taking into account the stochastic nature of the dynamics of the mortality rates. In this way we aim to capture the impact of longevity risk on the pricing of annuities. The outcome of our study demonstrates that an annuity price can be more than 4% under-valued when different assumptions are made on determining the survival curve constructed from the distribution of the forecasted death rates. Given that a typical annuity portfolio consists of a large number of policies with maturities which span decades, we conclude that the impact of longevity risk on the accurate pricing of annuities is a significant issue to be further researched. In addition, we find that mis-pricing is increasingly more pronounced for older ages as well as for annuity policies having a longer maturity. △ Less

Submitted 3 August, 2015; originally announced August 2015.

Comments: 9 pages; conference paper

Managing Systematic Mortality Risk in Life Annuities: An Application of Longevity Derivatives

Authors: Man Chung Fung, Katja Ignatieva, Michael Sherris

Abstract: This paper assesses the hedge effectiveness of an index-based longevity swap and a longevity cap. Although swaps are a natural instrument for hedging longevity risk, derivatives with non-linear pay-offs, such as longevity caps, also provide downside protection. A tractable stochastic mortality model with age dependent drift and volatility is developed and analytical formulae for prices of these lo… ▽ More This paper assesses the hedge effectiveness of an index-based longevity swap and a longevity cap. Although swaps are a natural instrument for hedging longevity risk, derivatives with non-linear pay-offs, such as longevity caps, also provide downside protection. A tractable stochastic mortality model with age dependent drift and volatility is developed and analytical formulae for prices of these longevity derivatives are derived. Hedge effectiveness is considered for a hypothetical life annuity portfolio. The hedging of the life annuity portfolio is comprehensively assessed for a range of assumptions for the market price of longevity risk, the term to maturity of the hedging instruments, as well as the size of the underlying annuity portfolio. The model is calibrated using Australian mortality data. The results provide a comprehensive analysis of longevity hedging, highlighting the risk management benefits and costs of linear and nonlinear payoff structures. △ Less

Submitted 1 August, 2015; originally announced August 2015.

Comments: 23 pages; under review

Quantum parameter estimation with general dynamics

Authors: Haidong Yuan, Chi-Hang Fred Fung

Abstract: One of the main quests in quantum metrology, and quantum parameter estimation in general, is to find out the highest achievable precision with given resources and design schemes that attain that precision. In this article we present a general framework for quantum parameter estimation which relates the ultimate precision limit directly to the underlying dynamics. With this framework we present sys… ▽ More One of the main quests in quantum metrology, and quantum parameter estimation in general, is to find out the highest achievable precision with given resources and design schemes that attain that precision. In this article we present a general framework for quantum parameter estimation which relates the ultimate precision limit directly to the underlying dynamics. With this framework we present systematical methods for computing the ultimate precision limit and optimal probe states. We further demonstrate the power of the framework by deriving a sufficient condition on when ancillary systems are not useful for improving the precision limit. △ Less

Submitted 20 May, 2016; v1 submitted 5 June, 2015; originally announced June 2015.

Comments: 14 pages. Comments are particularly welcome

Fidelity and Fisher information on quantum channels

Authors: Haidong Yuan, Chi-Hang Fred Fung

Abstract: The fidelity function of quantum states have been widely used in quantum information science and frequently arises in the quantification of optimal performances for the estimation and distinguish of quantum states. A fidelity function on quantum channel is expected to have same wide applications in quantum information science. In this paper we propose a fidelity function on quantum channels and sh… ▽ More The fidelity function of quantum states have been widely used in quantum information science and frequently arises in the quantification of optimal performances for the estimation and distinguish of quantum states. A fidelity function on quantum channel is expected to have same wide applications in quantum information science. In this paper we propose a fidelity function on quantum channels and show that various distance measures on quantum channels can be obtained from this fidelity function, for example the Bures angle and the Bures distance can be extended to quantum channels via this fidelity function. We then show that the distances between quantum channels lead naturally to a new Fisher information which quantifies the ultimate precision limit in quantum metrology, the ultimate precision limit can thus be seen as a manifestation of the distances between quantum channels. We also show that the fidelity on quantum channels provides a unified framework for perfect quantum channel discrimination and quantum metrology, in particular we show that the minimum number of uses needed for perfect channel discrimination is exactly the counterpart of the precision limit in quantum metrology, and various useful lower bounds for the minimum number of uses needed for perfect channel discrimination can be obtained via this connection. △ Less

Submitted 7 March, 2017; v1 submitted 2 June, 2015; originally announced June 2015.

Comments: 15 pages. Comments are particularly welcome

Universal time scaling for Hamiltonian parameter estimation

Authors: Haidong Yuan, Chi-Hang Fred Fung

Abstract: Time is a valuable resource and it seems intuitive that longer time should lead to better precision in Hamiltonian parameter estimation. However recent studies have put this intuition into question, showing longer time may even lead to worse estimation in certain cases. Here we show that the intuition can be restored if coherent feedback controls are included. By deriving asymptotically optimal fe… ▽ More Time is a valuable resource and it seems intuitive that longer time should lead to better precision in Hamiltonian parameter estimation. However recent studies have put this intuition into question, showing longer time may even lead to worse estimation in certain cases. Here we show that the intuition can be restored if coherent feedback controls are included. By deriving asymptotically optimal feedback controls we present a quantification of the maximal improvement feedback controls can provide in Hamiltonian parameter estimation and show a universal time scaling for the precision limit of Hamiltonian parameter estimation under the optimal feedback scheme. △ Less

Submitted 24 March, 2015; originally announced March 2015.

Comments: 5 pages. Comments and reference requests are welcomed

Journal ref: Phys. Rev. Lett. 115, 110401 (2015)