Showing 1–45 of 45 results for author: Zhaoxia

A Survey of Fragile Model Watermarking

Authors: Zhenzhe Gao, Yu Cheng, Zhaoxia Yin

Abstract: Model fragile watermarking, inspired by both the field of adversarial attacks on neural networks and traditional multimedia fragile watermarking, has gradually emerged as a potent tool for detecting tampering, and has witnessed rapid development in recent years. Unlike robust watermarks, which are widely used for identifying model copyrights, fragile watermarks for models are designed to identify… ▽ More Model fragile watermarking, inspired by both the field of adversarial attacks on neural networks and traditional multimedia fragile watermarking, has gradually emerged as a potent tool for detecting tampering, and has witnessed rapid development in recent years. Unlike robust watermarks, which are widely used for identifying model copyrights, fragile watermarks for models are designed to identify whether models have been subjected to unexpected alterations such as backdoors, poisoning, compression, among others. These alterations can pose unknown risks to model users, such as misidentifying stop signs as speed limit signs in classic autonomous driving scenarios. This paper provides an overview of the relevant work in the field of model fragile watermarking since its inception, categorizing them and revealing the developmental trajectory of the field, thus offering a comprehensive survey for future endeavors in model fragile watermarking. △ Less

Submitted 8 July, 2024; v1 submitted 7 June, 2024; originally announced June 2024.

Comments: Submitted Signal Processing

Exploring the Robustness of Decision-Level Through Adversarial Attacks on LLM-Based Embodied Models

Authors: Shuyuan Liu, Jiawei Chen, Shouwei Ruan, Hang Su, Zhaoxia Yin

Abstract: Embodied intelligence empowers agents with a profound sense of perception, enabling them to respond in a manner closely aligned with real-world situations. Large Language Models (LLMs) delve into language instructions with depth, serving a crucial role in generating plans for intricate tasks. Thus, LLM-based embodied models further enhance the agent's capacity to comprehend and process information… ▽ More Embodied intelligence empowers agents with a profound sense of perception, enabling them to respond in a manner closely aligned with real-world situations. Large Language Models (LLMs) delve into language instructions with depth, serving a crucial role in generating plans for intricate tasks. Thus, LLM-based embodied models further enhance the agent's capacity to comprehend and process information. However, this amalgamation also ushers in new challenges in the pursuit of heightened intelligence. Specifically, attackers can manipulate LLMs to produce irrelevant or even malicious outputs by altering their prompts. Confronted with this challenge, we observe a notable absence of multi-modal datasets essential for comprehensively evaluating the robustness of LLM-based embodied models. Consequently, we construct the Embodied Intelligent Robot Attack Dataset (EIRAD), tailored specifically for robustness evaluation. Additionally, two attack strategies are devised, including untargeted attacks and targeted attacks, to effectively simulate a range of diverse attack scenarios. At the same time, during the attack process, to more accurately ascertain whether our method is successful in attacking the LLM-based embodied model, we devise a new attack success evaluation method utilizing the BLIP2 model. Recognizing the time and cost-intensive nature of the GCG algorithm in attacks, we devise a scheme for prompt suffix initialization based on various target tasks, thus expediting the convergence process. Experimental results demonstrate that our method exhibits a superior attack success rate when targeting LLM-based embodied models, indicating a lower level of decision-level robustness in these models. △ Less

Submitted 16 July, 2024; v1 submitted 30 May, 2024; originally announced May 2024.

AutoBreach: Universal and Adaptive Jailbreaking with Efficient Wordplay-Guided Optimization

Authors: Jiawei Chen, Xiao Yang, Zhengwei Fang, Yu Tian, Yinpeng Dong, Zhaoxia Yin, Hang Su

Abstract: Despite the widespread application of large language models (LLMs) across various tasks, recent studies indicate that they are susceptible to jailbreak attacks, which can render their defense mechanisms ineffective. However, previous jailbreak research has frequently been constrained by limited universality, suboptimal efficiency, and a reliance on manual crafting. In response, we rethink the appr… ▽ More Despite the widespread application of large language models (LLMs) across various tasks, recent studies indicate that they are susceptible to jailbreak attacks, which can render their defense mechanisms ineffective. However, previous jailbreak research has frequently been constrained by limited universality, suboptimal efficiency, and a reliance on manual crafting. In response, we rethink the approach to jailbreaking LLMs and formally define three essential properties from the attacker' s perspective, which contributes to guiding the design of jailbreak methods. We further introduce AutoBreach, a novel method for jailbreaking LLMs that requires only black-box access. Inspired by the versatility of wordplay, AutoBreach employs a wordplay-guided mapping rule sampling strategy to generate a variety of universal mapping rules for creating adversarial prompts. This generation process leverages LLMs' automatic summarization and reasoning capabilities, thus alleviating the manual burden. To boost jailbreak success rates, we further suggest sentence compression and chain-of-thought-based mapping rules to correct errors and wordplay misinterpretations in target LLMs. Additionally, we propose a two-stage mapping rule optimization strategy that initially optimizes mapping rules before querying target LLMs to enhance the efficiency of AutoBreach. AutoBreach can efficiently identify security vulnerabilities across various LLMs, including three proprietary models: Claude-3, GPT-3.5, GPT-4 Turbo, and two LLMs' web platforms: Bingchat, GPT-4 Web, achieving an average success rate of over 80% with fewer than 10 queries △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: Under review

Semi-Supervised Disease Classification based on Limited Medical Image Data

Authors: Yan Zhang, Chun Li, Zhaoxia Liu, Ming Li

Abstract: In recent years, significant progress has been made in the field of learning from positive and unlabeled examples (PU learning), particularly in the context of advancing image and text classification tasks. However, applying PU learning to semi-supervised disease classification remains a formidable challenge, primarily due to the limited availability of labeled medical images. In the realm of medi… ▽ More In recent years, significant progress has been made in the field of learning from positive and unlabeled examples (PU learning), particularly in the context of advancing image and text classification tasks. However, applying PU learning to semi-supervised disease classification remains a formidable challenge, primarily due to the limited availability of labeled medical images. In the realm of medical image-aided diagnosis algorithms, numerous theoretical and practical obstacles persist. The research on PU learning for medical image-assisted diagnosis holds substantial importance, as it aims to reduce the time spent by professional experts in classifying images. Unlike natural images, medical images are typically accompanied by a scarcity of annotated data, while an abundance of unlabeled cases exists. Addressing these challenges, this paper introduces a novel generative model inspired by Hölder divergence, specifically designed for semi-supervised disease classification using positive and unlabeled medical image data. In this paper, we present a comprehensive formulation of the problem and establish its theoretical feasibility through rigorous mathematical analysis. To evaluate the effectiveness of our proposed approach, we conduct extensive experiments on five benchmark datasets commonly used in PU medical learning: BreastMNIST, PneumoniaMNIST, BloodMNIST, OCTMNIST, and AMD. The experimental results clearly demonstrate the superiority of our method over existing approaches based on KL divergence. Notably, our approach achieves state-of-the-art performance on all five disease classification benchmarks. By addressing the limitations imposed by limited labeled data and harnessing the untapped potential of unlabeled medical images, our novel generative model presents a promising direction for enhancing semi-supervised disease classification in the field of medical image analysis. △ Less

Submitted 7 May, 2024; originally announced May 2024.

FaceCat: Enhancing Face Recognition Security with a Unified Generative Model Framework

Authors: Jiawei Chen, Xiao Yang, Yinpeng Dong, Hang Su, Jianteng Peng, Zhaoxia Yin

Abstract: Face anti-spoofing (FAS) and adversarial detection (FAD) have been regarded as critical technologies to ensure the safety of face recognition systems. As a consequence of their limited practicality and generalization, some existing methods aim to devise a framework capable of concurrently detecting both threats to address the challenge. Nevertheless, these methods still encounter challenges of ins… ▽ More Face anti-spoofing (FAS) and adversarial detection (FAD) have been regarded as critical technologies to ensure the safety of face recognition systems. As a consequence of their limited practicality and generalization, some existing methods aim to devise a framework capable of concurrently detecting both threats to address the challenge. Nevertheless, these methods still encounter challenges of insufficient generalization and suboptimal robustness, potentially owing to the inherent drawback of discriminative models. Motivated by the rich structural and detailed features of face generative models, we propose FaceCat which utilizes the face generative model as a pre-trained model to improve the performance of FAS and FAD. Specifically, FaceCat elaborately designs a hierarchical fusion mechanism to capture rich face semantic features of the generative model. These features then serve as a robust foundation for a lightweight head, designed to execute FAS and FAD tasks simultaneously. As relying solely on single-modality data often leads to suboptimal performance, we further propose a novel text-guided multi-modal alignment strategy that utilizes text prompts to enrich feature representation, thereby enhancing performance. For fair evaluations, we build a comprehensive protocol with a wide range of 28 attack types to benchmark the performance. Extensive experiments validate the effectiveness of FaceCat generalizes significantly better and obtains excellent robustness against input transformations. △ Less

Submitted 14 April, 2024; originally announced April 2024.

Comments: Under review

Fragile Model Watermark for integrity protection: leveraging boundary volatility and sensitive sample-pairing

Authors: ZhenZhe Gao, Zhenjun Tang, Zhaoxia Yin, Baoyuan Wu, Yue Lu

Abstract: Neural networks have increasingly influenced people's lives. Ensuring the faithful deployment of neural networks as designed by their model owners is crucial, as they may be susceptible to various malicious or unintentional modifications, such as backdooring and poisoning attacks. Fragile model watermarks aim to prevent unexpected tampering that could lead DNN models to make incorrect decisions. T… ▽ More Neural networks have increasingly influenced people's lives. Ensuring the faithful deployment of neural networks as designed by their model owners is crucial, as they may be susceptible to various malicious or unintentional modifications, such as backdooring and poisoning attacks. Fragile model watermarks aim to prevent unexpected tampering that could lead DNN models to make incorrect decisions. They ensure the detection of any tampering with the model as sensitively as possible.However, prior watermarking methods suffered from inefficient sample generation and insufficient sensitivity, limiting their practical applicability. Our approach employs a sample-pairing technique, placing the model boundaries between pairs of samples, while simultaneously maximizing logits. This ensures that the model's decision results of sensitive samples change as much as possible and the Top-1 labels easily alter regardless of the direction it moves. △ Less

Submitted 12 June, 2024; v1 submitted 11 April, 2024; originally announced April 2024.

Comments: The article has been accepted by IEEE International Conference on Multimedia and Expo 2024

Adaptive White-Box Watermarking with Self-Mutual Check Parameters in Deep Neural Networks

Authors: Zhenzhe Gao, Zhaoxia Yin, Hongjian Zhan, Heng Yin, Yue Lu

Abstract: Artificial Intelligence (AI) has found wide application, but also poses risks due to unintentional or malicious tampering during deployment. Regular checks are therefore necessary to detect and prevent such risks. Fragile watermarking is a technique used to identify tampering in AI models. However, previous methods have faced challenges including risks of omission, additional information transmiss… ▽ More Artificial Intelligence (AI) has found wide application, but also poses risks due to unintentional or malicious tampering during deployment. Regular checks are therefore necessary to detect and prevent such risks. Fragile watermarking is a technique used to identify tampering in AI models. However, previous methods have faced challenges including risks of omission, additional information transmission, and inability to locate tampering precisely. In this paper, we propose a method for detecting tampered parameters and bits, which can be used to detect, locate, and restore parameters that have been tampered with. We also propose an adaptive embedding method that maximizes information capacity while maintaining model accuracy. Our approach was tested on multiple neural networks subjected to attacks that modified weight parameters, and our results demonstrate that our method achieved great recovery performance when the modification rate was below 20%. Furthermore, for models where watermarking significantly affected accuracy, we utilized an adaptive bit technique to recover more than 15% of the accuracy loss of the model. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Journal ref: The paper is under consideration at Pattern Recognition Letters, Elsevier, 2023

AdvFAS: A robust face anti-spoofing framework against adversarial examples

Authors: Jiawei Chen, Xiao Yang, Heng Yin, Mingzhi Ma, Bihui Chen, Jianteng Peng, Yandong Guo, Zhaoxia Yin, Hang Su

Abstract: Ensuring the reliability of face recognition systems against presentation attacks necessitates the deployment of face anti-spoofing techniques. Despite considerable advancements in this domain, the ability of even the most state-of-the-art methods to defend against adversarial examples remains elusive. While several adversarial defense strategies have been proposed, they typically suffer from cons… ▽ More Ensuring the reliability of face recognition systems against presentation attacks necessitates the deployment of face anti-spoofing techniques. Despite considerable advancements in this domain, the ability of even the most state-of-the-art methods to defend against adversarial examples remains elusive. While several adversarial defense strategies have been proposed, they typically suffer from constrained practicability due to inevitable trade-offs between universality, effectiveness, and efficiency. To overcome these challenges, we thoroughly delve into the coupled relationship between adversarial detection and face anti-spoofing. Based on this, we propose a robust face anti-spoofing framework, namely AdvFAS, that leverages two coupled scores to accurately distinguish between correctly detected and wrongly detected face images. Extensive experiments demonstrate the effectiveness of our framework in a variety of settings, including different attacks, datasets, and backbones, meanwhile enjoying high accuracy on clean examples. Moreover, we successfully apply the proposed method to detect real-world adversarial examples. △ Less

Submitted 3 August, 2023; originally announced August 2023.

Decision-based iterative fragile watermarking for model integrity verification

Authors: Zhaoxia Yin, Heng Yin, Hang Su, Xinpeng Zhang, Zhenzhe Gao

Abstract: Typically, foundation models are hosted on cloud servers to meet the high demand for their services. However, this exposes them to security risks, as attackers can modify them after uploading to the cloud or transferring from a local system. To address this issue, we propose an iterative decision-based fragile watermarking algorithm that transforms normal training samples into fragile samples that… ▽ More Typically, foundation models are hosted on cloud servers to meet the high demand for their services. However, this exposes them to security risks, as attackers can modify them after uploading to the cloud or transferring from a local system. To address this issue, we propose an iterative decision-based fragile watermarking algorithm that transforms normal training samples into fragile samples that are sensitive to model changes. We then compare the output of sensitive samples from the original model to that of the compromised model during validation to assess the model's completeness.The proposed fragile watermarking algorithm is an optimization problem that aims to minimize the variance of the predicted probability distribution outputed by the target model when fed with the converted sample.We convert normal samples to fragile samples through multiple iterations. Our method has some advantages: (1) the iterative update of samples is done in a decision-based black-box manner, relying solely on the predicted probability distribution of the target model, which reduces the risk of exposure to adversarial attacks, (2) the small-amplitude multiple iterations approach allows the fragile samples to perform well visually, with a PSNR of 55 dB in TinyImageNet compared to the original samples, (3) even with changes in the overall parameters of the model of magnitude 1e-4, the fragile samples can detect such changes, and (4) the method is independent of the specific model structure and dataset. We demonstrate the effectiveness of our method on multiple models and datasets, and show that it outperforms the current state-of-the-art. △ Less

Submitted 13 May, 2023; originally announced May 2023.

Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learning

Authors: Songning Lai, Jiakang Li, Guinan Guo, Xifeng Hu, Yulong Li, Yuan Tan, Zichen Song, Yutong Liu, Zhaoxia Ren, Chun Wan, Danmin Miao, Zhi Liu

Abstract: Designing an effective representation learning method for multimodal sentiment analysis tasks is a crucial research direction. The challenge lies in learning both shared and private information in a complete modal representation, which is difficult with uniform multimodal labels and a raw feature fusion approach. In this work, we propose a deep modal shared information learning module based on the… ▽ More Designing an effective representation learning method for multimodal sentiment analysis tasks is a crucial research direction. The challenge lies in learning both shared and private information in a complete modal representation, which is difficult with uniform multimodal labels and a raw feature fusion approach. In this work, we propose a deep modal shared information learning module based on the covariance matrix to capture the shared information between modalities. Additionally, we use a label generation module based on a self-supervised learning strategy to capture the private information of the modalities. Our module is plug-and-play in multimodal tasks, and by changing the parameterization, it can adjust the information exchange relationship between the modes and learn the private or shared information between the specified modes. We also employ a multi-task learning strategy to help the model focus its attention on the modal differentiation training data. We provide a detailed formulation derivation and feasibility proof for the design of the deep modal shared information learning module. We conduct extensive experiments on three common multimodal sentiment analysis baseline datasets, and the experimental results validate the reliability of our model. Furthermore, we explore more combinatorial techniques for the use of the module. Our approach outperforms current state-of-the-art methods on most of the metrics of the three public datasets. △ Less

Submitted 19 March, 2024; v1 submitted 15 May, 2023; originally announced May 2023.

Journal ref: International Joint Conference on Neural Networks (IJCNN) 2024

Multimodal Sentiment Analysis: A Survey

Authors: Songning Lai, Xifeng Hu, Haoxuan Xu, Zhaoxia Ren, Zhi Liu

Abstract: Multimodal sentiment analysis has become an important research area in the field of artificial intelligence. With the latest advances in deep learning, this technology has reached new heights. It has great potential for both application and research, making it a popular research topic. This review provides an overview of the definition, background, and development of multimodal sentiment analysis.… ▽ More Multimodal sentiment analysis has become an important research area in the field of artificial intelligence. With the latest advances in deep learning, this technology has reached new heights. It has great potential for both application and research, making it a popular research topic. This review provides an overview of the definition, background, and development of multimodal sentiment analysis. It also covers recent datasets and advanced models, emphasizing the challenges and future prospects of this technology. Finally, it looks ahead to future research directions. It should be noted that this review provides constructive suggestions for promising research directions and building better performing multimodal sentiment analysis models, which can help researchers in this field. △ Less

Submitted 3 July, 2023; v1 submitted 12 May, 2023; originally announced May 2023.

Comments: It needs to be returned for major modifications

Adversarial Examples Detection with Enhanced Image Difference Features based on Local Histogram Equalization

Authors: Zhaoxia Yin, Shaowei Zhu, Hang Su, Jianteng Peng, Wanli Lyu, Bin Luo

Abstract: Deep Neural Networks (DNNs) have recently made significant progress in many fields. However, studies have shown that DNNs are vulnerable to adversarial examples, where imperceptible perturbations can greatly mislead DNNs even if the full underlying model parameters are not accessible. Various defense methods have been proposed, such as feature compression and gradient masking. However, numerous st… ▽ More Deep Neural Networks (DNNs) have recently made significant progress in many fields. However, studies have shown that DNNs are vulnerable to adversarial examples, where imperceptible perturbations can greatly mislead DNNs even if the full underlying model parameters are not accessible. Various defense methods have been proposed, such as feature compression and gradient masking. However, numerous studies have proven that previous methods create detection or defense against certain attacks, which renders the method ineffective in the face of the latest unknown attack methods. The invisibility of adversarial perturbations is one of the evaluation indicators for adversarial example attacks, which also means that the difference in the local correlation of high-frequency information in adversarial examples and normal examples can be used as an effective feature to distinguish the two. Therefore, we propose an adversarial example detection framework based on a high-frequency information enhancement strategy, which can effectively extract and amplify the feature differences between adversarial examples and normal examples. Experimental results show that the feature augmentation module can be combined with existing detection models in a modular way under this framework. Improve the detector's performance and reduce the deployment cost without modifying the existing detection model. △ Less

Submitted 7 May, 2023; originally announced May 2023.

Robust image steganography against lossy JPEG compression based on embedding domain selection and adaptive error correction

Authors: Xiaolong Duan, Bin Li, Zhaoxia Yin, Xinpeng Zhang, Bin Luo

Abstract: Transmitting images for communication on social networks has become routine, which is helpful for covert communication. The traditional steganography algorithm is unable to successfully convey secret information since the social network channel will perform lossy operations on images, such as JPEG compression. Previous studies tried to solve this problem by enhancing the robustness or making the c… ▽ More Transmitting images for communication on social networks has become routine, which is helpful for covert communication. The traditional steganography algorithm is unable to successfully convey secret information since the social network channel will perform lossy operations on images, such as JPEG compression. Previous studies tried to solve this problem by enhancing the robustness or making the cover adapt to the channel processing. In this study, we proposed a robust image steganography method against lossy JPEG compression based on embedding domain selection and adaptive error correction. To improve anti-steganalysis performance, the embedding domain is selected adaptively. To increase robustness and lessen the impact on anti-steganalysis performance, the error correction capacity of the error correction code is adaptively adjusted to eliminate redundancy. The experimental results show that the proposed method achieves better anti-steganalysis and robustness. △ Less

Submitted 26 April, 2023; originally announced April 2023.

With Shared Microexponents, A Little Shifting Goes a Long Way

Authors: Bita Rouhani, Ritchie Zhao, Venmugil Elango, Rasoul Shafipour, Mathew Hall, Maral Mesmakhosroshahi, Ankit More, Levi Melnick, Maximilian Golub, Girish Varatkar, Lei Shao, Gaurav Kolhe, Dimitry Melts, Jasmine Klar, Renee L'Heureux, Matt Perry, Doug Burger, Eric Chung, Zhaoxia Deng, Sam Naghshineh, Jongsoo Park, Maxim Naumov

Abstract: This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning. It enables comparison of popular quantization standards, and through BDR, new formats based on shared microexponents (MX) are identified, which outperform other state-of-the-art quantization approaches, including narrow-precision floating-p… ▽ More This paper introduces Block Data Representations (BDR), a framework for exploring and evaluating a wide spectrum of narrow-precision formats for deep learning. It enables comparison of popular quantization standards, and through BDR, new formats based on shared microexponents (MX) are identified, which outperform other state-of-the-art quantization approaches, including narrow-precision floating-point and block floating-point. MX utilizes multiple levels of quantization scaling with ultra-fine scaling factors based on shared microexponents in the hardware. The effectiveness of MX is demonstrated on real-world models including large-scale generative pretraining and inferencing, and production-scale recommendation systems. △ Less

Submitted 12 April, 2023; v1 submitted 15 February, 2023; originally announced February 2023.

Adversarial Example Defense via Perturbation Grading Strategy

Authors: Shaowei Zhu, Wanli Lyu, Bin Li, Zhaoxia Yin, Bin Luo

Abstract: Deep Neural Networks have been widely used in many fields. However, studies have shown that DNNs are easily attacked by adversarial examples, which have tiny perturbations and greatly mislead the correct judgment of DNNs. Furthermore, even if malicious attackers cannot obtain all the underlying model parameters, they can use adversarial examples to attack various DNN-based task systems. Researcher… ▽ More Deep Neural Networks have been widely used in many fields. However, studies have shown that DNNs are easily attacked by adversarial examples, which have tiny perturbations and greatly mislead the correct judgment of DNNs. Furthermore, even if malicious attackers cannot obtain all the underlying model parameters, they can use adversarial examples to attack various DNN-based task systems. Researchers have proposed various defense methods to protect DNNs, such as reducing the aggressiveness of adversarial examples by preprocessing or improving the robustness of the model by adding modules. However, some defense methods are only effective for small-scale examples or small perturbations but have limited defense effects for adversarial examples with large perturbations. This paper assigns different defense strategies to adversarial perturbations of different strengths by grading the perturbations on the input examples. Experimental results show that the proposed method effectively improves defense performance. In addition, the proposed method does not modify any task model, which can be used as a preprocessing module, which significantly reduces the deployment cost in practical applications. △ Less

Submitted 16 December, 2022; originally announced December 2022.

High Capacity Reversible Data Hiding for Encrypted 3D Mesh Models Based on Topology

Authors: Yun Tang, Lulu Cheng, Wanli Lyv, Zhaoxia Yin

Abstract: Reversible data hiding in encrypted domain(RDH-ED) can not only protect the privacy of 3D mesh models and embed additional data, but also recover original models and extract additional data losslessly. However, due to the insufficient use of model topology, the existing methods have not achieved satisfactory results in terms of embedding capacity. To further improve the capacity, a RDH-ED method i… ▽ More Reversible data hiding in encrypted domain(RDH-ED) can not only protect the privacy of 3D mesh models and embed additional data, but also recover original models and extract additional data losslessly. However, due to the insufficient use of model topology, the existing methods have not achieved satisfactory results in terms of embedding capacity. To further improve the capacity, a RDH-ED method is proposed based on the topology of the 3D mesh models, which divides the vertices into two parts: embedding set and prediction set. And after integer mapping, the embedding ability of the embedding set is calculated by the prediction set. It is then passed to the data hider for embedding additional data. Finally, the additional data and the original models can be extracted and recovered respectively by the receiver with the correct keys. Experiments declare that compared with the existing methods, this method can obtain the highest embedding capacity. △ Less

Submitted 4 November, 2022; originally announced November 2022.

Knowledge Representation for Conceptual, Motivational, and Affective Processes in Natural Language Communication

Authors: Seng-Beng Ho, Zhaoxia Wang, Boon-Kiat Quek, Erik Cambria

Abstract: Natural language communication is an intricate and complex process. The speaker usually begins with an intention and motivation of what is to be communicated, and what effects are expected from the communication, while taking into consideration the listener's mental model to concoct an appropriate sentence. The listener likewise has to interpret what the speaker means, and respond accordingly, als… ▽ More Natural language communication is an intricate and complex process. The speaker usually begins with an intention and motivation of what is to be communicated, and what effects are expected from the communication, while taking into consideration the listener's mental model to concoct an appropriate sentence. The listener likewise has to interpret what the speaker means, and respond accordingly, also with the speaker's mental state in mind. To do this successfully, conceptual, motivational, and affective processes have to be represented appropriately to drive the language generation and understanding processes. Language processing has succeeded well with the big data approach in applications such as chatbots and machine translation. However, in human-robot collaborative social communication and in using natural language for delivering precise instructions to robots, a deeper representation of the conceptual, motivational, and affective processes is needed. This paper capitalizes on the UGALRS (Unified General Autonomous and Language Reasoning System) framework and the CD+ (Conceptual Representation Plus) representational scheme to illustrate how social communication through language is supported by a knowledge representational scheme that handles conceptual, motivational, and affective processes in a deep and general way. Though a small set of concepts, motivations, and emotions is treated in this paper, its main contribution is in articulating a general framework of knowledge representation and processing to link these aspects together in serving the purpose of natural language communication for an intelligent system. △ Less

Submitted 20 October, 2022; v1 submitted 25 September, 2022; originally announced October 2022.

Comments: 8 pages, 7 figures

Neural network fragile watermarking with no model performance degradation

Authors: Zhaoxia Yin, Heng Yin, Xinpeng Zhang

Abstract: Deep neural networks are vulnerable to malicious fine-tuning attacks such as data poisoning and backdoor attacks. Therefore, in recent research, it is proposed how to detect malicious fine-tuning of neural network models. However, it usually negatively affects the performance of the protected model. Thus, we propose a novel neural network fragile watermarking with no model performance degradation.… ▽ More Deep neural networks are vulnerable to malicious fine-tuning attacks such as data poisoning and backdoor attacks. Therefore, in recent research, it is proposed how to detect malicious fine-tuning of neural network models. However, it usually negatively affects the performance of the protected model. Thus, we propose a novel neural network fragile watermarking with no model performance degradation. In the process of watermarking, we train a generative model with the specific loss function and secret key to generate triggers that are sensitive to the fine-tuning of the target classifier. In the process of verifying, we adopt the watermarked classifier to get labels of each fragile trigger. Then, malicious fine-tuning can be detected by comparing secret keys and labels. Experiments on classic datasets and classifiers show that the proposed method can effectively detect model malicious fine-tuning with no model performance degradation. △ Less

Submitted 16 August, 2022; originally announced August 2022.

Comments: Published in 2022 IEEE International Conference on Image Processing (ICIP)

A novel view: edge isoperimetric methods and reliability evaluation of several kinds of conditional edge-connectivity of interconnection networks

Authors: Mingzu Zhang, Zhaoxia Tian, Lianzhu Zhang

Abstract: Reliability evaluation and fault tolerance of an interconnection network of some parallel and distributed systems are discussed separately under various link-faulty hypotheses in terms of different $\mathcal{P}$-conditional edge-connectivity. With the help of edge isoperimetric problem's method in combinatorics, this paper mainly offers a novel and unified view to investigate the $\mathcal{P}$-con… ▽ More Reliability evaluation and fault tolerance of an interconnection network of some parallel and distributed systems are discussed separately under various link-faulty hypotheses in terms of different $\mathcal{P}$-conditional edge-connectivity. With the help of edge isoperimetric problem's method in combinatorics, this paper mainly offers a novel and unified view to investigate the $\mathcal{P}$-conditional edge-connectivities of hamming graph $K_{L}^{n}$ with satisfying the property that each minimum $\mathcal{P}$-conditional edge-cut separates the $K_{L}^{n}$ just into two components, such as $L^{t}$-extra edge-connectivity, $t$-embedded edge-connectivity, cyclic edge-connectivity, $(L-1)t$-super edge-connectivity, $(L-1)t$-average edge-connectivity and $L^{t}$-th isoperimetric edge-connectivity. They share the same values in form of $(L-1)(n-t)L^{t}$ (except for cyclic edge-connectivity), which equals to the minimum number of links-faulty resulting in an $L$-ary-$n$-dimensional sub-layer from $K_{L}^{n}$. Besides, we also obtain the exact values of $h$-extra edge-connectivity and $h$-th isoperimetric edge-connectivity of hamming graph $K_{L}^{n}$ for each $h\leq L^{\lfloor {\frac{n}{2}} \rfloor}$. For the case $L=2$, $K_2^n=Q_n$ is $n$-dimensional hypercube. Our results can be applied to more generalized class of networks, called $n$-dim-ensional bijective connection networks, which contains hypercubes, twisted cubes, crossed cubes, Möbius cubes, locally twisted cubes and so on. Our results improve several previous results on this topic. △ Less

Submitted 24 March, 2022; originally announced March 2022.

Comments: 12 pages, 8 figures, 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS)

Universal adversarial perturbation for remote sensing images

Authors: Qingyu Wang, Guorui Feng, Zhaoxia Yin, Bin Luo

Abstract: Recently, with the application of deep learning in the remote sensing image (RSI) field, the classification accuracy of the RSI has been dramatically improved compared with traditional technology. However, even the state-of-the-art object recognition convolutional neural networks are fooled by the universal adversarial perturbation (UAP). The research on UAP is mostly limited to ordinary images, a… ▽ More Recently, with the application of deep learning in the remote sensing image (RSI) field, the classification accuracy of the RSI has been dramatically improved compared with traditional technology. However, even the state-of-the-art object recognition convolutional neural networks are fooled by the universal adversarial perturbation (UAP). The research on UAP is mostly limited to ordinary images, and RSIs have not been studied. To explore the basic characteristics of UAPs of RSIs, this paper proposes a novel method combining an encoder-decoder network with an attention mechanism to generate the UAP of RSIs. Firstly, the former is used to generate the UAP, which can learn the distribution of perturbations better, and then the latter is used to find the sensitive regions concerned by the RSI classification model. Finally, the generated regions are used to fine-tune the perturbation making the model misclassified with fewer perturbations. The experimental results show that the UAP can make the classification model misclassify, and the attack success rate of our proposed method on the RSI data set is as high as 97.09%. △ Less

Submitted 3 January, 2023; v1 submitted 22 February, 2022; originally announced February 2022.

Comments: Published in the Twenty-Fourth International Workshop on Multimedia Signal Processing, MMSP 2022

The PCG-AIID System for L3DAS22 Challenge: MIMO and MISO convolutional recurrent Network for Multi Channel Speech Enhancement and Speech Recognition

Authors: Jingdong Li, Yuanyuan Zhu, Dawei Luo, Yun Liu, Guohui Cui, Zhaoxia Li

Abstract: This paper described the PCG-AIID system for L3DAS22 challenge in Task 1: 3D speech enhancement in office reverberant environment. We proposed a two-stage framework to address multi-channel speech denoising and dereverberation. In the first stage, a multiple input and multiple output (MIMO) network is applied to remove background noise while maintaining the spatial characteristics of multi-channel… ▽ More This paper described the PCG-AIID system for L3DAS22 challenge in Task 1: 3D speech enhancement in office reverberant environment. We proposed a two-stage framework to address multi-channel speech denoising and dereverberation. In the first stage, a multiple input and multiple output (MIMO) network is applied to remove background noise while maintaining the spatial characteristics of multi-channel signals. In the second stage, a multiple input and single output (MISO) network is applied to enhance the speech from desired direction and post-filtering. As a result, our system ranked 3rd place in ICASSP2022 L3DAS22 challenge and significantly outperforms the baseline system, while achieving 3.2% WER and 0.972 STOI on the blind test-set. △ Less

Submitted 21 February, 2022; originally announced February 2022.

Comments: To appear at ICASSP 2022 (Accepted)

Reversible Attack based on Local Visual Adversarial Perturbation

Authors: Li Chen, Shaowei Zhu, Zhaoxia Yin

Abstract: Adding perturbations to images can mislead classification models to produce incorrect results. Recently, researchers exploited adversarial perturbations to protect image privacy from retrieval by intelligent models. However, adding adversarial perturbations to images destroys the original data, making images useless in digital forensics and other fields. To prevent illegal or unauthorized access t… ▽ More Adding perturbations to images can mislead classification models to produce incorrect results. Recently, researchers exploited adversarial perturbations to protect image privacy from retrieval by intelligent models. However, adding adversarial perturbations to images destroys the original data, making images useless in digital forensics and other fields. To prevent illegal or unauthorized access to sensitive image data such as human faces without impeding legitimate users, the use of reversible adversarial attack techniques is increasing. The original image can be recovered from its reversible adversarial examples. However, existing reversible adversarial attack methods are designed for traditional imperceptible adversarial perturbations and ignore the local visible adversarial perturbation. In this paper, we propose a new method for generating reversible adversarial examples based on local visible adversarial perturbation. The information needed for image recovery is embedded into the area beyond the adversarial patch by the reversible data hiding technique. To reduce image distortion, lossless compression and the B-R-G (bluered-green) embedding principle are adopted. Experiments on CIFAR-10 and ImageNet datasets show that the proposed method can restore the original images error-free while ensuring good attack performance. △ Less

Submitted 2 January, 2023; v1 submitted 6 October, 2021; originally announced October 2021.

High Capacity Reversible Data Hiding in Encrypted 3D Mesh Models Based on Multi-MSB Prediction

Authors: Wanli Lv, Lulu Cheng, Zhaoxia Yin

Abstract: As a new generation of digital media for covert transmission, three-dimension (3D) mesh models are frequently used and distributed on the network. Facing the huge massive of network data, it is urgent to study a method to protect and store this large amounts of data. In this paper, we proposed a high capacity reversible data hiding in encrypted 3D mesh models. This method divides the vertices of a… ▽ More As a new generation of digital media for covert transmission, three-dimension (3D) mesh models are frequently used and distributed on the network. Facing the huge massive of network data, it is urgent to study a method to protect and store this large amounts of data. In this paper, we proposed a high capacity reversible data hiding in encrypted 3D mesh models. This method divides the vertices of all 3D mesh into "embedded sets" and "prediction sets" based on the parity of the index. In addition, the multiple most significant bit (Multi-MSB) prediction reserved space is used to adaptively embed secret message, and the auxiliary information is compressed by arithmetic coding to further free up redundant space of the 3D mesh models. We use the majority voting system(MSV) principle to restore the original mesh model with high quality. The experimental results show that our method achieves a higher embedding capacity compared with state-of-the-art RDH-ED methods on 3D mesh models and can restore the original 3D mesh models with high quality. △ Less

Submitted 16 December, 2022; v1 submitted 3 October, 2021; originally announced October 2021.

Comments: Published in Signal Processing

Journal ref: Volume 201, December 2022, 108686

On the Robustness of "Robust reversible data hiding scheme based on two-layer embedding strategy"

Authors: Wen Yin, Longfei Ke, Zhaoxia Yin, Jin Tang, Bin Luo

Abstract: In the paper "Robust reversible data hiding scheme based on two-layer embedding strategy" published in INS recently, Kumar et al. proposed a robust reversible data hiding (RRDH) scheme based on two-layer embedding. Secret data was embedded into the most significant bit (MSB) planes to increase robustness, and a sorting strategy based on local complexity was adopted to reduce distortion. However, K… ▽ More In the paper "Robust reversible data hiding scheme based on two-layer embedding strategy" published in INS recently, Kumar et al. proposed a robust reversible data hiding (RRDH) scheme based on two-layer embedding. Secret data was embedded into the most significant bit (MSB) planes to increase robustness, and a sorting strategy based on local complexity was adopted to reduce distortion. However, Kumar et al.'s reversible data hiding (RDH) scheme is not as robust against joint photographic experts group (JPEG) compression as stated and can not be called RRDH. This comment first gives a brief description of their RDH scheme, then analyses their scheme's robustness from the perspective of JPEG compression principles. JPEG compression will change pixel values, thereby destroying auxiliary information and pixel value ordering required to extract secret data correctly, making their scheme not robust. Next, the changes in both bit plane and pixel value ordering after JPEG compression are shown and analysed by different robustness-testing experiments. Finally, some suggestions are given to improve the robustness. △ Less

Submitted 22 January, 2022; v1 submitted 24 September, 2021; originally announced September 2021.

Low-Precision Hardware Architectures Meet Recommendation Model Inference at Scale

Authors: Zhaoxia, Deng, Jongsoo Park, Ping Tak Peter Tang, Haixin Liu, Jie, Yang, Hector Yuen, Jianyu Huang, Daya Khudia, Xiaohan Wei, Ellie Wen, Dhruv Choudhary, Raghuraman Krishnamoorthi, Carole-Jean Wu, Satish Nadathur, Changkyu Kim, Maxim Naumov, Sam Naghshineh, Mikhail Smelyanskiy

Abstract: Tremendous success of machine learning (ML) and the unabated growth in ML model complexity motivated many ML-specific designs in both CPU and accelerator architectures to speed up the model inference. While these architectures are diverse, highly optimized low-precision arithmetic is a component shared by most. Impressive compute throughputs are indeed often exhibited by these architectures on ben… ▽ More Tremendous success of machine learning (ML) and the unabated growth in ML model complexity motivated many ML-specific designs in both CPU and accelerator architectures to speed up the model inference. While these architectures are diverse, highly optimized low-precision arithmetic is a component shared by most. Impressive compute throughputs are indeed often exhibited by these architectures on benchmark ML models. Nevertheless, production models such as recommendation systems important to Facebook's personalization services are demanding and complex: These systems must serve billions of users per month responsively with low latency while maintaining high prediction accuracy, notwithstanding computations with many tens of billions parameters per inference. Do these low-precision architectures work well with our production recommendation systems? They do. But not without significant effort. We share in this paper our search strategies to adapt reference recommendation models to low-precision hardware, our optimization of low-precision compute kernels, and the design and development of tool chain so as to maintain our models' accuracy throughout their lifespan during which topic trends and users' interests inevitably evolve. Practicing these low-precision technologies helped us save datacenter capacities while deploying models with up to 5X complexity that would otherwise not be deployed on traditional general-purpose CPUs. We believe these lessons from the trenches promote better co-design between hardware architecture and software engineering and advance the state of the art of ML in industry. △ Less

Submitted 26 May, 2021; originally announced May 2021.

High-Capacity Reversible Data Hiding in Encrypted Images using Adaptive Encoding

Authors: Wenjing Ma, Youqing Wu, Zhaoxia Yin

Abstract: With the popularization of digital information technology, the reversible data hiding in encrypted images (RDHEI) has gradually become the research hotspot of privacy protection in cloud storage. As a technology which can embed additional information in encrypted domain, extract the embedded information correctly and recover the original image without loss, RDHEI has been widely paid attention by… ▽ More With the popularization of digital information technology, the reversible data hiding in encrypted images (RDHEI) has gradually become the research hotspot of privacy protection in cloud storage. As a technology which can embed additional information in encrypted domain, extract the embedded information correctly and recover the original image without loss, RDHEI has been widely paid attention by researchers. To embed sufficient additional information in the encrypted image, a high-capacity RDHEI method using adaptive encoding is proposed in this paper. Firstly, the occurrence frequency of different prediction errors of the original image is calculated and the corresponding adaptive Huffman coding is generated. Then, the original image is encrypted with stream cipher and the encrypted pixels are marked with different Huffman codewords according to the prediction errors. Finally, additional information is embedded in the reserved room of marked pixels by bit substitution. The experimental results show that the proposed algorithm can extract the embedded information correctly and recover the original image losslessly. Compared with similar algorithms, the proposed algorithm makes full use of the characteristics of the image itself and greatly improves the embedding rate of the image. On UCID, BOSSBase, and BOWS-2 datasets, the average embedding rate of the proposed algorithm reaches 3.162 bpp, 3.917 bpp, and 3.775 bpp, which is higher than the state-of-the-art algorithm of 0.263 bpp, 0.292 bpp, and 0.280 bpp, respectively. △ Less

Submitted 8 October, 2021; v1 submitted 24 February, 2021; originally announced February 2021.

PICA: A Pixel Correlation-based Attentional Black-box Adversarial Attack

Authors: Jie Wang, Zhaoxia Yin, Jin Tang, Jing Jiang, Bin Luo

Abstract: The studies on black-box adversarial attacks have become increasingly prevalent due to the intractable acquisition of the structural knowledge of deep neural networks (DNNs). However, the performance of emerging attacks is negatively impacted when fooling DNNs tailored for high-resolution images. One of the explanations is that these methods usually focus on attacking the entire image, regardless… ▽ More The studies on black-box adversarial attacks have become increasingly prevalent due to the intractable acquisition of the structural knowledge of deep neural networks (DNNs). However, the performance of emerging attacks is negatively impacted when fooling DNNs tailored for high-resolution images. One of the explanations is that these methods usually focus on attacking the entire image, regardless of its spatial semantic information, and thereby encounter the notorious curse of dimensionality. To this end, we propose a pixel correlation-based attentional black-box adversarial attack, termed as PICA. Firstly, we take only one of every two neighboring pixels in the salient region as the target by leveraging the attentional mechanism and pixel correlation of images, such that the dimension of the black-box attack reduces. After that, a general multiobjective evolutionary algorithm is employed to traverse the reduced pixels and generate perturbations that are imperceptible by the human vision. Extensive experimental results have verified the effectiveness of the proposed PICA on the ImageNet dataset. More importantly, PICA is computationally more efficient to generate high-resolution adversarial examples compared with the existing black-box attacks. △ Less

Submitted 19 January, 2021; originally announced January 2021.

Attention-Guided Black-box Adversarial Attacks with Large-Scale Multiobjective Evolutionary Optimization

Authors: Jie Wang, Zhaoxia Yin, Jing Jiang, Yang Du

Abstract: Fooling deep neural networks (DNNs) with the black-box optimization has become a popular adversarial attack fashion, as the structural prior knowledge of DNNs is always unknown. Nevertheless, recent black-box adversarial attacks may struggle to balance their attack ability and visual quality of the generated adversarial examples (AEs) in tackling high-resolution images. In this paper, we propose a… ▽ More Fooling deep neural networks (DNNs) with the black-box optimization has become a popular adversarial attack fashion, as the structural prior knowledge of DNNs is always unknown. Nevertheless, recent black-box adversarial attacks may struggle to balance their attack ability and visual quality of the generated adversarial examples (AEs) in tackling high-resolution images. In this paper, we propose an attention-guided black-box adversarial attack based on the large-scale multiobjective evolutionary optimization, termed as LMOA. By considering the spatial semantic information of images, we firstly take advantage of the attention map to determine the perturbed pixels. Instead of attacking the entire image, reducing the perturbed pixels with the attention mechanism can help to avoid the notorious curse of dimensionality and thereby improves the performance of attacking. Secondly, a large-scale multiobjective evolutionary algorithm is employed to traverse the reduced pixels in the salient region. Benefiting from its characteristics, the generated AEs have the potential to fool target DNNs while being imperceptible by the human vision. Extensive experimental results have verified the effectiveness of the proposed LMOA on the ImageNet dataset. More importantly, it is more competitive to generate high-resolution AEs with better visual quality compared with the existing black-box adversarial attacks. △ Less

Submitted 13 January, 2022; v1 submitted 19 January, 2021; originally announced January 2021.

Multi-domain Reversible Data Hiding in JPEG

Authors: Zhaoxia Yin, Hongnian Guo, Yang Du

Abstract: As a branch of reversible data hiding (RDH), reversible data hiding in JEPG is particularly important. Because JPEG images are widely used, it is great significance to study reversible data hiding algorithm for JEPG images. The existing JEPG reversible data methods can be divided into two categories, one is based on Discrete Cosine Transform (DCT) coefficients modification, the other is based on H… ▽ More As a branch of reversible data hiding (RDH), reversible data hiding in JEPG is particularly important. Because JPEG images are widely used, it is great significance to study reversible data hiding algorithm for JEPG images. The existing JEPG reversible data methods can be divided into two categories, one is based on Discrete Cosine Transform (DCT) coefficients modification, the other is based on Huffman table modification, the methods based on DCT coefficient modification result in large file expansion and visual quality distortion, while the methods based on entropy coding domain modification have low capacity and they may lead to large file expansion. In order to effectively solve the problems in these two kinds of methods, this paper proposes a reversible data hiding in JPEG images methods based on multi-domain modification. In this method, the secret data is divided into two parts by payload distribution algorithm, part of the secret data is first embedded in the DCT coefficient domain, and then the remaining secret data is embedded in the entropy coding domain. Experimental results demonstrate that most JPEG image files with this scheme have smaller file size increment and higher payload than previous RDH schemes. △ Less

Submitted 10 November, 2020; originally announced November 2020.

Robust adaptive steganography based on dither modulation and modification with re-compression

Authors: Zhaoxia Yin, Longfei Ke

Abstract: Traditional adaptive steganography is a technique used for covert communication with high security, but it is invalid in the case of stego images are sent to legal receivers over networks which is lossy, such as JPEG compression of channels. To deal with such problem, robust adaptive steganography is proposed to enable the receiver to extract secret messages from the damaged stego images. Previous… ▽ More Traditional adaptive steganography is a technique used for covert communication with high security, but it is invalid in the case of stego images are sent to legal receivers over networks which is lossy, such as JPEG compression of channels. To deal with such problem, robust adaptive steganography is proposed to enable the receiver to extract secret messages from the damaged stego images. Previous works utilize reverse engineering and compression-resistant domain constructing to implement robust adaptive steganography. In this paper, we adopt modification with re-compression scheme to improve the robustness of stego sequences in stego images. To balance security and robustness, we move the embedding domain to the low frequency region of DCT (Discrete Cosine Transform) coefficients to improve the security of robust adaptive steganography. In addition, we add additional check codes to further reduce the average extraction error rate based on the framework of E-DMAS (Enhancing Dither Modulation based robust Adaptive Steganography). Compared with GMAS (Generalized dither Modulation based robust Adaptive Steganography) and E-DMAS, experiment results show that our scheme can achieve strong robustness and improve the security of robust adaptive steganography greatly when the channel quality factor is known. △ Less

Submitted 20 March, 2021; v1 submitted 16 July, 2020; originally announced July 2020.

Journal ref: IEEE Transactions on Signal and Information Processing over Networks, 2021, 7: 336-345

Reversible Data Hiding in Encrypted Images Based on Bit-plane Compression of Prediction Error

Authors: Youqing Wu, Wenjing Ma, Yinyin Peng, Ruiling Zhang, Zhaoxia Yin

Abstract: As a technology that can prevent the information from being disclosed, the reversible data hiding in encrypted images (RDHEI) acts as an important role in privacy protection and information security. To make use of the image redundancy and further improve the embedding performance, a high-capacity RDHEI method based on bit-plane compression of prediction error is proposed in this paper. Firstly, t… ▽ More As a technology that can prevent the information from being disclosed, the reversible data hiding in encrypted images (RDHEI) acts as an important role in privacy protection and information security. To make use of the image redundancy and further improve the embedding performance, a high-capacity RDHEI method based on bit-plane compression of prediction error is proposed in this paper. Firstly, the whole prediction error is calculated and divided into blocks of the same size. Then, the content owner rearranges the bit-plane of prediction error by block and compresses the bitstream with the joint encoding algorithm to reserve room. Finally, the image is encrypted and the information can be embedded into the reserved room. On the receiver side, the information extraction and the image recovery are performed separably. Experimental results show that the proposed method brings higher embedding capacity than state-of-the-art RDHEI works. △ Less

Submitted 24 September, 2021; v1 submitted 8 July, 2020; originally announced July 2020.

Reversible data hiding in encrypted images based on pixel prediction and multi-MSB planes rearrangement

Authors: Zhaoxia Yin, Xiaomeng She, Jin Tang, Bin Luo

Abstract: Great concern has arisen in the field of reversible data hiding in encrypted images (RDHEI) due to the development of cloud storage and privacy protection. RDHEI is an effective technology that can embed additional data after image encryption, extract additional data error-free and reconstruct original images losslessly. In this paper, a high-capacity and fully reversible RDHEI method is proposed,… ▽ More Great concern has arisen in the field of reversible data hiding in encrypted images (RDHEI) due to the development of cloud storage and privacy protection. RDHEI is an effective technology that can embed additional data after image encryption, extract additional data error-free and reconstruct original images losslessly. In this paper, a high-capacity and fully reversible RDHEI method is proposed, which is based on pixel prediction and multi-MSB (most significant bit) planes rearrangement. First, the median edge detector (MED) predictor is used to calculate the predicted value. Next, unlike previous methods, in our proposed method, signs of prediction errors (PEs) are represented by one bit plane and absolute values of PEs are represented by other bit planes. Then, we divide bit planes into uniform blocks and non-uniform blocks, and rearrange these blocks. Finally, according to different pixel prediction schemes, different numbers of additional data are embedded adaptively. The experimental results prove that our method has higher embedding capacity compared with state-of-the-art RDHEI methods. △ Less

Submitted 20 March, 2021; v1 submitted 8 July, 2020; originally announced July 2020.

Journal ref: Signal Processing, 2021, 187: 1-11

New Framework for Code-Mapping-based Reversible Data Hiding in JPEG Images

Authors: Yang Du, Zhaoxia Yin

Abstract: Code mapping (CM) is an efficient technique for reversible data hiding (RDH) in JPEG images, which embeds data by constructing a mapping relationship between the used and unused codes in the JPEG bitstream. This study presents a new framework for designing a CM-based RDH method. First, a new code mapping strategy is proposed to suppress file size expansion and improve applicability. Based on our p… ▽ More Code mapping (CM) is an efficient technique for reversible data hiding (RDH) in JPEG images, which embeds data by constructing a mapping relationship between the used and unused codes in the JPEG bitstream. This study presents a new framework for designing a CM-based RDH method. First, a new code mapping strategy is proposed to suppress file size expansion and improve applicability. Based on our proposed strategy, the mapped codes are redefined by creating a new Huffman table rather than selecting them from the unused codes in the original Huffman table. The critical issue of designing the CM-based RDH method, that is, constructing code mapping, is converted into a combinatorial optimization problem. This study proposes a novel CM-based RDH method that utilizes a genetic algorithm (GA). The experimental results demonstrate that the proposed method achieves a high embedding capacity with no signal distortion while suppressing file size expansion. △ Less

Submitted 16 July, 2022; v1 submitted 29 June, 2020; originally announced June 2020.

Reversible Adversarial Attack based on Reversible Image Transformation

Authors: Zhaoxia Yin, Hua Wang, Li Chen, Jie Wang, Weiming Zhang

Abstract: In order to prevent illegal or unauthorized access of image data such as human faces and ensure legitimate users can use authorization-protected data, reversible adversarial attack technique is rise. Reversible adversarial examples (RAE) get both attack capability and reversibility at the same time. However, the existing technique can not meet application requirements because of serious distortion… ▽ More In order to prevent illegal or unauthorized access of image data such as human faces and ensure legitimate users can use authorization-protected data, reversible adversarial attack technique is rise. Reversible adversarial examples (RAE) get both attack capability and reversibility at the same time. However, the existing technique can not meet application requirements because of serious distortion and failure of image recovery when adversarial perturbations get strong. In this paper, we take advantage of Reversible Image Transformation technique to generate RAE and achieve reversible adversarial attack. Experimental results show that proposed RAE generation scheme can ensure imperceptible image distortion and the original image can be reconstructed error-free. What's more, both the attack ability and the image quality are not limited by the perturbation amplitude. △ Less

Submitted 25 May, 2021; v1 submitted 6 November, 2019; originally announced November 2019.

Comments: 2021 International Workshop on Safety & Security of Deep Learning

Efficient Multi-robot Exploration via Multi-head Attention-based Cooperation Strategy

Authors: Shuqi Liu, Zhaoxia Wu

Abstract: The goal of coordinated multi-robot exploration tasks is to employ a team of autonomous robots to explore an unknown environment as quickly as possible. Compared with human-designed methods, which began with heuristic and rule-based approaches, learning-based methods enable individual robots to learn sophisticated and hard-to-design cooperation strategies through deep reinforcement learning techno… ▽ More The goal of coordinated multi-robot exploration tasks is to employ a team of autonomous robots to explore an unknown environment as quickly as possible. Compared with human-designed methods, which began with heuristic and rule-based approaches, learning-based methods enable individual robots to learn sophisticated and hard-to-design cooperation strategies through deep reinforcement learning technologies. However, in decentralized multi-robot exploration tasks, learning-based algorithms are still far from being universally applicable to the continuous space due to the difficulties associated with area calculation and reward function designing; moreover, existing learning-based methods encounter problems when attempting to balance the historical trajectory issue and target area conflict problem. Furthermore, the scalability of these methods to a large number of agents is poor because of the exponential explosion problem of state space. Accordingly, this paper proposes a novel approach - Multi-head Attention-based Multi-robot Exploration in Continuous Space (MAMECS) - aimed at reducing the state space and automatically learning the cooperation strategies required for decentralized multi-robot exploration tasks in continuous space. Computational geometry knowledge is applied to describe the environment in continuous space and to design an improved reward function to ensure a superior exploration rate. Moreover, the multi-head attention mechanism employed helps to solve the historical trajectory issue in the decentralized multi-robot exploration task, as well as to reduce the quadratic increase of action space. △ Less

Submitted 5 November, 2019; originally announced November 2019.

Reversible Data Hiding in Encrypted Images based on Pixel Prediction and Bit-plane Compression

Authors: Zhaoxia Yin, Yinyin Peng, Youzhi Xiang

Abstract: Reversible data hiding in encrypted images (RDHEI) receives growing attention because it protects the content of the original image while the embedded data can be accurately extracted and the original image can be reconstructed lossless. To make full use of the correlation of the adjacent pixels, this paper proposes an RDHEI scheme based on pixel prediction and bit-plane compression. Firstly, to v… ▽ More Reversible data hiding in encrypted images (RDHEI) receives growing attention because it protects the content of the original image while the embedded data can be accurately extracted and the original image can be reconstructed lossless. To make full use of the correlation of the adjacent pixels, this paper proposes an RDHEI scheme based on pixel prediction and bit-plane compression. Firstly, to vacate room for data embedding, the prediction error of the original image is calculated and used for bit-plane rearrangement and compression. Then, the image after vacating room is encrypted by a stream cipher. Finally, the additional data is embedded in the vacated room by multi-LSB substitution. Experimental results show that the embedding capacity of the proposed method outperforms the state-of-the-art methods. △ Less

Submitted 5 November, 2019; originally announced November 2019.

Separable Reversible Data Hiding Based on Integer Mapping and Multi-MSB Prediction for Encrypted 3D Mesh Models

Authors: Zhaoxia Yin, Na Xu, Feng Wang

Abstract: Reversible data hiding in encrypted domain (RDH-ED) has received tremendous attention from the research community because data can be embedded into cover media without exposing it to the third party data hider and the cover media can be losslessly recovered after the extraction of the embedded data. Although, in recent years, extensive studies have been carried out about images based RDH-ED, littl… ▽ More Reversible data hiding in encrypted domain (RDH-ED) has received tremendous attention from the research community because data can be embedded into cover media without exposing it to the third party data hider and the cover media can be losslessly recovered after the extraction of the embedded data. Although, in recent years, extensive studies have been carried out about images based RDH-ED, little attention is paid to RDH-ED in 3D meshes due to its complex data structure and irregular geometry. In this paper, we propose a separable RDH-ED method for 3D meshes based on integer mapping and Multi-MSB (multiplication most significant bit) prediction. The proposed method divides all the vertices of the mesh into the "embedded" set and "reference" set, and maps decimals of the vertex into integers. Then, we calculate the Multi-MSB prediction errors for the vertices of the "embedded" set and a bit-stream encryption technique will be executed. Finally, additional data is embedded by replacing the Multi-MSB of the encrypted vertex coordinates. According to different permissions, recipient can obtain the original plaintext meshes, additional data or both. Experimental results show that the proposed method has higher embedding capacity and higher quality of the recovered meshes compared to the state-of-art methods. △ Less

Submitted 16 November, 2019; v1 submitted 7 August, 2019; originally announced August 2019.

Image Encryption Algorithm Based on Facebook Social Network

Authors: Xiaoqing Liu, Yinyin Peng, Jie Wang, Zhaoxia Yin

Abstract: Facebook is the online social networks (OSNs) platform with the largest number of users in the world today, information protection based on Facebook social network platform have important practical significance. Since the information users share on social networks is often based on images, this paper proposes a more secure image encryption algorithm based on Facebook social network platform to ens… ▽ More Facebook is the online social networks (OSNs) platform with the largest number of users in the world today, information protection based on Facebook social network platform have important practical significance. Since the information users share on social networks is often based on images, this paper proposes a more secure image encryption algorithm based on Facebook social network platform to ensure the loss of information as much as possible. When the sender encrypts the image for uploading, it can first resist the third party's attack on the encrypted image and prevent the image data from leaking, simultaneously processed by some unknown processing such as compression and filtering of the image on the Facebook platform, the receiver can still decrypt the corresponding image data. △ Less

Submitted 21 May, 2019; originally announced June 2019.

Comments: in Chinese

An Improved Reversible Data Hiding in Encrypted Images using Parametric Binary Tree Labeling

Authors: Youqing Wu, Youzhi Xiang, Yutang Guo, Jin Tang, Zhaoxia Yin

Abstract: This work proposes an improved reversible data hiding scheme in encrypted images using parametric binary tree labeling(IPBTL-RDHEI), which takes advantage of the spatial correlation in the entire original image but not in small image blocks to reserve room for hiding data. Then the original image is encrypted with an encryption key and the parametric binary tree is used to label encrypted pixels i… ▽ More This work proposes an improved reversible data hiding scheme in encrypted images using parametric binary tree labeling(IPBTL-RDHEI), which takes advantage of the spatial correlation in the entire original image but not in small image blocks to reserve room for hiding data. Then the original image is encrypted with an encryption key and the parametric binary tree is used to label encrypted pixels into two different categories. Finally, one of the two categories of encrypted pixels can embed secret information by bit replacement. According to the experimental results, compared with several state-of-the-art methods, the proposed IPBTL-RDHEI method achieves higher embedding rate and outperforms the competitors. Due to the reversibility of IPBTL-RDHEI, the original plaintext image and the secret information can be restored and extracted losslessly and separately. △ Less

Submitted 25 November, 2019; v1 submitted 23 May, 2019; originally announced May 2019.

Journal ref: IEEE Transactions on Multimedia, 2019, 22(8): 1929-1938

Multiple reconstruction compression framework based on PNG image

Authors: Zhiqing Lu, Zhaoxia Yin, Bin Luo

Abstract: It is shown that neural networks (NNs) achieve excellent performances in image compression and reconstruction. However, there are still many shortcomings in the practical application, which eventually lead to the loss of neural network image processing ability. Based on this, this paper proposes a joint framework based on neural network and zoom compression. The framework first encodes the incomin… ▽ More It is shown that neural networks (NNs) achieve excellent performances in image compression and reconstruction. However, there are still many shortcomings in the practical application, which eventually lead to the loss of neural network image processing ability. Based on this, this paper proposes a joint framework based on neural network and zoom compression. The framework first encodes the incoming PNG or JPEG image information, and then the image is converted into binary input decoder to reconstruct the intermediate state image, next we import the intermediate state image into the zooming compressor and re-pressurize it, and reconstruct the final image. From the experimental results, this method can better process the digital image and suppress the reverse expansion problem, and the compression effect can be improved by 4 to 10 times as much as that of using RNN alone, showing better ability in application. In this paper, the method is transmitted over a digital image, the effect is far better than the existing compression method alone, the Human visual system can not feel the change of the effect. △ Less

Submitted 14 November, 2019; v1 submitted 22 May, 2019; originally announced May 2019.

Comments: The experimental results cannot reproduced

An Efficient Pre-processing Method to Eliminate Adversarial Effects

Authors: Hua Wang, Jie Wang, Zhaoxia Yin

Abstract: Deep Neural Networks (DNNs) are vulnerable to adversarial examples generated by imposing subtle perturbations to inputs that lead a model to predict incorrect outputs. Currently, a large number of researches on defending adversarial examples pay little attention to the real-world applications, either with high computational complexity or poor defensive effects. Motivated by this observation, we de… ▽ More Deep Neural Networks (DNNs) are vulnerable to adversarial examples generated by imposing subtle perturbations to inputs that lead a model to predict incorrect outputs. Currently, a large number of researches on defending adversarial examples pay little attention to the real-world applications, either with high computational complexity or poor defensive effects. Motivated by this observation, we develop an efficient preprocessing method to defend adversarial images. Specifically, before an adversarial example is fed into the model, we perform two image transformations: WebP compression, which is utilized to remove the small adversarial noises. Flip operation, which flips the image once along one side of the image to destroy the specific structure of adversarial perturbations. Finally, a de-perturbed sample is obtained and can be correctly classified by DNNs. Experimental results on ImageNet show that our method outperforms the state-of-the-art defense methods. It can effectively defend adversarial attacks while ensure only very small accuracy drop on normal images. △ Less

Submitted 30 December, 2019; v1 submitted 15 May, 2019; originally announced May 2019.

Comments: in Chinese

High Capacity Lossless Data Hiding in JPEG Bitstream Based on General VLC Mapping

Authors: Yang Du, Zhaoxia Yin, Xinpeng Zhang

Abstract: JPEG is the most popular image format, which is widely used in our daily life. Therefore, reversible data hiding (RDH) for JPEG images is important. Most of the RDH schemes for JPEG images will cause significant distortions and large file size increments in the marked JPEG image. As a special case of RDH, the lossless data hiding (LDH) technique can keep the visual quality of the marked images no… ▽ More JPEG is the most popular image format, which is widely used in our daily life. Therefore, reversible data hiding (RDH) for JPEG images is important. Most of the RDH schemes for JPEG images will cause significant distortions and large file size increments in the marked JPEG image. As a special case of RDH, the lossless data hiding (LDH) technique can keep the visual quality of the marked images no degradation. In this paper, a novel high capacity LDH scheme is proposed. In the JPEG bitstream, not all the variable length codes (VLC) are used to encode image data. By constructing the mapping between the used and unused VLCs, the secret data can be embedded by replacing the used VLC with the unused VLC. Different from the previous schemes, our mapping strategy allows the lengths of unused and used VLCs in a mapping set to be unequal. We present some basic insights into the construction of the mapping relationship. Experimental results show that most of the JPEG images using the proposed scheme obtain smaller file size increments than previous RDH schemes. Furthermore, the proposed scheme can obtain high embedding capacity while keeping the marked JPEG image with no distortion. △ Less

Submitted 18 November, 2019; v1 submitted 14 May, 2019; originally announced May 2019.

Reversible data hiding based on reducing invalid shifting of pixels in histogram shifting

Authors: Yujie Jia, Zhaoxia Yin, Xinpeng Zhang, Yonglong Luo

Abstract: In recent years, reversible data hiding (RDH), a new research hotspot in the field of information security, has been paid more and more attention by researchers. Most of the existing RDH schemes do not fully take it into account that natural image's texture has influence on embedding distortion. The image distortion caused by embedding data in the image's smooth region is much smaller than that in… ▽ More In recent years, reversible data hiding (RDH), a new research hotspot in the field of information security, has been paid more and more attention by researchers. Most of the existing RDH schemes do not fully take it into account that natural image's texture has influence on embedding distortion. The image distortion caused by embedding data in the image's smooth region is much smaller than that in the unsmooth region, essentially, it is because embedding additional data in the smooth region corresponds to fewer invalid shifting pixels (ISPs) in histogram shifting. Thus, we propose a RDH scheme based on the images texture to reduce invalid shifting of pixels in histogram shifting. Specifically, first, a cover image is divided into two sub-images by the checkerboard pattern, and then each sub-image's fluctuation values are calculated. Finally, additional data can be embedded into the region of sub-images with smaller fluctuation value preferentially. The experimental results demonstrate that the proposed method has higher capacity and better stego-image quality than some existing RDH schemes. △ Less

Submitted 13 May, 2019; originally announced May 2019.

Comments: 11 pages, 11 figures, 1 table

Journal ref: Signal Processing, 2019, 163: 238-246

Reversible Data Hiding in JPEG Images with Multi-objective Optimization

Authors: Zhaoxia Yin, Yuan Ji, Bin Luo

Abstract: Among various methods of reversible data hiding (RDH) in JPEG images, the consideration in designing is only the image quality, but the image quality and the file size expansion are equally important in JPEG images. Based on this situation, we propose a RDH scheme in JPEG images considering both the image quality and the file size expansion while designing the algorithm. The multi-objective optimi… ▽ More Among various methods of reversible data hiding (RDH) in JPEG images, the consideration in designing is only the image quality, but the image quality and the file size expansion are equally important in JPEG images. Based on this situation, we propose a RDH scheme in JPEG images considering both the image quality and the file size expansion while designing the algorithm. The multi-objective optimization strategy is utilized to realize the balance of the two objectives. Specifically, the cover is divided into several non-overlapping signals firstly, and after that, the embedding costs of signals are calculated using the knowledge of the JPEG compression. Next, the optimized combination of signals for embedding data is gained by the multi-objective optimization. Experimental results show the better performance of our proposed RDH compared with state-of-the-art RDH in JPEG images. △ Less

Submitted 9 May, 2019; originally announced May 2019.

Journal ref: IEEE Transactions on Circuits and Systems for Video Technology, 2020, 30(8): 2343-2352

Reversible Data Hiding in Encrypted Images based on MSB Prediction and Huffman Coding

Authors: Youzhi Xiang, Zhaoxia Yin, Xinpeng Zhang

Abstract: With the development of cloud storage and privacy protection, reversible data hiding in encrypted images (RDHEI) has attracted increasing attention as a technology that can embed additional data in the encryption domain. In general, an RDHEI method embeds secret data in an encrypted image while ensuring that the embedded data can be extracted error-free and the original image can be restored lossl… ▽ More With the development of cloud storage and privacy protection, reversible data hiding in encrypted images (RDHEI) has attracted increasing attention as a technology that can embed additional data in the encryption domain. In general, an RDHEI method embeds secret data in an encrypted image while ensuring that the embedded data can be extracted error-free and the original image can be restored lossless. In this paper, A high-capacity RDHEI algorithm is proposed. At first, the Most Significant Bits (MSB) of each pixel was predicted adaptively and marked by Huffman coding in the original image. Then, the image was encrypted by a stream cipher method. At last, the vacated space can be used to embed additional data. Experimental results show that our method achieved higher embedding capacity while comparing with the state-of-the-art methods. △ Less

Submitted 22 December, 2018; originally announced December 2018.

Comments: 20 pages, 8 figures, to submit to IEEE Trans. Multimedia

Journal ref: [J]. IEEE Transactions on Multimedia, 2019, 22(4): 874-884