Marktechpost Media Inc.

Imposter.AI: Unveiling Adversarial Attack Strategies to Expose Vulnerabilities in Advanced Large Language Models Researchers from Meetyou AI Lab, Osaka University, and East China Normal University have introduced an innovative adversarial attack method called Imposter.AI. This method leverages human conversation strategies to extract harmful information from LLMs. Unlike traditional attack methods, Imposter.AI focuses on the nature of the information in the responses rather than on explicit malicious inputs. The researchers delineate three key strategies: decomposing harmful questions into seemingly benign sub-questions, rephrasing overtly malicious questions into less suspicious ones, and enhancing the harmfulness of responses by prompting the models for detailed examples. Imposter.AI employs a three-pronged approach to elicit harmful responses from LLMs. First, it breaks down harmful questions into multiple, less harmful sub-questions, which obfuscates the malicious intent and exploits the LLMs’ limited context window. Second, it rephrases overtly harmful questions to appear benign on the surface, thus bypassing content filters. Third, it enhances the harmfulness of responses by prompting the LLMs to provide detailed, example-based information. These strategies exploit the LLMs’ inherent limitations, increasing the likelihood of obtaining sensitive information without triggering safety mechanisms. Quick read: https://lnkd.in/gJK6ZRGf Paper: https://lnkd.in/gy4GapPb #llms

Nvidia AI Introduces NV-Retriever-v1: An Embedding Model Optimized for Retrieval A team of researchers from NVIDIA have introduced a state-of-the-art embedding model called NV-Retriever-v1. It is a family of hard-negative mining methods that utilizes the positive relevance score, to remove false negatives more effectively. It performed exceptionally well, scoring an average of 60.9 across 15 BEIR datasets, and took first place on the MTEB Retrieval leaderboard when it was published there on July 11th, 2024. A key method for training this model is hard-negative mining, which is important for achieving top performance in text retrieval. This includes selecting the top-k most similar candidates to the query, after ignoring the positive passages, which is called Naive Top-K. The MTEB includes tasks like retrieval, reranking, classification, and clustering, so there is a requirement for various training datasets for it to perform well. The method NV-Retriever-v1 is fine-tuned using the E5-Mistral-7B embedding model for hard-negative mining with a maximum sequence length of 4096. The proposed method, TopK-PercPos is used to avoid false negatives, setting the negative relevance score threshold at 95% of the positive score. Moreover, there are two stages of instruction tuning. In the first stage, retrieval supervised data with in-batch negatives and mined hard-negative are used. In the second stage, the data for the retrieval task is combined with datasets from other tasks. Quick read: https://lnkd.in/g9YtZHza Paper: https://lnkd.in/gi6yp_r6 Model Card: https://lnkd.in/g2gWNSXW NVIDIA NVIDIA AI

Here are some upcoming AI Webinars from various AI companies Get the full list here: https://lnkd.in/gTEZRemD Databricks Zilliz CrateDB Snowflake Gretel SuperAnnotate SingleStore H2O.ai John Snow Labs Snorkel AI #ai #webinar #rag

LEAN-GitHub: A Large-Scale Dataset for Advancing Automated Theorem Proving Researchers from The Chinese University of Hong Kong propose LEAN-GitHub, a large-scale Lean dataset that complements the well-utilized Mathlib dataset. This innovative approach provides an open-source Lean repositories on GitHub, significantly expanding the available data for training theorem-proving models. The researchers developed a scalable pipeline to enhance extraction efficiency and parallelism, enabling the exploitation of valuable data from previously uncompiled and unextracted Lean corpus. Also, they provide a solution to the state duplication problem common in tree-proof search methods. Read our take on this: https://lnkd.in/gGnt8XmK Paper: https://lnkd.in/gKAYmiZa Dataset: https://lnkd.in/gCZYWA6M

This AI Paper from Cohere AI Introduces a Multi-faceted Approach to AI Governance by Rethinking Compute Thresholds Cohere for AI researcher has introduced a critical examination of these compute thresholds as a governance tool. They argue that current implementations are shortsighted and fail to effectively mitigate risks. They emphasize that the relationship between compute and risk is highly uncertain and rapidly evolving. Instead of relying solely on compute thresholds, they suggest a more nuanced approach to AI governance that considers multiple factors influencing AI’s risk profile. The proposed approach advocates for a dynamic and comprehensive evaluation of AI systems rather than fixed compute thresholds. This includes better specifying FLOP as a metric, considering additional dimensions of AI performance and risk, and implementing adaptive thresholds that adjust to the evolving landscape of AI capabilities. The researchers recommend enhancing transparency and standardization in reporting AI risks and aligning governance practices with the actual performance and potential harms of AI systems. This comprehensive method involves examining factors such as the quality of training data, optimization techniques, and the specific applications of AI models to ensure a more accurate assessment of potential risks. Read our take on this: https://lnkd.in/gNTUCki2 Paper: https://lnkd.in/grj9KrfT Cohere

Researchers at Google Deepmind Introduce BOND: A Novel RLHF Method that Fine-Tunes the Policy via Online Distillation of the Best-of-N Sampling Distribution Researchers at Google DeepMind have introduced Best-of-N Distillation (BOND), an innovative RLHF algorithm designed to replicate the performance of Best-of-N sampling without its high computational cost. BOND is a distribution matching algorithm that aligns the policy’s output with the Best-of-N distribution. Using Jeffreys divergence, which balances mode-covering and mode-seeking behaviors, BOND iteratively refines the policy through a moving anchor approach. Experiments on abstractive summarization and Gemma models show that BOND, particularly its variant J-BOND, outperforms other RLHF algorithms by enhancing KL-reward trade-offs and benchmark performance. Best-of-N sampling optimizes language generation against a reward function but is computationally expensive. Recent studies have refined its theoretical foundations, provided reward estimators, and explored its connections to KL-constrained reinforcement learning. Various methods have been proposed to match the Best-of-N strategy, such as supervised fine-tuning on Best-of-N data and preference optimization. BOND introduces a novel approach using Jeffreys divergence and iterative distillation with a dynamic anchor to efficiently achieve the benefits of Best-of-N sampling. This method focuses on investing resources during training to reduce inference-time computational demands, aligning with principles of iterated amplification. Quick read: https://lnkd.in/gRDFdnW7 Paper: https://lnkd.in/gkb5sJ4r Google DeepMind

Google AI Introduces NeuralGCM: A New Machine Learning (ML) based Approach to Simulating Earth’s Atmosphere GoogleAI proposes NeuralGCM to address the limitations in weather and climate prediction using general circulation models (GCMs). Traditional GCMs, which rely on physics-based simulations, are computationally intensive and struggle with long-term stability and accurate ensemble forecasts. These GCMs combine numerical solvers for large-scale atmospheric dynamics with empirical parameterizations for smaller-scale processes like cloud formation. Machine-learning models, trained on historical data like ECMWF’s ERA5, have demonstrated impressive short-term weather prediction capabilities at lower computational costs but fail in long-term forecasting and ensemble accuracy..... Read our take on this: https://lnkd.in/gbjyuwz2 Paper: https://lnkd.in/gY9NMQyr Details: https://lnkd.in/ggPFkYU2 Google

Yandex Introduces TabReD: A New Benchmark for Tabular Machine Learning Quick read: https://lnkd.in/gfU8Pmn7 Paper: https://lnkd.in/guVMmnBS GitHub: https://lnkd.in/gTQH_2nC Yandex #machinelearning #ai

This AI Paper from UC Berkeley Shows How Interfacing GPT with Prolog (Reliable Symbolic System) Drastically Improves Its Math Problem-Solving Abilities Researchers from the University of California, Berkeley, have proposed integrating a reliable, deductive reasoning module into their inference pipeline. In their study, researchers prompted the model to encode the constraints and relationships as a set of Prolog code statements in variables explained in the problem statement. The Prolog evaluates the generated code using a deductive technique to provide a definite answer to the problem. The method also provides the benefit of reflecting the probable human architecture of separate linguistic and reasoning systems. It also greatly enhances the performance of LLMs for mathematical reasoning. Moreover, researchers have introduced the Non-Linear Reasoning dataset (NLR), a new dataset created to test how well large language models (LLMs) can handle mathematical reasoning. This new dataset aims to address issues found in existing ones, such as overlap between test and training sets and the repetitive nature of reasoning patterns in current benchmarks. The NLR dataset ensures that it is not included in current models’ training sets, and each problem requires a unique and creative reasoning pattern to solve but is limited to basic arithmetic and algebra skills. This benchmark contains unique constraint problems, math word problems, and problems related to algorithmic instructions for updating a game model. Quick read: https://lnkd.in/gewfqH4c Paper: https://lnkd.in/gChZC9Fg