Gregory Renard

I think AI agentic machine translation has huge potential for improving over traditional neural machine translation, and am releasing as open-source a demonstration I'd been playing with as a fun weekend project. Using an agentic workflow, this demonstration (i) Prompts an LLM to translate from one language to another, (ii) Reflects on the translation to come up with constructive suggestions, (iii) Uses the suggestions to refine the translation. In our limited testing, this is sometimes competitive with, and sometimes worse than, leading commercial providers. But it gives a highly steerable translation system where by simply changing the prompt, you can specify the tone (formal/informal), regional variation (do you want Spanish as spoken in Spain or as spoken in Latin America?), and ensure consistent translation of terms (by providing a glossary). This is not mature software. But I hope the open-source community can make agentic translation work much better. Given how a simple reflection workflow already gives decent results, I think there's significant headroom to make agentic translation much better. Releasing an early software prototype like this is something new I decided to try to see if it is helpful to the developer community. I'd love any feedback on this. Thanks to Joaquin Dominguez, Nedelina Teneva, PhD and John Santerre PhD for help with this. https://lnkd.in/gjGANH6H

5,285

165 Comments

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

If you want to understand why LLMs are not the solution for AGI (Artificial General Intelligence), I highly recommend listening to Lex Fridman podcast with Yann LeCun : Meta AI, Open Source, Limits of LLMs, AGI & the Future of AI T͟L͟D͟R͟: 1. LLMs do not understand the physical world 2. LLMs have not the ability to remember 3. LLMs have no persistent memory 4. LLMs are not able to reason and to plan

27

5 Comments

New AI Agentic course! Learn to use LangGraph to build single and multi-agent LLM applications in AI Agents in LangGraph. This short course, taught by LangChain founder Harrison Chase and Tavily founder Rotem Weiss, shows how to integrate agentic search to enhance an agent's knowledge with query-focused answers in predictable formats. Also learn to implement agentic memory to save state for reasoning and debugging, and see how human-in-the-loop input can guide agents at key junctures. You'll build an agent from scratch, then reconstruct it with LangGraph to thoroughly understand the framework. Finally, you'll build a sophisticated essay-writing agent that incorporates all the learnings from the course. Sign up here! https://lnkd.in/gsQnySCA

5,205

135 Comments

👉🏼 Dual Causes Generation Assisted Model for Multimodal Aspect-Based Sentiment Classification 🤓 Rui Fan 👇🏻 https://lnkd.in/eC5m_4qv 🔍 Focus on data insights: - The proposed MDCA method tracks underlying causes behind expressed sentiments, providing reasoning cause (RC) and direct cause (DC) to explain emotions. - Utilizes large language models (LLMs) and visual-language models to construct MABSC datasets with RC and DC. - Implements a multitask learning framework leveraging cause data to train a generative model for sentiment prediction. 💡 Main outcomes and implications: - MDCA model achieves state-of-the-art performance in MABSC benchmark datasets. - Small fine-tuned model shows superior adaptability to MABSC compared to larger models like ChatGPT and BLIP-2. 📚 Field significance: - Advances in understanding sentiment analysis in multimodal data. - Enhances accuracy and interpretability of sentiment prediction models. 🗄️: #Multimodal #SentimentAnalysis #DataInsights #GenerativeModel

1

LLM-Select: Feature Selection with Large Language Models D P. Jeong, Z C. Lipton, P Ravikumar Carnegie Mellon (2024) https://lnkd.in/ei5v2myA - LLMs can perform effective feature selection for supervised learning tasks by just using feature names and task descriptions, without seeing any training data. - The authors propose 3 methods for LLM-based feature selection: scoring features by importance (LLM-Score), ranking features (LLM-Rank), and sequentially selecting features via dialogue (LLM-Seq). - Sufficiently large LLMs like GPT-4 can identify predictive features just as well as traditional data-driven methods like LASSO, even with simple prompting strategies like zero-shot scoring features one at a time. - LLM-based feature selection works well across diverse real-world datasets, suggesting LLMs encode rich knowledge about real-world relationships that can inform feature selection. - LLMs may be useful not only for selecting features after data collection, but also for deciding what features to collect, especially in expensive domains like healthcare. abs: In this paper, we demonstrate a surprising capability of large language models (LLMs): given only input feature names and a description of a prediction task, they are capable of selecting the most predictive features, with performance rivaling the standard tools of data science. Remarkably, these models exhibit this capacity across various query mechanisms. For example, we zero-shot prompt an LLM to output a numerical importance score for a feature (e.g., "blood pressure") in predicting an outcome of interest (e.g., "heart failure"), with no additional context. In particular, we find that the latest models, such as GPT-4, can consistently identify the most predictive features regardless of the query mechanism and across various prompting strategies. We illustrate these findings through extensive experiments on real-world data, where we show that LLM-based feature selection consistently achieves strong performance competitive with data-driven methods such as the LASSO, despite never having looked at the downstream training data. Our findings suggest that LLMs may be useful not only for selecting the best features for training but also for deciding which features to collect in the first place. This could potentially benefit practitioners in domains like healthcare, where collecting high-quality data comes at a high cost. #AI #machinelearning #computation #language #LLM

31

Interesting early approach by Anthropic for reducing transformers models complexity, helping with interpretability and (consequently) reducing hallucinations and errors #GenerativeAI #largelanguagemodels #artificialintelligence #airevolution

53

1 Comment

#Performance of #LLMs. The adoption of #LLMs is rapidly growing, but the difficulty arises when trying to apply them beyond experimental settings. Deploying LLMs in an actual environment is complex, and one of the significant challenges is effectively evaluating their performance. Evaluation of LLMs can be based on different factors like toxicity, helpfulness, correctness and more. This article by Surya Prakash Pathak from Red Hat explores various evaluation techniques and metrics employed in assessing the performance of LLMs, with a particular emphasis on retrieval-augmented generation (#RAG) use cases. RAG, a pivotal use case for different enterprises, focuses on combining retrieval methods with the generation capabilities of LLMs. This unique approach involves not only generating content based on learned patterns but also leveraging pre-existing information through retrieval mechanisms. Scott Creeley :: Stephen Flaherty https://lnkd.in/gJTBagjv

13

2 Comments

Function calling is a powerful way to extend the capabilities of LLMs and AI agents by letting them use external tools. Our new short course Function calling and Data Extraction with LLMs, created with Nexusflow and taught by Jiantao Jiao and Venkat Srinivasan, demonstrates how to prompt LLMs to form calls to external functions. You'll work with NexusRavenV2-13B, a 13B parameter open-source model that excels in function calling tasks while still being small enough to host locally. Learn to use function calling to extract structured data from unstructured text and access web APIs, and build an end-to-end application that processes customer service transcripts. You'll learn how to build LLM-powered applications that can analyze feedback, automate data entry, and enhance search. Please get started here: https://lnkd.in/g9FpiCuH

5,043

114 Comments

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

Hallucination is Inevitable in LLMs An important new paper out that shows that even with perfect data and an idealized architecture “LLMs will always hallucinate”. I suspect that this will over time be linked to Gödel’s Incompleteness Theorem [1] and Turing’s Halting Problem [2]. Every technology has trade-offs. We love LLMs but they need to be used in the right way. [1] Good pop science depiction in Oppenheimer (Gödel was Einstein’s walking companion) or also see Rebecca Goldstein’s book Incompleteness [2] Turing is depicted in The Imitation Game and also the progenitor of the famed Turing Test.

32

These findings motivate our design of a novel framework to investigate hidden representations in LLMs with LLMs, which we call Patchscopes. The key idea behind this framework is to use LLMs to provide natural language explanations of their own internal hidden representations. Patchscopes unifies and extends a broad range of existing interpretability techniques, and it enables answering questions that were difficult or impossible before. For example, it offers insights into how an LLM's hidden representations capture nuances of meaning in the model's input, making it easier to fix certain types of reasoning errors. While we initially focus the application of Patchscopes to the natural language domain and the autoregressive Transformer model family, its potential applications are broader. For example, we are excited about its applications to detection and correction of model hallucinations, the exploration of multimodal (image and text) representations, and the investigation of how models build their predictions in more complex scenarios. Google DeepMind

8

Check out the new work on Holistic Evaluation of Multimodal Foundation Models: Paper: https://lnkd.in/eHPd8ihB Code: https://lnkd.in/eSjs-5tF The goal is to systematically evaluate capabilities of multimodal models covering 3 dimensions: Basic multimodal skills: learning interactions across modalities, fine-grained alignment, multi-step reasoning, handling external knowledge. Information flow: how multimodal content changes during a task through querying, translation, editing, and fusion. Use cases: multimedia, affective computing, natural sciences, healthcare, human-computer interaction applications. With Paul Liang, Akshay Goindani, Talha Chafekar, Leena Mathur, Haofei Yu, and Louis-Philippe Morency.

96

2 Comments

We just released a blog on whether the use of multimodal feature transfer can detect deception (phenomena like sarcasm, irony, and condescension) in conversation. The short answer... Yes. This particular write-up profiles a subset of our work at Symbl.ai on multimodal large language models, and is research led by one of our summer interns, Benjamin R. (from Prof. Larry Heck's AVA Lab at Georgia Institute of Technology). We have a paper currently under submission on this work, and we can't wait to share the full details with all of you soon. Blog Link: https://lnkd.in/g6m_wFDi #multimodal #featuretransfer #LLMs #deception #conversation #sarcasm #irony #language #AI #ML #NLP #LargeLanguageModels #SummerInternship #ResearchPapers #ACL #Llama2 #SymblDotAI

16

👉🏼 Reliability of large language models for advanced head and neck malignancies management: a comparison between ChatGPT 4 and Gemini Advanced 🤓 Andrea Lorenzi 👇🏻 https://lnkd.in/eSW4Admy 🔍 Focus on data insights: - ChatGPT 4 generally outperformed Gemini Advanced in adherence to guidelines and comprehensive treatment planning. - Higher AIPI scores were observed with ChatGPT 4, indicating better overall performance. 💡 Main outcomes and implications: - Both large language models showed potential in aiding multidisciplinary management of head and neck oncology. - Discrepancies in critical areas emphasize the need for further refinement and validation against current clinical standards. 📚 Field significance: - Growing role of AI in enhancing clinical decision-making. - Necessity for continuous updates and validation to fully integrate AI into healthcare practices. 🗄️: [#largeLanguageModels #headAndNeckMalignancies #AIclinicalDecisionMaking]

1

1 Comment