Burag Cetinkaya

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

The Stanford Human-Centered AI Report is out. This is the 7th year that Stanford has published this report and it is a must-read if you're trying to keep track of the developments in this field. https://lnkd.in/gwsHtzsq

2

"What is AI anyways?" in this thought provoking and inspirational TED Talk, Mustafa Suleyman, CEO of Microsoft AI, attempts to answer his six-year-old nephew's question with a logical and radical metaphor: AI - a new digital species. Check it out! #NewDigitalSpecies #AI #homotechnologicus #microsoftadvocate

1

Very useful insights for folks who are starting to dig further and further into the realm of LLM. An easier GUI approach for fine tuning LLMs.

9

A new research paper by our incredible teams at Google Research and Google DeepMind shows the strong performance of Med-Gemini across a variety of multimodal tasks, including  2D and 3D imaging in radiology, histopathology, ophthalmology, and dermatology, and genomics. Med-Gemini ranked best in class on chest X-ray and CT scan report generation, and is able to perform genomics-based disease prediction and mortality risk more accurately than a standard linear polygenic risk score (PRS) based approach. This work couldn't have been possible without close collaboration with our partners, including a panel of specialists who assessed and rated our models as part of this research. These promising results showcase the potential of LLMs to transform medicine as we know it. https://lnkd.in/dHx8_DYh

282

5 Comments

The LLM will see you now: Google and DeepMind released a paper on Monday describing Med-Gemini, a group of advanced AI models targeting healthcare applications. The authors claim that Med-Gemini is outperforming competing models such as OpenAI’s GPT-4. During the pandemic we saw the rise of Telehealth which has largely come back to earth but could doctors be consulting more often with AI? (We know they already Google stuff) #Healthcare #AI

12

9 Comments

LLMs are not always cool in Apollo.io A recent trend at Apollo is the frequent integration of LLMs into production, with new instances appearing almost every two weeks. Interestingly, these implementations are not about generative AI or agent-based systems. Instead, LLMs are being utilized to enhance small aspects of the user experience. They help make empty or default states more reasonable, clean and normalize data more effectively, and bring greater nuance to user input validation. What's more interesting is that these improvements are often driven by non-ML engineers who find innovative ways to leverage LLMs to refine the product one interaction at a time. So it is not always about conversational interfaces or automating complex tasks. This micro-trend is about handling unstructured data more efficiently and subtly improving user interactions. And SDEs should and will add LLMs into their toolkit to build better features.

69

3 Comments

Over the course of each month, I spend a few moments each day or so collecting relevant links, and then, at the end of the month, I bundle them (with the help of some nifty automation scripts) into an easily digestible reading list. Here's everything that I found interesting in June 2024 about the irresponsible use of machine intelligence: #qfm #quantumfaxmachine #blog #irresponsible #ai #machine #intelligence #genai https://lnkd.in/efrC7bEU

5

I am excited to share the latest blog, co-authored by myself, Junhwi Kim and Vyas Sekar, detailing how we managed to successfully localized and improved LLM by combining Fine-Tuning and RAG. https://lnkd.in/gKDkgGPq

69

3 Comments

Great and detailed explanation of the forward and backward passes for the batch normalization coefficients.

4

AI “is not just another invention. AI itself is an infinite inventor.” Mustafa Suleyman, Microsoft’s new AI CEO, shares insight in this TED Talk on what AI is, what it may become, and the sweeping implications for all of us. I was particularly intrigued by his exploration of AI as a new “digital species” as a valuable but imperfect metaphor. What do you think? https://lnkd.in/g2rDiJ6m

35

Over the past few weeks I’ve been researching, and building a framework that combines the power of Large Language Models for text parsing and transformation with the precision of structured data queries over Knowledge Graphs for explainable data retrieval. In this third article of the series I will show you how to combine structured and unstructured semantic queries, and use LLMs to orchestrate question answering over a knowledge graph. https://lnkd.in/e2xvGkWd #LLM #Chat #OpenAI #KnowledgeGraphs #Neo4J #Concerto

485

21 Comments

There is so much research going into bringing the power of LLMs into the Enterprise context. The main techniques are k-shot learning or prompt engineering, RAG (Retrieval Augmented Generation) & Model Fine tuning (not to mention many variations of these e.g. Agentic RAG, GraphRAG etc.) However, I came across an interesting take on this from IBM and they are calling it "InstructLab" - where the claim is that they are able to teach the LLM "new skills" grounded in an Enterprise context. On the surface it looks like a k-shot learning + fine turning on steroids. Approach seems to be to utilize a small "skills training dataset" in order to generate a large synthetic dataset (perhaps utilizing a specialized LLM) that is then incorporated into the LLM weights natively. This could be a very interesting concept as long as the cost of adding new skills is not very high (when compared to fine tuning). Exciting times! #genAI #enterpriseAI

40

2 Comments

This approach could turn the Generative AI industry upside down. Most everyone knows that AI requires computers to do a lot of calculations and especially for Large Language Models (LLMs) like ChatGPT. The majority of these calculations are actually “Matrix Multiplications”. (You remember, a 3x3 matrix times another 3x3 matrix). If the researchers in the article are proven correct, they have demonstrated a way that Matrix Multiplication can be replaced with a more efficient alternative inside the LLM getting nearly the same results. The article positions this savings as a win for the environment (yeah environment), but I think they are missing the boat. More efficient means less electricity and potentially a smaller need for GPU resources, the bread and butter of stock darling Nvidia. When you combine this cost saving with reduced privacy concerns of hosted LLMs (Open AI) and innovative LLM licensing schemes (LibreChat - https://lnkd.in/gKQjMfz5) you can see the cracks in the current direction of the LLM industry. Q: If you could run your own LLM on your own private environment with MIT open-source licensing and without expensive GPUs, why exactly do you need Nvidia, Open AI and alike? Bold prediction: if the researchers get this working at scale, we could see a rapid shift and commoditization in the LLM and generative AI space. All the pieces are in place. TLDR details from the article, describing the innovation: “The researchers' approach involves two main innovations: first, they created a custom LLM and constrained it to use only ternary values (-1, 0, 1) instead of traditional floating-point numbers, which allows for simpler computations. Second, the researchers redesigned the computationally expensive self-attention mechanism in traditional language models with a simpler, more efficient unit (that they called a MatMul-free Linear Gated Recurrent Unit—or MLGRU) that processes words sequentially using basic arithmetic operations instead of matrix multiplications.” #AI #GenerativeAI #Innovation

22

KV-cache quantization is super useful when you have a large batch-size and/or longer context-windows. This requires on-the-fly quantization. Happy to see HQQ used in this context: https://lnkd.in/eyxyPnhP

70

Summarizing Clinical Practice Guidelines (CPGs) using Google's Gemini 1.5 Pro. If you've been working with large language models (LLMs), especially if you've tinkered with retrieval-augmented generation (RAG), you've likely encountered limitations with context windows. That's what I ran into when built PubmedAI using OpenAI's GPT-3 Turbo models when I was passing hundreds of pubmed abstracts to the model to summarize. Ultimately, I had to constrain the number of articles significantly to make it work. With the new 1 million tokens context window in Google's #Gemini 1.5 Pro, it made me think about applications that can be built on top of CPGs. Much easier using Google's web-based AI Studio to do it since you can directly upload a PDF, but I spent around an hour tinkering with their API and passed in the National Comprehensive Cancer Network® (NCCN®)'s 2024 Breast Cancer Guidelines to see how it performs. Results? 1. Basic summarization was great. Especially for quickly manipulating outputs for new CPGs. See screenshot below. 2. Answering specific questions was terrible. E.g. What systemic adjuvant treatment should I consider for HER2 positive breast cancer? Conclusion: Pretty cool to see the 1 million context window, but also the ability to pass in multiple PDFs and entire videos, for the LLM to process. Disappointed in how it performed on specific questions, but I think it's probably a bit on my end too since I did the PDF extraction to text, losing a lot of the figures from NCCN that contains some of the critical aspects. Nevertheless, curious to explore more. #healthcareonlinkedin #llm #rag #retrievalaugmentedgeneration #openai #cpgs #clinicalpracticeguidelines

42

7 Comments

Amazon Bedrock now supports compressed embeddings from Cohere Embed! 💡 Cohere Embed, a top text embedding model, is widely known for enhancing RAG & semantic search systems. The int8 and binary compressed embeddings now available empower developers and businesses to create more efficient #generativeAI applications without sacrificing performance. Explore more at: https://go.aws/4eBniXm #AWS ☁️

2

I loved reading this expertly told article; kudos to Yasemin Saplakoglu! It is the story of what might, in hindsight, be remembered as the most impactful innovation driven by machine learning of our time -- and yes, I have heard of that ChatGPT thing ;) (It is an interesting and funny aside that the Alphafold 2 paper talks about #machinelearning, but has zero mentions of #AI.. 😅)

15

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments