Jae Lee

CoreWeave and Lambda are growing 1,000% year-over-year reselling NVIDIA's compute to LLMs, AI startups, and anyone building AI features into their product. Why is NVIDIA sweet on CoreWeave and Lambda? - NVIDIA is arming these GPU cloud rebels with priority access to advanced chips to conduct a proxy war through them against cloud giants Amazon Web Services (AWS), Google Cloud and Microsoft Azure, which are all developing their own custom silicon. - AWS and other clouds have become GPU-constrained as a result. For example, even though Azure competes with CoreWeave on cloud GPU, it has signed a $2B+ contract with CoreWeave out of necessity to ensure that it has enough compute to meet its growing demand from OpenAI. - Around the wedge of the GPU, the GPU clouds (CoreWeave, Lambda and Together AI) are now building production-grade, AWS-like tooling for autoscaling, networking, and fine-tuning models to enable them to win on developer experience long after the GPU shortage is over. Check the comments for links to our full memos on CoreWeave, Lambda Labs, and Together AI.

27

6 Comments

The best things come in small packages 👝 🚨 Pay attention to the rise of Small Language Models (SLMs)! They will become important going forward: 1️⃣ Efficiency: SLMs achieve comparable or superior performance to LLMs with fewer parameters. Microsoft's new Phi-3 series outperforms larger models on benchmarks like MMLU & MT-bench (Google: https://lnkd.in/gn5zYZHg Microsoft: https://lnkd.in/ghK8fm7V Meta: https://lnkd.in/gkbM7pGs) 2️⃣ Specialization: Trained on curated, high-quality datasets. With 3.8B-14B parameters & advanced datasets, these transformer decoder models match GPT-3.5's capabilities in a compact form. Accurate benchmarking/eval will be important. 3️⃣ Interpretability: Neurons in SLMs are easier to analyze, improving model transparency. 4️⃣ Cost-effective: Less resource-intensive (vs training large models) I have a chance to meet a few exciting startups here. Reach out if you are working on this space 🙋🏻‍♂️ #startups #llm #slm #smalllanguagemodels #phi3 #largelanguagemodels #gpt4 #google #gemma #llama3 #ai #ml #edgeai

24

4 Comments

Please media types writing about the cost of AI: recognize there is a difference between huge GPU/TPU clusters needed to train foundation models vs. inference costs to run most apps on top of foundation models. The two are not synonymous, so you can dial back some of the doom and gloom about AI startups and your commingling of the two different AI business types.

11

In today's episode, the renowned RLHF thought-leader Dr. Nathan Lambert digs into the origins of RLHF, its role today in fine-tuning LLMs, emerging alternatives to RLHF... and how GenAI may democratize (human) education! Nathan:  • Is a Research Scientist at the Allen Institute for AI (AI2) in Seattle, where he’s focused on fine-tuning Large Language Models (LLMs) based on human preferences as well as advocating for open-source AI. • He’s renowned for his technical newsletter on AI called "Interconnects".  • Previously helped build an RLHF (reinforcement learning from human feedback) research team at Hugging Face. • Holds a PhD from University of California, Berkeley in which he focused on reinforcement learning and robotics, and during which he worked at both Meta AI and Google DeepMind. Today’s episode will probably appeal most to hands-on practitioners like data scientists and machine learning engineers, but anyone who’d like to hear from a talented communicator who works at the cutting edge of AI research may learn a lot by tuning in. In today’s episode, Nathan details:  • What RLHF is and how its roots can be traced back to ancient philosophy and modern economics.  • Why RLHF is the most popular technique for fine-tuning LLMs. • Powerful alternatives to RLHF such as RLAIF (reinforcement learning from A.I. feedback) and direct distilled preference optimization (dDPO). • Limitations of RLHF. • Why he considers AI to often be more alchemy than science. Many thanks to Amazon Web Services (AWS) and Crawlbase for supporting this episode of the "Super Data Science Podcast with Jon Krohn", enabling the show to be freely available on all major podcasting platforms as well as the video version we publish on YouTube. This is Episode #791! #superdatascience #machinelearning #ai #llms #rlhf #reinforcementlearning

74

10 Comments

Jensen: "Most code will become API calls to these powerful LLMs." It really does feel like there are just 2 strategies for building in AI 1) better LM ==> better product 2) better LM ==> product less useful?? At Crosshatch we're all in on (1). We're betting that AI gets faster, cheaper, and more capable. With AI changing so fast, Jeff Bezos' now-famous quote feels important "'What's not going to change in the next 10 years?' [This question] is actually the more important [one] because you can build a business strategy around the things that are stable in time." So with everything changing so fast right now, what's not going to change? - People want to feel known and understood. - We want to save time and money. - We want to feel in control. Excited for AI to help deliver a true hyper-personalized cyberspace.

8

#GenerativeAI becomes more and more accessible with Open source LLMs now available thanks to Meta & Mistral AI and many more. As a result we see more opportunities for developers to build their own agents. Yet, security & privacy still remain a bottleneck for companies that want to deploy those AI tools. If you are thinking leveraging public/private generative AI in your organisation and have concerns when it comes to privacy, prompt engineering or data protection .. Book a time with me. Either virtually or let's grab a coffee if you are in Sydney. Coffee is on us. https://lnkd.in/ge2vNN4a Aona AI Bastien Cabirou

10

Solitary LLMs are impressive, but several can work together as a Multi-Agent System to astounding effect and overcome solo LLM (GPT-4o, Claude 3, Gemini) limitations. Hear all about it in today's "Five-Minute Friday" episode. In a bit more detail: • In recent weeks, OpenAI's GPT-4o and Google's Project Astra showcased A.I. agents that engage in humanlike conversations and analyze real-time video. • Despite such impressive capabilities, single A.I. agents still struggle with many categories of complex tasks. • Thoughtfully linking individual LLMs together into a Multi-Agent System (MAS) enables them to work together, assign tasks, build upon each other's work, and deliberate to find solutions, in aggregate exceeding the capabilities of a single LLM on its own... often without any decrease in compute efficiency. • As an example, a DARPA experiment demonstrated 3 agents (named Alpha, Bravo, Charlie) collaborating to defuse (virtual!) bombs efficiently. • In an Massachusetts Institute of Technology study, two chatbots solving math problems outperformed a single agent through dialogue and updating each other's answers. • Microsoft developed a software-writing MAS with specialized roles (Commander, Writer and Safeguard; see the image for this post) wrote code 3x faster without sacrificing accuracy. • As always, there are potential risks, e.g.: illogical solutions cascading through the team, agents getting stuck in loops, and malicious exploitation. • Commercial interest in MAS is accelerating, with frameworks like AutoGen and Camel making it straightforward for you to get started with MAS today yourself. What in your industry could be automated or improved by MAS where single-LLM approaches aren’t sufficiently effective? 🤔 For more detail on all of the above, check out today's episode (#788)! The Super Data Science Podcast is available on all major podcasting platforms and a video version is on YouTube. #superdatascience #machinelearning #ai #llms #multiagentsystems

51

4 Comments

This is a good article on using LLMs in the real world. It covers many aspects that we learned at Nurish over the last few months and more. Using LLMs is an inexact science, but has its own needs in processes and tools. While most of the attention has been on the LLMS, their capabilities, and RAG, there are mundane issues that need to be taken care of, issues where the solutions for pre-LLM development are not that well-suited. These include version management, integration, testing, data management, quality assessment, and more. This article covers many of those practical aspects. Eugene Yan, Bryan Bischof, Charles Frye & more https://applied-llms.org/

9

Don't fight against AI. Help it to get latest info. 👇 is the talk I gave last week regarding Real-Time #RAG in Zilliz's Unstructured Data and #LLM meetup. "Talk is cheap, show me the code" This blog shows the key data flow and code blocks. The full example is available at https://lnkd.in/gUj349hh How it works? Input: Stream stories and comments from #HackerNews API. Pre-process: Retrieve updates and filter for stories/comments via #Bytewax. Retrieve Content: Download the html and parse it into usable text. Thanks to the awesome #Unstructured.io. Vectorize: Create an embedding or list of embeddings for text using #HuggingFace #Transformers. Output: Write the JSON document with #embeddings and other fields to a local #Timeplus #Proton server, into the data streams. Filter/Route: Use streaming SQL to filter/transform JSON data and send output to a local #Kafka server. Forward: Use #Milvus plugin for #KafkaConnect to forward the data to #Zilliz Cloud. Query: Build a #streamlit web application to query the live data with the same embedding model.

18

1 Comment

Prompting for scale will soon be obsolete... i think...i hope. Prompting is an error-prone, discovery-based process, unsuitable for scalable AI. Many folks rely on black-box LLMs behind API walls (e.g., Claude, GPT-4), lacking direct control over LLM model weights. Programmatically custom-pruned LLMs will be the future of controllable, production AI, not prompting. hint--> #steeringvectors

16

6 Comments

NVIDIA's Jensen said "ChatGPT democratized computing, Llama2 democratized generative AI" ...and now Llama3 is the next level 🔥 It's fascinating to watch Meta's strategic moves. With Llama3, they prioritized training on a massive 15T token dataset to pack all into a lean 70B param model, instead of building a massive model. This allows Llama 3 to match trillion+ parameter models like GPT-4, but at 1/10th the compute, storage and inference costs! 💰 This technique was published by Google DeepMind a few years ago https://lnkd.in/gx7VU8aA Meta is not a dark horse in AI anymore. They might be the top dog. https://lnkd.in/gRRJYuxt #llama2 #llama3 #llm #chatgpt #gpt4 #ai #ml

16

5 Comments

Aidan Gomez, CEO and co-founder of Cohere, has been instrumental in developing generative AI. His work at Google in 2017 on transformers laid the groundwork for today's AI advancements. In a recent interview, Gomez talked about Cohere's mission to create practical, easy-to-integrate AI models that drive profitability by automating tasks and enhancing efficiency. He stressed the scalability and accessibility of Cohere's AI solutions, aiming to democratize AI for all businesses. #AI #GenerativeAI #Cohere #TechInnovation #BusinessGrowth #Automation #AIIntegration #Scalability #AidanGomez

3

Meta's LLaMa 3.1 405B required an astounding 30.84M GPU hours to train. To put this in perspective: - On a single GPU, this would take ~3,520 years! 😮 - With 16,000+ H100 GPUs, Meta completed it in just ~80 days. 🌟 This showcases the incredible power of distributed computing in AI development. By leveraging parallel processing at scale, Meta compressed millennia of computation into a fraction of a year. Such advancements in training infrastructure are pushing the boundaries of what's possible in AI, enabling the creation of increasingly sophisticated models in remarkably short timeframes. By the way, though open-sourced/free, this new model outperforms the new ChatGPT-4o on a considerable number of benchmarks. Read more in the article! #AI #MachineLearning #DistributedComputing #LLaMa #MetaAI"

5

Zoom has been using the power of LLMs to enable novel platform features like AI Companion. Excited to see that #DBRX will enable enterprises like Zoom to cost-effectively build, train, and serve their own custom LLMs! Learn more about DBRX here👇

25

Narratize is delighted to announce that we have been accepted into NVIDIA's Inception program, an accelerator designed to help startups evolve faster through cutting-edge technology, opportunities to connect with venture capitalists, and access to the latest technical resources from NVIDIA. Since the start of 2024, the team has: ✅ Recruited a seasoned and diverse engineering, data science, and narrative science product team ✅ Focused on building DEI best practices into our data architecture, ensuring our uses are always safe, equitable, secure, and human-centric ✅ Launched the world’s first generative AI white paper builder ✅ Welcomed new enterprise customers across science, tech, and medical industries ✅ Traveled the world sharing insight and inspiration on GenAI transformation This spring, Narratize is bringing 20+ new use cases to Narratize’s Core Product Library as well as new features designed to streamline communications and collaboration among product, R&D, and marketing teams.   ‍ Narratize combines the power of narrative science with cutting-edge generative AI technology to ensure that no great idea is left untold. From optimizing research & development processes to scaling product launch, Narratize empowers brands with transformative ideas to unleash their full potential. Narratize brings together computer science, data science, and narrative science expertise to develop AI in a uniquely human, story-led way. You’re invited to experience the magic yourself, Try Narratize’s Core Product Library free for 14 days 👉 https://lnkd.in/e8HEfwvR #narratize #genai #innovation #nvidiainception #techstartup

5

LLM powered AI agents have been a hot topic this year. Inspired by John Yang's SWE-agent paper which framed agents as their own class of end users and showed a path to improving agent performance through agent-computer interfaces (ACI), I wanted to explore the parallels between AI agents and consumers. In my latest article, I lay out my vision of Web 4.0--a web characterized by AI operability and agentic markets, and how ACIs will enable these. Through examining some recent LLM research papers, I outline: - Why some AI agents are consumers - How AI agents being consumers is currently a problem - Why allowing AI agents to spend will lead to new economies and define Web 4.0 - Why agent-computer interfaces (ACI) is needed for Web 4.0 to come to life Check out the piece below and let me know your thoughts! Huge thanks to my friends Kevin Zhang, Hanyi . , Robin Huang, 🐙 Marc Mengler, Sheng Ho, Andrew Tan, Phillip An , Nadja Carina Reischel, and Luc de Leyritz for reviewing this piece and valuable suggestions! #AI #Agents #LLM #ACI #Web4

43

3 Comments

🌟 Had the pleasure of speaking with Ofer Medelevitch from Vectara on the Work AI podcast! 🗣️ We explored the intricate details of building effective RAG pipelines and their impact on reducing AI hallucinations. Whether you're in healthcare, law, or finance, this conversation provides invaluable insights into how Vectara is leading the charge in AI innovation. 📈 Tune in now to discover more! #AI #TechInnovation #RAGAsAService

1

The most robust, reliable, and productive uses of AI are when many models are used in coordination with each other, with "regular" programming mixed in. This leads to more capable, more reliable, and more interpretable AI systems. Most people building with AI already know this. But (unless you work for Google) the main barrier to realizing multi-step AI workloads is an infrastructure one. For the rest of us, we’re left creating an unwieldy mess of chained API calls to multiple providers, with many roundtrips in between. This state of affairs is stifling progress. Nobody wants to deal with more tooling and infrastructure… but everyone would benefit from simple, intuitive interfaces that abstract away a powerful system underneath. Substrate is the first inference API optimized for multi-step AI workloads. With Substrate, you can write less code, run more inference, and build high performance AI applications with zero infrastructure to manage. No tooling, no infrastructure – just elegant abstractions. Excited to support Ben Guo, Rob Cheung and the amazing Substrate team on their mission to democratize how AI applications are built.

40

4 Comments

🚀 The tech leaders panel at Headstarter AI was truly beneficial! I want to share some key takeaways from speakers: Dustin Beadle, Logan Havern, Nabeel Alamgir🦄, Shariar Kabir, and Miguel Acero. Dustin Beadle (Director at Lattice, Ex-Director AMD): - Communication is Key: Effective communication and active listening are essential. It’s important to value and understand others’ goals. - Attitude Over Hard Skills: While technical skills can be taught, a positive attitude and the ability to collaborate are what set you apart. - Global Perspective: Dustin’s international experience highlights the importance of collaboration and communication in diverse teams. Logan Havern (CEO of DataLogz, Raised $12M): - Master the Art of Pitching:** Pitching yourself and your ideas effectively is crucial. Logan’s perseverance, living off savings while pitching his startup, eventually led to raising over $12M. Nabeel Alamgir🦄 (CEO of LunchBox, Raised $100M VC): - Be Authentic and Resilient: Authenticity is your strongest asset. Success often involves setbacks, but how you handle failures and bounce back is what makes the difference. Shariar Kabir & Miguel Acero (Ruby Card - YC W23): - Fail Gracefully: Embrace failures as learning opportunities. Shariar and Miguel shared how they tried various ventures in college before finding success with their startup Ruby. Their journey included countless phone calls and emails before securing funding. 🌟 Although all the speakers share great advice, my favorite speakers are Shariar Kabir and Miguel Acero. Their stories have been incredibly motivating for me, especially as I have my own challenges and failures while trying to make progress. Thank you to Headstarter AI, and a special shoutout to Yasin Ehsan 🚀 and Faizan Ahmed for organizing this incredible panel! #Leadership #Communication #Pitching #Perseverance #Authenticity #FailForward #CareerGrowth #StartupSuccess #HeadstarterAI

13