Mithil Shah’s Post

Had a very insightful panel discussion on Generative AI with Chris Howard Cheyne W. and Sumith Jitta yesterday at the AWS AI day for Software companies. Some of the key learning were: 💰 Its important to find a use case which has a direct business value (think ROI, productivity gains) 🚄 Once we find the use case, its important to get to production fast. Companies need to establish a path to production and moving fast with an achievable use case is key. ⚙ GenAI can improve productivity in not just writing code, but the entire Software Development Lifecycle. Saeed Gharadaghi Luke Anderson Simon Peter Johnston Rajeev Muralidhar Michelle Ng

join us for an in-person workshop on Generative AI in Auckland on 2nd July!

Anthropic released Claude 3.5 Sonnet. Amongst other things, it introduced a capability called Artifacts 👨💻. When a user generates code, an artifacts window appears on the right and this shows the artifact as file 📁 in a dedicated window. This would eventually allow editing the AI generated content in real time. Here's how Anthropic envisions the future 💪 of development: "teams—and eventually entire organizations—will be able to securely centralize their knowledge, documents, and ongoing work in one shared space, with Claude serving as an on-demand teammate" In this video 🎥 I tell Sonnet to "Create cloudformation template to build the following 1) public subnet with internet gateway 2) private subnet 3) RDS postgres in private subnet 4) Nat gateway" Try out Claude 3.5 on Amazon Bedrock today! https://lnkd.in/gJfZBprN Ben Friebe Rajeev Muralidhar Mark Roy Timothy Downs Sanjay Tiwary

one of the challenges with RAG is evaluating it. check out this code from Amazon Science if you are building a RAG based system Romina Sharifpour, Ph.D. Baichuan Sun

Running an LLM on a smartphone! We have all heard about Apple intelligence, where apple devices will now have an LLM. However, very soon app developers will also be able to embed an LLM into their own app. This opens up excellent possibilities. PowerInfer(2) is one such library that makes it possible to run models as large as TurboSparse-Mixtral-47B on a phone. 47B seems too large to fit on a phone, so how does that work? 💡 PowerInfer2 breaks down the traditional matrix computation into smaller neuron cluster computations. 💠 💡 Each inference iteration requires only a subset of neurons, rather than the entirety of the model weights. 💠 💡 It builds an in-memory cache that stores most frequent neurons in memory. 💠 💡 It utilizes heterogenous cores such as heterogeneous XPUs present in contemporary smartphones, 1such as asymmetric big.LITTLE CPU cores, GPU, and NPU present in the phones. 💠 💡 The hot-activated neurons are kept in the GPU and cold neurons in the CPU. 💠 For more details, read this paper https://lnkd.in/gimuk6hH Here's the GitHub repo - https://lnkd.in/gpyj8FkK #largelanguagemodels #artificialintelligence #generativeai

⭐ Social dilemma for AI agents ⭐ In this article I summarize how 🖥 AI Agents deal with social dilemma in a multi agent system. Think of scenarios like 1/ driverless cars 2/ multiple drones working together 3/characters in a game 4/ LLM agents. Specifically the article talks about the intrinsic motivation and external motivation for AI agents. Intrinsic motivation includes concepts like Inequality aversion, Social value orientation, Altruism, Social Influence and Reputation External motivation includes peer rewards and agreements. The article also talks about how human - human interaction changes in presence of AI agents. #artificialintelligence #llm

📈 Popular or trending open source tools for Generative AI. 📈 Sometimes the best way to keep abreast of the latest trends in Generative AI is to look at the open source libraries. Here are six repositories on GitHub that thousands of users have found useful. 1️⃣ Ollama | github.com/ollama/ollama | 67K ⭐ Get up and running with large language models locally. 2️⃣ gpt4all | https://lnkd.in/gZsa6-Pt | 65K ⭐ Run customized LLMs that work locally on consumer grade CPUs and NVIDIA and AMD GPUs 3️⃣ AutoGPT | https://lnkd.in/g4eXHkF8 | 162K ⭐ A semi-autonomous agent powered by LLMs to execute (almost) any task for you. 4️⃣ PrivateGPT | https://lnkd.in/gcPdqP6G | 52K ⭐ RAG, even without internet 5️⃣ Llama.cpp | https://lnkd.in/gH3RFAVq | 58K ⭐ Inference of Meta's LLaMA model (and others) in pure C/C++ 6️⃣ MemGPT | github.com/cpacker/MemGPT | 10K ⭐ build and deploy stateful LLM agents with support for Long term memory