Hari Gangadharan

A few thoughts on #LLM usage in the small, prompted by my toying with #llama3 and #phi-3 on consumer hardware.

5

Artificial intelligence (AI) is rapidly evolving, and with it comes a lot of hype – and a healthy dose of fear. Watching some really thought-provoking videos https://lnkd.in/gAj88rEt , https://lnkd.in/gQvMhFVZ from renowned AI leaders like Kaifu Li and Yann Lecun , several ideas worth pondering: 1. AI's Potential is beyond human intelligence: Just think about the superior intelligence animals have, cats, dogs, and dolphins, and the potential for machines to imitate and incorporate all these intelligent abilities from them!!! 2. Risks Exist, But We Can Mitigate Them: Openness and accessibility in AI research are key to mitigating risks. Open Source is essential! 3. Humans Still Rule: Both videos remind us that even with advanced AI, human qualities like earning trust, empathy, compassion, and critical thinking will be irreplaceable. The future of AI is likely collaborative, with humans and machines working together, but the balance of power might shift as AI capabilities evolve. 4. Not everyone or everything in this universe wants to take over the world, certainly not machines. Here is my favorite line from Yann: “Intelligence has nothing to do with a desire to dominate. It’s not even true for humans,” he said. “If it were true that the smartest humans wanted to dominate others, then Albert Einstein and other scientists would have been both rich and powerful, and they were neither.” #AI #AIfears #futureofwork #responsibleAI

49

2 Comments

Google's $1 billion investment in Pacific connectivity is a major milestone in bridging the digital divide and fostering a more connected world. This initiative not only strengthens infrastructure but also holds the promise of transforming lives and economies across the Pacific region. As we move towards an increasingly digital future, investments like these are crucial in ensuring that everyone can benefit from the opportunities that connectivity brings. “In partnership with Governor Green’s administration, Hawaii's Connect Kakou initiative is working to ensure people from all walks of life have reliable and affordable access to high-speed Internet. The Pacific Connect initiative perfectly complements our planned efforts, and will significantly enhance our future connections from Hawaii to the continental U.S. and Japan as well as ensure communities across the Pacific have equitable and reliable access to digital services." - Sylvia Luke, Lieutenant Governor, State of Hawaii https://lnkd.in/gp849ihX https://lnkd.in/gcUwbJs7 Amanda Stange Jen Stanley Reymund Dumlao Jamar Thomas Sandeep Togrikar Steve Lujan Christopher Hein

38

🚀 Reflecting on Kafka Summit: Transforming Data Streams into Business Value Kafka Summit Bangalore 2024 was not only enlightening but also provided a delightful reunion with Jay Kreps, co-founder and CEO at Confluent, and my former colleague at NexTag. It's inspiring to see how Confluent is shaping the future of Streaming with Apache Kafka & Apache Flink Unified for the Data Streaming Era (Two(2) big communities coming together) Here are some pivotal learnings from the summit that resonate deeply with our objectives at MakeMyTrip and the broader trends within the tech industry: 1.  𝐒𝐢𝐦𝐩𝐥𝐢𝐟𝐢𝐜𝐚𝐭𝐢𝐨𝐧 𝐚𝐧𝐝 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐃𝐚𝐭𝐚 𝐎𝐩𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐬: Confluent is focused on making it easier for users to process data streams more quickly and efficiently than before, emphasizing the importance of moving beyond mere data movement to achieve transformative business impacts while Addressing Data Infrastructure Complexity. 2. 𝐔𝐧𝐢𝐯𝐞𝐫𝐬𝐚𝐥 𝐃𝐚𝐭𝐚 𝐏𝐫𝐨𝐝𝐮𝐜𝐭𝐬: Confluent aims to transition from a model where data consumers pull data from systems, to a model where producers present well-formed data that multiple downstream subscribers can use. This approach is embodied in their concept of "data products," which are well-curated, reusable data sets with a clear ownership and governance structure. 3. 𝐒𝐭𝐫𝐞𝐚𝐦𝐢𝐧𝐠 𝐏𝐥𝐚𝐭𝐟𝐨𝐫𝐦 𝐄𝐧𝐡𝐚𝐧𝐜𝐞𝐦𝐞𝐧𝐭𝐬: The new features in Confluent’s streaming platform are designed to federate data effectively across operational and analytical domains. The platform’s capabilities are extended to ensure data is not only real-time but also reusable, governed, and securely connected across all systems. 4. 𝐈𝐧𝐭𝐞𝐠𝐫𝐚𝐭𝐢𝐨𝐧 𝐨𝐟 𝐎𝐩𝐞𝐫𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐚𝐧𝐝 𝐀𝐧𝐚𝐥𝐲𝐭𝐢𝐜𝐚𝐥 𝐃𝐚𝐭𝐚 𝐄𝐬𝐭𝐚𝐭𝐞𝐬: Confluent team introduced a significant development called "TableFlow," which seamlessly integrates streaming data with analytical systems. This integration aims to make real-time data available in data lakes or warehouses instantly, supporting both operational responsiveness and analytical depth. Confluent added support for Iceberg to Confluent Cloud, allowing customers to convert Kafka topics, schemas, and metadata into Iceberg tables seamlessly. 5. 𝐅𝐮𝐭𝐮𝐫𝐞 𝐕𝐢𝐬𝐢𝐨𝐧 𝐟𝐨𝐫 𝐃𝐚𝐭𝐚 𝐒𝐭𝐫𝐞𝐚𝐦𝐢𝐧𝐠: The overarching theme was the strategic shift towards a holistic view of data management, where streaming is not just a method of transporting data but a fundamental aspect of how data is structured, processed, and consumed across businesses. While they might not be explicitly calling it Kappa architecture, their focus on data streams suggests they're on a similar path. 📸 Here’s a snapshot of Jay Kreps, me, Aditya & Confluent team at the event. Looking forward to many more such reunions and fruitful collaborations! #KafkaSummit #DataStreaming #Personalization #Confluent #MakeMyTrip

189

1 Comment

Concerned that your next presentation will be adorned with your AI's "artistic interpretation" (read hallucinations) of the data. Tired of AI assistants autocorrecting "world domination" to "world vacation"? Then join me and Abhishek Jawali V at AWS Summit Bengaluru on May 15th, 2024 at 4:00 PM for a talk on "Advancing Responsible AI in the Generative Era", where we will delve into the emerging risks of AI like hallucinations, toxicity etc. and to ensure it doesn't become the villain in our next sci-fi nightmare (unless, of course, it involves robot chefs who whip up delicious meals). P.S.: And I assure you that no AIs were harmed (or used to write) in the making of this post :) Register here: https://lnkd.in/d-MMK4KV #responsibleAI, #AWS, #AWSSummit, #generativeAI, #GenAI, #AmazonBedrock, #Guardrails, #awscloud, #AI, #CloudComputing, #MachineLearning

201

3 Comments

Software monitoring is a breeze when you have customers who move fast, and want control levers to tame High Cardinality data. 😍 Prathamesh Sonpatki writes on winning Replit and showcasing the power of a managed time series data warehouse. (Link in the comments) #Devops #SRE #observability #Alerting #Monitoring #o11y

9

1 Comment

"The good news is that we are at #AnsibleFest" -- asserted Sathish Balakrishnan, VP & GM of the Ansible Business Unit at Red Hat during his keynote today. Full House at the keynote where Balakrishnan explained how #Automation has come a long way and the critical role that #Ansible has played in that journey -- making automation more consumable with ease of use. "Automation is no longer a nice-to-have. Automation is mission critical" continued Balakrishnan. Looking ahead to the future, building upon what we heard from Matt Hicks at the Red Hat Summit keynote, Balakrishnan asserted that "#AI is a natural progression of automation!" How is your AI strategy coming along? But, wait, where are you with your Automation Strategy? These are the questions that we can address at this event and beyond. Kaete Piccirilli :: Moez Dharani :: Erich Morisse https://lnkd.in/gdwYQbGs

27

*New episode of Level Up Prodcast with Malthi SS" In this episode, I chat with Sonali Nanda, who is Senior Vice President at Exxat. Prior to Exaat, Sonali was product leader with High Radius and managed a fairly large team with #Product, #Design, #Analytics and Ops. We spoke about: 💠 Managing different things as a Product Leader - people, product, strategy. 💠How to Foster Innovation and Creativity 💠Balancing Speedy Execution and Strategic Thinking 💠Advice for Women in Product Management 💠Managing Diverse Teams and Priorities Check the episode for tons of insights #ProductLeadership #WomenInLeadership #TechInnovation #CareerDevelopment #ProductManagement #EmpoweringWomen #LeadershipDevelopment #ProfessionalGrowth

23

2 Comments

vAttention: Dynamic Memory Management for Serving LLMs without PagedAttention Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, Ashish Panwar Abstract Efficient use of GPU memory is essential for high throughput LLM inference. Prior systems reserved memory for the KV-cache ahead-of-time, resulting in wasted capacity due to internal fragmentation. Inspired by OS-based virtual memory systems, vLLM proposed PagedAttention to enable dynamic memory allocation for KV-cache. This approach eliminates fragmentation, enabling high-throughput LLM serving with larger batch sizes. However, to be able to allocate physical memory dynamically, PagedAttention changes the layout of KV-cache from contiguous virtual memory to non-contiguous virtual memory. This change requires attention kernels to be rewritten to support paging, and serving framework to implement a memory manager. Thus, the PagedAttention model leads to software complexity, portability issues, redundancy and inefficiency. In this paper, we propose vAttention for dynamic KV-cache memory management. In contrast to PagedAttention, vAttention retains KV-cache in contiguous virtual memory and leverages low-level system support for demand paging, that already exists, to enable on-demand physical memory allocation. Thus, vAttention unburdens the attention kernel developer from having to explicitly support paging and avoids re-implementation of memory management in the serving framework. We show that vAttention enables seamless dynamic memory management for unchanged implementations of various attention kernels. vAttention also generates tokens up to 1.97x faster than vLLM, while processing input prompts up to 3.92x and 1.45x faster than the PagedAttention variants of FlashAttention and FlashInfer. 👉 https://lnkd.in/dDJKrsea #machinelearning

1

Watch LIVE as Oracle's Edward Screven and Nipun Agarwal detail how Oracle HeatWave enables you to take advantage of #GenAI. Register now: https://lnkd.in/gXacma5U

1

#gemini-ai #generative-ai Ganesh Babu V Kalanidhi Gurunathan function calling feature of gemini is the one of the interesting space to watch out to get your llm with upto date information 1. https://lnkd.in/gmpYfan4 2. Hands on https://lnkd.in/gdsCTP3a

12

1 Comment