Jyothi Nookula

#MAJOR #AI 💡 break-through #SURFACED THIS WEEK 💬 While majority of the folks (and candidates I am having for job interviews) dream about riding the #LLM's horses of #GPT-type, in the background there are new trends really emerging. I have been watching closely work of Sepp Hochreiter and his team in giving #GenAI new hope by "resurrecting" LSTM. 🔭 For those of you, who don't follow the AI tech deeply, Long-Short-Term-Memory (or #LSTM) models have been the pre-cursors of the GPT technology. Most of the translators and other language bots being on market before launch of ChatGPT have been actually some form of LSTM application. When OpenAI broke through with Generative Pretrained Transformer (hence GPT) line of models, they looked like good-bye kiss to LSTM. GPT could work with so many more tokens, achieved better precision in tasks and was also faster in inference. 🪜 However, Sepp and his team recognized also one strong weakness of GPT family of models. Their learning is #quadratic, which makes push for more and more parameters prohibitive expensive. That also means that GPT's already got close to their (reasonable) ceiling of how big single model can be. (This is reason why they have been geared into round-of-experts model platforms, but that will be subject for another post of mine later). 💥Not succumbing to GPT dominance, Sepp Hochreiter worked hard on upgrading the LSTM by adding #exponential #gating, #matrix #memory and parallel training. This week they published paper on #xLSTM brand new generation of LSTMs that actually competes well with GPT-technology and are much cheaper to train. And that is after much of the hype a real technological advancement. Large Language Models are not peek of AI, they are just transition to another phase. And this whitepaper is major step on that pathway. Read full detail here :

5

Thank you to everyone who came for the AI Horizons roundtable! We welcomed an impressive group of GenAI builders from Tesla, Clari, Qlik, Moveworks, Amazon, and founders from Cogniswitch, Repello, and EventHQ My favorite moments from the session were: 🔐 Hearing from Saurabh Shintre about the early days of AI security and how he sees it evolving with GenAI models 🤖 Insights from Baris Aksoy on AI agents can automate enterprise security compliance 🎯 Real-life examples from Aryaman Behera of Repello AI on how his company partners with GenAI model creators to identify vulnerabilities Attendees particularly appreciated: 💡 Small-group format that fostered deep, insightful discussions on AI security 🗣️ Candid conversations peppered with real-life examples 🌐 Opportunity to network with fellow GenAI builders A huge shout-out to Suhas Motwani from The Product Folks and Manthan Shah from WestBridge Capital for their partnership! Interested in contributing or attending our next event? Drop a comment below or send me a DM!

86

6 Comments

I participated in a “SMART MANUFACTURING” panel with Gayathri Radhakrishnan, Rohini Chakravarthy and Abhijit Supekar (moderator) last week at TIECon 2024. The coolest part of the panel was 3 Women VCs talking about a topic that has not traditionally been discussed by women. I never imagined I’d see this in my lifetime. Things have indeed changed! We had a very lively discussion of whether we can expect lights out factory in 5-10 years. Automation is happening in all stages but full automation manufacturing is tough because it requires human like adaptability. It is not converting text into instruction or code - it is about precision. Robotics + AI is making this happen and we will get there. AI is going to bring the cost down for complete solution for example: there are robotic arms that can do more tasks and learn like a human and not just some specific tasks. Also check out our portfolio companies Micropsi Industries and RGo Robotics that are making it happen in this space. Thanks to Anshu Bahadur Hrishikesh Sathawane Nilesh Dixit for organizing the panel and Anita Manwani and Surbhi Kaul for making this happen! #TiECon2024 #AIUbiquity

95

6 Comments

Example of how increased inference speed creates qualitatively new opportunities: Have a conversation with Vapi at https://vapi.ai. Thanks to ultra-fast inference of things like Llama-3 running on Groq, you can have a back-and-forth conversation that feels very human. You can even interrupt Vapi mid-conversation and it will just handle that seamlessly. Pretty awesome.

4

2 Comments

1. IMO, agent patterns are one of the most interesting ways to extend LLMs. memary is based on leveraging a graph database (neo4j) for long-term memory (to get around the limited memory inherent in even the largest context windows in LLMs) and augmenting it with other tools like web search and Computer Vision. https://lnkd.in/gfFcBqJx Looks interesting. 2. More about how consolidation effects the entire healthcare industry: https://lnkd.in/gXJKqNJB The Change hack/ransomware attack not only affected claims, service, refills, etc. but now CMS & NCQA are delaying the entire HEDIS reporting process (used for Medicare/Medicaid quality ratings) due to the impact. 3. NARA bans ChatGPT use agency-wide. https://lnkd.in/gW8T94JU Federal Government agencies see this as a risk, including their information leaking and being used to train models (which may then output the wrong information or show PII/PHI other proprietary information). 4. Oh great, now I can't even be angry anymore :) https://lnkd.in/gcrN9mjx

1

1 Comment

From where I sit in the SF Bay Area, cradle of innovation for at least the last 30 years, I'm a true believer in the promise of the #AI revolution. That said, I find it fascinating to consider views on the opposing side. I'm sharing below two opinion pieces from the last two weeks suggesting that the AI revolution is fundamentally overhyped and a bubble ripe for bursting, and that AI tools are actually not useful or transformative. Contrast this with venture capitalist Kai-Fu Lee's prognostication that 50% of white collar jobs will be displaced by #AI within three years, and that OpenAI and the other cutting edge movers in the space are headed to a trillion+ dollar valuation. Which side will win out? Where will we be by 2030 in the AI race? I'd love to hear your thoughts in the comments.

7

3 Comments

#WIP24 is amazing this year with 3000 product professionals from around the world. I am honored to have shared the panel with Dena Saunders, Deena Minwalla, and Shubha Rao to discuss AI product leadership. This year, AI is a central theme in many sessions at WIP, highlighting the evolving roles of product managers in the age of AI. We discussed how PMs determine AI investments, assess feasibility, and discussed strategies for developing and scaling 0-1 AI products. Among other conference highlights, I enjoyed Marily Nika, Ph.D, Deboshree Duttaand Farah Akoum's session, which featured live interactions with GPT-4. I wholeheartedly agree with Marily's prediction that every product manager will become an AI product manager in the future! And the best part was listening Eunice Kim and Elizabeth Stone at Netflix. It's inspiring to see two influential roles led by women. My key takeaway from their session was the emphasis on the synergy between Product and Engineering. I often times see this common misconception that PMs are solely responsible for the WHAT and WHY, while engineers handle the HOW. In true product cultures, engineers are equally invested in the WHAT and WHY, collaborating closely with product managers. This approach allows teams to identify possibilities and feasibility early on, enabling engineers to understand the user and business context and develop products that users love. Now even more excited for the Day 2!!! #aiproductmanagement

32

2 Comments

I haven’t tried this model fine tune and due to issues with gguf quant fine tune conversion, still working on adjustments. That being said llama3 and mixtral are my go to models. I’m using a context tuned version to 32k. Just not practically to run a 70b parameter local (yet) #llm #llama3 #genai #models

4

1 Comment

We had a full house in a fireside chat with Arvind Jain, Glean's CEO and co-founder, at Stanford University Graduate School of Business! It was an honor to talk with him about building successful companies in the dynamic #AI era. He shared invaluable lessons from his remarkable journey: building Google search infra, founding Rubrik (which went public last month!), and founding Glean, recently nominated by Forbes as one of the Top 50 AI companies! 🎯 Start with the problem, not the solution Arvind is very bullish on Glean's mission to solve app fragmentation and workplace knowledge. He saw his customers at Rubrik struggle with it and took the time to validate it. People tend to undermine the importance of a problem and get too excited about the solution. However, if the solution solves the wrong problem, users will not adopt it. 🤝 Validate hypotheses with people you don't know Friends want to support and might avoid critical feedback. Arvind and the team spent the first year of Glean sending cold LinkedIn reach-outs to potential customers to validate the problem and the solution. While he could easily leverage his network, he wanted unbiased feedback from people who didn't know him. 🎧 When it's noisy outside - stay focused on your customers. That's your moat. With every week's announcements of AI innovations, Glean's Northstar is to be customer-driven and leverage new technologies where there's a fit. Even though they are busy with the growth struggles of a Series D company, Glean teams are working on 70 % of core work : 30% of experimental projects, to stay ahead of the curve. 📈 Companies should plan how to embrace AI internally As a 2nd-time founder, Arvind is looking for new opportunities to use AI at Glean and improve #productivity, beyond using Glean Assistant. With so much innovation, customers are very excited about AI demos but get disappointed when they try to use new tools for their specific use cases. It will take a (short) while before we see a significant impact on productivity.

187

10 Comments

The Economic Times's article on GENAI features my conversation with Himanshi Lohchab on why Cost (especially inference cost) and high quality Conversational Data are the key barriers to make LLMs ready for India Scale. TWO.AI's mission with #ChatSUTRA and #SUTRA models is to bring energy-efficient language models that are truly multilingual. https://lnkd.in/gYshMMNz

66

1 Comment

🚀 Day 15 of My 25-Day Generative AI and LLM Challenge 🤖🔍🚀✨ The Role of LLMs in Transforming Conversational AI ✨🚀 with my friends HARSH JAVIA and Maanan Jagani! 👉 In the ever-evolving landscape of technology, Large Language Models (LLMs) are revolutionizing the way we interact with machines. 🧠🤖 🔍 What are LLMs? LLMs, like GPT-4, are advanced AI models trained on vast amounts of text data. They excel in understanding and generating human-like text, making them indispensable in various applications. 📚💡 💬 Impact on Conversational AI: ➡ Enhanced User Experience: LLMs enable more natural and engaging interactions, providing users with human-like conversations. 🌟🗣️ Whether it's customer service chatbots, virtual assistants, or interactive voice response systems, LLMs create smoother and more intuitive user experiences. ➡ Personalization: By understanding context and user preferences, LLMs offer personalized responses, improving user satisfaction. 🎯🤩 Imagine a virtual assistant that remembers your preferences and provides tailored recommendations – that's the power of LLMs at work! ➡ Multilingual Capabilities: Breaking language barriers, LLMs can communicate in multiple languages, broadening accessibility. 🌍🗺️ This is crucial for global businesses looking to provide consistent customer support across different regions and languages. ➡ Automation & Efficiency: From customer service to content creation, LLMs streamline processes, saving time and resources. ⏱️💼 Businesses can automate routine tasks, allowing human employees to focus on more complex and creative endeavors. ➡ Understanding Nuances: LLMs can grasp subtle nuances in language, including idioms, slang, and context-specific meanings. This allows them to handle complex queries with greater accuracy. 🧩🔍 ➡ 24/7 Availability: Unlike human agents, AI-driven systems powered by LLMs can operate around the clock, providing uninterrupted service and support to users anytime, anywhere. 🌐🕒 📈 Future Prospects: The potential of LLMs in Conversational AI is immense. With continuous advancements, we can expect even more sophisticated and intuitive interactions, driving innovation across industries. 🚀🔮 Imagine AI that can not only understand but also predict user needs, offering proactive solutions and insights. 🔧 Challenges & Considerations: While the benefits are significant, it's essential to address challenges such as data privacy, ethical use of AI, and ensuring the accuracy and fairness of AI responses. 🤔⚖️ By prioritizing responsible AI development, we can harness the full potential of LLMs while mitigating risks. 🌐 Join the Conversation! How do you see LLMs shaping the future of Conversational AI? Share your thoughts and let's explore the possibilities together! 💬🤝 #ConversationalAI #AI #MachineLearning #Technology #Innovation #LLMs #MultilingualAI #Automation

11

What if your queries required access to multiple sub-related data from different documents, for an LLM to parse and make sense of before returning a coherent and accurate output? We often ask complicated questions, right? 🔹Well to solve this problem, these researchers released Multi-Head RAG (MRAG) which leverages activations of Transformer’s multi-head attention layer, instead of the decoder layer, as keys for fetching multi-aspect documents. They are taking advantage of the fact that multi-head attention can learn to capture different data aspects. 🔹The interesting part is that MRAG is both cost-effective and energy-efficient. In other words, it does not require additional LLM queries, multiple model instances, increased storage, or multiple inference passes over the embedding model. 🔹Moreover, MRAG can be seamlessly integrated with existing RAG frameworks and benchmarking tools like RAGAS as well as different classes of data stores. 🔹The researchers provide a comprehensive evaluation methodology, including specific metrics, synthetic datasets, and real world use cases to demonstrate MRAG’s effectiveness. 🔸Do check it out! #genai #technology #artificialintelligence Click #linkedangle and follow for more content Set the 🔔 notification on my page, don’t miss a post!

181

10 Comments

🚀 We've open-sourced TimesFM – a decoder-only foundation model for time-series forecasting 📈 Key Highlights: ✅ Large-Scale Training Data: Trained on a corpus of over 100 billion real-world time points, including Google Trends and Wikipedia pageviews. ✅ Zero-Shot Forecasting: Offers high-quality forecasts even on unseen datasets, without requiring fine-tuning or additional training. ✅ Outstanding Benchmark Performance: Achieved competitive results across a comprehensive suite of benchmarks. 🤗 Explore the model on Hugging Face and GitHub. I'm encouraged by the early feedback and look forward to seeing how #TimesFM will be used in real-world applications for forecasting and anomaly detection. 🙏 A special thanks to the talented research team behind this achievement: Abhimanyu Das Rajat Sen Yichen Zhou Weihao Kong

42

I am so proud of the work that is being delivered by our team in Search that was showcased at I/O this week. Highlights were... ⚡Plan Ahead with Generative AI in Search ⚡ Thanks to a new Gemini model customized for Google Search, we are able to do SO MUCH more. With planning capabilities *directly* in Search, you can get help creating plans for whatever you need, starting with meals and holidays. Search for something like “create a 3 day meal plan for a group that’s easy to prepare,” and you’ll get a starting point with a wide range of recipes from across the web. And if you don’t like the suggestions, Search will continue working on your behalf. So if you want to swap the dinner suggestion for a vegetarian option, just like that, Search will customize your meal plan. On top of this, there are lots more updates in Search so — for the curious  — have a read of the blog by Head of Search Elizabeth Reid below 🔽 https://lnkd.in/en6_7fvX #GoogleIO #GenAI #GoogleSearch

177

7 Comments

AI Appreciation Day celebrates the advancements and impact of artificial intelligence on our lives, industries, and future. This special day acknowledges the remarkable progress in AI technologies, from machine learning and natural language processing to generative models and autonomous systems. On this day, we honor the researchers, engineers, and visionaries who contribute to the AI field, driving innovation and addressing ethical considerations to ensure AI benefits all of humanity. It is also a moment to reflect on the responsible use of AI, emphasizing the importance of ethical frameworks and inclusive development. Let's celebrate the milestones achieved and look forward to the endless possibilities AI holds for a brighter tomorrow. #AIDay #IndustryInnovation #IntelligentAutomation #AIAppreciationDay #ArtificialIntelligence

16

"Last year, more than 50% of all global venture funding for AI-related startups went to companies headquartered in the Bay Area, Crunchbase data shows, as a cluster of talent congregates in the region." In other news, water is wet. There was a moment during the pandemic when it looked like everyone was going to work remotely and there was an opportunity for startups to be founded anywhere. I think that time has gone: the San Francisco Bay Area is once again the place to found any kind of technology startup. Yes, there are always exceptions, but the confluence of community density, living conditions, universities, and mindset make for a perfect storm. NYC and London - and maybe Boston / Cambridge - are pretty good too, for what it's worth, but the sheer volume of startup activity in the area gives San Francisco the edge. This is something I fought earlier in my career: my first startup was proudly founded in Scotland and largely run from England. I wish we'd just moved to San Francisco. This isn't to completely sing the praises of the city: the cost of living is now astronomical, and there's a contingent of right-wing activists that seem to want to paint it as some doom spiraling hellhole, as if its progressive past isn't something to be proud of. But there is still beauty, there is still that can-do sense of adventure, and if I was founding something new, that's probably where I'd be.

2

Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam University of Chicago, Carnegie Mellon June 2024 https://lnkd.in/eai4ZFwa - Large Language Models (LLMs) like GPT can store and recall facts, but this ability is not robust and can be easily manipulated by changing contexts without altering factual meanings. - Example: When prompted with "The Eiffel Tower is in the city of", GPT-2 answers Paris. But if the prompt is changed to "The Eiffel Tower is not in Chicago. The Eiffel Tower is in the city of", GPT-2 answers Chicago, even though the factual meaning is unchanged. - This "context hijacking" suggests LLMs may behave like associative memory models, where tokens in the prompt serve as clues to retrieve facts, even if associations are not semantically meaningful. - To study this, the authors propose a synthetic "latent concept association" task, where the output token is related to sampled prompt tokens through latent semantic concepts. - Through theory and experiments on transformers, the key findings are: - The value matrix acts as associative memory, gathering information from the prompt. - Embedding training is crucial, with embeddings capturing semantic similarity. - Attention can select relevant tokens as clues for memory retrieval. - These results provide insights into how transformers and LLMs perform fact retrieval via associative memory in the latent semantic space. The unintuitive associations formed make models susceptible to context hijacking. abs: Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be easily manipulated by changing contexts, even without altering their factual meanings. These findings highlight that LLMs might behave like an associative memory model where certain tokens in the contexts serve as clues to retrieving facts. We mathematically explore this property by studying how transformers, the building blocks of LLMs, can complete such memory tasks. We study a simple latent concept association problem with a one-layer transformer and we show theoretically and empirically that the transformer gathers information using self-attention and uses the value matrix for associative memory. #AI #machinelearning #computation #language #LLM #transformers

7

"So getting back to NVIDIA, I don’t see any force stopping the juggernaut in the next few years, other than a huge decline in the need for AI compute, and that seems unlikely. Any startup competitor could easily be acquired by NVIDIA for tens of billions of dollars and it would barely be material given NVIDIAs market cap. And while the other major chip companies are mounting a challenge, particularly AMD, the demand for GPUs is so great, and will be for the next few years, there is room for massive growth for multiple players. When NVIDIA eventually slows down, I suspect it will come from one of two areas. The first is a surprising shift in market use cases. Given the long development cycles of computer chips, it’s hard to be as responsive to the market demands as software companies can be. And product designers have to choose some level of predictability and reliability in their components and their partners, which is why they favor proven tech over innovative new designs most of the time. But the way NVIDIA benefited because their chips for graphics cards happened to be a good match for new AI workloads - I think something similar could happen to slow them down. Some new popular AI workload will be a better fit for something other than a GPU, but a chip that is already established in the market in other ways. Or secondly, it comes from a company that is building their own chips and decides to get into the broader semiconductor business. This means Apple, Amazon, Facebook, or most likely - Google. But all those companies, even if they can compete technically with NVIDIA, lack the infrastructure to sell and support chips and those who develop them. Building those capabilities will be slow." https://lnkd.in/dFpj5hqv

1

1 Comment