Sujatha Sagiraju

This article raises an interesting question: Is hybrid cloud the answer to hosting hybrid large language model (aka your Generative AI estate, enriched by your own data and prompts)? Given the nature of large language models, any prompt used in a public cloud large language model (LLM) setting can conceivable become part of the next cycle of learning. This is a different concern than traditional privacy and security considerations - and can naturally lead to the question of how to best host your own LLM (expensive as that may be), whether specialized or not. Personally I am hopeful that the emergence of small language models will ultimately resolve the cost aspect, until then we may have to revisit our cloud hosting strategies. https://lnkd.in/gjPM6dae

42

4 Comments

💡Scaling Generative AI: Technical Insights from Experience (Part 1) 🚀 As I've been working with customers on their Generative AI (GenAI) initiatives, I've noticed some familiar challenges surfacing - reminiscent of the early cloud adoption days. As enterprises are gearing up to launch their first few GenAI workloads in production, they want to get ahead of potential roadblocks for smooth scaling and adoption across lines of business (LoBs). Here are a few crucial considerations based on recent engagements: Robust Identity and Access Management (IAM) has surfaced a few times already. IAM is key in ensuring users can securely access only the necessary resources (LLMs, APIs, Vector Databases, Data, etc.) - most LoBs don't openly share data/assets across BU boundaries. Additionally, tracking and allocating resource consumption for each line of business (LOB) promotes fairness and manages costs across departments. Learning from the Cloud adoption days, organizations want the usual FinOps best practices on 'Metering Usage', 'Show Backs' and 'Charge Backs' sorted before mass adoption. Organizations are starting to realize the need early on for a flexible enterprise GenAI platform that supports both commercial and open-source Large Language Models (LLMs) - allows users to choose the best-suited models for their specific use cases, and selecting the most cost-performant LLM based on business needs is important. Who wants to pay $$$ for vacation policy lookup? Also, ability for the organization to evaluate LLMs based on performance, accuracy, and relevance is emerging as an essential capability for enterprise customers. This is because enterprises are already getting humbled by the various creative ways in which employees/LoBs are considering leveraging GenAI. So much variety warrants choice and flexibility! Enterprises want to be able to put necessary guardrails in place for what prompts can be sent to the LLM and what outcome can be shared back with the employees. As such, platform's capability to assessing the quality, ensuring safety of prompts, and adherence to corporate guidelines is critical for GenAI production launch. Likewise, implementing thorough testing and validation automation is surfacing as a core capability for path to production. To adhere with audit and compliance, organizations want to implement Large Language Model Operations (LLMOps) practices for model management, tuning, deployment, and monitoring streamlines operations. Stay tuned for Part 2 of this post, where I'll share additional technical insights that my customers are considering to help build a robust, scalable, and secure enterprise GenAI platform. 💬 Do share your GenAI experiences and technical learnings on the path to production in the comments below! 📢 Subscribe to my newsletter to get access to strategies and practical guidance on accelerating adoption of generative AI within your organization. Get started here: https://lnkd.in/g3bdneR7  #genai #ai #aws

14

The Evolution of AI: From Automation to Replacement AI systems typically progress through three key phases. Understanding where a solution stands in this progression can be insightful, especially amidst the buzz around LLMs and Generative AI. Here’s a breakdown: ▪ 1. Automate  The initial phase focuses on automating routine tasks to save time and reduce errors. For example, at Lucid Lane, we’ve automated administrative tasks, aiding our therapists by summarizing sessions and sending reminders to patients and therapists alike. While workflow automation tools exist, they’re just the beginning of the AI journey. ▪ 2. Enhance  As AI matures, it begins to augment human capabilities, allowing for scalability. The sophistication of AI in this phase correlates with fewer errors and greater human efficiency. Examples include Microsoft Copilot, GitHub Copilot, and Google’s auto-fill features. At Lucid Lane, our AI analyzes patient history and suggests medication dosages, aiding therapists and doctors in decision-making. Real-time suggestions during sessions can also enhance therapist-patient interactions, exemplifying AI systems with a human-in-the-loop. ▪ 3. Replace  The final phase is when AI becomes reliable enough to replace certain human functions. While companies may be cautious about claiming their solutions can fully replace humans due to the fear and negativity it may evoke, some are boldly making such claims. For instance, Devin, the touted first AI software engineer, represents this claim. Although I approach these assertions skeptically, it’s undeniable that some AI systems will reach this phase. It will become evident as AI enhancements repeatedly surpass the need for human intervention. These phases signify deeper AI integration into operational processes, aiming to improve efficiency, accuracy, and outcomes. However, it’s crucial to remember that despite AI’s potential to replace some functions, human oversight, ethical considerations, and handling complex situations remain indispensable. This progression not only reflects the future of work but also raises questions about human judgment and the ethical implications of AI, particularly in sensitive sectors like healthcare. What are your thoughts on the future of AI in your industry? Have you witnessed these phases in action? #generativeai #LLMs #EthicalAI #ai

19

1 Comment

Very good point - especially important also in the medical context: Key for the value-add of AI/LLMs is clean structured data which is relevant for the use case. I.e. data models yielding well-structured data remain very relevant, absolute LLM size seems less so.

16

1 Comment

🚀 Accelerating the Future of Healthcare with AI 🚀 I recently had the pleasure of tuning into an insightful episode of The Chris Voss Show featuring Lana Feng, Ph.D., CEO and co-founder of Huma.AI. 🗝 Here are some key takeaways that highlight the transformative power of AI in drug development and precision medicine: 🧠 Introduction to Huma.AI: Dr. Feng’s company, Huma.AI, leverages generative AI to automate data analysis, speeding up the development of life-saving drugs. Their mission? To reduce the time and cost involved in bringing these drugs to market. 🌟 💡 Challenges in Drug Development: Traditional drug development is a 10-year, $2.6 billion process plagued by data bottlenecks and regulatory complexities. AI can revolutionize this by handling vast amounts of unstructured data from various sources efficiently. 📊 🤖 The Role of AI: Utilizing advanced AI models like GPT-4, Huma.AI ensures data privacy, accuracy, and transparency. This leads to more reliable insights, faster clinical trial enrollments, and ultimately, better patient outcomes. 🏥 🌍 Real-world Applications: From identifying the right treatments in precision medicine to accelerating clinical trials, AI’s impact on healthcare is profound. It’s especially crucial in areas like cancer treatment and chronic disease management. 🎯 📈 Industry Recognition: Huma.AI has garnered significant recognition, being named a leader in generative AI for life sciences by Gartner. 🏆 🔮 Future Directions: The integration of AI in healthcare requires responsible usage, industry-wide collaboration, and continuous efforts to mitigate risks like data privacy issues and AI hallucination. 🤝 🌟 AI for Good: Dr. Feng emphasizes the importance of using AI responsibly to lower drug prices and improve healthcare outcomes. This approach ensures AI serves as a powerful ally in our quest for better health. ❤️ I’m excited about the potential of AI in transforming healthcare and making life-saving treatments more accessible and affordable. Let's embrace this technology for a healthier future! 🚀 #HealthcareInnovation #AI #PrecisionMedicine #DrugDevelopment #TechForGood #FutureOfHealthcare #ChrisVossShow #HumaAI #Leadership

11

Ai : Hunanity's greatest creation for Good and Bad. To secure ourself #zerotrustai In the journey, MS brings up VASA-1 will AI model which essentially spins a selfie image and turn it into a talking clip of you. All you have to do is upload a photo along with a voice note and let the AI model do the talking for you.

2

Brilliant and Timely: Why Data is the Top Priority in the Life Sciences AI race? Verbatim from Health Catalyst Capital (HCC): Over the past several decades, the life sciences industry has faced a troubling decline in R&D productivity. The cost to bring a new drug to market has increased from about $400 million in the 1980s to over $2 billion in the 2010s, adjusted for inflation. This cost increase is largely due to increasingly complex biology and the inefficiency of the traditional drug development process that relies on a “trial and error” approach. AI stands out as a potentially transformative technology for life sciences companies, capable of analyzing vast amounts of complex data and predicting experimental and clinical outcomes. NVIDIA CEO Jensen Huang has even stated that digital biology will be one of the most significant revolutions in human history.   And yet, the life sciences industry is plagued by a problem preventing it from unlocking the full potential of AI, a problem that other parts of the healthcare ecosystem know all too well: siloed data. According to a McKinsey survey, the lack of integrated data sources was cited by pharma leaders as the largest hurdle to scaling digital and analytics capabilities. Furthermore, nearly half (48%) of senior decision-makers in pharmaceutical companies surveyed by Aspen Technology stated that data silos derailed the efficiency of cross-functional collaboration in their organization.   Data is often siloed within different departments, systems, and repositories in life sciences companies. Without an integrated infrastructure, it’s difficult to pinpoint existing data, access it quickly, and leverage it across the organization, which often stifles decision-making and innovation. Data is the cornerstone of AI models and remains a top priority in the life sciences AI race. A key strategic differentiator is the ability to generate, acquire, and control large datasets from diverse sources for training models in areas such as omics, imaging, and health records.4   Before life sciences companies can achieve digital and analytics transformation, they must embrace data transformation, which is where DataOps comes in. Thanks for sharing Charles Boorady! Check out Unipr to learn how you can lead the charge by using Operations AI to raise R&D productivity in Life Sciences!

6

4 Comments

Hear what Dr. Alpan Raval has to say about creating impact with AI, from enabling someone in a rural area to open a bank account using their regional language without the hassle of filling out complex forms, to using computer vision AI for monitoring baby growth in rural India. This transformative work is made possible by skilled AI engineers and scientists collaborating with government stakeholders to identify problems and develop scalable, technology-driven solutions. At the core of these solutions is the need for robust data and compute infrastructure. ✅ Data infrastructure involves the curation of training data, whether through labeled datasets or synthetic approaches. ✅ Compute infrastructure refers to the hardware provided by cloud services, including GPUs. #socialgood #computervision #llms #trainingdata #malnutrition #ctd #anthropometry #asr

6

1 Comment

Y Combinator, an influential early-stage startup accelerator once led by OpenAI's Sam Altman, has notably increased its focus on AI, reflecting a broader trend where AI integration is becoming crucial for startups. Despite the more selective nature of its recent cohorts, the 2024 winter batch saw 64% of startups focusing on AI-related themes. Renowned for its impact in shaping the tech landscape, Y Combinator has nurtured successful companies like Airbnb and Coinbase. Its success formula includes supporting tech-savvy founders within a collaborative environment and making strategic investments across diverse, innovative projects. The evolving focus suggests that AI is increasingly seen as essential for new startups to gain traction and compete effectively in the rapidly evolving tech ecosystem, thereby extending YC's influence across many sectors and pushing the boundaries of AI applications. 🌐⇢ https://lnkd.in/dB7562fc

42

2 Comments

📢 Exciting insights from Google CEO Sundar Pichai in his recent interview on the future of AI! 🌐 In this compelling discussion, Pichai delves into Google's evolving AI strategies, emphasizing the immense potential of generative AI and the importance of ethical and responsible deployment. Google's journey from a search engine pioneer to a comprehensive Cloud and Enterprise platform underscores its adaptability and innovation in the tech landscape. As the internet has grown, so has Google's role, shaping not just how we search for information, but how businesses leverage technology for growth. For those interested in the intersection of AI, cloud computing, and enterprise solutions, this interview is a must-watch! Check it out here: https://lnkd.in/ea_eEJtP #AI #CloudComputing #TechInnovation #GoogleAI #FutureOfAI #EnterpriseSolutions #Leadership #DigitalTransformation

21

2 Comments

A couple of recent college graduates from India against AI finetuning giants like Anyscale and Run:ai? I met Arko C at a NASSCOM event last October. He was building Xylem AI (YC S24) , a platform that allows you to train and deploy LLMs in production. So naturally most Indian VCs rejected him. Because how could a team of three recent college grads compete with the likes SV startups like Anyscale that have raised hundreds of millions of dollars, have teams of experienced AI researchers and a stash of H100 GPUs? Arko hustled and closed two Fortune 500 companies for paid contracts. I personally know unicorns that haven't managed to do this. He then got into YC in the last round in May. There's a lot to still be proved out, but Arko exemplifies how valuable grit is in building startups. I’m doing a webinar with him tomorrow on challenges of using LLMs in production. Link to register in the comments. Disclaimer: I'm a small investor in Xylem.

150

4 Comments

I've been thinking that #MachineLearning models often perform better in healthcare and last longer in production. This is because the human body is a controlled environment, and there aren't continuous shifts in the way the human body works fundamentally. In business, things are more dynamic. As the company grows, so does the nature and the volume of the produced data. Let's not forget external factors such as the economy, social changes, and shifting market demands. This means models need regular updates to stay accurate, which makes them even more complex and costly. A project's lifecycle doesn't finish when an #ML model goes into production. It's essential to regularly check in to ensure the model is still meeting the business requirements, the data powering the model is still relevant, and its performance delivers added value. Chip Huyen very briefly touches on this subject in one of the SuperDataScience podcast episodes, discussing the impact of feature drift in production-ready models. However, in systems based on #GenAI, where #LLMs are not developed internally, handling changes can differ because businesses rely on vendors for updates and improvements. I think in this scenario, an LLM-agnostic approach can be helpful. System architects should be able to constantly evaluate and replace LLMs with ones that perform better in the new circumstances. link to episode 👉🏼 https://lnkd.in/ebXB5Maq #AI #DataScience #MLOps #HealthcareAI #BusinessAI #ArtificialIntelligence #LLM #GenAI

38

3 Comments

UBS predicts the AI industry's revenue will grow at a ~72% CAGR from $83B in 2024 to $420B in 2027. The semiconductor ecosystem is complex, but here is a simplification of #ShayBoolor’s 20 favorite companies within this space -- ranked from most to least expensive based on 2025 metrics https://lnkd.in/efnaDusP

9

1 Comment

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

"Look, this is not ready for prime time" Last week, The Information met customers and vendors at Google's annual cloud conference, Google Cloud Next. They found that while Google and other vendors provided examples of tasks that could be fully automated using #GenAI (e.g. writing code and creating ad copy), the vast majority of customers are still using AI for relatively mundane and low-level tasks (e.g. summarizing conversations or finding information in internal docs). The level of inaccuracy and hallucination for higher-order tasks is still too high today and so is the risk of a bad customer experience. It takes too much time and ultimately cost (compute, energy, and labor) for customers to train models that fully automate tasks within the enterprise. That's why I believe #startups that provide a combination of software and services (i.e. an in-house AI-assisted human worker to meet a specific need) can create the biggest leverage for customers today. In the infrastructure software market, this could mean a next-gen identity lifecycle management, pen testing, or MDR platform that completes an end-to-end task for customers leveraging both humans and AI. Please reach out if you're a #founder working on a new startup with this concept of software and services.

37

8 Comments

This is a great list, and a good example of fractal complexity inherent in seemingly "solved" problems. But: most frameworks handle the majority of these out of the box for you — except for SSO / SAML, which should probably not be a pre-launch concern. More broadly: software is about making decisions and trade-offs with varying degrees of confidence. Sometimes those decisions are very clearly "build vs. buy"-shaped; sometimes they're more esoteric. You owe it to yourself as an engineer and operator to make those kinds of decisions not because you think "well, this is what everyone says you gotta do", but because you've weighed the pros and cons and come to a conclusion, even if you don't know everything about the problem space.

1

🚀 The need for India-focused GenAI platforms has led to a host of indigenous developments. Let's take a look at 8 Made-in-India LLMs that are shaping the country's future. 🔸 Recognizing the socio-cultural gap in existing systems, entrepreneurs and developers from India are working on a GenAI ecosystem tailored for Indian consumption, yielding better contextual outputs across all formats. 🔸 A handful of homegrown conglomerates, such as Reliance (Jio), Tata Consultancy Services, Infosys, and Mahindra & Mahindra, are already working on a slew of GenAI projects. 🔸 Indigenous developments like BharatGPT, Ola Krutrim, and Project Indus (a Tech Mahindra venture including 40 different Indic languages), are addressing the language challenges pan-Indian users face in real life and now in the AI world. According to Inc42 data, India’s GenAI market is expected to grow exponentially in the next few years, surpassing $17 Bn by 2030 from $1.1 Bn in 2023, growing at a CAGR of 48%. With BharatGPT and its ilk taking charge and cementing their positions in India’s conversational AI landscape, India is well-prepared for a new era of digital empowerment where language barriers are dissolved and access to information is democratized. 🌟 Thoughts on this trend: Cultural and Linguistic Relevance: These platforms address India's diverse socio-cultural and linguistic nuances, ensuring inclusivity and effectiveness. Local Innovations and Entrepreneurship: Investments by companies like Reliance, TCS, Infosys, and Mahindra & Mahindra highlight India's growing tech ecosystem and global AI positioning. Economic Growth and Opportunities: The rapid expansion of the GenAI market signifies substantial economic opportunities, leading to job creation and skill development. Digital Empowerment and Accessibility: By providing AI-driven solutions in multiple Indian languages, these platforms enhance digital literacy and inclusivity. Addressing Real-World Challenges: Indigenous AI platforms deliver solutions closely aligned with the needs and preferences of the local population, improving customer service, education, and healthcare support. The emergence of India-focused GenAI platforms is a promising development driving technological, economic, and social progress in the country. Let's look forward to a digitally empowered and inclusive future! 🚀🌐 #GenAI #AI #India #DigitalTransformation #Innovation #Technology #BharatGPT #OlaKrutrim #ProjectIndus #DigitalEmpowerment #FutureTech #AIinIndia

28

3 Comments

Bang on about generating value with #genai Are you a 'taker' or a 'shaper'? McKinsey & Company reveals how a combination of 'taker' and 'shaper' CIOs will drive the quickest value for most of the outcomes and value.

9