Sanjay Dhar

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

2

NVIDIA NIM™ — inference microservices that provide models as optimized containers — to deploy on clouds, data centers or workstations, giving them the ability to easily build generative AI applications for copilots, chatbots and more, in minutes rather than weeks. Available to download today! #TDSYNNEX #NVIDIA #DESTINATIONAI

14

1 Comment

Better Synthetic Data by Retrieving and Transforming Existing Datasets Saumya Gandhi, Ritu Gala, Vijay Viswanathan, Tongshuang Wu, Graham Neubig Abstract Despite recent advances in large language models, building dependable and deployable NLP models typically requires abundant, high-quality training data. However, task-specific data is not available for many use cases, and manually curating task-specific data is labor-intensive. Recent work has studied prompt-driven synthetic data generation using large language models, but these generated datasets tend to lack complexity and diversity. To address these limitations, we introduce a method, \textit{DataTune}, to make better use of existing, publicly available datasets to improve automatic dataset generation. DataTune performs dataset transformation, enabling the repurposing of publicly available datasets into a format that is directly aligned with the specific requirements of target tasks. On a diverse set of language-based tasks from the BIG-Bench benchmark, we find that finetuning language models via DataTune improves over a few-shot prompting baseline by 49\% and improves over existing methods that use synthetic or retrieved training data by 34\%. We find that dataset transformation significantly increases the diversity and difficulty of generated data on many tasks. We integrate DataTune into an open-source repository to make this method accessible to the community: 👉 https://lnkd.in/dDybDHWZ #machinelearning #datascience

1

Here come the SLMs (small language models)! The ultimate architecture for a future app would be having local SLMs handle 90% of the work while calling one or more cloud LLMs for the rest. Ideally, the next step would be for operating systems or even browsers to come with a few default models optimized for the device so that we don't end up with a bunch of 5GB+ downloads. https://lnkd.in/gv6HaByu

5

2 Comments

Thanks DevOps.com for publishing my latest piece on why #LLMs fall short for #rootcauseanlaysis! Some useful guidelines for leaders looking to harness AI for observability – plus why it’s hard to convert structured graph data into the type of context-rich input needed for LLMs. (All of it based on lots of sweat, tears, and lessons learned the hard way. 😅) https://lnkd.in/dzTYNSDC

25

A Marriage Made in Heaven - LLM's and DADA-X by Decision-Zone The integration of task-oriented Large Language Models (LLMs) with the dynamic capabilities of DADA-X is indeed a transformative development in the realm of distributed systems management. These advanced LLMs, with their profound ability to simulate intricate Operational Technology (OT) systems, are revolutionizing the way mission-critical processes are conceived and executed. When these processes are fed into DADA-X, they are ingeniously converted into causal event patterns, which act as the backbone for autonomous applications operating in the real world. This convergence is unlocking unprecedented levels of real-time control and management. Organizations can deploy these sophisticated event patterns within DADA-X, enabling them to fully exploit the capabilities of their distributed systems. The result is a paradigm shift in operational efficiency, with DADA-X autonomously orchestrating complex missions, optimizing performance, and proactively mitigating risks. The emergence of the Artificial General Intelligence (AGI) market is a direct consequence of the synergy between LLMs and DADA-X. This market is poised to offer organizations the ability to construct intelligent systems that not only comprehend but also autonomously execute complex tasks with remarkable precision and efficiency. The potential here is immense, with implications across various sectors and industries. As we move forward, the traditional approaches of manual intervention and centralized control are becoming obsolete. The combination of LLMs and DADAX equips organizations to confidently step into the future of distributed systems management. This transition heralds the end of inefficiencies and the beginning of an era characterized by autonomy, efficiency, and reliability. The opportunity presented by this technological evolution is significant. Organizations that act swiftly can transform their distributed systems into agile, intelligent networks capable of managing the complexities of contemporary operations. The power of LLMs and DADA-X is not just theoretical but is a tangible force driving the AGI market forward. To be at the forefront of this revolution is to be a pioneer in the AGI market. It's an invitation for organizations to empower themselves with the necessary tools and technologies to succeed in an increasingly intricate and dynamic landscape. The future of distributed systems management has arrived, and it is powered by the innovative fusion of LLMs and DADA-X. The call to join this revolution is clear, and it promises a journey towards a smarter, more autonomous world.

5

Truth. This blog articulates the shortcomings of most tiered storage solutions in delivering predictable performance at scale. Without some rebuild that harnesses cloud primitives like AWS S3 and GCP GCS, mere incremental adjustments to existing products won't adequately serve users. From a product manager's perspective, sometimes incremental adjustments isn't a path to sustainable product adoption and success.

9

Google Cloud has taken a significant step in enhancing Vertex AI's grounding capabilities for more reliable AI responses. The introduction of dynamic retrieval for Grounding with Google Search is a standout addition, now generally available. This innovative feature empowers Gemini, Google's advanced large language model, to intelligently determine whether to ground user inquiries in Google Search or rely on its intrinsic knowledge. This strategic approach strikes a balance between response quality and cost efficiency, given that grounding with Google Search incurs additional processing costs. Gemini leverages its understanding to decide if the requested information is likely static, slowly changing, or rapidly evolving. Check out the details here: #GoogleCloud #VertexAI #AI #GoogleSearch #TechnologyInnovation

16

🤖 GenAI transformed the way AI Engineering was evolving. The key change is that there is no model-building process for most use cases like Machine Learning Engineering. 💻 Model building process is replaced by identifying the right technology vendor and partners. 👩🏽‍💼 The cost of hiring the best AI talent is shifter to upskill your Engineering with GenAI API and frameworks. 🎯 The missing part between security and data is that RAG systems show all the weaknesses of a minimal secure data system. What we need is SRAG or Secure RAG for GenAI. #genai #ai #aiml #generativeai #llm https://lnkd.in/dqUk4V5j

11

1 Comment

#Microsoft’s latest open-source #AI model, I must say is quite amazing! The company’s new #Phi-3 family of small language models (SLMs) is set to revolutionize the way we interact with machines.The rise of small language models like Phi-3 marks a significant shift in the AI landscape. These models are more accessible, cost-effective, and easier to integrate into enterprise legacy workflows. As the technology continues to evolve, we can expect to see widespread adoption of SLMs across various industries.

3

Interesting article about humongous foundational LLMs potentially reaching the ceiling in terms of improving model quality. I believe that smaller, domain specific models are the hew frontier that might offer better ROI in terms of costs and model quality for a specific domain/use case. This should be especially true for enterprise use cases, when companies possess significant proprietary data corpuses. #artificialintelligence #largelanguagemodels #airevolution

52

8 Comments

Big news: Anthropic's most intelligent model yet, Claude 3.5 Sonnet, is generally available on Google Cloud’s Vertex AI! Get all the details on the announcement and learn how to get started ⬇️

Google Cloud and Anthropic made some exciting announcements: 📣 Claude 3 Opus, Anthropic’s most intelligent model, is now generally available on Vertex AI ⚙️ Tool use is available for the Claude 3 model family on Vertex AI, enabling Claude to act as an autonomous agent 📈Provisioned throughput is available for the Claude 3 model family on Vertex AI

19

1 Comment

"How Meta trains large language models at scale" - Great article by Meta that provides a thorough overview of the infrastructure challenges and innovations involved in their efforts to train large language models at scale for generative AI. Some key highlights that stood out: 👉 They had to modify their Grand Teton platform that uses NVIDIA H100 GPUs on a very tight schedule, increasing the GPU TDP to 700W and moving to HBM3. This required mechanical and thermal design changes plus a validation cycle, all while staying within the constraints of their existing air-cooling infrastructure. Making significant hardware changes like this under time pressure and resource constraints is a really impressive feat. 👉 Meta decided to build two 24k GPU clusters simultaneously, one with RoCE and one with InfiniBand, to gain operational learnings and inform their future directions. Pursuing both paths in parallel at that scale is a significant undertaking, showing real commitment to thoroughly evaluating the tradeoffs. 👉 They mention observing various hardware failure modes during their scaling journey, such as GPUs "dropping off" the PCIe bus and uncorrectable memory errors. Monitoring, identifying "repeat offenders", and proactively mitigating these issues at scale required comprehensive telemetry and automation. 👉 The article notes that in the coming years Meta anticipate "handling even larger volumes of data, and dealing with longer distances and latencies." This suggests they can predict new networking and data management challenges as they continue to increase scale and potentially distribute training across larger geographical areas. #ai #generativeai #llms #enterpriseai #aitraining https://lnkd.in/eQsGmndf

5

1 Comment

https://lnkd.in/gWUpZECK Mixture of Agents (MoA) is an approach to harness the collective strengths of multiple LLMs to improve state-of-the-art quality. According to Together AI, it achieves a score of 65.1% on AlpacaEval 2.0, surpassing prior leader GPT-4o (57.5%).

5

*Mixture of Agents (MoA)* is an innovative way to boost Large Language Models’ performance by leveraging the strengths of multiple agents as a team. Layered Architecture: MoA arranges many LLMs in layered architecture, each layer composed of numerous individual agents(LLMs). Collaboration: These agents generate responses that are based on outputs from the previous layer’s agents. They repeatedly refine and enhance the last output. Diverse Insights: By integrating different capabilities and insights from various models, MoA achieves a more powerful combined model that is secure against diverse risks. Performance Boost: MoA outperforms standalone LLMs by a wide margin. For instance, it set a new benchmark with 65.1% on AlpacaEval 2.0 as against GPT-4 Omni at 57.5%, using only open-source models. In short, if we combine language modelers’ strengths then it results in increased efficiency and versatility. Here is a nice implementation of MOA by Together AI Github source: https://lnkd.in/dMCVRS3D #AI #LLM #LargeLanguageModels #MachineLearning

59

3 Comments

Big Thinking. AI and Computing in the Future. WEC Yann LeCun . https://lnkd.in/gfzTHhSH Abundance. Open Source. Emergence. Assistants. Efficiency. Sustainability. A new human renaissance. #ai #compute #memory #energy #knowledge #evolution

2