Magnifico

Excited to learn more about NeMo and possibly implement it for customers!

This is amazing!

The AI pin from hu.ma.ne is out, the first LLM-native consumer hardware device. I think it's a great stride towards "ambient intelligence", where AI fades into the background and emerges naturally when you need it. I can imagine having the GPT app store streamed to the device, switching agents depending on the multimodal context around you at the moment. The note-taking feature is awesome: I want to remember important conversations and contacts at conferences without explicitly typing notes on my phone. The privacy concerns are huge though, despite the safeguard mechanism. Google Glass was dead partially because of the social stigma. How will Humane AI pin perform in the mass market? I'm curious what you all think!

👏

Can we pre-train LLMs with Retrieval Augmentation? 🤔 RETRO was a research by Google DeepMind, which included retrieval into the pre-trainng process. Now NVIDIA continues this research by scaling RETRO to 48B, where they continued pretraining a 43B GPT model on an additional 100 billion tokens using the Retrieval augmentation method by retrieving from 1.2 trillion tokens. 🤯 𝗜𝗺𝗽𝗹𝗲𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻: 1️⃣ Pretrain an LLM using the next token prediction on 1.1T token 2️⃣ Continue pretraining of the LLM with retrieval augmentation (Retro-fitting) on an additional 100 billion tokens while retrieving from whole pretraining dataset. 3️⃣ Instruction tune the model, with retrieval augmentation. Only update the weights of the decoder. 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀: 📈 Continued pretraining of LLMs with retrieval yields better decoders for QA. 🚀 Up to 10% improvements doing retrieval augmented pretraining 📚 Pretraining with retrieval improves the incorporating of context 🧮 Retrieval database had 19 billion chunks, with each chunk containing 64 tokens 🐌 Retrieval pretraining is slower and more complex due index and retrieval step Check out the full paper: https://lnkd.in/d2R4zsQ5 Remember that these are just my personal findings. Make sure always to conduct your own research and analysis. 🤗

A paper that really illustrates both the unexpected power, and unexpected risks, that come from LLMs. Given text of anonymous posts on Reddit, GPT-4 can infer things like income, gender & location with 85%+ accuracy at 1% of the cost, and 0.25% of the time required by human raters. https://lnkd.in/emBn9wgx

Wanna 𝗱𝗲𝗽𝗹𝗼𝘆 an 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗲 𝗟𝗟𝗠? 🚀 ↓ ---- Hi there! It's Pau 👋 Every week I share free, hands-on content, on production-grade ML, to help you build real-world ML products. 𝗙𝗼𝗹𝗹𝗼𝘄 𝗺𝗲 and 𝗰𝗹𝗶𝗰𝗸 𝗼𝗻 𝘁𝗵𝗲 🔔 so you don't miss what's coming next #machinelearning #mlops #realworldml #llm #llmops

NVIDIA AI TensorRT-LLM just went live on GitHub! 🔥   Let us know via the repo if there are any questions you have about the new toolchain. Models you would like supported, features you're looking for, bugs you run into, and we'd love to hear about your experience working with the toolchain after you've been able to run your experiments. TLDR: TensorRT-LLM is a opensource acceleration engine for LLM Models with support for Multi-GPU & Multi-node. Support for Inflight Batching, Quantization, mixed precision on NVIDIA #Ampere and #Hopper H100 #GPUs. Looking forward to hearing about your experience! TensorRT-LLM https://lnkd.in/gYiP4N_S Triton Inference Server Backend for TensorRT-LLM https://lnkd.in/garWpi3g TensorRT-LLM Documentation: https://lnkd.in/gTUr3cB7 The quantization toolkit for TensorRT-LLM Ammo: https://lnkd.in/gWbFYmTt

🔵 How Graphs Could Shape the Future of Vector Search 🔵 With ongoing advancements in Large Language Models (LLMs) such as ChatGPT, vector-based search mechanisms are rapidly transitioning from being auxiliary features to core functionalities in many platforms. Vector search is now found not only in specialised stores like Pinecone and Weaviate but also in search platforms such as Elasticsearch and databases like MongoDB. Notably, both these platforms have employed an algorithm called Hierarchical Navigable Small Worlds (HNSW) to deliver efficient vector search. HNSW is a graph-based algorithm, its power lies in the ability to transform continuous embedding vectors into a discrete, layered graph. 🔵 Discrete and Continuous Semantics Traditionally, fuzzy matching strategies are often implemented in conjunction with discrete filters. In search, this is referred to as 'faceting' (think of searching for 'shiny black shoes' on eBay and then selecting a specific brand from a dropdown menu). This hybrid approach has proven effective and is being widely adopted for vector search as well. For example, one might restrict documents based on geographical origin or timeframe and then use vector-based search to gauge sentiment only within that subset. 🔵 A Graph-Based Revolution Traditional filtering is typically based on tabular (rows in a database) or tree-like (JSON documents) data formats. The landscape changes significantly when the data itself is structured as a graph. When employing HNSW in a graph-based setup, both continuous vectors and discrete facets become vertices in the same graph. This allows for more nuanced relationships and more efficient alignment. Furthermore, the upper layers within HNSW represent a form of compression. With your data in a graph, you can move beyond the classic HNSW node-degree compression algorithms to consider more semantic forms of compression, which take domain-specific ontologies into account. This could prove to be very powerful. 🔵 Key Takeaways for Organisations I posit that transitioning to graph-based data structures is the next logical step in the evolution of search and knowledge representation. Therefore, my advice to organisations looking to stay ahead in the data management and analytics game is to transition as much of their core data into a graph structure as quickly as possible. ⭕ HNSW: https://lnkd.in/eH7JqEyZ ⭕ Continuous and Discrete: https://lnkd.in/ex8HA_Nj ⭕ Embrace Complexity: https://lnkd.in/ejZikEGp ⭕ Semantic Router: https://lnkd.in/eucZUjrV

There's a lot to be optimistic about with AI.