Piotr Dabkowski

New Hidden Layers is here! Ron Green is joined by two AI veterans, Paco Nathan and Dr. Steve Kramer to talk all things graph neural networks (GNNs.) Together they talk through how GNNs reduce computation time, how to tackle the cold start problem in recommendation engines, and how they even aid in election integrity. Learn about real-world applications in pharma, healthcare, and cybersecurity, and the promising future of hybrid AI architectures merging symbolic and neural approaches. Listen/watch below!

28

5 Comments

🔥🚀 Exciting News from MIT: Introducing FlowMap for Deep Learning-based Structure from Motion! 🔥🚀 MIT unveils FlowMap, a groundbreaking self-supervised, end-to-end differentiable SfM method delivering COLMAP-level accuracy for 360° scenes. FlowMap revolutionizes the process by precisely solving camera poses, intrinsics, and dense depth of video sequences without relying on unique image features or manual matching. This innovation outperforms prior gradient-descent based methods and rivals COLMAP, marking a significant advancement in SfM technology. 👉 Explore the project page, paper, and code: https://lnkd.in/gyyinuTu #Reconstruction #SfM #AI #DeepLearning #Innovation

7

🔍 The Future of AI with OpenRLHF 🔍 As AI technology advances, aligning large language models (LLMs) with human values and intentions becomes increasingly critical. Reinforcement Learning from Human Feedback (RLHF) is a powerful technique addressing this challenge. However, traditional RLHF frameworks struggle with the complexity and resource demands of training models exceeding 70 billion parameters. Launching this week is **OpenRLHF** is an open-source framework designed to overcome these limitations by leveraging cutting-edge technologies such as Ray, vLLM, and DeepSpeed. Here's why OpenRLHF is a game-changer for AI development and why I am so excited! 1. Scalability - OpenRLHF efficiently scales RLHF training for LLMs beyond 70 billion parameters by distributing models across multiple GPUs, optimizing memory use, and minimizing computational overhead. 2. Performance Optimization - By integrating Ray for model scheduling, vLLM for accelerated sample generation, and DeepSpeed for enhanced training, OpenRLHF ensures superior performance and reduced training time. 3. Versatility and Usability - Fully compatible with the Hugging Face library, OpenRLHF supports various alignment techniques like Direct Preference Optimization (DPO), Kahneman-Tversky Optimization (KTO), and rejection sampling. It offers user-friendly, one-click training scripts for diverse models and algorithms. 4. Resource Efficiency - OpenRLHF’s scheduling and memory management reduce GPU memory fragmentation and communication overhead, enabling larger batch sizes and more efficient training processes. OpenRLHF sets a new standard for AI development and ResponsibleAI. Explore the full potential of OpenRLHF and its strategic benefits in the detailed report. Read more https://lnkd.in/eCF7Q9Pb #Innovation #AI #Leadership #RLHF #OpenSource #TechAdvancement #AILeadership #FutureOfAI

5

✨🦙 𝗟𝗟𝗮𝗺𝗮 𝟯 𝗶𝘀 𝗵𝗲𝗿𝗲! There's been a fair bit of speculation that #LLama3, the successor to AI at Meta's Llama 2 was coming and that appears true with the announcement below! Availability is coming soon on AWS, GCP, Hugging Face plus others and the numbers look promising - it will be interesting to get hands on with this! Meta's goal here was to "build the best open models that are on par with the best proprietary models available today". ❓🙋🏽‍♂️ One question I come away with this however is... on the release page there are comparisons with everything barring GPT-4? (they even compare vs GPT-3.5) #Meta #Llama #Ai #LLM #OpenSource

6

1 Comment

A small win. We (/w Ritwik Raha) wanted to read about diffusion models for some time now. After the NeRF tutorial series on PyImageSearch, we were thinking of something that was out of our comfort zone. We came across DDPM and were excited to see how we could break the paper down for the community. Well our poor formal math education would prohibited us from understanding the paper. We quickly moved into covering other things, and the idea got shelved. We came back to the idea of diffusion models early this year. There were far more expamples from the community to ease our way into it. We decided on taking two approaches. I would go on the math route, while Ritwik Raha takes the code route. How do I know that I am on the right track? I run an experiment on myself, note what I feel and then run the same expriment later. This time I read Lilian Weng's blog post on diffusion models. Notes on Feb 3 2024: need to understand probability theory. Notes on Apr 25 2024: - This is so thorough. - Is the forward process close to CLT? - Equating the reverse process steps as latent variable model is amazing! I am genuinely happy that I understand something from scratch and can communicate my ideas with the people in the field without having to search for words. Expect something from PyImageSearch very soon.

54

4 Comments

I was chatting about translation with Andrew Ng at an event several weeks ago. It's fun to see him post a workflow with the seed of the AI Review concept that's live in LILT today. === This is along the path to what many in the industry are doing. Two comments: 1. Replace step 2 with a formalized reflection approach such as AutoMQM. 2. BLEU isn't discriminative at the accuracy levels modern MT systems achieve. Use a trained metrics like BLEURT or COMET to see real differences between systems. 20 years ago we called this idea automatic post-editing, where you'd train a second string transducer to rewrite initial MT output. However, those systems were very difficult to train and cascade. Using LLMs in cascades is significantly more effective.

42

We just evaluated our 1M context length AI at Meta Llama-3-8B model using RULER to get a more comprehensive view on quality. Until recent our team has primarily relied on NIAH (Needle-in-a-Haystack), but this method may not fully represent how a long context model will perform in practical use cases. Our results show that: 🔗 https://lnkd.in/gxru4hNK ✅ Our model is behind only GPT-4 and Yi in the retrieval and Q&A tasks ✅ It’s ranked 1st amongst models < 30B parameters Shoutout to Crusoe for sponsoring the compute for these evals. #Gradient #GradientAI #LLMs #LongContext #RULER #NIAH #EVALS #RAG

32

1 Comment

It’s day 4 of our 11 Days of Inference Acceleration Techniques. Today, we’re moving on to runtime level optimization best practices. 𝐅𝐨𝐮𝐫𝐭𝐡 𝐭𝐢𝐩: 𝐓𝐚𝐤𝐞 𝐚𝐝𝐯𝐚𝐧𝐭𝐚𝐠𝐞 𝐨𝐟 𝐠𝐫𝐚𝐩𝐡 𝐜𝐨𝐦𝐩𝐢𝐥𝐚𝐭𝐢𝐨𝐧 📈 Graph compilers such as TVM, Tensor-RT, and OpenVino work by getting a computation graph of a specific model and generating an optimized code adjusted for the target hardware. Graph compilation can optimize the graph structure by merging redundant operations, performing kernel auto-tuning, enhancing memory reuse, preventing cache misses, and more. But few things to be aware of: 📝 Not all models compile equally. 📝 The impact of compilation on model performance can vary. Hence, make sure to check the architecture's ability to compile early on in the process to avoid wasting time and resources on training a model that can’t be optimized for fast inference. – What’s the #11DaysofInferenceAccelerationTechniques? The Deci team is posting, for 11 days, a series of inference acceleration techniques for deep learning applications. If you’re looking for practical tips and best practices for improving inference, follow Deci AI so you won’t miss an update. #deeplearning #machinelearning #neuralnetworks #computervision

9

📊 Exciting News: TransformerFAM is Revolutionizing Sequence Modeling! 🤖 There's a powerful new architecture on the block that's taking transformer models to the next level. Introducing TransformerFAM - the game-changing approach that incorporates a feedback attention mechanism (FAM) to supercharge transformers with working memory capabilities. 🔑 Key Highlights: - FAM enables transformers to process sequences of unlimited length by attending to their own latent representations - Prompt tuning allows efficient learning of FAM weights with minimal additional parameters - Memory-efficient implementations make FAM computationally feasible even for large-scale models - Experiments show significant performance gains on long-range context and coherence tasks Curious to dive deeper? Check out the groundbreaking research paper linked below: https://lnkd.in/dmhXUuZZ #DataScience #MachineLearning #DeepLearning #TransformerFAM #NLP

15

if you know, you know NVIDIA annual shareholder meeting today: june 26 2024 09:00 pst in about an hour we'll have a chance to listen to new developments. it's breath-taking. the speed of not only NVIDIA's ability to navigate (well i guess they must be using their own (ai) to decide what where how to do and then some. huh?) more importantly just as Apple Microsoft defined the pervasiveness of computers. their applications. we believe NVIDIA holds that position currently. they have been handsomely rewarded in the markets. a valuation as the most valuable in the world. 3.3 trillys. and going. so every and all announcements are truly where the centre of the universe of chip design, manufacturing, exists. it's more than simply offering state of the art chip design for (ai) modelling. where Alphabet Inc. organizes the world's information. NVIDIA makes sense of it. and everyone else applies it, to their subject matter expertise. this is the change that is afoot. at this moment. where it was a race to offer advertisers of services/products a better understanding of their clientele, now the lens is sharper. there is no need to suggest age/sex/ethnicity/generational differences. there are behaviours. theses behaviours are independent of what data was used to point adverts to. to get these people to buy more sh^t. that's the base of the u.s. economy. it's apparently consumer driven. and so in this post pandemic world where it literally stopped. and we were forced to sit still. wait it out as the storm clouds passed. in hopes no one. not us. or loved ones would perish. that time was a generational shift. we all felt. it. no matter what class you were. NVIDIA is the fulcrum Google is the waterfall or Google is the head NVIDIA the neck wherever the neck turns that is where the head has their attention. the neck keeps the attention of the head there. so it appears as if in many ways Google has in essence offered the search part of it's business up for grabs. this is why Perplexity exists, and others. a new dynamic pricing auction model is about to emerge. it will as it is by nature adversarial. let's not fool ourselves. we are designed to be adversarial. it's our nature. and in coveting $$$$$ as the prime objective. money for the sake of money this will destroy a lot of thoughtful, mindful, intentionally beneficial innovation. why? well. it's not playing according to the whims of a recommendation machine that rewards those that are part of it's invisible army. today is a crowning. it's a crown. upon the head of jensen. this is akin to king charles and his crowning. we wonder who will be at the crowning. to recognize this eventful day. it's good to be king. ;) < . > source: https://lnkd.in/eumX4num

1

NVIDIA Introduces RankRAG 👉 #NVIDIA and Georgia Tech researchers introduce RankRAG, a novel framework for #RetrievalAugmentedGeneration (#RAG). 👉 RankRAG instruction-tunes a single Large Language Model (LLM) to perform both context ranking and answer generation. 👉 The framework uses a two-stage instruction tuning process, incorporating diverse datasets. 👉 During inference, RankRAG uses a retrieve-rerank-generate pipeline. 👉 RankRAG outperforms existing models, especially on challenging datasets such as long-tailed QA and multi-hop QA. 👉 The 8B parameter version of RankRAG competes with larger models, while the 70B version surpasses strong baselines. This looks interesting as it appears to unify context ranking and answer generation in a single #LLM. RankRAG is still using a retrieval step to fetch relevant contexts (from an external knowledge base) but, as per the article, uses a "retrieve-rerank-generate pipeline" which first retrieves top-N contexts, then reranks them to select the most relevant top-k, and finally generates answers based on these refined contexts. The key innovation seems to be that it uses a single instruction-tuned LLM to perform both the reranking of retrieved contexts and the generation of answers, rather than using separate models for these tasks. #ai #enterpriseai https://lnkd.in/gq5GNR6R

4

We’ve recently contributed FP8 support to vLLM in collaboration with Neural Magic -- with this feature, you can see up to a 1.8x reduction in inter-token latency, with >99% accuracy preservation! A common concern with FP8 is whether users will experience accuracy degradation. To address this, Neural Magic has produced many checkpoints for key models with >99% accuracy preservation across a wide range of benchmarks (https://lnkd.in/gTimN5dZ), including: - Llama3-70b - Mixtral 8x7b - Llama3-8b You can easily try this out on vLLM, and read more about the feature here -- https://lnkd.in/gzKJqerB

139

3 Comments

FP8 quantization is now available in vLLM - check it out! Quantized inference is one of the best ways to reduce the costs of LLM deployments.

5

NVIDIA's Jensen said "ChatGPT democratized computing, Llama2 democratized generative AI" ...and now Llama3 is the next level 🔥 It's fascinating to watch Meta's strategic moves. With Llama3, they prioritized training on a massive 15T token dataset to pack all into a lean 70B param model, instead of building a massive model. This allows Llama 3 to match trillion+ parameter models like GPT-4, but at 1/10th the compute, storage and inference costs! 💰 This technique was published by Google DeepMind a few years ago https://lnkd.in/gx7VU8aA Meta is not a dark horse in AI anymore. They might be the top dog. https://lnkd.in/gRRJYuxt #llama2 #llama3 #llm #chatgpt #gpt4 #ai #ml

16

5 Comments

Introducing RouteLLM: a sophisticated routing framework developed with Berkeley-LMSys in collaboration with Anyscale. RouteLLM optimizes query handling by dynamically selecting between high-performance proprietary LLMs and cost-effective open-source models, cutting costs by over 2x without sacrificing quality. Using human preference data and LLM-as-a-judge for data augmentation, our routers evaluate query complexity to choose the appropriate model. Rigorous testing on benchmarks like MMLU and GSM8K confirms our cost-efficient, high-quality performance. Explore our open-source code, models, and preference data on GitHub: https://lnkd.in/gJyijZRx, and try our online demo: https://lnkd.in/gME-GC2a Explore More about RouteLLM and learn how it can revolutionize your LLM applications: Read our Blog here: https://lnkd.in/gfPD-u-y LLMsys Blog here: https://lnkd.in/ga_MgERE Full research paper here: https://lnkd.in/gqRy7Pjy

80

1 Comment