Mitodru Niyogi

Prometheus-2 (Congratulations to our chief scientist Minjoon Seo) aims to overcome the limitations of existing open evaluator LMs by addressing two critical shortcomings: (1) divergent scoring from human judgments and (2) the lack of flexibility to perform both direct assessment and pairwise ranking, which are prevalent evaluation methods. The model achieves high correlation and agreement with humans and proprietary LM judges on various direct assessment and pairwise ranking benchmarks, outperforming other tested open evaluator LMs. The availability of Prometheus-2 promotes transparency, fairness, and accessibility in the field of language model evaluation. Do you know what's even more important? The evaluation of multimodal models. Stay tuned for what the research team at Twelve Labs is cooking up in this area 😉

51

1 Comment

Introducing RouteLLM: a sophisticated routing framework developed with Berkeley-LMSys in collaboration with Anyscale. RouteLLM optimizes query handling by dynamically selecting between high-performance proprietary LLMs and cost-effective open-source models, cutting costs by over 2x without sacrificing quality. Using human preference data and LLM-as-a-judge for data augmentation, our routers evaluate query complexity to choose the appropriate model. Rigorous testing on benchmarks like MMLU and GSM8K confirms our cost-efficient, high-quality performance. Explore our open-source code, models, and preference data on GitHub: https://lnkd.in/gJyijZRx, and try our online demo: https://lnkd.in/gME-GC2a Explore More about RouteLLM and learn how it can revolutionize your LLM applications: Read our Blog here: https://lnkd.in/gfPD-u-y LLMsys Blog here: https://lnkd.in/ga_MgERE Full research paper here: https://lnkd.in/gqRy7Pjy

80

1 Comment

Unlock the potential of edge data. Check out Sai Prashant Vajja's session from DevoxxUK and discover how to train ML models using data at the edge while preserving user privacy. Learn about transfer and federated learning with a practical example involving cute dogs! View talk @ devoxx.co.uk/talk?id=230

4

🚀 Research Paper Highlights: Let's find out impact of full fine tuning on LLMs! Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? by Zorik Gekhman et al from Israel Institute of Technology and Google Research. 1. Acquiring New Knowledge and Hallucinations: This study investigates how fine-tuning LLMs with new factual knowledge affects their tendency to generate hallucinations relative to their pre-existing knowledge. When LLMs undergo supervised fine-tuning, they encounter new information not seen during pre-training, potentially leading to the generation of factually incorrect responses. 🚫 2. Challenges and Risks: The research highlights that LLMs struggle to acquire new knowledge through fine-tuning, learning it significantly slower than pre-existing information. Additionally, this process increases the model’s propensity for hallucinations, underscoring the risks of introducing new factual knowledge ⚠️ and the critical role of pre-training for knowledge acquisition. 3. Knowledge Assessment with SliCK: To evaluate the impact of new knowledge, the SliCK framework categorizes information into Known and Unknown types. Known examples are further divided into HighlyKnown, MaybeKnown, and WeaklyKnown. This hierarchical method facilitates a controlled study on closed-book question answering (QA), assessing the model's ability to integrate new information. 4. Practical Mitigation Strategies: Early-stopping can significantly reduce the risk of hallucinations, which typically emerge during later training stages. Alternatively, filtering out Unknown fine-tuning examples effectively minimizes overfitting risk without compromising performance. ✅ In conclusion, LLMs struggle to integrate new factual knowledge through fine-tuning, suggesting that fine-tuning is less effective for introducing new information. This emphasizes the importance of robust pre-training for factual knowledge acquisition. 💡 Further reading- https://lnkd.in/e4FZQ7SR 🌟 Stay tuned for more updates on upcoming research and analysis in this rapidly evolving landscape of Generative AI. #LLMs #finetuning #hallucinations #innovation #research

8

Here's a snippet demonstrating how you can use Mirascope to make your LLMs take conditional actions based on the classification of an input. Check out the full example at https://lnkd.in/eV8wE5C2

11

2 Comments

Separating Fact from Logic: Test of Time ToT Benchmark Isolates Reasoning Skills in LLMs for Improved Temporal Understanding Quick read: https://lnkd.in/gbc5CqN8 Paper: https://lnkd.in/giu72CwW HF Page: https://lnkd.in/g7H6dgY9

17

A Survey of RAG and RAU: Advancing Natural Language Processing with Retrieval-Augmented Language Models Quick read: https://lnkd.in/gnGqk44W Paper: https://lnkd.in/g8-VftbX #llms #largelanguagemodels #ai

20

Exploring Offline Reinforcement Learning RL: Offering Practical Advice for Domain-Specific Practitioners and Future Algorithm Development Data-driven methods that convert offline datasets of prior experiences into policies are a key way to solve control problems in various fields. There are mainly two approaches for learning policies from offline data, imitation learning and offline r... https://lnkd.in/eAPbG2DZ #AI #ML #Automation

1

🤔 Facing challenges with traditional Retrieval-Augmented Generation systems in your LLM projects? 🚀 Introducing GraphRAG: A Solution to Power Up LLMs with Knowledge Graphs: https://lnkd.in/gUpyKgSy Traditional retrieval systems often struggle with the complexity and volume of data, leading to suboptimal integration of information into LLMs. GraphRAG, developed by Microsoft Research, revolutionizes this by leveraging knowledge graphs to enhance data structuring, relevance, and retrieval efficiency. 📅 Join us at the DataHack Summit 2024 for an in-depth session: Topic: "GraphRAG: Powering Up LLMs with Knowledge Graphs" The speaker of this session will be Siddhant Agarwal, Developer Relations Lead - APAC at Neo4j. In this session, Siddhant will explore how GraphRAG addresses these challenges by: ➡️ Creating Knowledge Graphs: Extract entities and relationships to form a structured knowledge base. ➡️ Semantic Clustering: Organize the graph into semantic communities for better data cohesion. ➡️ Enhanced Query Processing: Generate comprehensive, diverse answers relevant to user queries. #AnalyticsVidhya #DataHackSummit2024 #DHS2024

132

4 Comments

Advancing Ethical AI: Preference Matching Reinforcement Learning from Human Feedback RLHF for Aligning LLMs with Human Preferences Quick read: https://lnkd.in/gr-238BN Paper: https://lnkd.in/gJddxRws

15

Now on Arxiv: Linsen Li, Pratyush Anand and Kaiming He tackle real-time estimation & control in quantum systems using reinforcement learning and Transformer models. Their theoretical research acieves: 1. Quantum Resource Scheduling with Monte Carlo Simulation: Formulating the scheduling problem in a digitized environment, allowed them to develop a rule-based greedy heuristic that significantly outperforms random scheduling baselines. 2. Reinforcement Learning & Transformer-on-QuPairs Framework: By training reinforcement learning agents in an interactive probabilistic environment, they utilized a self-attention mechanism on sequential information of inhomogeneous qubit pairs. The Transformer-on-QuPairs scheduler achieved over 3x improvement compared to rule-based methods! 3. Scalable & Versatile Framework: This approach is scalable, handling larger sets of input variables and supporting longer sequences as the number of qubits increases. This paves the way for enhanced co-design of physical and control systems in quantum communication, networking, and computing. This RL-based framework for system optimization is versatile, applicable to various large-scale quantum systems, and maximizes the effectiveness of existing hardware platforms. Read the full paper here: https://lnkd.in/gjVuZ3vM Thank you Dirk Englund, MIT Research Laboratory of Electronics, MITRE and National Science Foundation (NSF)! #QuantumComputing #ReinforcementLearning #AI #QuantumScience #Innovation #Research #QuantumTechnology

8

Fine-tuning LLMs with Reinforcement Learning from Human Feedback This blog post explores how Reinforcement Learning from Human Feedback (RLHF) can be used to fine-tune Large Language Models (LLMs) to align with human preferences. It delves into the concept of Proximal Policy Optimization (PPO) and how it is used in this context. The key takeaway is that RLHF offers a powerful framework to train LLMs that can produce human-aligned text, while still maintaining creativity. This paves the way for more natural and nuanced language generation in the future of AI and natural language understanding. Read the article: https://lnkd.in/d9EGf579 #AI #machinelearning #goML

8

Unlock the potential of Meta’s Llama3 for biotech applications with torchtune, a PyTorch-native library that simplifies fine-tuning of large language models. Whether you're looking to enhance drug discovery, genetic analysis, or scientific writing, our latest blog provides a guide to leverage this powerful tool effectively. 🔗 Learn more here: https://lnkd.in/g9WnrNSN #Biotechnology #Llama3 #torchtune #AI #MachineLearning

6

Researchers at CMU Introduce TriForce: A Hierarchical Speculative Decoding AI System that is Scalable to Long Sequence Generation Quick read: https://lnkd.in/gr_he4kw Researchers from Carnegie Mellon University and Meta AI (FAIR) Introduce TriForce, a hierarchical speculative decoding system designed for scalable long sequence generation. TriForce utilizes the original model weights and dynamic sparse KV cache via retrieval as a draft model, serving as an intermediate layer in the hierarchy. Maintaining the full cache allows for superior KV cache selection using retrieval-based drafting, characterized as lossless compared to eviction-based methods like StreamingLLM and H2O. The hierarchical system addresses dual memory bottlenecks, pairing a lightweight model with a StreamingLLM cache for initial speculations to reduce drafting latency and accelerate end-to-end inference. TriForce introduces a hierarchical speculative decoding system with retrieval-based KV cache selection. The hierarchical system addresses dual bottlenecks, enhancing speed-up. Retrieval-based drafting segments the KV cache, highlighting relevant information. Lightweight models with StreamingLLM cache accelerate initial speculations, reducing drafting latency. TriForce utilizes model weights and KV cache to enhance LLM inference speed for long sequences. The implementation utilizes Transformers, FlashAttention, and PyTorch CUDA graphs, maintaining full layer sparsity while minimizing kernel launching overhead. Paper: https://lnkd.in/ga45hspw Machine Learning Department at CMU

26

🎤 Submit to speak at THE event covering cutting-edge open source machine learning framework - PyTorch Conference comes to San Francisco September 18-19! We're looking for experts on: 🌱 PyTorch Framework & Ecosystem 📚 Deep Learning Applications 💾 Software Applications ✏️ Training & Inference ♻️ Model Lifecycle & Management 🔏 Ethical, Social, & Regulatory Aspects 🤖 AI in Industries See alll suggested topics and submit your talk by June 7: https://hubs.la/Q02t4Pkc0. 🎟️ Register by April 26 & save US$350: https://hubs.la/Q02t4Nnh0. #PyTorchConf #PyTorch #PyTorchFoundation #AI #MachineLearning #ML #DeepLearning #OpenSource #OpenSourceSoftware #OpenSourceDevelopment #OpenSourceCommunity #OSS #LinuxFoundation #events #linux #software

63

2 Comments

🚀 Check out the latest Medium post by our student Tianyunxi Yin: "Quantization Fundamentals for Model Compression". Explore the essentials of quantization and learn how it helps in compressing machine learning models for better efficiency without compromising performance. Read more here: https://lnkd.in/gHbbR_J4 #USFMDS #DataScience #MachineLearning #ModelCompression #Quantization

16

Our code for RAG evaluation is now publicly available at Amazon Science website: https://lnkd.in/g-HKUNZZ. This is the implementation of our evaluation mechanism performed by scoring the RAG on an automatically-generated synthetic exam composed of multiple choice questions based on the corpus of documents associated with the task. #OpenSource #OpenSourcing #AmazonScience #ICML2024 #Evaluation #LLMEvaluation

213

6 Comments