Jiri Dobes

💡 For several years, generative AI has been used by Lizeo R&D to analyse and understand texts. Currently, AIs such as GPT-4, Llama-3, or Mistral-8x7b are coupled with our own algorithms to automatically extract and qualify the topics of a text.🤖 As Stéphane Bonnevay, Senior Data Scientist at Lizeo pointed out, «the continuous evolution of Large Language Models presents both challenges and opportunities. We are committed to testing and comparing new models to ensure our tools remain at the cutting edge.» 🔬 #AI is at the heart of our R&D. To find out more, visit 👉 https://lnkd.in/diZZRzD8 ______ Context of the chart: Comparison of these models on a set of textual data compared with a human analysis. #generativeai #artificialintelligence #llm #gpt #llama #mixtral

18

Happening July 11 at Arize:Observe: Sandeep Subramanian from Mistral AI will be talking about CodeStral–LLMs for code... This talk will cover some of the important aspects of building LLMs for code. Specifically, it will discuss the ingredients of CodeStral-22B, a code model from MistralAI that is trained with FIM, allowing it to act as a "tab-complete" model as well as a chat model. The Research Track at Arize:Observe covers cutting-edge research, emerging techniques, and theoretical advancements in AI observability and LLM evaluation. Explore the latest findings and methodologies pushing the boundaries of what’s possible in understanding, evaluating, and improving AI-powered systems. Join us SHACK15 on July 11 in SF for latest cutting-edge research, real-world insights from AI engineers, and useful workshops to evaluate and troubleshoot AI in production. See more speakers and register: https://lnkd.in/gpEitQb5

68

6 komentářů

☝️ From tips 1-5, there’s an assumption that the model is fixed, but it gets even more complicated when you can change the model. On Day 6, we’re exploring model level optimizations, which include any method that changes the model or architecture itself. Since neural networks often contain many redundancies, it’s essential to exploit or streamline these redundancies to accelerate the inference process. What can you do to optimize inference at the model level?   𝐒𝐢𝐱𝐭𝐡 𝐭𝐢𝐩: 𝐂𝐨𝐦𝐩𝐫𝐞𝐬𝐬 𝐦𝐨𝐝𝐞𝐥𝐬 𝐰𝐢𝐭𝐡 𝐦𝐨𝐝𝐞𝐥 𝐝𝐢𝐬𝐭𝐢𝐥𝐥𝐚𝐭𝐢𝐨𝐧. 🧑‍🏫 Widely applicable in various AI domains, including NLP, speech recognition, visual recognition, and recommendation systems, model distillation is a training technique that trains small models to be as accurate as larger models by transferring knowledge. 🔤 A classic example of model compression can be seen in various BERT models that employ knowledge distillation to compress their large deep models into lightweight versions of BERT. For instance, BERT-PKD, TinyBERT, DistilBERT, and BERT-BiLSTM-based model compression techniques solve multilingual tasks using lightweight language models. – What’s the #11DaysofInferenceAccelerationTechniques? The Deci team is posting, for 11 days, a series of inference acceleration techniques for deep learning applications. If you’re looking for practical tips and best practices for improving inference, follow Deci AI so you won’t miss an update. #deeplearning #machinelearning #neuralnetworks #computervision

14

New chapter out! You can now access Chapter 16, “Ownership and maintenance” of Machine Learning System Design by Valerii Babushkin and Arseny Kravchenko. #Ml #MLSystems #ManningBooks

11

Automatically creating a knowledge graph with LLMs is hard - one issue is you end up with a lot of duplicates. Here’s a seriously cool Cypher snippet by Tomaz Bratanic and others at Neo4j that performs entity deduplication for you, using a combination of text embeddings and word distance.

2

I'm starting a new series where I break down new upcoming CV models and explain why they're special/different! More detailed explanation in the comments on how Florence-2 actually works thanks to Piotr Skalski and Roboflow!

84

5 komentářů

Kolmogorov-Arnold Networks look interesting and explainable. But weren’t we doing this already in engineering school? I mean curve matching and fitting. The innovation here is that they are doing it on a interrelated multiple equation basis, where the “constants” (activation functions) are being changed dynamically, which is pretty neat. Definitely will be more explainable than multilayer perceptrons or simple neural networks. KANs “explainable” nature is equivalent to explain why the constants in the equations are what they are. In other words, this is not the path to AGI, or discovering news laws of physics either, although the method is elegant. In my opinion, where the “gold” resides in Kolmogorov-Arnold Networks is this: Being trained with reasonable good data and then use the trained network to (i) predict a more accurate output when any of inputs is out of whack; (ii) discard inputs that are out of range; (iii) qualify the inputs before a prediction is applied; (iv) get a notion of the extend at which the inputs are diverging from the ground truth One case that comes to my mind is Virtual Flow Meters where data (inputs) coming from well tests is very inconsistent due to uncertainties ocurring during measurement. In the abstract the authors got carried away a bit: " KANs are shown to be useful collaborators helping scientists re-discover mathematical and physical laws." It seems incorrect to me. You don’t rediscover the laws of physics with this ML algorithm. You get a hint of what external factors could be affecting the input data that make the output deviate from the laws of physics. #machineLearning #physics #artificialIntelligence #AI #AGI #spe #petroleumEngineering #VirtualFlowMeters #dataScience

47

10 komentářů

"Learn Your Reference Model for Real Good Alignment," introduces Trust Region Direct Preference Optimization (TR-DPO). Developed by a visionary team led by Alexey Gorbatovski at Tinkoff, this paper pioneers a method that significantly advances the training of language models by updating the reference model during the training process itself. Why This Matters: ➡️ Enhanced Model Alignment: TR-DPO introduces dynamic updates to the reference model, which allow for more accurate and adaptive training compared to the traditional Direct Preference Optimization (DPO). ➡️ Improved Performance Metrics: Through rigorous testing, TR-DPO has demonstrated a 19% improvement in performance over DPO, making it a groundbreaking tool for achieving higher coherence, correctness, and detail in model outputs. Key Insights: ➡️ Dynamic Reference Model Updates: Unlike static methods, TR-DPO continuously adjusts the reference model during training, which keeps the model relevant and highly effective across changing data sets and conditions. ➡️ Human-Centric Results: The model notably excels in producing outputs that are coherent, detailed, helpful, and harmless—key qualities for any application from chatbots to complex decision-making systems. #analyticsvidhya #datascience #generativeai #researchpaper

16

There is a lot of conversation on X about how llms just aren't intelligent enough. A new paper, "Can language models be used as world simulators?" on Arxiv. The paper asks the question: can current llms accurately serve as world simulators by accurately predicting how actions change world states? it findings are that we are some distance away from skynet type scenarios. current got 4 for example is only ~60% accurate at simulating state changes based on common-sense tasks, like boiling water. #genai #ai

1

1 komentář

🚀 Want to employ the power of #PyTorch for your scientific models without a steep learning curve? 👀 Check out our latest blog, where Travis Oliphant discusses how PyTorch's integration with the Array API makes it easy to use with familiar libraries like NumPy and scikit-learn.🔥 https://lnkd.in/dDRkEY3v #Innovation

10

Does Fine-tuning Increase Hallucinations? Perhaps... From this paper: https://lnkd.in/gMZUUwVg "We demonstrate that large language models struggle to acquire new factual knowledge through fine-tuning, as fine-tuning examples that introduce new knowledge are learned significantly slower than those consistent with the model's knowledge." Okay, this is what we expect, more or less. And then there's this: 👉 "However, we also find that as the examples with new knowledge are eventually learned, they linearly increase the model's tendency to hallucinate" Uh-oh - fine-tuning can increase hallucinations! What's the takeaway? Anyone paying attention to LLMs should realize by now that your evaluation strategy and effective search for an optimal solution is vital to success. Following a formula, like "RAG + Fine-tuning = My-domain LLM" is merely a starting point. Experiments have consistently shown that "out of the box", such formulations can perform very badly even though they give the impression, upon initial inspection, of being incredibly powerful. Don't be fooled by "demo wow-factor-ness"!

2

🚀 New video alert: Evaluating LLMs with MLflow by Miloš Švaňa 🚀 In the journey to integrate LLM functionality into our applications, we face an array of options - GPT, Gemini, Claude, or even open-source models from Hugging Face. But how do we evaluate these diverse solutions? That's exactly what Miloš explores in our latest video! 📌 Key Highlights: - Using similarity to evaluate different LLMs - Leveraging MLflow to track experiments and compare different LLM solutions In this video, we dive into a specific task: using an LLM to analyze HTML and suggest a title. We evaluate 3 different models to see how well they perform this task. 🙂 Curious to learn more? Watch the full video ➡ https://lnkd.in/exdsHn9e #MachineLearning #LLMs #languagemodels #MLflow #AI #TechInnovation #ExperimentTracking #ModelEvaluation #profiq #techresearch

6

💡 GPU? Lazy eval? No problem! 💻 Write dataframe code that works beautifully across pandas, Polars, cuDF & more with Narwhals. 🔥 Check out our in-depth blog, by Marco Gorelli on making data science more interoperable: https://lnkd.in/dedkSPPy #Polars #DataFrame #Pandas #Narwhals

14

Learn the basics of nonlinear dimensionality reduction, kernel PCA, and multidimensional scaling by following along Biman Chakraborty's clear, hands-on Python tutorial.

18

Most people are aware that transformers (the "T" in "GPT") are effectively next-token sequence predictors. They have been trained on long sequences of tokens that we call "paragraphs" and "documents," and they use that training to predict the next most likely tokens from a series we provide (a question, an instruction, or a story). When you ask ChatGPT, "Who was the 40th President of the United States," it will respond with, "Ronald Reagan was the 40th President of the United States" in the same way that if you sing the alphabet melody "A-B-C-D-E-F-G," your favorite kindergartner will respond with "H-I-J-K-LMNO-P." The notion of a transformer as a context-aware sequence predictor isn't just useful for text sequences, and tokens are not limited to short letter sequences. Tokens can actually be any kind of data structure that can be represented with a consistent vector embedding of high dimensionality, like a data frame of unlabeled values. It turns out that this category of sequence predictors nicely models multi-variate time sequences, and that rather than building a model for every kind of data you might want to predict, you can use transformers to predict time series (like of seasonally-dependent beach house rental prices) as general-purpose predictors trained on a variety of time series data. The research behind MOMENT rests on a couple of earlier papers, including "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers," which introduced the novel idea that time series forecasting with transformers can be made both more efficient and more interesting if you group multivariate events into shorter sequences (usually called "patches") that bundle parts of the sequence together that—for example—might represent a day's worth of measurements across all variables. Your transformer then predicts the following "tokens"—what the next day's worth of events would look like—rather than what the next 24 tokens might be if your sampling rate was once per hour. This does two things: 1️⃣ It improves the efficiency of your prediction over the same span of time. So in this example, if you are predicting 7 days of measurements, you generate only 7 tokens, not 168. 2️⃣ It binds observations together that occur during the context of the patch that are probably tightly related. If, for example, you are forecasting outside temperature for the next 10 days, chances are the hourly measurements on any given day are already contextually related; grouping them trains the model on that context. What's revolutionary about using large and diverse training sets in this way to teach a transformer to predict things like beach house rental prices is that you don't need to specifically teach it about the seasons, inflation, or the economics of beach front property. It has learned those concepts implicitly, and applies them to any time series data it is given—sometimes, more reliably than custom models built by data scientists on those specific series!

6

"Alignment of MPNNs and Transformers" Bao Nguyen, Anjana Yodaiken, and Petar Veličković "As the complexity of machine learning (ML) model architectures increases, it is important to understand to what degree simpler and more efficient architectures can approximate the same hypothesis space. In this paper, we investigate the degree to which a Message Passing Neural Network (MPNN) can operate the same way as a Graph Transformer. We do this by training an MPNN to align with the embeddings of a Relational Transformer (RT). We explore variations of the standard MPNN and assess the impact of different components on the degree of alignment. Our findings suggest that an MPNN can align to RT and the most important components that affect the alignment are the MPNN permutation invariant aggregation function, virtual node and layer normalisation." Paper: https://lnkd.in/dZ5MuErd #graphneuralnetworks #transformers #attention

32

1 komentář

Parsing structured data from LLMs can be frustrating for anything beyond toy problems. I recently compared three frameworks: Instructor, Fructose, and Langchain, on three problems that are more complex than the normal toy examples in tutorials. https://lnkd.in/eJCNndyj 1. Extreme Multilabel Classification with 60 classes 2. 3-level deeply nested Named Entity Recognition 3. 3-level deeply nested Synthetic Data Generation Recently, Khuyen Tran shared about Mirascope, a relatively new framework for structured data parsing: https://lnkd.in/eE6MssQ6, so I had to test it out using the same methodology I had used to crown Fructose as the best out-of-the-box framework for these tasks. Unfortunately, Mirascope did not perform as well as Fructose. Details below. Following the documentation on the respective frameworks, I ran 20 runs of the same input and collected some metrics, as shown in the picture. - Instructor: Has concise code, uses Pydantic for data structure definitions, has good latencies, and is excellent for extreme multilabel classification tasks. - Mirascope: Has slightly more verbose code but not by much, and it also uses Pydantic for data structure definitions. However, it has comparable or worse accuracy than Instructor with generally worse latency. - Fructose: Has concise code, generally good latencies (except for synthetic data generation) and is excellent for all three complex tasks. However, it uses Dataclasses, which I’m not a big fan of. - Langchain: I want to love Langchain, but it struggles out of the box with more complex problems like nested fields. Also, debugging is a challenge with LCEL, as I couldn’t figure out how to place breakpoints. The documentation is also vague and broken at times. In conclusion, ✅ Fructose is still the most well-rounded out-of-the-box option for Multilabel Classification, deeply-nested NER, or Synthetic data generation. ✅ Instructor is good, too, if you want to use Pydantic instead of Dataclasses. ❌ Mirascope is comparable to or worse than Instructor for these tasks, so I’d stick to Instructor if I needed to use Pydantic instead of the Dataclasses that Fructose requires. ❌ I don’t recommend Langchain for these tasks. 🔗 Medium friend link to all the code and evaluation methodology: https://lnkd.in/gUcWrZNx 🌟 Fructose Github: https://lnkd.in/gadEKyNZ 🌟 Instructor Github: https://lnkd.in/g__dT2Gr 🌟 Mirascope Github: https://lnkd.in/efp9FDNH Follow me for more tips on building successful ML and LLM products! Medium: https://lnkd.in/g2jAJn5 X: https://lnkd.in/g_JbKEkM #generativeai #llm #nlp #artificialintelligence #mlops #llmops

147

19 komentářů