Marc Planagumà i Valls

How I import very wide CSV data into SQL Server, without using the headache-inducing wizard! https://lnkd.in/eKRiX9-p

5

According to Anthropic's announcement, Claude 3.5 surpassed all the competitors' top-tier models. But does it actually perform better in an actual use case in our hands? Here is my quick experiment using LangChain's Agent framework. #llm #claude #gpt #genAI #langchain

13

1 comentario

From D Hwang [Google LLC] (2024) FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information https://lnkd.in/ecCv_Ccs - Adam optimizer can be considered as an approximation of natural gradient descent through the use of diagonal empirical Fisher information matrix (FIM). - The diagonal FIM results in loss of covariance information but retains variance. This approximation seems to work well in practice. - The second momentum term in Adam is actually the diagonal empirical FIM. However, Adam uses the square root of FIM instead of the direct inverse. This is to get a more stable estimate. - The FIM relies on empirical approximations using training data rather than model distribution. This works better for discrete distributions (like text) than continuous (like images). - Some flaws in original Adam algorithm are uncovered based on the analysis. Proposed fixes are: use invariant natural gradient for momentum, adjust bias corrections, clip gradients. - Weight decay should be applied as a natural gradient too based on FIM. This explains the success of AdamW which decouples weight decay. - Overall modified algorithm is named Fisher Adam (FAdam) which shows strong performance on text, speech and image tasks. abs: This paper establishes a mathematical foundation for the Adam optimizer, elucidating its connection to natural gradient descent through Riemannian and information geometry. We rigorously analyze the diagonal empirical Fisher information matrix (FIM) in Adam, clarifying all detailed approximations and advocating for the use of log probability functions as loss, which should be based on discrete distributions, due to the limitations of empirical FIM. Our analysis uncovers flaws in the original Adam algorithm, leading to proposed corrections such as enhanced momentum calculations, adjusted bias corrections, and gradient clipping. We refine the weight decay term based on our theoretical framework. Our modified algorithm, Fisher Adam (FAdam), demonstrates superior performance across diverse domains including LLM, ASR, and VQ-VAE, achieving state-of-the-art results in ASR. #AI #LLM

24

From IBM Granite Code Models: A Family of Open Foundation Models for Code Intelligence https://lnkd.in/easN99Pz - Introduces Granite Code models, a series of code foundation models ranging from 3B to 34B parameters, for code intelligence tasks like code generation, fixing bugs, explaining code, etc. - Trained using a two-phase strategy: Phase 1 on 3-4 trillion tokens of 116 programming languages, Phase 2 on 500 billion tokens of code and natural language data. - Two main variants released: Granite Code Base (foundation models) and Granite Code Instruct (finetuned on instructions using curated data). - Evaluated extensively on diverse tasks like code generation, explanation, fixing, editing, translation across major programming languages. - Shows strong performance across tasks and languages, often exceeding recent models like CodeGemma, StarCoder2 despite being smaller. - Larger models like 34B can handle complex tasks like application modernization, while smaller ones suitable for on-device uses. - Advantages: versatile for enterprise software tasks, trained ethically on permissible data, released under Apache 2.0 license. abs; Large Language Models (LLMs) trained on code are revolutionizing the software development process. Increasingly, code LLMs are being integrated into software development environments to improve the productivity of human programmers, and LLM-based agents are beginning to show promise for handling complex tasks autonomously. Realizing the full potential of code LLMs requires a wide range of capabilities, including code generation, fixing bugs, explaining and documenting code, maintaining repositories, and more. In this work, we introduce the Granite series of decoder-only code models for code generative tasks, trained with code written in 116 programming languages. The Granite Code models family consists of models ranging in size from 3 to 34 billion parameters, suitable for applications ranging from complex application modernization tasks to on-device memory-constrained use cases. Evaluation on a comprehensive set of tasks demonstrates that Granite Code models consistently reaches state-of-the-art performance among available open-source code LLMs. The Granite Code model family was optimized for enterprise software development workflows and performs well across a range of coding tasks (e.g. code generation, fixing and explanation), making it a versatile all around code model. We release all our Granite Code models under an Apache 2.0 license for both research and commercial use.

19

Wanna know where Data teams keep getting it wrong after all these years? They keep boasting about their infrastructure, tooling, processes, how fast and reliable their technology, how beautiful and accurate their models. My (somewhat contrarian) view is that the Data teams that will win are all about their products and customer experience. Data leaders: it’s time to start really building great products and customer experiences. Learn to talk to customers, learn to build products that delight your users and become part of their lives, every day.

26

4 comentarios

Here’s the 2nd and final part to our series on building data products with snowflake and how to leverage a data developer platform to optimise compute with Animesh Kumar. Do give it a read. And do follow Modern Data 101 for more such articles. #DataProducts

12

1 comentario

🚀 New Article Alert! 🚀 I'm thrilled to share my latest article, "Open File Formats: Parquet"! 📄✨ In this article, I dive deep into the world of Parquet, an open-source, columnar storage format designed for efficient data processing. Whether you're a data engineer, analyst, or simply interested in data storage solutions, this piece will provide valuable insights into: 🔍 The advantages of using Parquet 📊 How it optimizes data storage and retrieval ⚙️ Practical applications and use cases with Python and Apache Arrow 📈 Performance benefits for large-scale data analytics Check it out to understand how Parquet can enhance your data workflows and drive better performance in your projects. https://lnkd.in/d6TYNaAa #DataScience #DataEngineering #BigData #Parquet #OpenSource #DataStorage #TechInsights #DataAnalytics #ApacheArrow #Data #Technology

6

"Databricks Ushers In New Era for Open #Data And #AI Governance With Unity Catalog", click to get the full story via the News Desk over on Elnion. https://lnkd.in/gDnArSJ5 #dataaisummit #data #datalake #ai #ml #gpt #genai #cloud #analytics #bi #datascience #datascientist

1

One of Generative AI’s greatest strengths is the ability to analyze new data provided to it. However, because of a limited context size there is lots of work of getting the right data to the AI at the right time. RAG has emerged as its own area of research and study. However, this doubles the problems that need to be solved by the AI team. First getting the right data to the model then having the model produce the answer. However, it looks like Google may have solved this problem. Google claims to give large language models (LLMs) the ability to work with text of infinite length.  https://lnkd.in/e4_aunK4 While only a research paper this space is moving so fast it wouldn’t surprise me to this implemented by someone in the next several months. Imagine the problems that can be solved by an AI that can keep entire data sets in its mind at one time. It’s an exciting time to be in this field.

4

Data Scientists, This is BIG! Snowflake's Arctic LLM is Here – and Open-Source! 🤯 Get your hands on it now: Interactive Demo: Explore Arctic's capabilities with this Streamlit app → https://lnkd.in/gJ7y38Ng Replicate: Ready to experiment? Find it on Replicate → https://lnkd.in/gyDXiMju Why This Matters: Arctic lets you quickly build demos of your models. This is a game-changer for showcasing your work and making your models more accessible! Huge thanks to Snowflake for making this happen! Are you excited to try Arctic? Let me know in the comments! 👇 #LLM #OpenSource #DataScience #Streamlit #Snowflake

8

Microsoft has published their GraphRAG repository. Repo: https://lnkd.in/dPma5iuj Description of their approach: https://lnkd.in/djt994Zv Azure accelerator: https://lnkd.in/dETv4uac Hierarchical Leiden algorithm: https://lnkd.in/dVVksyDk Gephi: https://gephi.org/ Note that a 45-minute deployment of an Azure RG to be able to use a GraphRAG API is extensive. Personally, I think it's worth taking the time to study the repo and emulate the approach, offering more control and flexibility, e.g., if you want to use Neo4j instead of CosmosDB. Also, note that entity resolution (by an LLM, this they plan to modify) is not enabled by default. It's a cool idea they've had to use the hierarchical Leiden algorithm for generating communities and subsequently an LLM for generating reports of those communities, to then apply a UMAP dimensionality reduction for a 2D visualization of the graph - for which they recommend using Gephi for displaying. It'll be exciting to follow the coming updates.

9

2 comentarios

👉🏼 Dual Causes Generation Assisted Model for Multimodal Aspect-Based Sentiment Classification 🤓 Rui Fan 👇🏻 https://lnkd.in/eC5m_4qv 🔍 Focus on data insights: - The proposed MDCA method tracks underlying causes behind expressed sentiments, providing reasoning cause (RC) and direct cause (DC) to explain emotions. - Utilizes large language models (LLMs) and visual-language models to construct MABSC datasets with RC and DC. - Implements a multitask learning framework leveraging cause data to train a generative model for sentiment prediction. 💡 Main outcomes and implications: - MDCA model achieves state-of-the-art performance in MABSC benchmark datasets. - Small fine-tuned model shows superior adaptability to MABSC compared to larger models like ChatGPT and BLIP-2. 📚 Field significance: - Advances in understanding sentiment analysis in multimodal data. - Enhances accuracy and interpretability of sentiment prediction models. 🗄️: #Multimodal #SentimentAnalysis #DataInsights #GenerativeModel

1

Did you miss this awesome demo Hugo Lu did with Youssef Mrini over at Databricks 🏘 In Orchestra you can trigger and monitor Databricks Workbooks, Workflows, data quality tests, and lots more. Our users also leverage Orchestra's overarching monitoring and alerting capabilities to link upstream jobs like data ingestion (e.g. ADF) through Databricks into downstream jobs like Reverse ELT with Hightouch or Power BI dataset refreshes. Check it out here 🔗 https://lnkd.in/dU2grpum #dataengineering #databricks

7

1 comentario

Our new philosophy for standardizing metadata has been established, marking the start of our journey towards FAIR data. #data #daphne4nfdi

45

3 comentarios

🌟 Exciting News in AI Forecasting! 🌟 Google Research has unveiled a groundbreaking approach to time series forecasting. Their new decoder-only foundation model is setting a new standard in accuracy and flexibility, capable of leveraging vast datasets across multiple domains seamlessly. This innovative model bypasses the need for domain-specific adjustments, offering a robust solution that adapts to various applications from finance to weather forecasting. Imagine the possibilities when these predictions become more precise and reliable! For professionals involved in data science, AI, and predictive analytics, this could mean more accurate forecasts, efficient resource management, and ultimately, better decision-making. Eager to get my hands dirty with it. Do you think that the future of forecasting will be in tied to the evolution of foundation models vs current approaches? #ArtificialIntelligence #Innovation #GoogleResearch #Forecasting #FoudnationModels

76

5 comentarios

Read how PyTest can be used to enhance #MLOps with data validation techniques. Ensuring the integrity and quality of data is fundamental to machine learning. In his latest article, our machine learning scientist Marcello Politi shows how to effectively use PyTest for data validation, focusing on both deterministic and non-deterministic testing strategies. This approach allows for comprehensive validation of datasets, ensuring that models can handle dynamic changes in data over time. ⬇ #machinelearning #deeplearning

1

Comprehensive Steps for Building LLM-Native Apps: From Initial Idea to Experimentation, Evaluation, and Productization. #llms #llmops #chatgpt4 #gemini #ai #ml #rag https://lnkd.in/da8uYKPf

4

Please check my first articles on Polars out on Medium. #pydata #datscience #pydata #python #rstats

12