Roger Barga

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

A really awesome blog from Celebal Technologies showing the power of enhancing RAG with fine-tuning. This is how you build customized AI using compound systems! https://lnkd.in/gp4z3zTa

216

5 Comments

🔍 Azure introduces Model monitoring for generative AI applications! 🌟 Now available in preview, this feature supports key metrics including: - Groundedness: evaluating alignment with input information - Relevance: assessing relevance to given questions - Coherence: ensuring smooth output flow and human-like language - Fluency: measuring language proficiency and correctness - Similarity: comparing generated text to ground truth For detailed insights on each metric and their use cases, check out the link below! 👇 [Link to Azure Model Monitoring for Generative AI Applications preview](https://lnkd.in/dsbhGdqg)

4

"Smaller is better." There are now Small Language Models (SLMs) that are “smaller” in size compared to LLMs. SLMs are trained on 10s of billions of parameters, while LLMs are trained on 100s of billions of parameters. More importantly, SLMs are trained on data pertaining to a specific domain. They might not have broad contextual information, but they perform very well in their chosen domain -- write Iyengar Ashok and Praneet Adusumilli in this IBM blog. One area that has not seen much innovation is at the far edge and on constrained devices. However, there are smaller models that have the potential to innovate gen AI capabilities on mobile devices. Enterprises that operate in specialized domains, like telcos or healthcare or oil and gas companies, have a laser focus. While they can and do benefit from typical gen AI scenarios and use cases, they would be better served with smaller models. Because of their smaller size, these models can be hosted in an enterprise’s data center instead of the cloud. SLMs might even run on a single GPU chip at scale, saving thousands of dollars in annual computing costs. While running SLMs on-premises seems practical and tiny LLMs on mobile edge devices are enticing, what if the model requires a larger corpus of data to respond to some prompts? Hybrid cloud computing offers the best of both worlds. Might the same be applied to AI models? Click below to read the article and find out! #LLM #SLM #LargeLanguageModel #SmallLanguageModel   https://lnkd.in/gFzyHhfH

13

1 Comment

There's a big difference between Database Modeling and Semantic Layers- How would you explain it? Even though they look alike, the underlying meaning is vastly different. ▶ The type of data modeling you would do with database modelers like SqlDBM and semantic layers are interrelated in that we help design and organize database structures in a way that supports the creation of a semantic layer. However: ▶ A database modeling solution is focused on the 𝐩𝐡𝐲𝐬𝐢𝐜𝐚𝐥 and 𝐥𝐨𝐠𝐢𝐜𝐚𝐥 aspects of database modeling, ensuring the data structures align with technical requirements and business concepts. ▶ A semantic layer operates at a higher level of abstraction often within 𝐁𝐈 𝐭𝐨𝐨𝐥𝐬 such as PowerBI, Looker, ThoughtSpot, and Tableau. It maps the technical details of data that were designed by tools like SqlDBM or not properly designed at all, to business-friendly terms and concepts. ▶A tool like SqlDBM will essentially lay the 𝐟𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧𝐚𝐥 𝐝𝐚𝐭𝐚 𝐚𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞 that a semantic layer translates into an easily navigable and understandable format for end-users. ---- "Enabling non-technical business users to perform data analysis without needing to understand complex database queries or structures. However, leveraging a data modeling tool and having a good data model will make the complexity of building and running a semantic layer more manageable and efficient." - Keith Belanger, 𝘚𝘲𝘭𝘋𝘉𝘔 𝘗𝘳𝘰𝘥𝘶𝘤𝘵 𝘌𝘷𝘢𝘯𝘨𝘦𝘭𝘪𝘴𝘵 & 𝘚𝘯𝘰𝘸𝘧𝘭𝘢𝘬𝘦 𝘋𝘢𝘵𝘢 𝘚𝘶𝘱𝘦𝘳𝘩𝘦𝘳𝘰 ---- This is why you hear someone say "We do our data modeling in Power BI" - they are modeling their 𝐬𝐞𝐦𝐚𝐧𝐭𝐢𝐜 𝐥𝐚𝐲𝐞𝐫 in Power BI. This is an image from Power BI, if it had Dark Mode it would look even more similar! #DataModeling #DataArchitecture #SqlDBM

55

1 Comment

Interesting post by João Salgado about partitioning vs. sharding in #Postgres which lists trade-offs for each. It's great looking at the list of cons for partitioning and seeing that #TimescaleDB actually helps with most of them - Complexity ⬇️ because we transparently manage partition creation and lifecycle for you as data is ingested - Optimization Challenges ⬇️ because we have planner hooks that help with handling data exclusion (you still do need to pay attention to your partitioning key though). - Storage Overhead ⬇️ (although Postgres doesn't really have much overhead for partitions) because you can just choose to compress your data (we often see a 95x reduction in size). Timescale 🚀

18

1 Comment

When I speak to customers on AI, they often ask where do you even start? This was a great conversation of matching a project to the AI resources. We also dive a bit into what it takes to build an AI Cloud

16

Large language models (LLMs) have great potential to improve various tasks. Explores different approaches to leverage your data with LLMs, including prompt engineering, retrieval-augmented generation (RAG), supervised fine-tuning, reinforcement learning from human feedback (RLHF), and distillation. https://lnkd.in/g5thXecU The best method depends on your specific needs, such as data availability and desired level of control. You can even combine these methods for optimal results! Experiment on Google Cloud's Vertex AI to find the best approach for your application. #AI #MachineLearning #GoogleCloud #VertexAI #finetuning #rag #sft #rlhf

8

This is interesting! Now, PostgreSQL functions (and procedures) can be written in something called PRQL: see https://prql-lang.org/ PRQL is some kind of modern declarative data language that compiles directly to SQL. Here's how that looks like in PostgreSQL: https://lnkd.in/d5UXQwsw create function match_stats(int) returns table(player text, kd_ratio float) as $$ from matches filter match_id == $1 group player ( aggregate { total_kills = sum kills, total_deaths = sum deaths } ) filter total_deaths > 0 derive kd_ratio = total_kills / total_deaths select { player, kd_ratio } $$ language plprql; Note that select part comes last, and from first, probably to utilize IDE capabilities batter. In any case the list of languages that can be used to write PostgreSQL functions (and procedures) is as follows: - SQL - plpgSQL (PostgreSQL procedural SQL) - This new PRQL thing I just mentioned. - C and C++ - JavaScript - Python - Rust - C# - GO (I think, probably) And that is only that I'm aware of, probably even more. And then you can use NpgsqlRest client (download here: https://lnkd.in/d3UqP_Sq) to turn those functions (and procedures) into REST endpoint with one command. I'm just going to say it: PostgreSQL functions (and procedures) are what microservices were meant to be.

19

21 Comments

Exciting news! From the White House: "Microsoft will invest $3.3 billion in a new datacenter. Microsoft is committed to creating good-paying jobs with growth pathways and comprehensive benefits such as paid family leave. Microsoft will pair its datacenter investment with a commitment to investing in innovation and workforce in Racine and statewide. Microsoft will partner with Gateway Technical College to develop a Datacenter Academy that trains 1,000 Wisconsinites for datacenter and STEM roles by 2030, and will employ up to 2,000 people in permanent roles at its Racine facility. And, Microsoft will build a Co-Innovation Lab in southeast Wisconsin, as well as team up with the startup accelerator Gener8tor to train 1,000 business leaders to adopt AI in their operations, so that Wisconsin manufacturers can seize AI’s promise. Together, these investments will position the industrial heartland to lead the way in industries of the future."

88

2 Comments

Just in time for our Global Modeling Webinar tomorrow, 9 a.m. PST! Learn how collaboration will fast track your modeling practices🌎 Link in comments! Featuring: T.y. Li 🌟 #sqldbm #datamodeling #cloud #snowflake

17

6 Comments

🚀 Elasticsearch - Enhancing Retrieval-Augmented Generation (RAG) with Advanced Chunking! 🧠 My latest blog on intelligent RAG and how to use Elasticsearch Vector Database for advanced chunking to improve LLM accuracy. Learn about fetching surrounding chunks (N-1, N+1) for better context. The blog includes a complete end-to-end example via a #google Colab notebook. #elasticsearch #vectorSearch #llm #ai #machinelearning #vectordb

51

1 Comment

We've got some exciting announcements today with Anthropic 🎉 Take a look: 📣 Claude 3 Opus, Anthropic’s most intelligent model, is now generally available on Vertex AI. ⚙️ Tool use is available for the Claude 3 model family on Vertex AI, enabling Claude to act as an autonomous agent to handle tasks requiring real-time data or complex computations. 📈 Provisioned throughput is available for the Claude 3 model family on Vertex AI to get assured performance and predictable costs for your production workloads. Learn how to get started → https://goo.gle/4aHWL7U

6

There is some confusion in the industry about causal reasoning versus other methods involved in observability. It's nuanced, but also clearly differentiated. If you want to see it in action, here's a blog on it https://lnkd.in/dk9xn33K

7

1 Comment

Great post by Celebal Technologies on implementing RAFT on top of Databricks fine-tuning to out-perform RAG: https://lnkd.in/gqfjeMTV We think this is a key tool for building compound AI systems that get the best possible quality on *your* data -- mixing the best of retrieval, fine-tuning and other AI techniques to maximize application quality. RAFT paper: https://lnkd.in/g3vg5tpw

169

4 Comments

Imagine seamlessly ingesting incremental data from *any* API right into #DeltaLake via Structured #streaming. This article by Irfan Elahi goes deep into the new Custom Data Source API in #apachespark, showing how you can create your own data source for streaming - with checkpointing capabilities to achieve exactly-once guarantees! https://lnkd.in/gEvr5kQJ

17

Microsoft's AI model, Aurora, quickly predicts levels of six major air pollutants worldwide, including carbon monoxide and particulate matter. Read more in this article from Nature. #msftadvocate #ai https://lnkd.in/g-AppzdB

18