Rick Wedgeworth

I presented about Apache XTable (Incubating) & how it brings interoperability to #lakehouse table formats at Dremio's Subsurface. Check out the recording of the talk in comments! #dataengineering #softwareengineering

49

3 Comments

A completely new neural network architecture just dropped - the Kolmogorov-Arnold Network (KAN). I don’t mean a new type of neural network model, but a foundation change in the core architecture of NNs. Multi-layer perceptrons have been core to all deep-learning frameworks. At its core, they are simple units that sum the weighted input from other neurons and apply some form of non-linear activation (most of the time RELUs). At a foundational level, it relies on the theory of universal approximation. KANs on the other hand rely on The Kolmogorov–Arnold representation theorem, which states that any continuous multivariate function can be represented as a composition of a small number of univariate functions and a small number of operations on those functions In practice, KANs do not have any edge weights, instead, they have a trainable univariate function parameterized by a B-spline. This seemingly small but foundation change makes KANs much better than MLPs in terms of speed of training, model complexity, and explainability. I have seen some incredible results coming out in the short time since this was published. The paper comes from one of my favourite scientists Max Tegmark and you can read it here (Liu et al., 2024, https://lnkd.in/d8iSTAGb). Fair warning, while the concept is simple the paper is quite verbose and detailed.

46

IMVC 2024 just released Dr. Amir Alush great talk about going from messy visual data to high quality datasets: https://lnkd.in/dGBDUUHQ . Try it yourself on your own data at https://lnkd.in/ds4vwrpP!

24

The coding capability of Claude v3 Opus is absolutely insane. I am working on a small project to create a POC app, and in about 4 hours iterating with Opus I have generated nearly 1300 lines of code and 4 different apps/packages. Opus is making calls back to itself for GenAI tasks. I have edited maybe 30 lines of code in total. I am iterating the same way I do when developing... start small and continue to add function(ality)s with each new feature I want. I will take the code base, feed into Opus, and ask it to add the functionality I want. I'm completely blown away in it's ability to generate code and also fix code when I encounter issues/bugs during runtime.

70

3 Comments

An interesting article in Computerworld from Steven Vaughan-Nichols "Worse still, this will eventually lead to a time when those genAI tools start consuming their own garbage. This is a known problem that will cause model collapse. Or, as neuroscientist Erik Hoel pithily describes the end result: “synthetic garbage.” He’s not alone; many AI engineers think a little bit of AI-generated data can poison their LLMs." #ai #genai #aiethics #search #LLM #teckedin https://lnkd.in/gDU6eZ5X

1

1 Comment

In just a couple of days, thanks to the work of Osma Suominen (and the Annif tool - suggested by Juho Inkinen), there are some very compelling results (> 0.95 recall for policy area, and F1 of nearly 0.74 for subject areas), demonstrating the power of open data on Hugging Face and the excellent international Library community! In follow-up discussions with Marci Harris and Leonard Park, my team at DreamProIT plans to add Committee referrals to the data set. This could allow a model to recommend committees, based on prior bill referrals... See https://lnkd.in/gww39QeG

16

The context window of a Large Language Model (LLM) determines the amount of text it can consider when generating a response (the number of words it can "keep in mind," figuratively speaking). For example, when a conversation grows longer than the context window, the LLM is unable to refer back to earlier parts of that conversation (in simplified terms). Modern models have extensive context windows, equivalent to the text of multiple novels. The article in question examines the performance of LLMs on a task when provided with hundreds or thousands of examples of the task (known as in-context learning). The accuracy improves as the number of examples increases, and in some cases, even surpasses the performance of finetuning (where certain parts of an LLM are retrained). However, finetuning requires an even larger dataset. Interesting. This suggests that when you have access to thousands of examples but not tens of thousands, in-context learning can serve as an alternative to finetuning. #generativeai #lllm https://lnkd.in/gpX-Ugft

5

We now have an img2txt (image to text, also known as, image tagging) model available on PixelFlow. Feel free to reach out to me or go to cloud.segmind.com to get access today! #img2txt #AI #generativeai #pixelflow #segmind

2

Free video primer on the good, bad, and ugly of #XAI with SHAP: https://bit.ly/3J2if3U. If you enjoy the primer, sign up for my Maven XAI training, "Explainable AI for Decision-Making Applications." In that course, we'll cover how to build accurate, transparent and reliable #machineleaning (ML) models. Topics include: - Explainable feature engineering - Explainable #ML models - Post-hoc explanation - Important background and compliance information Sign up here: https://bit.ly/4cXFeej!

45

1 Comment

Graph databases, with their ability to map intricate relationships and dynamic interactions, are essential for understanding the hidden patterns typical of fraudulent activities. TigerGraph's #GraphRAG leverages this by enabling real-time, deep link analysis, which is a game-changer for identifying suspicious patterns that traditional methods might miss. With TigerGraph #CoPilot, the process is further streamlined, allowing financial institutions to rapidly deploy sophisticated models that can predict and prevent fraud before it impacts operations. This not only enhances security but also ensures compliance with evolving regulatory requirements. The unique way that TigerGraph CoPilot AI leverages #GenAI unlocks incredible new user experiences for #banking customers, #FinancialCrime investigators, #Compliance officers, #CallCenter responders, as well as business developers alike. Visit us at https://lnkd.in/drxn-7pQ

70

8 Comments

https://lnkd.in/gvdBb_Ay Important. graph models have different properties than other models. Some speculation. there's a potential given The nature of graphs is that in the interconnections between notes, semantic information can become encoded such that the edges have as much value as the nodes. It is possible that ILGNNs could represent a form of metarags, It could be combined in mixture of experts implementations of LLMS. Based on the evolution of LLM's,. it is also possible that Small Generative Language Model could be created with GPTs. It would be very interesting to see if LGGMs could be used to create or recreate knowledge graphs that exist today. That would be an interesting kind of benchmark. I may be overreaching on some of these but I think it's very important still. Note l: Based on the work of Daniel Dennett and Judea Pearl, There are potentials of causality being determined with this type of model and what it can generate. GNNs and causality, were.one of the newest most popular new elements at Neurips in Dec 2023. Things seem to be emerging in that direction. some of these are variants of CNNs Stay tuned this is an interesting area. #ai #gnn #tracking #aiprediction #ai

4

Check out this May 14 Webinar >> Complex #AI Agents — Use #GenAI to Automate Complex Tasks. Register now at: https://lnkd.in/eDiR5D5H -- presented by @Abacus.AI Join Abacus in this engaging 2-hour webinar on how to use advanced AI agents to automate and revolutionize your business operations. With the help of @Abacus.AI, you can develop agents capable of automating intricate tasks. These agents will formulate a strategy, which you can implement upon approval. These state-of-the-art AI Agents are designed with a planning feature that allows you to: 🟣Streamline complex business workflows: automate both routine and intricate business tasks. 🟣Achieve seamless integration and communication with other enterprise systems. 🟣Enhance automation across various departments including customer support, marketing, technical support, IT, sales, and content creation. Participants in any of the demonstrations will receive certificates. 🟣🟣🟣🟣 #GenerativeAI #DeepLearning #MachineLearning #LLMs #DataScience #DigitalTransformation #CDO #CTO

6

1 Comment

Check out this May 14 Webinar >> Complex #AI Agents — Use #GenAI to Automate Complex Tasks. Register now at: https://lnkd.in/eDiR5D5H -- presented by @Abacus.AI Join Abacus in this engaging 2-hour webinar on how to use advanced AI agents to automate and revolutionize your business operations. With the help of @Abacus.AI, you can develop agents capable of automating intricate tasks. These agents will formulate a strategy, which you can implement upon approval. These state-of-the-art AI Agents are designed with a planning feature that allows you to: 🟣Streamline complex business workflows: automate both routine and intricate business tasks. 🟣Achieve seamless integration and communication with other enterprise systems. 🟣Enhance automation across various departments including customer support, marketing, technical support, IT, sales, and content creation. Participants in any of the demonstrations will receive certificates. 🟣🟣🟣🟣 #GenerativeAI #DeepLearning #MachineLearning #LLMs #DataScience #DigitalTransformation #CDO #CTO

2

Why is clustering analysis so hard? In his paper “an Impossibility Theorem of Clustering” (2002) Jon Kleinberg articulated that there is no clustering model that can satisfy all three desirable axioms simultaneously: namely, scale invariance, richness, and consistency. (Kleinberg, 2002) To cut a long story short, Kleinberg demonstrated that a mathematically satisfactory clustering algorithm simply does not exist. This might be a death sentence for clustering analysis to some theoretical fundamentalists. Nevertheless, ever since Kleinberg released his impossibility theorem, many methodologies for clustering evaluation have been proposed from the field of engineering (e.g. applied mathematics, information theory, etc.). To pursue pragmatism in filling the gap between theoretical/scientific limitations and practical functionality is the domain of engineering. That said, easier said than done. As Kleinberg’s Impossibility Theorem suggests, it is very hard to evaluate results of clustering algorithms. As a matter of fact, different validation techniques could yield confronting assessments about the same clustering results under investigation. In addition, evaluation metrics requires a certain set of pre-requisites in order to generate any meaningful information: in other words, a violation of one single pre-requisite can impair the validity of the metric. Remember this famous aphorism: “All models are wrong, but some are useful.” (https://lnkd.in/dj_xj69T )When we use any model, this caveat should echo and resonate in our mind. In this context, I encountered a paper written by Julio Omar P. & Fernando Berzal (2019), “Evaluation Metrics for Unsupervised Learning Algorithms” ( https://lnkd.in/dj_xj69T ) in which they proposed a clustering validation framework in their attempt to better evaluate the quality of clustering performances under the mathematical limitations posed by “the Impossible Theorem”. To promote our pragmatic use of clustering algorithms, I thought that it would be constructive to share my study note about their evaluation framework in my new article at Towards Data Science. Given the imperfect conditions under which we live in, let’s promote practical knowledge with vigilance in the spirit of Aristotelian Phronesis (https://lnkd.in/dKnihn9g.). #clustering #validation #machinelearning #datascience #pragmatism #phronesis #evaluation #metrics #impossible #trilemma #violation #engineering #imperfection #impossibility https://lnkd.in/dWtuRTPi

9

This is a win for #sustainability. In effect, ML applications may better leverage GPU capabilities, saving more time and energy 🚀

1

RAG with CPU only Inference https://lnkd.in/e2jR2p2H

3

Don't be scared by #NVDA stock "crash". The journos can call Nvidia's recent drop a bloodbath or whatever big, scary word they’d like. But you have to remember how #Nvidia got here... It’s truly been a run for the ages. The stock is up more than 750% since the beginning of last year. It was a $1.5 trillion company five months ago. It’s worth over $3 trillion today. We’re dealing with BIG numbers here, which magnifies the recent drop. But this is normal and expected. NVDA was due for a breather. In fact, last week I said: "While the company’s stronger than ever... caution is warranted from a sentiment perspective (how investors are feeling and acting). A mood change was all it would take to drop NVDA’s price by 30%. And that would be a buying opportunity." Nvidia dropped by 18% shortly after my comments. While it's not quite the 30% I would like, these are the kind of dips you want to buy.

5

2 Comments

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3