Miguel Paz

SWARM Community has built an extensive library of interviews and webinars with experts on #AI and #data We are thankful that Joaquin Melara is sharing this valuable content in the Teckedin knowledge base ___________________________________- 'Join Santona Tuli, Head of Data at Upsolver, as she talks through the different levels of AI initiatives and how much each level of involvement requires in terms of commitment. Her broad and extensive knowledge and experience working with data is a testament of her expertise in this area, from working on massive event data from particle collisions at CERN, to improving the development experience for data scientists and machine learning operations in Astronomer, to her present work leading data engineering and science operations. Throughout it all, Santona has been passionate about empowering others to build scalable and useful AI systems.' ABSTRACT: Incorporating AI into the business doesn't look the same for every business. This talk will enumerate the different levels of AI initiative and incorporation into business products and practices. We'll dig into the amount of commitment required for each level and what expected returns might look like. You will come away from the talk with a criteria for determining the level of AI investment that's right for your business. _______________ #artificialintelligence #analytics #dataanalysis #automation #datascience Santona Tuli, Ph.D. https://lnkd.in/gpNU7nBd

7

2 Comments

We have a new working paper with Miguel Perez Guerra on #AI and #antitrust. This time, while discussing AI, we drew a reference to 'I, Robot,' much like Benoît Coeuré's mention of 'Terminator.' We hope you will enjoy reading it as much as we enjoyed writing it. (Link in the comment)

56

3 Comments

This graph is also a representation of the many use cases of #AI in #technicalwriting.

4

Here are the key points from the paper "Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet" by Anthropic: - Anthropic has scaled their sparse autoencoder (SAE) method to extract interpretable features from the large language model Claude 3 #sonnet. This allows them to decompose the model's activations into a sparse set of "feature directions" that represent abstract concepts. - The SAE consists of an encoder that maps model activations to a high-dimensional feature space, and a decoder that reconstructs the original activations from the feature activations. Training minimizes reconstruction error and an L1 penalty to encourage sparsity. - Anthropic found a diversity of highly abstract features in Claude 3 Sonnet, including features for famous people, countries, cities, and code type signatures. Many features are multilingual, and multimodal, and encompass both abstract and concrete instantiations of concepts. - The optimal number of features and training steps for the SAE scales as a power law with the compute budget used to train it. Larger SAEs achieve lower reconstruction losses. - Analyzing the learned features reveals that they become less specific as their activation strength decreases, possibly reflecting the model's confidence in a concept being present. - Extracting interpretable features from large language models like Claude 3 Sonnet could make them safer by revealing how specific features relate to concepts and affect the model's behavior. In summary, Anthropic has made significant progress in scaling its interpretability techniques to #llms, extracting millions of interpretable features that provide insight into the models' inner workings and representations. Source: https://lnkd.in/gqCsGR-Z

4

This article explores the future of AI models, spotlighting trends, potential impacts, and emerging ethical considerations. #ai #innovation #ethics

3

This paper proposes a system where LLMs automatically generate scientific hypotheses, and then test those hypotheses with simulated AI human agents This can be a brutal game changer for science and innovation https://lnkd.in/dTDKPFyR

5

3 Comments

I had a lot of fun yesterday at the KCD Argentina! I talked about the crucial aspect of deploying AI models in production, highlighting the importance of efficient workflows and streamlined processes. We delved into MLOps and FM/LLMps during the talk, exploring their roles in bridging the gap between prototypes and production environments. Eduardo Spotti and team: Congratulations! The event was great! Looking forward to the next one! Rossana Suarez Emil Santurio Héctor Fernández it was really nice to see you! #ML #AI #MLOps #FMOps #LLMOps #AWSCommunityBuilders #GenAI #RAG #LLM #AWS #MachineLearning #ArtificialInteligence #GenerativeAI

70

7 Comments

I decided to see if Claude 3.5 Sonnet is as good as Anthropic claims. I came up with 5 tests to pit it against OpenAI's GPT-4o. 1. Handwriting: My handwriting is terrible so I wrote my prompt in a notebook. https://lnkd.in/d-q4Rprj 2. Making a complete game: Using Python create a tower defense game from scratch. https://lnkd.in/d7ZaiyP8 3. Vector art: Make me a cool spaceship logo. https://lnkd.in/dbxQN3Jn 4. A funny story: Write a tale about a humorous cat. https://lnkd.in/dAyrmT8x 5. The big debate: Should AI have rights? https://lnkd.in/dK4EG7GE

8

2 Comments

Must-read #GenAI analysis from the brilliant Benedict Evans addressing "the single core question about the future of Generative AI - is this a new general purpose tool, where one product from one company does the work of hundreds of pieces of software from hundreds of companies, or is this a generic technology that will enable features inside products from hundreds or thousands of companies?" Evans shows how Apple's strategy reveals they believe the latter, with profound implications. At ModuleQ, our experience deploying #AI in global financial institutions has led us to the same conclusion: personal context is key, and #LLM tech is often most effective behind the scenes to enable focused, high-value use cases. #UnpromptedAI https://lnkd.in/g-t92rup

14

5 Comments

“To be able to fight disinformation…[we]…must understand it. But research on disinformation is limited in several key respects…A big obstacle for researchers is the lack of access to data. The best data is not in public hands, but is “sitting in private networks in Silicon Valley,” says Phil Howard…of the International Panel on the Information Environment (IPIE). And collecting relevant data is becoming more difficult. After Elon Musk bought Twitter…in 2022 the company shut down the free system that let anyone download information on posts and accounts, and began charging thousands of dollars a month for such data access. Meta announced in March that it would be retiring CrowdTangle its platform-monitoring tool that lets scientists, journalists and civil-society groups access data, though the company says academics can still apply for access to certain datasets. Such changes have seriously hampered researchers’ ability both to detect disinformation and to understand how it spreads… In America…efforts to fight disinformation have become caught up in the country’s dysfunctional politics…fighting disinformation requires a co-ordinated effort by tech platforms, academics, government agencies, civil-society groups and media organisations. But…any co-ordination of this kind has come to be seen, particularly by those on the right, as evidence of a conspiracy between all those groups…Researchers studying disinformation have been subjected to lawsuits, attacks from political groups and even death threats. Funding has also diminished…All this has had a chilling effect on the field...“It is difficult to avoid the realisation that one side of politics—mainly in the US but also elsewhere—appears more threatened by research into misinformation than by the risks to democracy arising from misinformation itself” Disinformation is a sprawling problem, requiring co-ordinated action from multiple sectors of society. Unfortunately, the analysis of it tends to be siloed and there is a lack of agreement on things like terminology. This makes it hard to join the dots and find lessons that apply more broadly. Phil likens the situation to the early days of climate science: lots of people are trying to tackle the same problem from different perspectives, but it is difficult to see the whole picture…The current lack of a joined-up response to disinformation is a problem, but also an opportunity: co-ordinating research and action should lead to better detection and mitigation of deceptive content, because modern disinformation campaigns all work in similar ways. But, as with climate change, cleaning up the world’s information environment presents a daunting, long-term challenge.” https://lnkd.in/gb_g9CKf Ellen Doowan Mark Nina Roman Emerson Alan Jevin Renee Sheldon Justin J.D. Klon John Brandon Ed Sasha Alex Jakub Jill Clint The Economist Celine

35

4 Comments

One of our research focuses is the field of mechanistic interpretability: think of it as a bit like the "neuroscience of genAI". It's an emerging field but a lot of progress was made in 2023 as researchers used small models to test ideas such as the linear representation hypothesis, superposition, and dictionary learning. While these ideas are technically complex, they are conceptually straightforward. The big question at the end of 2023 was whether these techniques would scale, and it seems the answer is yes! We're excited about the potential of these advancements to enhance our understanding and application of AI. Today Anthropic released a really important paper - the ideas and practices that worked for small models applied on Claude 3 Sonnet. It's really worth a read - and before you do, a high level description of some of the key terms. Linear Representation Hypothesis: Neural networks, particularly large ones like transformer language models, represent concepts or features as directions in their internal activation space. Imagine activation space as a high-dimensional space (like a 3D space but with many more dimensions). In this space, each meaningful concept can be thought of as pointing in a specific direction. For example, in a neural network trained on language, there might be a specific direction representing the concept of "cat." Superposition Hypothesis: This builds on the linear representation hypothesis by suggesting that neural networks can represent more features than there are dimensions in their activation space. This is possible because, in high-dimensional spaces, many directions can be nearly orthogonal (almost at right angles to each other), allowing the network to store multiple concepts in a way that they don't interfere with each other. Think of it like being able to pack more information into a small space because the information is encoded in ways that don't overlap much. Dictionary Learning: This is a method used to find a set of basis vectors (a "dictionary") that can represent data efficiently.When applied to neural networks, dictionary learning helps identify the directions (features) in the activation space that correspond to meaningful concepts. Sparse Autoencoders: These are a specific type of neural network used for dictionary learning. They aim to learn a compressed representation of the input data by enforcing sparsity, meaning most of the activations are zero. This sparsity helps in identifying clear, distinct features because only a few neurons are active at any time, making it easier to interpret which features are being represented. #AI #MachineLearning #NeuralNetworks #GenerativeAI #Innovation #TechResearch #ArtificialIntelligence #TransformativeAI #MechanisticInterpretability #TechInnovation #AIResearch #FutureOfAI #DeepLearning #TechTrends #AIExplained #anthropic #claude https://lnkd.in/gHMuUNwt

9

5 Comments

⚡️ A new edition of ICONCLASS is released! 🖼️ https://iconclass.org/ What is new in this edition of the Browser? This fourth edition continues to use the ever expanding dataset of sample images, but presents them in a different way, allowing you to zoom in and inspect the metadata for selected images. New data and new functionalities Visual search: based on computer vision technology, algorithms help to find visually similar pictures with re-usable Iconclass encoding. The Iconclass Bibliography is now integrated with the Browser and continuously updated. Updating the Iconclass Bibliography also means we are creating deep links to other subject and art authority systems, such as the Warburg Institute's Iconographic Database, the Princeton Index of Medieval Art, Wikidata, and Getty's Art & Architecture Thesaurus. Primary textual sources important for iconography are also tagged with Iconclass, providing subject access to e.g. the text of the Bible, Ovid's Metamorphoses, Cesare Ripa's Iconologia and Andrea Alciato's Emblematum libellus. For more information: https://lnkd.in/ejeGvhat

58

1 Comment

Nathan Lambert wrote an article about chatbot arena with a lot of important observations I am a proponent of side by side evaluation methods that work well for many problems The article is not about side by sides but about chatbot arena, in particular but there are a lot of details that are genetically applicable to any side by side comparison. For example , about interpretation of Elon points , what do they mean for understanding difference in models accuracies One of the reasons why I do not like Elo system for side by side ranking, the Elo system was designed for particular game tournaments and models comparisons are very different in their nature Even for tournaments , there were many better models were invented after the Elo Good reading : There was also a good article from Anthropic about arena evaluation

4

Peeking inside the #AI "black box", #Anthropic’s groundbreaking discovery 🤜🤛 Anthropic researchers have made a significant leap in understanding the inner workings of #largelanguagemodels (#LLMs) 🌐 For the first time, they've identified specific neural patterns corresponding to distinct concepts within their AI model, #Claude 🤖 This breakthrough allows them to decode and even manipulate these "features," potentially making AI safer and more reliable 💪 By peering into these neural nets, we’re closer to solving the AI "black box" problem, paving the way for innovations in AI safety and functionality 🚀🚀 #TechInnovation

2

"A Primer on the Inner Workings of Transformer-based Language Models" by Javier Ferrando Monsonís, Gabriele Sarti, Arianna Bisazza, Marta Ruiz Costa-jussà "The rapid progress of research aimed at interpreting the inner workings of advanced language models has highlighted a need for contextualizing the insights gained from years of work in this area. This primer provides a concise technical introduction to the current techniques used to interpret the inner workings of Transformer-based language models, focusing on the generative decoder-only architecture. We conclude by presenting a comprehensive overview of the known internal mechanisms implemented by these models, uncovering connections across popular approaches and active research directions in this area." Paper: https://lnkd.in/eKKTTi-z #transformers #languagemodels

34

Heads up Climate Tech startups and VCs: The Compute For Climate Fellowship run by Amazon Web Services (AWS) and IRCAI - International Research Center on Artificial Intelligence under the auspices of UNESCO is now accepting applications for our 2nd cohort of fellows! If you’re working on a groundbreaking new project that needs a great deal of Advanced Cloud Computing like generative AI, artificial intelligence and machine learning (AI/ML), and high performance computing (HPC), this is for you. 👇 Apply here by June 7 2024 to be considered for a funded POC build this year. https://lnkd.in/gcJV6ksh The fellowship offers: ✅ Funding in the form of AWS cloud credits for a 2-3 month POC build ✅ Access to advanced computing services, such as Generative AI, AIML, HPC, and quantum computing ✅ A team of expert advisors from AWS and IRCAI who are experts in your field and GenAI, AIML, HPC ✅ Design advice to build POCs under the guidelines of UNESCO’s Ethics Impact Assessment for Artificial Intelligence to ensure that each solution is built with safe, trustworthy technology. Speed is critical to addressing the climate crisis before even more damage is done. We set up the Compute For Climate Fellowship to help climate tech startups move even faster at tackling global warming. Lisbeth Kaufman | Davor Orlic | Aidan O'Sullivan | Ian Sikora #AWSStartupCommunity #startups #climatetech #cloudcomp #generativeartificialintelligence #generativeai #proofofconcept #aws #AIML #quantumcomputing

15