Marc Dupuis

I wrote a blog post on a thorny but underappreciated problem in ML-based marketing personalization: event attribution. It's how you go from the actions you take and customer behavior you observe to creating training experiences for your model. Good attribution makes a huge difference in driving uplift, there are a lot of degrees of freedom in configuring it, and at the same time, it's rarely discussed in ML/bandits/RL literature.

38

4 Comments

Announcing our new feature: Clean Up Your Documents with RaijinAI’s Intelligent Formatting! 🌟 We know how much y’all love nice, clean transcripts😎. At Raijin.ai - AI Analysis Assistant, we are constantly pushing ourselves to make the best experience for anyone who wants to do quick + good research and insight mining. We can now automatically format documents and web links that you upload into our platform without any additional prompts or instructions from you. What this means is that you get enhanced readability without the hassle of trying to split your notes or web content into paragraphs manually. The example here is from the recent Fortune magazine interview with NVIDIA's legendary CEO Jensen Huang. This was done as simply as copying and pasting the URL link into our platform. Come try this out by signing up for our Beta Launch (link in comments section)! #AI #generativeAI #LLM #userresearch #discovery #productivitytools #insights #UIUX #productmanagement #productdevelopment #voiceofcustomer

14

1 Comment

With Redpanda Data's recent acquisition of Benthos I've gotten some questions about how it compares to Arroyo. In short: Benthos is *stateless*, while Arroyo (like Flink or RisingWave) is *stateful*. This may sound small, but adds up to a fundamental difference in how these systems are designed, built, and used. In a streaming engine, statefulness combines two inter-related features: * Repartitioning data (GROUP BY in SQL) * Remembering previously seen events With just these two features, we can support most of normal SQL (window functions, joins, aggregation, etc.) plus time-oriented streaming operators (time windows, event-time semantics, watermarks). Stateless engines on the other hand are limited to simple transformations and filters (SELECT and WHERE in SQL terms) that operate on individual events. But all of this is a tradeoff; stateful systems are much harder to operate. So if you don't need joins, aggregates, or windows, a stateless system can be a better choice. They also work well as infra glue, routing data between different systems—Benthos in particular has a huge connector ecosystem. For a deeper dive on this, I wrote down some thoughts on our blog:

56

4 Comments

As the AI hype cycle continues, you're also starting to see the "data is the new oil" make its way back into the zeitgeist. It's undeniable that data is valuable, especially proprietary datasets, but how valuable? Up until recently there weren't data marketplaces so it was difficult to gauge how the market would value a dataset. But now with the Snowflake, and other, data marketplaces we can get a sense of what people are willing to pay for data. If you check out the marketplace another interesting aspect is how many datasets are packaged up public data. For example most of the weather data, https://lnkd.in/euhS4GbB would be available for free from NOAA. We've been experimenting with this as well. We've aggregated all the public data from Dutchie eCommerce sites and made it available on Snowflake, https://lnkd.in/eJRTZd5S BDSA sells a full product based on this dataset, https://lnkd.in/ejMqCWuc Broadly, it feels like there's some interesting opportunities here to aggregate, normalize, and sell public data. Have you used any public datasets that are re-packed into a paid product?

2

Having worked at Cruise and Waymo on self-driving taught me many things. I hope to bring some of these learnings to healthcare so it too can soon have the magical, "no-driver-at-the-wheel" moment. Here's a short conversation I had with Louie Cai, MD on this topic!

12

1 Comment

#ObsessivelyEnthusiasticInvestor #UnreasonablyEnthusiasticInvestor #IrrationallyEnthusiasticInvestor at REFASHIOND Ventures: The Industrial Transformation Fund. We look for the people we believe are doing the best work in Industrial Transformation & Supply Chain Technology; Attempting the impossible. Then we go all-in on them. With obsessive and unreasonable enthusiasm. Join us. #IndustrialTransformation #SupplyChain #Innovation #Technology #Startups #VentureCapital #DemandTheImpossible #UnreasonableEnthusiasm #ObsessiveEnthusiasm #IrrationalEnthusiasm 

6

"Data democratization" (an iconic buzzphrase) has been around for a long time. Tons of companies have promised to deliver it... with mixed results. Lots of dashboards, heavy implementation, data siloes, the list goes on. But advancements in AI have made "simple data access for all" a reality. Check out Bessemer Venture Partners's Data Shift Right roadmap. Thanks for the shout out Lindsey Li, Yael Schiff, and Amit Karp.

14

Three things AI (and Generative AI too) can do to help engineering teams to deliver higher velocity for changes. All the recent conversations around AI magically producing code means that developers and QA have even greater pressure to find problems and fix them faster. In short, triaging failures is becoming a bigger problem. In this episode, I talk about Launchable's viewpoint and solution to help teams. 1. Catch issues earlier 2. Priortize tests 3. Classify failures It was a wonderful conversation with Joe Colantonio who is a veteran in the software testing field.

19

4 Comments

We're checking out Meta's newest model, Llama 3. We want to see how it stacks up against its predecessor, Llama 2. At a glance, they seem very similar, but there are a few differences. Llama 3 clocks in with slightly higher performance on benchmarks. While I wouldn't call it a gigantic step forward, it is indeed an improvement. When you compare it to other models currently available, like Google's Gemma or the Mistral model, Llama 3 often outperforms them. So if you're in the market for an open source model in the 8 to 70 billion parameter range, you might want to consider Llama 3. One fascinating detail is how Llama 3 and Llama 2 match up directly. More often than not, Llama 3 comes out on top, but there are surprising times when Llama 2 fares better. This was unexpected considering the newer model should at least tie up with, if not outstrip, the older one. As a user of Mistral myself, I was interested to see that the performance difference between it and Meta Llama 3 came down to the single digits. Despite this, it's encouraging to see Meta pushing their models to new frontiers with Llama 3. #metallama3 #modelcomparison #aiimprovement

15

2 Comments

Emily M. Bender does a great explanation in this article. I like how she uncovers the marketing term #AI with the real underlying concept of #automation. BTW, software automation was being done before the widespread use of ML. The bigger issue, as she rightly points out, is how it is hard to understand the bias that comes from training with data a system that is going to be used to make a decision.

4

1 Comment

Hey, I'd like to give a shout-out to ClearML for helping us at Vimeo develop AI-based systems. We have been happy customers for a few years now and I'd like to share a bit about how we use ClearML. We use ClearML Datasets to cache a dataset of video transcripts, and run testing loading directly from ClearML Datasets. This allows us to simultaneously speed up the data loading and ensure consistency. Moreover we use ClearML to save our benchmark annotations in one place and track our system performance on the benchmark every-time we run an experiment or change our prompts. ClearML allows us to see the different parameters and prompts that were used for each experiment and to monitor improvements or regressions in our performance. We also use ClearML to run large-scale tests and help with statistical evaluation of our methods. For example, we developed a RAG (Retrieval Augmented Generation) Q&A system and wanted to verify that the LLM will not answer certain questions or user queries that are outside the scope of the video. We used ClearML to collect and analyze the RAG responses on many videos for predefined user queries that were outside the scope of the videos and got good visibility into the performance of the system. Also, the comparison feature on ClearML is great for tracking the improvement of our metrics along the progressing versions of our systems.

28

1 Comment

The most important metrics might just be the ones you consider “dumb” 👀 When working with data, it's so easy to show off all that glorious, mathematical complexity. It's much, much harder to keep things simple — but all the more rewarding 🏆 This Wednesday, I'll be presenting my thoughts on easily-explainable metrics and intuitive visualizations. Check out out my session with Srini Kadamati at Open Data Science Conference (ODSC) Boston. https://lnkd.in/dD2rhKSs

25

4 Comments

👀 Want a sneak peak into how the Lume (YC W23) App automates your data pipelines? Our walkthrough video guides for the Lume App are now in our docs 📑 What's in there: - How to create a new data pipeline - How to auto-map new data with Lume's AI - How to review and edit mapping logic and transformed data, from natural language prompting to manual edits - Handling erroneous data Now you can use either our API or our app to easily onboard customer data, normalize data, or create integrations. Link in comments.

44

1 Comment

Learn the basics of how to build agentic AI products with this short course by Andrew Ng and Jerry Liu. DeepLearning.AI courses are easy to follow and serve as a fantastic starting point for developers who want to begin experimenting in the AI space.

2

1 Comment

AI is not slowing down, and the scaling laws continue to hold. Claude 3.5 Sonnet is now probably the most powerful model available, but Sonnet isn't even Anthropic's biggest model—Opus is. A particularly impressive number here is 59.4% on GPQA diamond questions. GPQA is a benchmark of PhD-level questions for which you can't Google the answers, and they don't make the data available online for people to easily train on, so the contamination risk is lower. In November 2023, GPT-4 scored 38.8% on GPQA with few-shot prompting. Claude 3.5 Sonnet gets 59.4% with 0-shot prompting (while GPT-4 has improved, with GPT-4o, to 53.6%).

2

Kyle Poyar recently shared some counterintuitive advice on building AI products. He pulled insights from 20+ product builders including Scott Belsky, Elad Gil, Rahul Vohra and Sarah Guo. Here are some insights that were shared: 1. Think differently: ↳ Data and interfaces may outweigh models in importance. ↳ Companies with deep vertical data understanding will lead. 2. Solve real problems: ↳ Cool AI demos ≠ user value. ↳ Focus on pain points and delightful experiences. 3. Prioritize design: ↳ Great UI/UX can impact metrics as much as model improvements. ↳ Guide users through AI's open-ended nature. 4. Value proprietary data: ↳ Unique, structured datasets are gold. ↳ They may become more valuable than the models themselves. 5. Segment by attitude: ↳ Consider AI embracers vs. skeptics in your testing. ↳ Don't let averages mislead you. Definitely worth reading Kyle’s post: https://lnkd.in/eHYb7Fxn There’s A LOT of AI product builders in the Rewardful community… we’d love to hear your thoughts! #AI #ProductDevelopment

3