Matt Samson

"How can I help?" A guide to good intentions Since I've been working on Motif Analytics, many people I talk to about it will ask if they can help you in some way. I've thought about this a lot now -- here's my ranking of things you can do in order of helpfulness: 1. Become a customer. This is the tallest order, but early stage startups need revenue and active users. If our product seems like a good fit for your domain, give us a chance to win your business. It takes belief in the vision and the team to be an early customer. We appreciate that a ton and work really hard to make sure our customers are getting a ton of value. 2. Use our product and give us feedback. Early products need lots of iteration and it's hard to know what you're doing well and doing poorly unless you can find users who give your product a serious try, are willing to be critical, and proactively communicate. Direct knowledge of our product helps you explain it to others and have more credibility in doing so. 3. Provide a warm introduction to a potential customer. You likely know a lot of people we don't know, and many of them could get a lot of value from our product. To introduce us convincingly, you have to invest a bit in understand what we're building and how to pitch it to people you know. We are always happy to explain and give you information to share to make it easier. 4. Tell people about our product and why it's awesome. Not everyone's a potential customer or user, but broad awareness of new products is generally quite low. The more people that are aware of and interested, the better. This kind of help pairs very well with (1) and (2). It's really encouraging how many people root for you when you're building something new and putting it out in the world, and the generosity they demonstrate is incredible. Just want to express my enormous gratitude for all the folks who've helped us so far in this journey.

93

2 Comments

I hope to see you next week at my free lightening lesson on Prioritizing Research for Impact on Maven! Join me on Thursday, May 16th at 10:30 am PT for 30 packed minutes, where I'll share an overview of my upcoming prioritization workshop and answer your questions. Topics: - Know when research is needed and when it's not - See how a proven prioritization framework works - Know how to involve partners in prioritization Register now at https://lnkd.in/gbNsGdih #Leadership #Impact #Research #CustomerInsights #UX

20

We've been working on a series of retention and conversion prediction models. (And by "we", I mean Eleanor Hanna Hanna has been doing the lion's share of the work and I've made a few suggestions). One of the challenges of doing this kind of work is that Aampe has lots of different customers, each of whom has different definitions of conversion, different dynamics on their app, and different levels of instrumentation of their app data - which means very different feature sets for a prediction model. The models had performed very well for the customers whose data we used for the initial model development, but when we then trained the same model of other customers' data, we saw some instances of less-then-exciting performance. However, this got me thinking: as an absolute measure of performance, precision and recall below 0.4 might not be anything to write home about, but the pertinent *business* question here has to do with *relative* performance. The question is: "is the model better than the alternative?" Which begs the question: what is the alternative? Thinking about this reminded me of Rob Hyndman's work on time-series forecasting (link to summary document in comments). Hyndman recommends a measure of forecast accuracy that compares model predictions to a naive prediction that just carries forward the most recent value. So, in the case of a week-over-week retention model, if a user had engaged the previous week, the naive prediction would assume that the user would also engage next week. Taking the time to set up a naive prediction changes that way we think about predictions. It's very possible that a app's instrumented events might not contain all of the information needed to get a very accurate prediction. But even a less-than-good prediction is useful if it beats what we'd be able to accomplish if we didn't have the model in the first place.

19

2 Comments

Here's your reminder that AOV is a fully blended metric. It's in the name, "average" order value. As an analyst I'm a bit alergic to averages since they often hide a lot of useful information. The way to better understand an average is to break it up into smaller parts. The report below is a great example of breaking up AOV into smaller pieces. The columns represent order value buckets while the rows represent the order creation date. We can now see the distribution of orders by order size and track changes over time. This allows us to a) better understand AOV, b) easily track our ability to change it, and c) quickly raise an alarm if the percentages move to the left (often as a result of over discounting).

24

1 Comment

AI progress isn't slowing. Looking at my past predictions for today, I've actually UNDERESTIMATED where we'd be. This time last year, Anthropic released Claude 2. I made predictions in my journal for where we'd be today. Here they are, relative to reality: Prediction: We'd have GPT-4.5 or something like it that's multimodal and has at least a 32K context window. Reality: GPT-4o is better than GPT-4, is multimodal, and has a 128,000 token context window. Prediction: Claude 2.5 or 3 will exist, and it will be at parity with GPT-4, or better. Reality: Claude 3.5 Sonnet exists, and is better than GPT-4. In fact, it's the best model right now for some things, such as coding. Prediction: Google will have unveiled, if not made available, a Gemini model as powerful as GPT-4 or better, with novel capabilities. Reality: Gemini 1.5 Pro is the best model after GPT-4o and Claude 3.5 Sonnet, and has novel capabilities in a 2 million token context window. Prediction: Generative AI will be baked into office products like Google Docs and Microsoft Office. Most people will be using them regularly in some way or another. Reality: Generative AI is baked into Google Docs and Microsoft Office, and surveys show 75% of white collar workers use generative AI. Prediction: There will be at least early hints of job impact. Companies will be hiring fewer people because they’ll need fewer people. Reality: PwC reported that AI-exposed jobs are growing 27% more slowly than non AI-exposed jobs. Government stats show that AI-exposed occupations are growing more slowly as well, and in some cases declining. Freelance jobs for copywriters, translators, and customer service agents on Upwork have declined up to 33%. Prediction: On the multimedia side, AI will be generating full songs that are 75% as good as human songs, and short videos of 15-30 seconds. Reality: Suno and Udio generate songs that are in many cases 100% as good as human songs. Sora generates 60-second videos. I was quite surprised. I've been bullish on generative AI for a long time, but even my own predictions, which some might have called optimistic, proved to be conservative. Perhaps I have to recalibrate and be even more aggressive in anticipating what's to come.

9

3 Comments

Introducing Advantage+ for Max+ AI advantage plus a lot of slides with neurons and brains. How about we drop the bs and compare F1 scores of some models for incremental sales. How about we bakeoff on the data rights section of our MFAs. Guess what folks -- these companies aren't actually very good at machine learning. They are one-trick ponies. Here's how it's gonna go: Either you drive a prediction machine or someone else's will run you over. Either you get on top of the AI wave and look out over oceans of data, or stare up in dumb wonder and inevitably get soaked. https://lnkd.in/eQkCdDXG ...

16

I think Apple's generative AI features will significantly accelerate the technology's mainstreaming and usefulness. "AI for the rest of us," as Apple's Craig Federighi says. I can't possibly summarize everything here that they announced at WWDC today, and you can find such summaries elsewhere. But I think the most important things to note are: • Generative AI will be everywhere on Apple devices. Literally, everywhere. But in smart ways, I think. They're focused on useful and fun use cases. • Personal context is a key differentiator for Apple. Its generative AI can use all of your personal information—in a privacy safe way (see below)—to generate more relevant outputs. This will make the technology much more useful. • Apple's approach to privacy is a major differentiator. They do whatever they can on device, and offload to cloud-based servers when needed, but in a privacy-safe way. • Siri will indeed get much more powerful. You'll be able to talk to it naturally, it can see your screen, and it can take actions within apps. This will unlock the potential of generative AI models to do things on your behalf. • Other AI tools, like ChatGPT, will be integrated into Apple's experience. Siri will be able to suggest using ChatGPT to answer a question, for example. Users will be able to access ChatGPT free, or sign in to get access to paid features if they're a subscriber. I think this will all go a long way to mainstreaming the technology, making it accessible to average users. By this time next year, my guess is that the majority of iPhone users will be using generative AI in some way, entrenching the technology into people's lives in a way that probably wouldn't have happened otherwise.

15

Highly encourage MRX and UX folks alike to tune in for this presentation. I expect there will be much to peal back and discuss post presentation as well as a wealth of knowledge and insight shared during the presentation itself. Michaela Mora, IPC, MSMR, UXMC, looking forward to listening in! #Insights #MRX #UX #Growth #MarketResearch

Excellent study from Damian Rollison and the SOCi, Inc. team which seems to confirm my longtime hypothesis around citations (https://lnkd.in/gxZQwdx3): the only citations that are valuable are the ones that rank in Google for the keywords you want to be known for. https://lnkd.in/g49DMSeN

39

11 Comments

“AI agents are going to drive demand for APIs through the roof.” Scott Brinker, explores the landscape of integration in the #martech stack (but with considerations that go much beyond that). A peek into a future where apps will have to expose APIs and get over the concern that this will “reduce the usage of their human-oriented UI.” As always, insightful stuff from Scott!

19

1 Comment

We're moving to "prompt-to-product," not just task automation. I think AI's longer term economic impact won't just be automating an increasing number of STEPs. It will be converting many PROMPTs directly to PRODUCTs. For example, in the movie industry, right now people think about AI automating things like script writing, scene generation, soundtrack generation, speech generation, all as discrete steps or tasks. But I no longer think that's the longer term outcome. Within a few years we'll have increasingly multimodal models that can do EVERYTHING in one step. Like, you'll be able to enter a prompt, and get back a full movie, with the script, video, soundtrack, and speech all generated together. I may be wrong, but this seems to be the way things are trending. For example, the full GPT-4o multimodal model can output text, speech, sound effects, and images (and possibly even more), and combine them, such as by including sound effects while narrating a story that it's generated. I think we're going to see this across an increasing number of modalities, ultimately leading us to a prompt-to-product future.

6

9 Evidence-Based Ways (Most) Brands Grow 👇 I just read How Brands Grow Part 2 and the data is compelling. Authors Romaniuk and Sharp are marketing professors and Directors at the Ehrenberg-Bass Institute. They synthesize decades of research into evidence-based fundamentals of buying behavior and brand performance that we can use to drive growth and improve marketing productivity. Here are 9 of my favorite, often counterintuitive takeaways: 1. Focus on market penetration, not customer loyalty. The “double jeopardy law” states that brands with less share also have less loyalty. And the way to increase loyalty is actually to increase share.    2. Position within a larger category. This widens the potential market. For example, one meat-free protein company shifted from positioning as a vegetarian substitute (relevant to ~7% of the market) to a healthy eating brand (relevant to ~70%) and drove considerable growth.   3. Compete against the top brands. The “duplication of purchase law” states that brands share customers in line with other brands’ penetration. So, to grow, you likely will be taking customers from the biggest brands, and they from you. Worry less about the smaller players.   4. Build links between your brand and the most common category entry points (CEPs). A CEP is a way that a customer mentally enters the buying process. If you’re hungry, a CEP might be that you want a snack, and Snickers might come to mind thanks to their “You’re not you when you’re hungry” campaign connecting themselves with that common CEP.   5. Use distinctive brand assets consistently. If your brand doesn’t stand out, attract attention, and look like only you, your marketing could cue people to think about others. The best assets have high fame and uniqueness, like the GEICO gecko.   6. Reach as many category buyers as possible. “It doesn’t matter how many irrelevant people you reach, what matters is the cost of how many relevant buyers you reach.” The broader your reach, the less response rate, but often the greater absolute gain (a 1% impact on 10K people is bigger than a 5% impact on 1K).   7. Reach category buyers consistently but not excessively. Not everyone is in-market all the time, and most buy infrequently (especially in B2B), so market steadily. Plus, the first exposure within a period has the greatest sales effect, and spaced exposures cumulate in more lasting impact.   8. Maximize physical availability. This means distributing where buyers buy, being prominent in those channels, and having a product portfolio that satisfies the largest buying contexts.   9. Overcome the real barriers to acquisition: obscurity, forgetfulness and distraction. Most people don’t buy because they don’t know you, forget about you, or are distracted from doing so—not because they think poorly of you or consciously choose someone else. While there are some exceptions, the evidence suggests the exceptions prove the rules. #yourweeklymap #marketing #howbrandsgrow

23

6 Comments

Clarifying my position on using LLMs as a replacement for human participants in market research: I’ve found firsthand that they work extraordinarily well for select use cases. Here's one: We (Marketing Architects) are using them to pretest TV spots in place of human survey panel data. DISCLAIMER: We have 10+ years of in-market data against REAL humans to validate our results. We are NOT just trusting it at face value. It took months of software development, hundreds of hours of prompt engineering, and thousands of a/b tests run through our model to ensure we had a truly special product. Using LLMs without fact-checking or first-party data is risky. We found synthetic audiences accurate and useful for this use case. It saves time and lets us focus on producing more quality creative work. I’m not saying you have to accept or adopt it. Everyone tests and trusts new technology at different times - see the image from Craig Chelius below. But I think sharing evidence of new things in marketing is helpful and fun. I wouldn’t expect anything less than a skeptical and challenging audience on LinkedIn. 😊

74

29 Comments

We're nearly two years into building Dewey. In that time, I've had hundreds of meetings with data companies and seen every iteration of their attempts to support academic researchers imaginable. Each license structure, data delivery choice, and long-term resourcing decision is riddled with challenges since this niche audience operates so much differently than any other customers they have. I've also had the opportunity to listen to librarians and hear their frustrations trying to source data from vendors for their researchers. Regardless of this struggle, they both keep coming back to the table! These groups clearly want to work together, yet despite their hard work, deals rarely get done. There has to be a better way. I wrote a quick blog post summarizing data vendors' and librarians' main challenges when licensing data. By highlighting the pitfalls, I hope more data companies can successfully navigate the channel. It also briefly covers how Dewey is a new solution to these challenges. HUGE thanks to Steve Cramer for allowing me to share this post on his blog, This Liaison Life. I've been a long-time subscriber and have always appreciated the honest and insightful perspective on a business librarian's day-to-day challenges and responsibilities. https://lnkd.in/gfWuWtuk

104

1 Comment

One of my favorite Maven Showcase projects this week is the LA County Crime Mapping Tool by Tai C. Here's what I love about this one: 👉 The analysis is directly tied to a clear and specific objective – targeting locations in LA County to launch decriminalization ad campaigns. 👉 The output is simple (a single map visual) but provides the exact information that stakeholders need 👉 The interactivity gives users the ability to drill up or down to explore trends at different levels of granularity My only feedback might be to include some specific takeaways and recommendations in the project description itself – such a great way to showcase those analytical thinking skills 💪 Link to the project details in the comments – check it out and leave Tai some love!

44

1 Comment

When shifting from legacy to agentic methods for engaging users, one particularly important change in thinking seems particularly difficult for customers to wrap their minds around. It's the shift from campaign-based thinking to message-inventory-based thinking. In legacy systems, everything revolves around the campaign, which embodies the message you want to send, the users you want to send it to, the time you want to send it. Once you send the campaign, you're done - next time you want to send something, you create a new campaign. In an agentic system, however, everything revolves around strategy. You communicate your various offerings and their value to different users based on preferences and contexts. Those things get stored in a permanent inventory, so agents can pull the right message in the right context for the right user. In camapign-based thinking, audiences, triggers, and message sequences are just appendages to limit sends to users for whom the campaign's message is appropriate. In message-inventory-based thinking, audiences, triggers, and messages sequences are all different forms of elibigility - each user can go in and out of eligibility for different messages under different circumstances. From a UI standpoint, it doesn't make a difference - people are used to thinking about audiences, triggers, and sequences as largely different things, so any UI (for the time being) will probably have different flows for each flavor of eligibility. But if you build entirely separate *backend* processes for audiences, triggers, and sequences, then you have to have a way to de-conflict those processes, hardening the system around arbitrary user-facing distinctions. In the long run, it petrifies the system. Systems that are difficult to change fall behind systems that are easier to to change. That's why we architected Aampe's eligibility engine to be different from other systems. It took extra work - in terms of thinking as well as implementation - to design a unified eligibility pipeline. It involves decomposing audiences, triggers, and sequences into sets of modular protocols that can be recombined to efficiently identify which messages are appropriate for which users at any given time. Having that architecture gives us the flexibility to handle new flavors of eligibility on the front-end without having to build a new process we adapt to customer asks. People who use our UI don't see any of this back-end work. It operates silently. But that thing our customers never see gives us the flexibility to build new features and fix bugs faster, which is something customers do see.

23

1 Comment

Hemingway has a quote about going bankrupt that describes the process as “gradually, then suddenly.” It applies to exponential change, and feels very apt for current technological, economic, and social trends. I think we’re in the early part of several changes that will intersect. For example: increasing adoption of AI tools in businesses, growing popularity of four-day work weeks, persistent reluctance to return-to-office, early signs of AI’s impact on jobs (especially for copywriters, translators, and customer service agents), and more experiments with universal basic income. Exponential change feels gradual, then sudden. We’re currently on the flatter end of the AI curve, where people can criticize predictions of major disruption because they don’t see it now. But I think that would be a mistake. Exponential change doesn’t usually feel exponential initially.

8

5 Comments