Edison Liu

Level up your research game with our ultimate guide, "Everything You Need to Know About Quantitative Research." Uncover the secrets to crafting killer research questions, automating like a pro, and nailing data quality. Get ready to revolutionize your quantitative research - grab your copy now: https://bit.ly/4aPQE1k #MarketResearch #QuantitativeResearch #DataInsights #ResearchTips

7

Level up your research game with our ultimate guide, "Everything You Need to Know About Quantitative Research." Uncover the secrets to crafting killer research questions, automating like a pro, and nailing data quality. Get ready to revolutionize your quantitative research - grab your copy now: https://bit.ly/4cbdX6O #MarketResearch #QuantitativeResearch #DataInsights #ResearchTips

6

AI generated synthetic data can help us avoid or reduce the impact of bad respondent / bots. So how big of an issue is bad data? It depends on your study type. Let's take a look at a few examples. 1. Low incidence studies have higher proportions of bad respondents. If you answer randomly, you are more likely to randomly select than you own a helicopter than a good faith respondent (also, sometimes they can detect the screening intentions of the survey writer). We recently implemented a new smart machine learning method of automatic bad respondent detection (very proud of this! Might have to make a whole post on it). In short, it detects self consistency patterns across responses to multiple questions and gives a quality score. We found that in a high incidence study, there actually were much fewer bad respondents than with other low incidence studies. 2. Virtual shopping studies (e.g., conjoint, shelf tests) are significantly impacted by bad respondents. If you answer randomly, by definition, you don't make selections based on price. To the underlying mathematical model, this doesn't randomly affect the betas, it specifically targets the pricing coefficient to suppress it. If ignored, this leads to simulators with very low price elasticity (of course, some categories are legitimately low price elasticity). 3. Segmentation studies are also impacted in a specific way. Bad respondents aren't internally consistent so they usually cluster together near the origin in multidimensional space (see pink group in the image below in 2d space). If bad data are not removed, this group sometimes appears to be very weird, over indexing on low incidence stuff and under indexing on high incidence stuff. If you also have a low incidence product, they'll appear very interested and you have a sinister combination! This isn't a doomsday scenario. Bad respondents aren't impossible to deal with if you know how they manifest. But we are still working on high quality AI generated synthetic data solutions to improve results further. 

31

2 Comments

Be inspired by top Insights & Analytics leaders via The Path https://lnkd.in/gtcBymxV. Our April leader is from Visa – Jagruti Jain is Head of North America Insights & Analytics and Measurement. Hear why Jagruti thinks market research is like having a crystal ball. Register to receive regular updates on featured leaders throughout 2024. #thepathtoinsights

1

This article has great suggestions on how to incentivize respondents and increase response rate. Virtual Incentives

17

So why is permissoned data so important? here's my article on Gen Z permissoned data published in MarTech Zone: https://lnkd.in/dmKV_Pvi

14

Having experienced the future of data collection with Conversational Surveys, first hand! I must admit that it is set to change insights gathering process by engaging customers in natural conversations. We used Chatbot to create dynamic surveys that adapt to responses, enhancing engagement and data quality. With this approach, we can help with enhanced user engagement, higher quality data, and improved completion rates. #ConversationalSurveys #CustomerEngagement #MRX #MarketResearch #MachineLearning #AI

14

Check out The Path https://lnkd.in/eiWKhfBt for career reflections and advice from top Insights & Analytics leaders, across a variety of industries. What does it take to become the Head of North America Insights & Analytics and Measurement for Visa? Take a look at Jagruti Jain’s career path for inspiration. Register to receive regular updates on featured leaders throughout 2024. #thepathtoinsights

1

It's been one week since the Analytics & Insights Summit hosted by Sawtooth Software, and I'm still savoring all that I learned and the new contacts I made. Curious about some of the take-aways? Take a look at this short piece I wrote to summarize the event. Enjoy! TRC Insights #marketresearch #analytics #insights

21

5 Comments

For over a decade, Caleb Benningfield has been at the forefront of developing customer data solutions. His post details the ongoing struggle to dismantle the longstanding silos in enterprise data management and describes how the lakehouse model is a significant advancement, combining the benefits of data lakes and warehouses while adding seamless data sharing capabilities. This integration allows each application to optimize its compute capabilities and features, while all data can be accessed through a single, unified interface. This not only simplifies data governance but also enhances flexibility, freeing up data engineering teams to refine and expand the data ecosystem rather than being bogged down with integrations. Do take a moment to read Caleb's post and explore how these insights could influence your strategies and solutions.

1

Last week, we published results from a gen AI synthetic data experiment we ran. We used gen AI synthetic respondents to quantify preference for different movies at the box office. We compared those preferences to the actual domestic box office sales, which we considered a "ground truth" for underlying preference. We hypothesized that the synthetic respondents would accurately capture movie preferences when the model has those preferences baked into its training data with the inclusion of social media posts and articles about the movies. But we hypothesized that the synthetic respondents would not be effective to tell us preferences for movies the AI model was not aware of yet. Methodology Like good data scientists, we designed our experiment to have a train, validate, and test component. We used movies from 2018 as a pseudo "train" dataset. We refined our prompts, synthetic sample plan, and pipeline for 2018. When that worked accurately, we then used movies from 2019 as our "validation" dataset to verify our process for 2019. Lastly, we applied the same process to 2023. (We skipped movie years 2020-2022 because of the impact of the pandemic on the movie industry.) 2023 is important, because we had AI models that were trained up until Sept 2022. Results 1. We accurately quantified 2018 and 2019 movie preferences. This means our synthetic data generation process worked. 2. We could not accurately predict 2023 movie preferences. This is further exacerbated if we remove sequels from the 2023 movie list (the model knows sequels do well, but that's not exactly enlightening, as movie execs already know that and pump out sequels every year). Implications The gen AI synthetic data process can work effectively as a layer on top of social media analysis. However, that did not extend to making predictions about the future. Of course, there are many ways to create synthetic respondents. But we used the same process to accurately predict movies that the AI model already knew about and other movies it didn't know about and it performed far worse for forecasts compared to backcasts. Dig Insights Upsiide

58

18 Comments

Myth: AI enabled Market Research relies solely on synthetic data, which does not reflect genuine user behaviors, opinions, interactions, and trends. Fact: AI enabled Market Research predominantly uses real data to provide valuable insights into real-world user behaviors and preferences. This data is collected from various sources, including social media, review platforms, e-commerce platforms, Google, first party data, and other forms of user-generated content. By analyzing this real data, AI can uncover patterns and trends that help businesses understand their audience and make informed decisions. While synthetic data is an important tool in the AI toolkit, it serves a different purpose. Synthetic data is artificially generated and is typically used for training machine learning models, testing systems, preserving privacy, and developing algorithms. It is especially useful when dealing with limited, sensitive, or confidential data. In short, AI-enabled market research relies on real data to capture the authentic voices and actions of users, offering a true reflection of the market landscape. Synthetic data, on the other hand, is a separate concept used primarily to support the development and testing of AI systems.

24

1 Comment

This shows the trade off between realism and diversity in image models, a very important ethical topic to study. If this generalizes to LLMs, it is additional evidence that gen AI synthetic data will perpetuate inherent bias in the models. The world is _very_ diverse (and real). But the training data that goes into the models is not nearly as diverse. This is a big issue. Until the internet is equal access across cultural groups, this could continue to be a problem. Realism in image models is different than realism in language models, but it’s been demonstrated over and over that language models have a bias towards the Western cultures that make up the primary user base. The current capabilities of LLMs do not support realistic writing in the voice of diverse cultures. It’s an analogous issue. This is a solvable problem with higher and higher quality and quantity of diverse original content, but that is so far proving very difficult to find. The full article is here https://lnkd.in/guKg9vw7 Dig Insights Upsiide

13

3 Comments

Delving into the Rise of "Delve" in Research Writing An analysis by Philip Shapira has uncovered a notable surge in the use of the word "delve" in the titles and abstracts of scientific papers indexed in PubMed and other scholarly databases.[1] Shapira found that while mentions of "delve" were gradually increasing over the past decade, there was a sharp spike starting in late 2022 that coincided with the release and rapid adoption of ChatGPT and other AI language models.[1] This pattern suggests that many researchers may employ AI writing assistants like ChatGPT to help draft their manuscripts. These language models are known to overuse certain words like "delve" to sound more formal or authoritative.[1][2] Key Takeaways The frequency of "delve" in PubMed publications rose by around 400% in late 2022 compared to previous years, mirroring ChatGPT's emergence.[1] This spike likely reflects the increasing use of AI writing tools by researchers, which tend to overuse words like "delve".[1][2] While AI assistance can be valuable, there are concerns about transparency if authors do not properly disclose the involvement.[1] The study highlights AI's growing impact on academic publishing and the need for norms around ethical AI usage.[1] As generative AI capabilities continue advancing, Shapira's insightful analysis provides an important window into the rapidly evolving landscape of scientific writing and communication.[1] #AIinAcademia #ResearchInnovation #EthicalAI #AcademicWriting #TechInEducation Citations: [1] https://lnkd.in/eTXT9jz8 [2] https://lnkd.in/emRdpw22 [3] https://lnkd.in/eXEU2tat [4] https://lnkd.in/ejE_mAWT [5] https://lnkd.in/et9rJujF

2

Crafting The Future. Featuring Arun Rajagopal Ep. 39 OUT NOW - https://lnkd.in/eAqUXyjF In this episode, Arun Rajagopal, Senior Director of AI and ML at adMarketplace, discusses the integration of AI in advertising and search optimization. Snippet question - "Are you able to share insights into the newest AI-driven feature of innovation at adMarketplace?" #ai #artificialintelligence

27

3 Comments

Mintel has launched an AI to help you tap into our massive depth of data and market reports in seconds. I am truly proud of Mintel and extremely excited about this application of AI. The hype around LLMs such as Chat GPT has been incredible this past year. And for once, I think the hype is actually well founded. I'm proud of how quickly Mintel has responded to the general availability of this technology. Mintel Leap is an actually useful and trustworthy implementation of generative AI. While generative AI has been shown to produce some very interesting content, I have long been an advocate that this technology should augment human intelligence, not replace it. Mintel Leap answers your market questions by citing Mintel's deep research, data and human-generated market reports. It helps you synthesize our data and get you to the insight faster. Like, super fast. I'm excited for how this is going to impact our customers. Customers that have long trusted our data and research and who can now shave days/weeks from their processes.

95

3 Comments

This essay gives voice to the Silent Majority of SMEs in data-centric roles who are seeing organizations say and do things regarding AI that are probably very silly. Too many great quotes to pick out just 1-2, so do yourself a favor and read the whole thing. It's a breath of fresh air -- easily the best thing I've read on the topic in a long time. https://lnkd.in/g8PtmYVp

29

8 Comments

Eye-Tracking + Conjoint Virtual Aisle Enhanced Data Quality By integrating both methods, researchers can gain a holistic view of decision-making processes, combining what consumers say they prefer with how they visually interact with options.   Improved Insights Eye-tracking data can validate conjoint responses by showing whether stated preferences align with visual attention patterns. Eye-tracking can reveal which attributes or features draw the most attention, helping to interpret which factors truly influence consumer choices in a conjoint analysis. Informed Decisions Group, Inc #Conjoint #EyeTracking #ShopperInsights

4