Josh Klahr

This is a huge step towards aligning the open data lakehouse formats. Recently Microsoft announced compatibilty with Snowflake for their fabric OneLake which was already compatible with Databricks. The future of data lakehouses is open and interoperable… your storage format should not constraint you from using your favorite compute engine..

14

The Information has a great write-up about the intense battle for foundational data storage companies namely Snowflake and Databricks and who really has the pole position. It is a great read for a part of the technology stack that frequently does not get the headlines they deserve. What do you think? #mangement #venturecapital #privateequity #cloudcomputing #capitalmarkets

1

Modern GTM Data should look like modern data for LLMs Harry Stebbings episode with Alexandr Wang a few weeks ago smacked us in the face with this. Alex is the leader in the #AI data space. He said frontier data is what is going to power these models going forward. New data, complex data and data that is created by leveraging our intellectual capabilities in reverse engineering how we as humans would interact with models.  Chain-of-thought prompting is a prompt engineering technique that aims to improve language models' performance on tasks requiring logic, calculation and decision-making by structuring the input prompt in a way that mimics human reasoning. This is required after you apply the reasoning that enables AI models to break down complicated problems into easy-to-handle intermediate steps, leading them to solve complex multi-step reasoning problems. Wait? Didn't I just do this in Clay in a demo two weeks ago? Isn't this all the rage in GTM data? YES - it is. GTM Architecture isn't changing - it has changed. The architect is the most important player right now. We need to reverse engineer how to be successful with GTM data which means we get meetings, we provide value and we get a mutual exchange with our target audience in a consistent and repeatable manner.  You need the basic building blocks to chain these thoughts together: 𝐃𝐚𝐭𝐚: Can't just use the same old data providers and win. Dozens of data sources need to be combined, enriched, verified and 𝐚𝐧𝐚𝐥𝐲𝐳𝐞𝐝 𝐭𝐨𝐠𝐞𝐭𝐡𝐞𝐫. 𝐄𝐱𝐞𝐜𝐮𝐭𝐢𝐨𝐧: This can all be automated. No need to pay someone for outcomes you expect from them when they have to design this manually and at best will take them 40% of their time but they have no experience in this. It isn't their fault! 𝐈𝐧𝐩𝐮𝐭: What do you do to the data that makes it more meaningful? Hiring, spending, technology, patterns, competitive information.  So much more. 𝐀𝐈: Yes. We need #AI and it should be used. However, prompting is a skill and needs to be done right and yes - humans need to be in the loop 𝐄𝐦𝐚𝐢𝐥 𝐒𝐞𝐪𝐮𝐞𝐧𝐜𝐞: You can't rely on the old way of delivering. Need multiple domains, need them rotating, need verified emails and you need a system that runs lights out that you don't support 𝐓𝐫𝐢𝐠𝐠𝐞𝐫: When is the best time to fire my cannon? 𝐑𝐞𝐥𝐞𝐯𝐚𝐧𝐜𝐲: Why does this even matter, at what point, what is the value? 𝐕𝐨𝐥𝐮𝐦𝐞: Less will be more. Shotgun approach and spray and pray looking at open rates and applying the wrong math won't work. A lot has changed. The challenge is fun. Data used in AI systems can teach us GTM folks a lot. When patterns tell you what is going to happen - listen to them.

22

1 Comment

Sitting in on an insightful talk from Noah Arliss and Hossein Ahmadi on core platform performance improvements at Snowflake. If you want to learn about the impact of these improvements (as measured by the Snowflake Performance Index) and also catch up on all of our latest platform announcements, follow this link: https://lnkd.in/grNNtsQe

117

ATTN Snowflake ❄️ and Databricks 🧱 users 👇 There's a new company in town called Espresso AI ☕ Very simply put: They are leveraging #GenAI and #LLMs to optimize SQL queries and cut your Snowflake bill by 50-70%. (If you're a Databricks customer they have a waiting list). I sat down w/ CEO & Co-Founder Ben Lerner (ex #Google and #Uber). They have a highly technical founding team with *deep* expertise 🤖 If you're leveraging a data warehouse then checking out their solution is a no-brainer -- it's a 10 minute install that can save your business hundreds of thousands of dollars. And now I bring you Espresso AI on the 10K podcast: https://lnkd.in/eD-78UHM

14

1 Comment

Databricks disclosed that they are expecting to cross $2.4B in revenue run-rate growing ~65% Y/Y by H1’25 (Jan FYE). This was alongside top-tier metrics, including 80%+ GMs, 140%+ NDR, and a multi-product story where ~50% of customers use 6+ products (incl. their Data Warehouse product which generated $400M in revenue up 4.0x Y/Y). Given their accelerated growth, Databricks is among the few scaled software companies, alongside hyperscalers, capitalizing on the AI boom. Equally noteworthy is their market share gain against competitors. Considering their growth trajectory, we explored various scenarios to determine when Databricks will surpass Snowflake in scale. The question is not if, but when, and we believe it is sooner than most anticipate.

1,005

26 Comments

Really interesting read on the evolution of Data Warehouses, Data Lakes and Lake Houses and how Databricks and Snowflake are battling for the top spot. https://lnkd.in/egg4P5VG This is one of the more complete analysis I've seen on the evolution of the analytics space and it's big gorilla's. In the end, it gets into the AI discussion of big LLM's vs. small models and how SLM's will have a place in the analytics world, which I completely agree with.

9

1 Comment

"Every business wants to be an AI business" - I had an opportunity to attend the Data+AI Summit last week and was blown away by the pace of innovation and the business outcomes the companies can drive for their customers. I put down some of my thoughts in this blog post. I couldn't be more excited about our mission and the platform we are building to help our customers in this journey along with our partner Databricks.

21

Our team at Databricks keeps pushing the bar higher on Data Intelligence. Data practitioners can now leverage AI in their data journey using the Databricks Assistant, which is now generally available to our customers. Very proud of the team's accomplishment 🎉 #Databricks #DataIntelligence #AI

5

#Snowflake has invested in #Metaplane, a Boston-based startup that provides an AI-powered data observability platform. The investment will lead to tighter integration between Metaplane's offering and the Snowflake data cloud, enabling Snowflake customers to monitor data quality issues across their data pipelines and downstream applications, including AI projects. Snowflake's investment in Metaplane looks a smart move that addresses a critical pain point for enterprises embarking on AI initiatives. Data quality issues can derail AI projects, leading to inaccurate insights, flawed decision-making, and ultimately, wasted resources. The deeper integration with Snowflake should allow customers to monitor data assets moving through different stages of the pipeline, receive alerts about issues, and get guidance on resolving them. #AI #EnterpriseAI https://lnkd.in/eyg6p7TW

7

We are very excited to hear about Snowflake Artic. Of many possible applications, one key data pipeline step it can revolutionize is data cleanup. Here are a few data cleanup steps I’ve built that could have been replaced by LLMs: ▪️ Standardizing a person’s name across different data sources. Ex: Mapping Robert, Rob, Bob, R. => Robert ▪️ Standardizing a job title across sources. Ex: Mapping Software Engineer, Sr. Software Developer, Software Hacker => “Software Engineer” ▪️ Extracting the City, Zip, Country from the address. Ex: “123 W 83rd Street, NYC, 10024, US” => City:New York, Zip: 10024, Country: USA ▪️ Standardizing dirty event data: Ex: product_clicked, product_clicked_ios, product_clicked_top_of_page, product_clk => product_clicked Often data teams waste time building and maintaining these data cleanup pipelines. RudderStack Profiles makes it easy to integrate LLMs into your c360 data pipeline to handle cleanup. What data cleanup tasks do you want to eliminate with LLMs?

90

3 Comments

Big announcement today- we’re unlocking Snowflake data for everyone!  With Snowflake Native Amplitude, you get all the benefits of Amplitude without your data ever leaving Snowflake. That means one single source of truth for all your enterprise, financial, and behavioral data to inform better product decisions. Learn more:

354

34 Comments

*** ANNUAL SNOWFLAKE SUMMIT PREDICTION THREAD *** Snowflake Summit is next week - and you know what that means: an “avalanche” of big announcements from one of the world’s most important technology companies. Matthew Baron, Jake Graham and I sat down and put together our list of predictions for some of the big swings we'll see from Snowflake. Here are our top five: → Snowflake is going to make a series of big, AI-focused announcements centered on Cortex. This will likely include the GA of features currently in preview, the launch of new features into private preview, and real success stories of enterprises using Cortex as a centerpiece of their AI strategy. → Snowflake will double down on their recent OneLake integration. All these platforms are trying to play nice, because the modern enterprise uses several platforms, not just one. → Snowflake makes a major acquisition announcement, but it's NOT something to do with AI. Instead, it’s something that reinforces its core data lake/warehouse foundation. This acquisition will expand on their efforts to integrate seamlessly with the various data lake formats (Delta/Iceberg/Hudi/X-Table) and extend their interoperability across other data platforms (Fabric/BigQuery/Databricks). → Snowflake is going to GA Snowpark container services, along with the launch of native applications running in the Snowpark container service. Hex is going to be front and center of that list, given the recent funding announcement. → Snowflake is going to launch a new, highly-strategic connector. They already have ServiceNow, Salesforce, Google Analytics. Our best guess? Workday. Sadly, this year's summit won't be in Vegas, so we're unable to place any actual bets on these announcements; LinkedIn prognostication will have to suffice for now. But curious for your view - are there any predictions we missed? #snowflake #snowflakesummit #snowpark #onelake #ai

35

3 Comments

For those who missed the Snowflake Data Summit keynote; here’s my GeneAI summarization :-) 1) Sridhar the new Snowflake CEO is smart as a pup and amazing academic and career pedigree, but has very little charisma or public speaking skills. Hope he improves on the latter over time. 2) Jensen zoomed in for his Keynote because he is still in Taiwan for Compudex. Jensen has labeled Taiwan the heart of GenAI Compute, all of Nvidia (and AMD)’s major suppliers are in Taiwan. It’s amazing this little island the size of California holds the future to AI. 3) Snowflake and Nvidia announced their partnership. Basically it’s combining Snowflake LLM Datastore with Nvidia LLM training and inference/generate. This is revolutionary. Snowflake pioneered the separation of compute from storage in massive EDWs, and now the circle is completely with bringing Nvidia compute training and inference/generation close to storage, eliminating bandwidth cost, latency and data policy/privacy issues. And as someone with Information Retrieval background, I see GenAI has come full circle with RAG - it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again. Except this time with supercharged GPU SoC that is improving at 16x Moore’s Law. We live in exciting times.

29

8 Comments

Thanks to Snowflake for selecting Immuta as Data Security Partner of the Year for the second year in a row. I cannot say enough positive things about how our teams have worked together to deliver the best experience possible for our customers. From JP Morgan to Roche to Booking.com -- our partnership is driven by the most complicated customers you could imagine. This year we have jointly worked to optimize query performance this year (boring but a huge deal to our largest customers), added granular security support for Iceberg, and integrated Horizon + Polaris. Oh, and we were able to integrate into Cortex to provide native RAG controls with dynamic tag-based filtering in just a couple weeks. Our success is purely due to our customer-oriented cultures. As customers hit snags, we work together to deliver fast, no matter how big or small the issue. The humility and mission focus allows us to solve customer problems together that others can't. Thanks Christian Kleinerman, Sridhar Ramaswamy, Raja Balakrishnan, John Sapone, Matthew Rose, Lisa Arbogast, Vikas Jain, Artin Avanes, Alex Jendzejec, Steven Touw, Moritz Plassnig

488

26 Comments

Headed to #DatabricksSummit? Check out definity's mission critical data observability platform and ping Roy!

7

Yesterday’s keynotes at the Databricks Data + AI Summit were electrifying. The atmosphere was buzzing with anticipation as Ali Ghodsi took the stage. It was great to be at the event in the company of Jordan Legg. The official announcements are available on Databricks' site, but I find the real value in the golden nuggets of insight from the speakers. One striking takeaway was that 85% of AI Proof of Concepts (POCs) never reach production due to poor governance. This highlights the need for robust frameworks to ensure ethical and effective AI use. As a serverless technology advocate, I was thrilled to see Databricks transition fully to serverless over two years, removing the need for cluster management and reducing operational overhead. Their next project is enabling data ingestion on GPUs, promising faster data processing and lower power use, impressive given their daily ingestion of over 2 Exabytes! Kasey Uhlenhuth’s live demo, despite technical hiccups, was handled with grace, earning commendation from Jensen Huang. Her demo showcased Databricks' impressive capabilities, and I’m eager to try it myself. Jensen Huang's interview with Ali Ghodsi was insightful, especially their discussion on GPU-accelerated data ingestion—a joint effort with NVIDIA to push data processing boundaries. Professor Fei-Fei Li’s talk from Stanford was the most inspiring. Her insights on Spatial Intelligence and AI's infancy were enlightening. It's easy to lose sight of the big picture amid daily AI development tasks. Ali Ghodsi’s final thought resonated: “Every company is an AI company,” emphasizing that businesses are sitting on untapped data. Perhaps it's more accurate to say, “Every company COULD be an AI company.” The future will see those who harness AI's power and those left behind. Jensen Huang summarized the urgency perfectly: “The field is moving so fast, you don’t want to be watching an exponential trend. Get on the train.” At takara.ai, our mission is to help companies ethically leverage AI and turn those POCs into production. We tailor AI solutions to your data and needs. The key message? Don’t wait. Start now.

12

6 Comments

Fantastic to see our good friend Dash Desai walk through an example of how to build sophisticated apps in Snowflake with RelationalAI Our goal at RelationalAI has always been to power every decision with intelligence and enable users to do more with less effort. Thanks to our partnership with Snowflake and our deep integration with the AI Data Cloud, that has never been simpler. The best has yet to come, stay tuned! #RelationalAI #KnowledgeGraphs #Snowflake #AIDataCloud

43

4 Comments