Roy Ben-Alta

Gemini in BigQuery changes the game for data engineers and analysts! More productivity ✔️ Improved query performance ✔️ Optimized costs throughout the analytics lifecycle—from ingestion and pipeline creation to deriving insights with AI ✔️ #GoogleCloudNext

2

Open has become the state-of-the-art (for now), Meta have changed the game - yet again - with the release of the Llama3.1 suite of models. The acceleration in this industry is breathtaking. The headlines: * Llama 3.1 models are immediately available in Cortex today. ✅ * 405B packs a punch - out performing other leading flagship models across a variety of benchmarks. 📈 * 8b & 70b models have been updated too. The already capable models have seen significant jumps, particularly in math and coding, in just 3 months. 🚀 * Meta's 3.1 suite will be made available for serverless fine tuning. 🤯 * We've also released Snowflake Cortex Guard (GA) to safeguard LLM applications. 🛡

8

3 Comments

We are excited to announce the launch of Ursa engine, our Kafka API-compatible data streaming engine that is built on top of Lakehouse. Ursa is not something entirely new; it is built on our many years of experience building and operating #ApachePulsar and #ApacheKafka. To learn more about the #Ursa engine, I recommend our #PulsarSummit keynote presentation and announcement blog post. Here, I’d like to delve deeper into some of its features: 1. Making Kafka API a first-class citizen: In #ApachePulsar, we developed a unified messaging model combining queuing and streaming semantics into one protocol, aimed at enhancing data sharing between business applications. Although powerful and increasingly adopted by enterprises and unicorns, the requirement for application rewrites is impractical in today’s economy. Instead of building data gravity around a protocol, we open up access to the robust stream storage layer by extending support to the Kafka protocol, transitioning from a single protocol to a multi-protocol platform, facilitating greater flexibility and integration. 2. Achieve Stream-Table duality via Lakehouse Storage: Data streams commonly end up in data lakehouses, where integrating and maintaining them can be complex and resource-intensive. Ursa simplifies this by storing data streams as lakehouse tables in object storage. This integration not only streamlines operations but also allows seamless access to operational data for analytical processes. Thanks to its multi-modal storage engine which fosters data sharing across teams and departments. 3. Object storage as the primary storage solution: To better align with cloud economics, particularly for large workloads, Ursa makes ZooKeeper and BookKeeper optional and utilizes a cost-optimized write-ahead log based solely on object storage with leaderless and stateless brokers. This architecture leverages object storage for both data replication and storage, minimizing inter-AZ data transfers and reducing costs for latency-relaxed, high-volume workloads. By integrating these features at the multi-tenant level rather than at isolated cluster levels, Ursa eliminates costly data duplication and movement across clusters, allowing teams to map directly to tenants and be able to share data across teams and departments more efficiently. We believe that the future of data streaming platforms lies in being open and multi-faceted, fundamentally different from traditional data platforms. They are designed not just to store and process data, but to enhance organizational capabilities by fostering data sharing across the organization and integration with other systems. More and more data streaming platforms will look more like Ursa, with multi-protocol, multi-tenant, and multi-modal capabilities. Check out our announcement blog post about Ursa: https://lnkd.in/ghEKTYE6 #ApacheKafka #Lakehouse #DataStreaming

100

1 Comment

Kapil Khangaonkar, CEO of Clodura.AI, shares how Google Cloud fueled their growth: 71% less tech debt, 12x faster app delivery, & significant customer growth. 🚀 Ready to transform your startup? The Google for Startups Cloud Program now offers new benefits for AI startups, giving you the technology and resources to build easily. Take advantage of our open AI ecosystem and tap into the best of Google’s infrastructure, AI products, and foundation models.   Sign up today and unlock your startup's potential! #startups #cloud #GoogleCloud #CloduraAI #techstartup #startuplife #AI #buildwithAI #GoogleAI #GoogleforStartups #GoogleCloudforStartups https://goo.gle/GFSCP_CS Clodura.AI Google for Startups

3

❄️ Snowflake Is in Talks to Buy Reka AI for $1 Billion Reka supports multimodal input (image, video w/ audio, text, and documents such as PDFs)

2

Really interesting read on the evolution of Data Warehouses, Data Lakes and Lake Houses and how Databricks and Snowflake are battling for the top spot. https://lnkd.in/egg4P5VG This is one of the more complete analysis I've seen on the evolution of the analytics space and it's big gorilla's. In the end, it gets into the AI discussion of big LLM's vs. small models and how SLM's will have a place in the analytics world, which I completely agree with.

9

1 Comment

https://lnkd.in/dJW9sZpx SelfExtend introduces a novel approach to expanding the context window of LLMs. It leverages a two-tiered attention mechanism—grouped attention for long-range dependencies and neighbor attention for local interactions—both derived from the model's original self-attention. This method allows seamless context window extension without requiring fine-tuning, and extensive experiments have demonstrated its effectiveness across various benchmarks.

12

1 Comment

Google Cloud Vertex AI is rolling out multiple updates to make AI accessible for every enterprise. Up to 85% lower input costs and 80% lower output costs, and 100+ new languages coming to Gemini 1.5. #GoogleCloud #VertexAI #Gemini

9

Google Cloud Vertex AI is rolling out multiple updates to make AI accessible for every enterprise. Up to 85% lower input costs and 80% lower output costs, and 100+ new languages coming to Gemini 1.5. #GoogleCloud #VertexAI #Gemini

6

Cohere Command R/R+ models are now available on Amazon Bedrock. The models are particularly well suited for RAG use cases and are multilingual. #AWSome #GenAI #AmazonBedrock #Cohere https://lnkd.in/dTJC5Pix

4

🔐 Boost Your Snowflake Security with Key Pair Authentication In our new blog, we highlight how key pair authentication is transforming data security for Snowflake users. This method offers a robust, efficient way to manage access and protect sensitive information. Learn how you can implement this feature to enhance your security framework.

3

Looking to supercharge your product analytics? Snowflake's ASOF Join might be the feature you've been missing. ASOF join provides a performant alternative to window function queries for time-series data and joining based on proximity. It's a great fit for product analytics use cases. for example, you're tracking user sessions and feature releases for your product. You want to know which version of the product each user interacted with during their sessions. This query joins each user session with the most recent feature release at the time of the session's start. It allows you to: 1. Analyze user behavior in the context of product versions 2. Measure the impact of new features on session duration 3. Identify which product versions are associated with higher engagement Have you used ASOF join in your analytics? Snowflake blog post about ASOF join in the comments. #snowflake #DataAnalytics #TimeSeries

20

1 Comment

Exciting news! Snowflake has unveiled a cutting-edge system for large language models, featuring the powerful Llama 3.1 405B from Meta. This innovation brings a new level of efficiency and capability to AI applications. At 7Rivers, we're excited to harness this technology, enhancing our Data-Native™ approach to deliver unparalleled AI solutions for our clients. Our commitment at 7Rivers is to drive business value through use of intelligent data and advanced AI. Snowflake's latest offering, which allows real-time AI processing with minimal infrastructure, perfectly complements our mission. This advancement enables us to provide our clients with scalable, cost-effective AI applications, enhancing operational efficiency and driving impactful outcomes. The integration of Meta’s Llama 3.1 405B within Snowflake Cortex AI aligns with our Data Native™ Model, empowering us to create transformative solutions, optimizing processes, and creating a competitive advantage. Together with Snowflake, we're revolutionizing data usage to create intelligent, augmented business experiences. #AI #DataNative #7Rivers #Snowflake #GenerativeAI #BusinessTransformation

24

1 Comment

Dremio’s Subsurface Conference for Data Lakehouse is right around the corner and is free for you to attend on May 2-3. So you can’t miss it! 🔹Join 7K+ data experts in New York City or virtually to learn about the latest data lakhouse trends, including SOTA architectures, Genai in Analytics, Data Mesh, and more! 🔹Register here: https://lnkd.in/gxmBiGSc 🔹Hear what leaders such as Gable CEO Chad Sanderson and Nextdata CEO Zhamak Dehghani discuss trends and get inspired to deliver the next wave of data lakehouse innovation. 🔹There will be 8+ sessions on AI and 10+ exploring real world use cases. 🔸Bonus points for those who register. You’ll receive a free digital copy of O’Reill “Apache Iceberg: The Definitive Guide.” This list awesome of speakers only means you’re in for a treat, so hope to see you there! #data #genai #analytics #technology Click #linkedangle and follow for more content Set the 🔔 notification on my page, don’t miss a post!

49

18 Comments

Using Kumo.AI in your Snowflake environment is like peanut butter and chocolate - better together! Check out this webinar to learn more.

10

1 Comment

How India’s digital public infrastructure is enabling the emergence of state-backed ‘alt big tech’ "India Stack" Much of what was described as digital public infrastructure in this context flowed from India’s experiences with projects like Aadhaar for digital identification, the Unified Payments Interface for digital payments, and more recently, the account aggregator framework for data management in the financial sector. A host of such interventions built around the idea of standardisation of “open” application programming interfaces, or APIs, are collectively described as the “India Stack”. #digitalindia #upi #DPI #bigtech https://lnkd.in/g7tqyjSe

8

Really fun conversation with Joe Reis 🤓 on a variety of topics across the evolution in data architecture patterns, open table formats, the growing preference for object stores and data lakehouses. And of course the fun bits and challenges of building a company in the data infrastructure space. I've always enjoyed Joe's informal, no nonsense, common sense approach to making sense of the data world and was super kicked to have this conversation! #dataengineering #datalake Apache Iceberg Apache Hudi Delta Lake Apache XTable (Incubating)

37

1 Comment

This week marked a watershed moment for Snowflake: We launched #SnowflakeArctic, a state-of-the-art large language model uniquely designed to be the most open, enterprise-grade LLM on the market. Arctic not only outperforms many leading open models, but we brought it to market at one-eighth of the cost and in less than three months compared to other models. Arctic will significantly enhance our ability to deliver reliable, efficient AI to our customers. By delivering industry-leading intelligence and efficiency in a truly open way to the AI community, we are furthering the frontiers of what open-source AI can do. https://lnkd.in/eueT5Wcs

78

This is an interesting read. The changes that the broader AI ecosystem will bring to existing business practices and verticals will be a transformational journey for many industries in the years to come. Some of the capabilities highlighted in this article will eventually become mainstream. A few things that the entire tech industry has to work on to make all of the concepts outlined as mainstream are: Data Quality and Governance - Garbage in equals garbage out. There is lot of business "junk" that can lead to model hallucinations, inaccuracies, etc. that require various re-learning techniques and feedback loops to be implemented today. Without improving data quality and governance practices through automation, intelligent systems, you will have low quality outcomes which eventually won't augment your workforce. This vertical becomes inherently more complex with voice and video getting added to text-based data. New business models - A lot of products in the market need to rethink how they optimize for the outcomes and which outcome they want to charge their customers for. Most customers don't want to sign up for an outcome based model which charges for activities which are in the business workflow without achieving the final outcome. And sometimes the final outcome like closing a sale is too far in the cycle especially in the B2B space that it's not sufficient to provide a robust cloud-based solution. Cost-of-goods-sold - AI models are expensive to run and operationalize and over the years, this will evolve and R&D across the industry will help drive down the cost. This would need buyers to re-think how they buy and plan for costs, forecasting and planning and sellers to re-think their business models and monetization strategies. Multi-modal data management - For your models to be efficient from a business context, you need to be able to process all types of data that is generated: data stored in data management systems, voice-related data from calls, presentations and video-related data from trainings, knowledge bases, news, etc. Managing multiple data management systems was complex with the data sets different industries like banking, financial, energy, healthcare, auto, etc. generate. It will be inherently harder to manage the additional types of data in a cost effective manner. We will see a boom of additional cloud computing power and a different level of computing efficiency (performance and price-per-performance) that would have to evolve much more than what we have today in the market. These are just a few of the innovation areas that will define how AI shapes up and becomes mainstream. This is a generational transformation that we are living through and a lot of innovation will have to happen in the above areas in the coming years. #ai #industrytranformation #data

32

1 Comment