Ishpreet Pandher

I am speaking at #DES2024. If you guys are interested please attend my session. It would be about how we can use the Embeddings for creating semantic search, custom #RAGs and how do we store them in #VectorDbs for faster Retrieval. I will be giving some practical examples of how we are using them in our products at #Bizom.

63

Recently had the pleasure of interviewing Kai Waehner, Confluent’s Global Field CTO for Dataquest at the Kafka Summit Bangalore. We discussed how Confluent is pushing the boundaries of streaming analytics and data management. Our conversation highlighted Confluent's commitment to robust data flow, seamless collaboration, and enhanced security. Exciting times ahead for real-time data! #DataAnalytics #Streaming #KafkaSummit #Confluent #datastreaming #kafka #ApacheFlink

47

In our industry, there are data "so-called experts" and hands-on data folks who solve complex issues and bring insights quickly. Naveen belongs to the latter group. When he pitched the idea of ChatWithCensus leveraging LLM-based technology, I immediately decided to join forces. After months of hard work, debates, and perseverance, ChatWithCensus is live. So, what does it do? ChatWithCensus simplifies engagement with ABS data. Users can ask questions in plain language, and our custom-built NLU engine converts these queries into SQL for rapid data retrieval.  Because it is simple and doesn't require complex loading or SQL coding, we hope this becomes the go-to tool for smaller research organisations. A huge thank you to those who made ChatWithCensus possible: - Naveen Y., my partner in crime, Thanks, mate! - Pere Martra, thank you for stepping in when we faced our first hurdle. - Stephen Johansen your mentorship and generosity are invaluable. - Belal Hossain, we wouldn't be here if it weren't for your late-night coding. - Asher Hussain Thank you for SSL magic! Again, thank you! - Prof. Jing and PhD student Haoyang Li, for their insights. Finally, I would appreciate it if you try it out and share your feedback: https://chatwithcensus.ai/

29

14 Comments

The latest from Belle Lin: Databricks is acquiring Tabular, a data-management startup that helps companies use a variety of open-source data formats, as the company looks to win artificial intelligence customers by making it easier for them to use their own data with AI. Databricks declined to specify how much it paid for Tabular, but said the price was between $1 billion and $2 billion. The deal comes as Databricks and other cloud data companies like Snowflake are on spending sprees��for the same AI clients—enterprises looking to spend big on using AI for business processes and building their own AI capabilities. Both Databricks and Snowflake provide a platform for storing, organizing and analyzing data across multiple cloud providers. “What we want to do is create custom AI on all of your data. That’s what we call data intelligence,” Ali Ghodsi, chief executive and co-founder of Databricks, said in an interview. “For that, we need all the data. The more data the better.”

15

2 Comments

Some thoughts/observations from my recent visit to the Databricks Data+AI summit. If you're a small enterprise:- Looking for a data lake/mesh+BI+AI/ML platform? Just buy Databricks and be done with it. At best, do a POC with another platform e.g. Snowflake and go for the lowest bidder. These are evolving rapidly and feature parity is a constant catchup so don't go only based on what exists today. If you're a query engine/federation tool (e.g. Starburst/Dremio):- Delta (the table format) has been open source for a while but Photon (the engine) isn't (and may never be). So Databricks has been under pressure from the vendor/partner ecosystem as well as large enterprises to support non-vendor-lock-in options. It's acquisition of Tabular makes this space interesting and here's hoping the ambition to support Iceberg as a first-class citizen is genuine. Snowflake also made the Polaris announcement for Iceberg support recently. Freshen up you're price/performance stats, you're going to need it. If you're a large enterprise Governance/Unity Catalog Soo many presentations all claiming Unity as the 'single' point of governance solving all their problems. But is any large customer really all-in on Unity? Nobody I spoke with can claim that 'single' is a reality. Now they all have one more catalog. Every large enterprise that uses Databricks also uses <name-your-fav-alternative>. While Unity helps with governance when data/models are accessed from within Databricks - it doesn't push down to the underlying external platform - e.g. it isn't going to generate Lake Formation grants for you. So you'll need two enforcement points for folks who access that Glue table from LF vs from Databricks/Unity. Immuta - can you wake up quickly to this opportunity pls? ABAC (Attribute Based Access Control) Zeashan Pappa/Kristen Wilder had an interesting talk on what's coming with Unity and ABAC. They're going with tags at the Account level as part of the design. Most Databricks customers will have one account for the entire org. Some very large customers probably have one account per LOB within the org. Either way, curious to know how many such orgs have governance centralized at such a high level instead of being federated deeper. If you are more federated, are we back to CS's fav problem - naming things? Domains/namespaces etc. E.g. two different parts of an org may not interpret 'client' the same way. ML capabilities and integrations Every AutoML vendor will need to compete with code-gen, it will not be sufficient for any auto-generated model to be adopted without a need to tweak the underlying code. The Unity and git integrations around MLFlow make it very compelling. How they continue to evolve Mosaic and better integrate into the platform will be interesting to follow. Still feels like very early days on that - feels a bit square-peg-round-hole still. All in all - tremendous pace of innovation. Lots of hype but much of it is deserved.

61

2 Comments

#AIforBusiness —> Right Model for the right use-case running on the right inferencing infrastructure to ensure unit marginal value > unit marginal cost. The probability of successfully moving from POV to scale deployment grows with —> (1) Trust in the data used to train the model… (2) Efficiency and accuracy in model performance and (3) flexability of deployment options. The IBM #Granite model family stands out for it’s performance attributes in these dimensions. See how it stacks up and how this is achieved…. Https://lnkd.in/eMkuX2an

34

Wow, what an incredible experience! While AI mega-conferences are everywhere, it was refreshing to join Nicole Kaufman, Tisson Mathew, Matt Marshall, and many other AI practitioners for an intimate discussion on real-world AI adoption. Many are delivering on board-level mandates to invest their firm's money and, more importantly, their teams' time in AI. It is also noteworthy how consistently conversations focus on return on investment--gone are the days when AI was purely an R&D effort!  However, those leading AI transformations must address a long list of challenges before an attractive return on investment can be realized. One that I have yet to hear discussed much is the behavioral design challenge à la IDEO frog etc. Often, teams need to adapt how they work before the full potential of your AI-powered system can be realized.   Marketing campaigns, for example, are not launched the instant that copy is created! If you want eye-popping marketing efficiency gains, your AI-powered system must be intentionally designed to help your teams adapt how they work. The good news is that your teams can do the behavioral design work while they address the data, technology, and governance work. In fact, it ought to be done before much of the data, technology, and governance work is undertaken. Thanks to the VentureBeat team for pulling together a great group for this leg of the AI Impact Tour. Thanks to Edge & Node for luring us all to the Presidio on a fantastic, sunny day. #ModernStrategy #EnterpriseAI #AI #DataCentric #Design #FutureTrends

29

👋Credo AI at Databricks Data + AI summit ! Last day today, come say hi and learn about Responsible AI governance ! We are at Booth E16. Dm Michelle Schifrin Brian Sweeney to learn more or see Credo AI in action.

104

2 Comments

Thrilled to share that kipi.bi has partnered with Dataiku to enhance our data science, artificial intelligence, and machine learning solutions on Snowflake! Tapping into the combined platform and services of Snowflake and Dataiku, we’re empowering organizations to extract actionable insights from their data like never before. Discover more about this strategic partnership here: #DataPartnership #Analytics #Dataiku #Snowflake

14

1 Comment

Faster, more accurate, less data needed = BaseModel.AI

2

The hunt for a pool of data scientists might be holding your digital transformation back. A new breed of leader is emerging: the AI Enabler. Here’s what you need to know about building your own class of AI Enablers: http://spklr.io/6040qSon #AI #GenAI #DigitalTransformation

2

The time of reckoning is upon the AI industry: Doug Gray and my gut wrenching and deeply researched book is now up for prepurchase on Routledge: https://lnkd.in/gfhH4WMM For those in my network who want to get a copy first, put a comment below "Discount Code" and I will DM you a 20% discount code while the book is on preorder. Once out, it will be too late for me to send. Early reviews are out and this book is going to create some big waves. #aifails #artificialintelligencefail #datascience #ai #analytics #artificialintelligence

41

15 Comments

In light of growing concerns about Delhi's increasing pollution levels, I recently undertook an analysis of real-time data collected from various monitoring stations on June 22, 2024 (Saturday). Utilizing Python for an initial data overview and deriving key insights, I then leveraged Tableau to create an interactive dashboard that visually represents the findings. For a detailed view of the analysis and to explore the dataset, please check the links in the comments section.

54

2 Comments

Good one. thanks for sharing Mark Henman. Early 2000's we read about these algorithms and probalistic models as great way to find popular patterns. Fast forward to now with great advancements in computing power, LLMs we can do much effective and faster way for automated topic model evaluation. #LLMs

1

1 Comment

This will contribute in making the GenAI race somewhat sustainable with reduced carbon footprint.

4

Meta has released pre-trained models that utilize a novel multi-token prediction approach for #LLM, potentially revolutionizing AI development. This technique, which predicts multiple future words simultaneously instead of just the next word, promises enhanced performance and significantly reduced training times. The new approach could lead to more efficient and sustainable AI, addressing concerns about computational power demands and environmental impact. Meta's multi-token prediction models may offer a more nuanced understanding of language structure and context, potentially improving tasks like code generation and creative writing. The company has released these models under a non-commercial research license on Hugging Face. The initial release focuses on code completion tasks, reflecting the growing market for AI-assisted programming tools. https://lnkd.in/eMBCXu8p

7

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

1

1 Comment