Herve Marcellini

Now we have two first class data catalogs integrated with MS Fabric now available in Azure. Of course DataBricks is the heavy heavy favorite in the enterprise lakehouses as well as AI (as of the conference that just finished 🤓)

6

pgvectorscale: A Game-Changer for Vector Similarity Search 🚀 Timescale's #pgvectorscale extension for PostgreSQL has achieved a remarkable milestone, matching the performance of Pinecone, a dedicated vector database, while being 75% more cost-effective. 💰 In their latest blog post, Timescale showcases the impressive speed and efficiency of pgvectorscale, which leverages the power of PostgreSQL to deliver lightning-fast vector similarity search. This breakthrough opens up new possibilities for applications like semantic search, recommendation systems, and more. 🔍 Key takeaways: 🌟 pgvectorscale now matches Pinecone's performance in vector similarity search 💸 pgvectorscale is 75% more cost-effective compared to Pinecone 🔧 Easy integration with existing PostgreSQL databases 🌐 Scalable solution for handling billions of vectors ⚡ Optimized for fast and efficient search results pgvectorscale's seamless integration with PostgreSQL allows developers to incorporate vector similarity search into their applications with ease, while benefiting from the robustness and scalability of a proven database system. 🗄️ If you're looking for a high-performance, cost-effective solution for vector similarity search, pgvectorscale is definitely worth exploring. 🔍💡 #pgvectorscale #postgresql #vectorsimilaritysearch #semanticsearch #recommendationsystems #timescale https://lnkd.in/gsfgCK32

4

I've been playing around with creating an opinionated AI Deep Tech Scale to help non-experts think about their strategy. Exploring this scale: Understanding how companies strategize their identity, team structures, costs, and activities is important. Success isn't confined to creating "God Models"; companies can achieve significant success by "connecting power to the people" and maintaining model agnosticism. For example, some might argue that SaaS companies are merely wrappers around database technologies, yet it's undeniable that they have generated significant value. #AI #DeepTech #Strategy #NonExperts #SaaS 

8

Less experienced engineers tend to rely too heavily on AI model outputs. However, these solutions are only as good as the data they were trained on and the prompt given by the user. Blindly trusting AI can lead to costly mistakes. My advice? Use AI tools to brainstorm and explore different designs to solve the problem. Seek the concepts behind the proposed solutions and craft the implementation together with the tools. Focusing on the problem is the right approach, and it's great to know that professors are investing time in this area, https://lnkd.in/eRUVvwmi.

11

2 Comments

If you're running a data or AI-focused startup, connect with Stephan Goupille and his team! When three of the industry's brightest data minds join forces to back your venture, you get more than just funding – you gain invaluable operator expertise. They understand the challenges and opportunities you face because they've been there themselves.

6

Data is like crude oil. Any companies working on foundation models would protect it fiercely - it’s a competitive edge. This is unlikely to change any time soon. Oil refinery needs raw material. Absent data, a foundation model underfits. The leading models on LMSYS Chatbot Arena Leaderboard are fundamentally based on transformers. If model architecture does not give an edge, what does? Yup, one big area is data! This is especially the case where data is scarce: 1) highly specialized domains (financial trading, biomedical research, medical information, etc) 2) enabling new behavior (how the model determines the right function to call, how model understands in-depth video content, etc) These limitations cannot be solved with more compute. The model will perform poorly on areas lacking enough data. Furthermore, for new model behavior, the model cannot even do it to start with. Even if it can, how reliable is it (for practical applications)? Even with more public sourced datasets, the competitive edge will still be the additional data that your competitors do not have. Lastly, a fundamental pinning is that machine learning needs data. In fact, lots of high quality data. Even with a new paradigm shift, this will remain true. https://lnkd.in/gwbUFyFR #llms #foundationmodels #datasets #datageneration

9

4 Comments

The afternoon Keynote @ Stripe Sessions was Fireside Chat between Patrick Collison (CEO of Stripe) and Jensen Huang (CEO of NVIDIA). Patrick was an excellent moderator — asked Jensen really probing, nuanced questions that Jensen answered with his usual sharp wit and unique (and impressively well rationalized) perspective. A couple of quotes among many that stuck with me: - “Current CPU installed market is $1 trillion of hardware, all of which will be upgraded to GPUs over the next 5 years. And I would estimate about another $1 trillion will be spent on top of that during the same time period as companies try to do more with AI. Add those together — and I think at least $2 trillion will be spent on GPUs in the next 5 years.” - “I like to go after Zero-Billion Dollar Markets — nonexistent markets that I envision & can rationalize well, but will be very hard, expensive & time-consuming to achieve.”

1

At the core of effective engineering leadership is the ability to stay connected with your team. This is perhaps the most challenging aspect because retaining the respect of your people is difficult unless you remain technically involved yourself. In my journey as an engineering leader, staying true to the individual contributor role that originally earned their respect has kept me close. This approach extends beyond technical expertise; it's about maintaining a deep connection with the core activities that propel product development forward at Gearflow. One of my biggest professional fears is becoming irrelevant or disconnected from my team and our work. This fear motivates me to continually reinvent and retool in response to market demands. Next month, I'll be sharing about my adventures generating synthetic data recently. And while I'm no expert when I start preparing for these talks, the process is incredibly valuable. It's less about delivering a perfect talk and more about how the preparation expands my thinking and exposes me to new capabilities. This helps me connect the dots to opportunities at Gearflow that bring real innovation to our customers. Speaking regularly holds me accountable and pushes me to learn ahead of the curve, ensuring I remain relevant and impactful day-to-day. Every talk is a journey of discovery, helping me to lead with humility, communicate clearly, and inspire a culture of innovation through a deep, technical understanding of our work.

19

Check out our blog comparing SingleStore vector search performance to Milvus, pgvector. https://lnkd.in/gfS82kmd We're on par with Milvus and way faster than pgvector, and we're getting better fast -- 2x to 4x perf gains or more in near term. And our index build times are much better.

27

Large Language Models (#LLMs) are evolving rapidly, and the demand for running them at scale is skyrocketing. #Ollama is stepping up to meet this need with its official container image on Docker Hub, excelling in local #deployments. However, there’s a catch for containerized environments: downloading #models requires the ollama serve command to be running within the #container. This extra step complicates automated deployments via #CI #CD #pipelines, making seamless model access challenging. To take your #model to production, it needs to be #container-ready—a feature Ollama has yet to fully support. The gap in #AIOps for smooth deployments gets blurred with the inference services available as an easy way to take it to production. Goal is to overcome obstacles and deliver solutions 😉 so here is to get you started on having your (O)#llama #production ready in a private #deployment from your #local to #cloud all in #oneshot . Enjoy Medium: https://lnkd.in/g9jqBQ8d dev.to: https://lnkd.in/gi_gGwam #AI #LLM #Ollama #llama3 #phi3 #mistral #S6overlay #AIOps #CI #CD #oneshot #container #docker #podmanEnjoy

11

Over the course of each month, I spend a few moments each day or so collecting interesting links, and then, at the end of the month, I bundle them (with the help of some nifty automation scripts) into an easily digestible reading list. Here's everything that I found interesting in May 2024 about the irresponsible use of machine intelligence: #qfm #quantumfaxmachine #blog #irresponsible #ai #genai https://lnkd.in/e2uE49up

10

I spoke with Eric Ignasik and Michael Sols from The Software House about dealing with silos at work. 🎙️ We discussed how Technology Managers can help their engineers gain deep operational expertise and take full responsibility for product/feature delivery. The best practices I described in this interview come straight from Payhawk. It doesn’t mean they can simply be copied — but they might be inspiring to some. Happy reading 👋🏼 ➡️ hubs.ly/Q02xn9j-0 #softwaredevelopment #cto #softwaredelivery

152

4 Comments

Check out this fascinating interview with Daniela Miao, CTO at #gomomento, where my colleagues Dave Neary and Aaron Williams dive deep into a discussion about caching to reduce service time, pubsub, tail latencies and more. In the interview, Daniela shares her experiences in porting and optimizing on #Ampere. "the portability was really amazing – we didn’t have to do much, and it just worked." Don't miss this insightful conversation about #cloudnative technology!

25

One of the challenges of incorporating AI into systems is that in some respects it's very similar to traditional software development, and in other respects it is not. Our software supply chain teams have been working for some time on how to translate software supply chain principles and practices to fit the needs of the AI supply chain. I'm very proud of the work our team and partners have done here. This report describes the key insights they have made. #ai #hardening

20

Can you trust AI? 🤔 For most companies the answer is no - at least using current foundational models out of the box. ❌ Problems like sensitive data leakage and toxic content are major barriers to implementing production genAI solutions. The route towards trust lies in system architecture and the good news is the field is advancing rapidly, including increasingly easily accessible toolkits of solutions to increase trust in AI systems. 🔥 This is the hot area for new releases from major AI platform players (along with AI security), including AWS (https://lnkd.in/e45S-ehS), DataRobot (https://lnkd.in/eCWvppWR), and Databricks (https://lnkd.in/e6A7cTcH). 👨‍⚖️ Guardrail or judge models - leveraging a second AI model to police the content coming out of an original - are probably the most promising scalable solution. For many the best route is likely to be a new class of foundational models specifically designed for this task such as Llama Guard and Databrick’s new judge LLM. There is also the ability to fine tune a model (think a discriminator model from a GAN architecture) or use a general purpose foundational model. 👮‍♂️ Classic filtering techniques for sensitive data can also be applied to the result of an LLM, whether for PII, company IP, or inappropriate content. This can include regex, classic PII models (such as Amazon Comprehend), etc. 🚻 Human in the loop techniques can also be used, but are far more scalable when used to calibrate automated solutions such as those described above rather than manually verifying outputs in realtime. What’s still missing? 🔎 🎯 While techniques to improve accuracy and reduce hallucination such as RAG and AI agents have made great strides, there remains a big gap in conveying a model’s confidence in its output to end users. 💡One idea is to leverage the probability distributions used for top-p (nucleus sampling) at the token level instead at greater aggregations to convey confidence levels for entire model outputs or significant segments (e.g. each step in a chain of thought structure or AI agents framework). #ai, #artificialintelligence, #GenAI, #GenerativeAI, #llm, #datascience, #machinelearning, #mlops, #bigdata

4

1 Comment

Have you digested the Situational Awareness paper yet? Have you recovered? An amazing document and analysis, even if it may feel hyperbolic. If you haven’t read up on it, Leopold Aschenbrenner, a former Open AI safety researcher, claims that we are on a path to achieve AGI and superintelligence by 2030. This analysis rests on a projection of the capital investment going into compute for AI and the likelihood of continued advancements in AI models from both algorithmic improvements and scaling compute. We can all pick holes at specific parts of the analysis (like I am about to), but I don't think any of us can ignore the possibilities raised in this paper. The AI arms race has truly begun and we dearly need our policymakers to be grappling, hard, with those implications. Now a bit about the paper itself.. Issue 1 - Leopold claims that GPT4 has “college-student level intelligence” based on the ability of the model to beat many of the standard tests we use to access college students. My problem is that beating a standardized assessment is not tantamount to “having the intelligence” of a typical human asked to take that test. I do think it is an amazing feat, but the LLM brings to bear processing power and training data volume vastly surpassing that of our human student. I am pretty confident there are lots of things the human student can do with their intelligence that GPT4 cannot. A simple example is the continuous learning from new info that the human can do easily. Leopold then makes the leap that if GPT5/6 shows a similar jump in performance relative to GPT2->4, then we will have “PhD intelligence” level models that could be trained as “AI researchers”. Once we activate these researchers then AI will self-improve at a dramatic rate until they achieve superintelligence. This doesn’t sound impossible, but again the idea that GPT6 wil be able to beat every “PhD level assessment” test we can give it does not necessarily mean that you suddenly have human-level AI researchers in the box. I think Leopold needs to revisit the Chinese Room and its implications. Moving further into the paper, Leopold talks about the dramatic national security implications of the first nation state to acquire “even early cognitive superintelligence”. This whole part sounds super scary - very similar to arguments that were made about the atomic bomb during WWII. My problem here is that the whole notion of superintelligence is that the machines are smarter than humans. I presume Leopold is too young to have any kids yet, but any parent will tell you that as your kids get smarter (by getting older), they increasingly refuse to do anything you tell them! So the idea that some government is going to order their new superintelligent AI to go attack their adversaries seems highly optimistic! Not that we won’t suffer some other terrible fallout from such a breakthrough, but it’s not obvious to me that super-AGI will just be easily corralled as a new weapon.

23

1 Comment

Tinkering fuels innovation and curiosity. It allows us to truly understand the tools and systems we work with at a granular level. Recently, I've been tinkering with Rust, a language gaining popularity for building reliable, safe, and sustainable systems at scale. The best way to learn is by doing. So, I turned our simple summarization proof of concept, Distill, into a CLI leveraging Rust's performance and memory safety advantages. As builders, it's our responsibility to continuously look for optimizations in everything we do. To ask questions and challenge the status quo. To learn and be curious. I encourage you to give Distill CLI a try, tinker with it, and share your feedback. And as always – now, go build! #AWS #tinkering https://lnkd.in/e7HUQwEt

628

20 Comments