Andy Humeniuk

Implementing Real-time Stream Processing in ClickHouse with Kafka

2

CEO Spotlight: Lukas Gentele, CEO, Loft Labs: Creators of vCluster. $24million Series-A, April, 2024. ** Multitenancy in Kubernetes with vClusters ** * Why VCs are So Excited About vCluster? (13:03) (Remaining Time) * How vCluster Different from All Cloud Mgmt Tools? (11:56) * Why are vClusters More Secure? (10:06) * Why do vClusters = K8s Uptime and Resiliency? (9:24) * How big is the Future Demand for vClusters? (8:42) * How does vCluster make K8s Easier for Developers? (6:37) * New Partnerships? Rancher, Coreweave, Vault, TF, etc. (4:59) * What is the Ideal Customer Profile for vCluster? (3:40) * Preferred Method for People to Learn More? Slack. (1:04) #kubernetes #vcluster #loftlabs

33

6 Comments

Announcing Firefly Workflows—simplify the way you deploy infrastructure using Terraform with your existing CI/CD solution. 🚀 GitOps Automation → Meet IaC Accelerate your cloud operations and manage infrastructure with unparalleled confidence. With Workflows you can enhance your IaC pipeline with GitOps automation for superior efficiency and reliability, all while leveraging your existing tech stack. 🌟 Features of Firefly Workflows: ✅ Simplify IaC Automation: Automatically generate a new IaC pipeline or integrate into your existing pipeline in GitHub, GitLab, and more.  ✅ Gain Visibility and Insights: Visualize all planned changes to your cloud resources and show their relationships. ✅ Prevent Undesired Changes: Secure your code by automatically preventing policy violations and excessive costs. ✅ Get a Post-Deployment Rundown: View post-provisioning insights into cloud activities, streamline debugging, and manage your asset inventory. Couple Workflows with Firefly’s cloud control plane to solve cloud complexity. Link to try in comments 👇

59

3 Comments

Real numbers from AWS to Bare Metal migration AWS i3en.metal - 36 core, 96 vCPU - 768 GB Ram - 8x 7500 GB NVMe - 100 Gbps NIC $7,924.30/mo On-Demand / $5,396.82/mo 1 yr RI Bare Metal - 64 core, 256 vCPU - 1 TB Ram - 10x 7680 GB NVMe - 4x 100 Gbps NIC $1,126.00/mo Do you see it??? 3x the containers per node, 79% less $ per node 🤯 Now let's add in the egress AWS 20 PB Egress $0.027/GB $540,000/month Datacenter 65 Gbps 95th Percentile $0.25/Mbps = $16,250/month AWS $1.08M/month vs Bare Metal $50.03k/month And put it all together 👇 AWS $12.96M/year - EC2 100x $5,396.82 = $539,682 - Egress $540,000 Bare Metal $600.36k/year - 30x $1,126 = $33,780 - BW = $16,250 1 year savings = $12.356M What would you do with an extra $12M per year? How about hire more people and moving faster! 🚀 This is why I'll never stop talking about this If you have questions I'll share the numbers Comment, DM, or email me

25

12 Comments

AIQCon is coming up fast and we have a new speaker lined up for you! Cindy Xing, Software Engineer Manager at Google, is presenting "Open Model and Its Curation over Kubernetes". Open Generative AI (GenAI) models are transforming the AI landscape. But which one is right for your project? What are the quality metrics for one to evaluate his/her own trained model? For application developers and AI practitioners enhancing their applications with GenAI, it’s critical to choose and evaluate the model that meets both quality and performance requirements. Cindy will examine customer scenarios and discuss the model selection process. She will explore the current landscape of open models and collection mechanisms to measure model quality and share insights from Google’s experience. There are a limited amount of tickets left! Use code "BeOurGuest" to secure your seat for 20% off! https://lnkd.in/gxTzZSXR

5

Linkerd made a mighty splash at KubeCon EU, packing rooms with lots of service mesh talks. In their presentation, Linkerd maintainer Matei David and Kong Inc.'s Michael Beaumont explored the past, present, and future of sidecar containers in Kubernetes. They dove into the surprisingly long wait for native sidecar support in Kubernetes, tracing the KEP-753 (a proposal for a sidecar mode) from its 2019 inception to its 2023 arrival. The talk breaks down how the new sidecar functionality works, how it compares to previous workarounds, and the differences between Kubernetes versions 1.28 and 1.29. #Kubernetes #ServiceMesh #CloudNative #OpenSource

17

Speed up semantic search performance with token pruning for our ELSER model: https://lnkd.in/gePkdDpn #ElasticSearchLabs #SemanticSearch

19

This is such an important insight and a key reason why NATS.io is differentiated in this space. The Kafka model of topics, partitions, and keys are optimized for writes, but reads (consumers) don't have the ability to granularly filter on a subset of keys. The consumption pattern requires consuming everything on the partition and dropping data on the floor. NATS.io takes a different approach to persisted messages, we call streams. All messages are addressed by subject (https://lnkd.in/egvGwDaW) and this enables subject-based server-side filtering to be configured for consumers. As a side note, interesting you note EventStore! NATS.io have a growing community of developers that apply event sourcing and CQRS who are adopting NATS since it supports OCC on streams on a per-subject basis, messaging (for really performant pub-sub including request-reply), and materialized views like key-value and object stores. Folks will definitely want to check out the `event-sourcing` channel in the NATS Slack (https://slack.nats.io).

14

#Profesia per la #digitaltransformation:WSO2’s API Platform for Kubernetes Open Source Future with Sanjeewa Malalgoda Join WSO2’s Director of Engineering, Sanjeewa Malagoda, as he discusses WSO2’s commitment to open source software and our newest open source release: API Platform for Kubnetes (APK). Topics will include: - The future of API management. - Why WSO2 built the API Platform for Kubernetes...

1

Is write once, run once becoming a reality? Does #WebAssembly / #WASM present an alternative to creating and maintaining secure and optimised containers to run application services? #kubernetes #cloudnative https://lnkd.in/eEAUHeDm

🌟Happy new and (hopefully) spring month!🌷 I was fortunate to attend Postgres Day in windy Chicago this past weekend, and I had no idea what to expect but I think that made my brain more open to the content. For those who may or may not know, PostgreSQL - or Postgres, is a free, powerful, and open-source object-oriented database management system with a community actively involved in its development. Postgres is used by analysts, developers, database architects/administrators, and so on, and has incredible capabilities. As a junior analyst, I was intimidated by the experience and expert-level technical know-how in the room. But instead of shrinking into a corner, I decided to try and fill my knowledge gap. Here's a snippet of some interesting sessions I attended: 🤖 Postgres with OpenAI: Building & Monetizing Generative AI Plugins What do you get when you mix a database, JAVA, and an LLM? If your answer was "no idea", let's be friends. I don't consider myself much of an app or backend developer by any means, however, this interactive session by Denis Magda empowered me to feel like I could develop an app connected to a database, and integrate it into an LLM (GPT-4 for example), with the potential of turning it into a lucrative Gen-AI plugin. 🗺 Map of Amazing Postgres Extensions I definitely didn't know about If you thought Postgres was only good for SQL and databases, think again. While that is at its core, Postgres has numerous extensions that add to its functionality and provide user extendability. Claire Giordano wonderfully presented these extensions as a mind map split by category. As a visual learner, this helped a lot in understanding what the extensions do, if/when you'd need them. I also learned about Citus, an extension that scales Postgres by distributing data & queries. There's a separate conference for it called "POSETTE" and I'm eager to learn more. 👱🏻‍♀️ Trying to be Barbie in Ken's Mojo Dojo Casa House I knew immediately what this session was about, but that's also because I've seen the Barbie movie. So if you haven't, take this as a sign because Karen Jex and Stacey Haysler (bada** women by the way) use this movie to mirror and present important statistics related to trends of women in tech. It was disappointing, but not surprising, to see how many women enter (and leave) the tech industry, and how women, POC, and other diversities are treated while they are in tech. This session made me emotional and I'm super grateful to Karen and Stacey for creating a space to shed some tears and rant. As a whole, I'm very grateful for this experience, the connections, and the knowledge shared. Special mentions to Henrietta Dombrovskaya for organizing the event and having a warm and welcoming presence the whole time. Autumn Nash Thank you for posting about the event and the free ticket (clearly not all heroes wear capes)😊 Looking forward to attending more conferences, networking, and professional growth in 2024! #postgresql

38

8 Comments

🥁 Llama3 is out 🥁 8B and 70B models available today. 8k context length. Trained with 15 trillion tokens on a custom-built 24k GPU cluster. Great performance on various benchmarks, with Llam3-8B doing better than Llama2-70B in some cases. More versions are coming over the next few months. https://lnkd.in/eZQpTEQ3

2

At Ajourney (YC S22), we recognized early on that CI (Continuous Integration) could become a productivity bottleneck if not properly planned. Here’s the playbook we follow: - Utilize Postgres Template Database: We test against a real database to catch real-world cases. - Utilize Vitest Shards: Automatically distribute test cases among N CI jobs. As of today, we have 2,295 unit test cases, each testing against 𝐚 𝐫𝐞𝐚𝐥 𝐢𝐬𝐨𝐥𝐚𝐭𝐞𝐝 𝐝𝐚𝐭𝐚𝐛𝐚𝐬𝐞, which: - Takes about 2.6 minutes to complete locally. - Takes about 6 minutes to complete in CI. Without these optimizations, we would face over 30 minutes of waiting time for each deployment. We're glad we planned ahead. Can this be improved? Absolutely. Engineering is about continuous improvement.

73

11 Comments

Embrace the future of API management with Gloo Gateway! While traditional tools like Apigee work well within Google Cloud, it's time to align with modern, cloud-native strategies. Gloo Gateway by Solo.io is your ticket to robust and lightweight API management. Learn more here: ℹ️ https://lnkd.in/gw99Wjtt #GlooGateway #APImanagement #APIgateway #CloudNative #Apigee

15

Explore our comparative analysis of Percona Server for MongoDB on bare metal vs. Percona Operator for MongoDB in Kubernetes. Learn how containerized databases sustain performance using the Operator and access the complete report on the efficiency of cloud-native databases in Kubernetes environments. https://hubs.ly/Q02v9ZLh0 #OpenSource #CloudNative

3

Learn how to build agentic RAG with semantic caching to speed up common queries! In this collaboration with Redis, Tyler Hutcherson and Laurie Voss explain how to build world-class RAG applications optimized for quality, efficiency and cost. https://lnkd.in/g7UwUaGX

124

1 Comment

🚨 Last Chance to Register! 🚨 Don't miss out on our upcoming webinar: Kubernetes Applications: Best Practices for Cost-Effective Scaling" happening tomorrow, May 9th at 1pm ET (10am PT). We’ll cover optimizing resource usage, managing workloads, and avoiding costly pitfalls. 🔗 https://lnkd.in/d9kDw3wK #Kubernetes #FinOps #Kubecost

8

Redis recently adopted a dual-source-available licensing approach, joining a notable trend observed among giants like HashiCorp, Couchbase, Cockroach Labs, Sentry, MariaDB, Confluent, MongoDB, and Elasticsearch. This collective movement away from conventional open-source licenses towards models imposing restrictions on commercial usage indicates a significant shift in the trajectory of open-source development. It reflects a growing desire among creators to seek fairer compensation for their contributions. https://lnkd.in/e4xqdR8v #Tandochain #opensource #blockchain #developers

11