Ram V.’s Post

“Your margin is my opportunity” #Jeff Bezos The fallacy of Infiniband is the premise that it can build a lossless network that works at large scale. Here is why it does not work…. Large scale deployments have a lot of active physical components. For example, in a 4000 GPU cluster, there are at least 8000 optics. Anyone who has dealt with optics knows that they are prone to failure. It is not unusual to experience a failure rate of over 5% per year. This translates to many failures per week in a cluster of only 4K GPUs. So, while Infiniband is trying to engineer a lossless network, the reality is that we will have physical layer failures. Trying to have the Infiniband Fabric Manager deal with physical layer failures is like shuffling chairs on the deck of the Titanic! Enter the expensive Infiniband cables! This is what I refer to as “Organic Water”. It is just a cable sold at exorbitant prices. It does not solve the problem. Ethernet has been built for and deployed at scale. It is built on the premise that there will be failures and the system quickly converges around failures. Ethernet is also priced for volume deployments. Ethernet says to Infiniband “your margin is my opportunity” Happy scaling!

Paying homage to one of the greatest minds on human behavior and behavioral economics: Dr. Daniel Kahneman. If only I could follow some of his insights more often in my daily being: - reversion to the mean - tendency to value possible loss twice as much as possible gains - basing long term projections on short term information: i.e. tendency to extrapolate current state well into the future - overconfidence on success rates that is inconsistent with historic data RIP Dr. Kahneman

Got Organic Water? #machinelearning #ai #ethernet #infiniband

Ethernet for Scaling Large Language Model Training Summary: - Ethernet is the network of choice - It is economical, widely available, easy to operate, and at least 2x faster than Infiniband. https://lnkd.in/gHW_UuXT

Committed to Ethernet: Rain or Shine Not long ago, 5G was the talk of the town. Now it is AI. In the infrastructure business, a few basics differentiate the leaders vs the rest: - a long term strategy based on first principles and persistence - relentless focus on innovation and execution - willingness to disrupt oneself - an amazing team At Broadcom, we made a decision over ten years ago that we need to have distinct products line for the Cloud Scale Data Centers, Telco Service Providers, and the Enterprise customers. This is was strategy based on first principles that one product cannot serve multiple markets and we stuck with it. We have consistently delivered successive generations on our Tomahawk, Jericho/Qumran, and Trident product lines. Our focus on execution has resulted in consistently doubling the bandwidth / performance of our chips about every 2 years. This compounding has helped reduce the cost and power requirements of networking by 90%+. A router that not too long ago consumed about 20,000 watts of power and cost over $1 Million, now consumes less than 1000 watts and costs less than 1/20th of what it used to cost. While we build our Jericho chips for large chassis systems and Disaggregated Distributed Chassis ( DDC), we also acknowledge that most customers’ needs can now be met with high performance single chip 1RU and 2RU systems. This is where we are happy to disrupt our own by building highly scaleable single chip routers such as our recently announced Qumran3D chip. But in many cases, even our single chip 25Tetabits per second routers (Qumran3D) are a little too much. So, we have just introduced a mini-me version of it… Qumran3A. I recently read that there is about 300Terawatts of Data Center capacity in operation. If you subscribe to projections that over the next 5-7 years this capacity will be doubled, one can only imagine the amount of network capacity needed to enable the new generation of distributed computing workloads (example: Machine Learning). We are a firm believer that this network will be built on Ethernet and we are working relentlessly on it. We are grateful for the amazing team at Broadcom and everyone else in the Ethernet ecosystem. 🙏 Committed to Ethernet!

A Beautiful Beast Not too long ago, a router with the capability to handle 25 Terabits per second would have come at an exorbitant cost, demanding power equivalent to illuminating a thousand 60-watt bulbs. Over the past decade at Broadcom, we've relentlessly striven to democratize the economics of merchant silicon for Ethernet Switching. Our Trident and Tomahawk product lines have been pivotal in extending this affordability to Cloud Providers. In the same vein, we've made strides to bring this economic advantage to Telecommunications Service Providers like AT&T through our Jericho line of Routing products. And now, we're thrilled to take this journey even further, advancing the accessibility, simplicity, and high-performance attributes of our Routing portfolio with the introduction of our cutting-edge Qumran-3D chip. A couple of years back, when we embarked on the journey of developing this product, our objectives were straightforward: to create a single-chip pizza box router that seamlessly incorporates deep buffering, expansive routing table capacity, and top-notch security, all without requiring compromises. While this may seem like a self-evident goal, it has proven to be far from common practice in the industry. We anticipate that routers based on the Qumran-3D chip will not only make the world more environmentally friendly but also accelerate the internet's speed by replacing the many thousands of slow, power-hungry refrigerator-sized boxes. Our heartfelt gratitude extends to our valued customers and dedicated teams for making this remarkable achievement possible. https://bit.ly/46OvZZM

Tiny & Perfect 👌 4 Terabits per second might seem like rounding error in the age of 50 Terabits per second switches. However, not everyone needs a 50 Terabits per second switch. Our new chip, Trident4-X7, rated at 4 Tbps has been built optimized for Top-of-Rack deployments in the Enterprise Data Center. Even at 4 Terabits per second, the entire data in the American Library of Congress ( 15TB) can be transmitted in just about 30 seconds! All this with a low power consumption of sub 100 watts. Design focus of Trident4-X7 switch has been on low power, low cost, investment protection, and deep network insights while enabling connecting to servers in a rack each at up to 50 Gigabits per second speed per lane. At Broadcom we believe in building the most optimized chip for a given use case. Trident4-X7 is yet another example of this. Democratizing Networking… one chip at a time! 🙏 For more details, please see:

The Age of Ultra (Ethernet) Ethernet is uniquely positioned to be the networking technology of choice for AI interconnect, period! This is why nine industry leaders, supplying and operating some of the world's largest AI and HPC networks, have formed Ultra Ethernet Consortium (UEC). No other technology has the performance, scale, economics, and extensive community involvement of Ethernet. Current Ethernet switches have double the bandwidth and double the port speed compared to any other networking technology. Broadcom’s Tomahawk 5, at 51.2Tbps, and Jericho3-AI, supporting high-performance AI fabrics up to 25.6 petabits per second, are proof points of Ethernet’s capabilities and are available today. Accelerating the evolution of AI (and HPC) networks is the reason UEC was formed. The goals of UEC include the ability to scale to a huge number of endpoints, up to 1,000,000 nodes in one training cluster. The UEC transport will improve network efficiency by enabling traffic multipathing/spraying and congestion notification optimized for AI traffic. Additionally, recognizing that not every company has a dedicated team of engineers available to tune the network and software for each AI training job, one of the goals of UEC is to simplify network management by enabling configuration-free end-to-end congestion control. UEC will also simplify software development by providing streamlined APIs (application program interfaces) for AI and HPC. Broadcom has already embraced several UEC goals by, for example, supporting traffic multipathing/spraying and end-to-end congestion control in the Jericho3-AI fabric. This fabric is available today with huge scale, supporting AI clusters of 32,000 nodes with 800GbE per endpoint. Similarly, Tomahawk 5 provides multipathing, load balancing, and improved congestion management with its Cognitive Routing capabilities. Broadcom is contributing technology to UEC to enable improvements in all aspects of the network stack. One of Ethernet's strengths lies in its extensive community of contributors. Broadcom is proud to partner with other industry leaders in UEC to continue evolving Ethernet in an open, inclusive, and innovative way. In the age of AI, Ethernet has emerged as the networking technology of choice, thanks to its unparalleled performance, scalability, and community involvement. With continued advancements and broad industry support, Ethernet is set to play a pivotal role in shaping the future of AI interconnect. Go ultra with Ethernet!

Ethernet for AI Workloads…. Agreed https://lnkd.in/gmWx5iqv