Shubu Mukherjee

AI workflows are driving demand for memory and storage to prepare data, train the model and create useful inference engines.  This is driving announcements of storage and memory products that support the various elements of these AI workflows.

51

4 Comments

"A study published in Opto-Electronic Science discusses high-intensity spatial-mode steerable frequency up-converter toward on-chip integration. Integrated photonic devices consisting of micro-lasers, amplifiers, optical waveguides, frequency converters, and modulators on a single chip, enabling control over photon's spatial modes, frequencies, angular momenta, and phases, are essential for preparing high-dimensional quantum entangled states, high-capacity photon information processing, all-optical communication, and miniaturization of photonic computing. However, current nonlinear waveguide devices, integrating spatial modes and photon frequency conversions, heavily rely on external optical path control and spatial light modulators, failing to meet the crucial requirement of on-chip integration for photonic devices." #photonics

1

As part of POLITICO’s “Future in Five Questions” series, Sr. Director, Product Management for AI & ML Dr. Vinesh Sukumar discussed Qualcomm’s AI leadership, his passion for machine learning, and how anyone — even his own mother — can use #AI tools. Take a look:

87

3 Comments

Some thoughts about the current Nvidia AI networking wave. BroadCom has been the king of networking for the last 20+ years. Is Nvidia going to be able to topple them? Nvidia has their own niche "AI networks" which enables them to sell the chip/equipment with such a high margin they are able to simply out-spend their competitors? Kind of like BroadCom has done the last years in almost all other network segments? The competition is on , who will be able to build the best AI network chips?

7

3 Comments

As we are moving into the AI/ML era of computing, we need to build larger and larger compute hardware for executing the workloads to scale. This requires building larger and larger chips to make these systems works within performance requirements but fit into cost and power limits. The key challenge in building the next set of chips is that as geometries are shrinking it is getting harder to build silicon. One solution to the challenge of building trillion transistor system is to use the 2.5D to 3D technologies. These solutions are already emerging. As these solutions emerge on much wider scale there is a need to accelerate technology development and collaboration. Here are a few challenges that need to be addressed to solve these problems. #chiplets #AI #ML

22

2 Comments

Last week’s DAC showcased exciting announcements from startups that are revolutionizing semiconductor design however these rising stars overshadowed a major update from an industry stalwart - Alphawave Semi - whose news is poised to accelerate the proliferation of chiplets and give rise to a “chiplet marketplace”. Alphawave Semi announced a breakthrough with the tapeout of the industry's first off-the-shelf, multi-protocol I/O connectivity chiplet. This technology can seamlessly integrate into larger SoCs or multi-chip modules, giving designers increased flexibility and more connectivity options for different applications. Reflecting on the Chiplet Summit this past spring, I remember listening to Tony Chan Carusone, Alphawave’s CTO, as he passionately described the significance of chiplets and how Alphawave will foster a more open and democratic design ecosystem. Now, just one quarter later, his team has launched a technology that brings this vision much closer to becoming a reality. Adding to their momentum, Alphawave also announced an expanded partnership with Samsung Foundry to advance the industry's progression toward cutting-edge semiconductor nodes, crucial for the evolution of advanced compute. Samsung’s roadmap in sub-4nm technologies includes FinFET and Gate-All-Around, which are designed to maximize performance in the post-Moore era. Initially targeted for data centers and HPC, these innovations may trickle down to low-cost, AI-enabled edge devices, putting the transformative power of AI into the hands of billions of people worldwide. This news follows Alphawave’s early June announcement of their new partnership with Arm, aimed at unlocking the potential of processors. Alphawave will give Arm access to their robust portfolio of connectivity solutions and silicon IP, which Arm will integrate into their proven portfolio of compute cores. At a time when AI technology is accelerating at breakneck speed, many companies are searching for ways to boost compute performance and address the challenge of moving data between compute and memory more efficiently. While multiple startups offer unique solutions, many customers want battle-tested technology to avoid the risk of investing in solutions that may become obsolete by the time they reach tapeout. Alphawave has developed vital solutions and in doing so, has increased design flexibility, built bridges to new process technologies, and shortened the path to market. “Connectors” are often the catalysts of innovation and Alphwave’s is a testament to that word. I’m encouraged by what I’ve seen from Alphawave and strongly believe they stand at the forefront of an emerging chiplet marketplace. More broadly, I believe their technology serves to democratize design technology and will give emerging companies more options to bring their brilliant ideas to life. #semiconductorindustry #ai #chiplets

110

Jim Keller, former Intel, AMD, and Tesla engineer, aims to challenge Nvidia's dominance in AI with Tenstorrent's energy-efficient chip design. Tenstorrent's upcoming second-generation AI chip promises superior energy and cost efficiency, presenting a viable alternative to Nvidia's offerings. The innovative design of Tenstorrent's chip features independent decision-making cores, potentially reshaping AI chip technology across diverse applications. #AI #Nvidia #semiconductor #semiconductors #chip #technology #innovation #research #computing Tenstorrent NVIDIA https://lnkd.in/g6a3vAu4

26

RISC was supposed to circumvent barriers around ARM, x86 etc it is doing that and more - it is allowing innovative spirits to run without bothering too much about the CPU IP itself too much open source has truly come to chip design https://lnkd.in/dUFy9V4v

10

1 Comment

In the fiercely competitive semiconductor industry, companies are in a constant battle to retain top talent because in the heart of every good engineer is a passionate problem solver, looking for their next challenge. For them, innovation is the currency of success. Yet companies like Qualcomm, Intel, and Broadcom attract tens of thousands every year by dangling lucrative compensation packages to lure them from competitors. Startup leaders don’t have those resources, so they have to work harder to attract the best talent and then fight like hell to retain them. But some of the most successful startup leaders I know take a different approach. Rather than viewing attraction and retention as a fight, they leverage the inherent qualities of their companies as their advantage. They use their nimbleness, bias toward innovation, and their collaborative cultures like superpowers to attract the right people and those usually end up being the reasons why people stay. Take Richard Fung, CEO of The Six Semiconductor, for example. I had a conversation with Richard and asked him to share his approach to motivating his team. His response was like a mini-masterclass on the topic of talent attraction and retention. Richard emphasized that a culture that encourages engineers to dive deep into the intricacies of their craft, understand the systems they’re building, and connect with the end customer is a successful culture. Richard has been using this approach and it's been paying dividends for The Six Semiconductor so in his honor, here are his six key concepts for creating this type of organization: 1. Foster a Learning Culture: Support ongoing learning and skill development beyond employees' core responsibilities. 2. Prioritize Customer Focus: Embed a customer-centric mindset, and encourage employees to enhance user experiences through their work. 3. Champion Problem-Solving: Empower employees to creatively solve challenges with a focus on customer needs. 4. Promote System-Level Understanding: Encourage employees to grasp the bigger picture of the company's offerings and their role within it. 5. Break Down Silos: Promote collaboration to ensure a comprehensive understanding of projects across teams. 6. Offer Meaningful Challenges: Provide intellectually stimulating projects that make a tangible impact on the company's success. What Richard highlighted is that success goes much deeper than getting great people to join. It's about unleashing the full potential of every individual on your team to drive innovation and help shape the company, and ensuring your team heads in the right direction together. I’m thrilled I got to learn Richard’s perspective and believe other leaders can benefit from hearing it, so I shared the video below. My "challenge question" to other startup leaders is this: what are you doing to foster this type of culture in your organization? Either share below or, feel free to DM me. #semiconductorindustry #startup #leadership

67

2 Comments

Automotive engineers need real-time processing, so check out this webinar from Synopsys on their optimized RISC-V processors, May 30, 10-11AM PDT. #SemiIP https://lnkd.in/gvkHkW8p

2

The rise of AI is transforming semiconductor manufacturing again as fabs adapt to designs geared toward disaggregated computing. AI chip companies are leveraging new schemes to maximize power and performance, like the use of chiplets, chip stacking, and advanced interconnects to arrange memory and compute in new architectures. These approaches require shrinking dimensions and much higher precision, making it a challenge for chipmakers to not only identify the known defects but also spot all the new ones that are popping up. This is why in our industry, yield, test, and inspection equipment is worth far more than its weight in gold. We’re fortunate that a handful of companies are pushing the boundaries of what this equipment can do. Onto Innovation is one of those innovative companies and their decades of expertise in optics and software have enabled them to build equipment renowned for its inspection, metrology, lithography, and yield optimization capabilities. Last month Onto announced another milestone achievement; they released a new sub-surface inspection capability for their Dragonfly G3 2D/3D inspection and metrology platform that enables whole wafer inspection. This means fabs can find defects in full 3D, which is crucial as chip architectures start looking more and more like skyscrapers under magnification. In addition, this enhancement enables detection at production speed, ensuring companies can ramp up their new designs efficiently and confidently. “Customers are demanding 100% inspection capability with production-worthy throughput,” said Mayson Brooks, VP/GM of Onto’s inspection business. Investing and innovating like Onto has created net positive results and momentum for our entire industry. By providing manufacturers with the insights they need to fine-tune processes, Onto not only supports the rapid development of new technologies but also their deployment. This in turn strengthens the competitive positions of newer, smaller companies developing the next generation of chips and quickens their time to market. Creating an ecosystem where mistakes can be eliminated and costs can be reduced brings down barriers for all companies, which means innovation can emerge from companies of all sizes. An Onto continues pushing the boundaries of what’s possible in a behind-the-scenes manner, let’s acknowledge all the hard work and creativity of that goes Onto, or rather, into, making those chips possible. Kudos to their team for this remarkable achievement! #semiconductorindustry #metrology #inspection

6

Thought provoking research paper- Enhancing the ubiquitous FPGA BRAMs with in-memory compute-capabilities. Specifically note, the "proposed architectural change results in 1.6× and 2.3× increase in the peak multiply-accumulate throughput of a large Stratix 10 FPGA, at a minimal cost of only 1.8% increase in the FPGA die size and no change to the BRAM's interface to the programmable routing." In practical terms, this work proposes enhancing the capabilities of FPGA (Field-Programmable Gate Array) on-chip memory, specifically BRAMs (Block RAMs), by enabling them to perform computational tasks in addition to storing data. This enhancement aims to significantly increase the efficiency and performance of data-intensive applications, such as deep learning inference. For example, in the field of deep learning, the proposed architecture, RIMA, could enable faster and more efficient execution of inference tasks on FPGA devices compared to traditional methods or even state-of-the-art solutions like GPU-based processing. This advancement could have significant implications across various industries where deep learning inference is applied, such as healthcare for medical imaging analysis, autonomous vehicles for real-time object detection, or finance for fraud detection in large datasets. Essentially, it allows for faster and more efficient processing of complex tasks, which can lead to improved performance and potentially new capabilities in a wide range of verticals.

1

The Chip Insider®   Nvidia’s Grand Strategy Summary:   In his keynote at Computex 2024 last week, Jensen Huang proclaimed to a cheering crowd, “We are in a new Industrial Revolution.” As everyone knows, Jensen’s strategy for Nvidia shifted from … But his grand strategy is far more encompassing than the vision of the AI Revolution. The Industrial Revolution made the transformation to making things with machines that had been made with human hands. The AI Revolution takes this transformation to the next step by making things with machines that could only be made with the human mind. But this is only a vision – not a strategy. One cannot monetize a vision without a strategy. Watching Jensen’s keynote, I realized he has a grand strategy based on Cochrane’s Theorem of the Technology Treadmill, which is what my college thesis used to develop a semiconductor cost model that showed economic underpinnings of why fab tool costs would rise with Moore’s Law …      “Strategy requires a sense of the whole that reveals the significance of respective parts.”                                              – John Lewis Gaddis For subscribers to The Chip Insider®, the full write-up can be found here: https://lnkd.in/g6w8YDec Copyright © 2024 TechInsights Inc. All rights reserved

29

people pondering why we don't have 128bit systems in reality a huge number of "computers" being shipped annually, possibly a majority, isn't even 64 bits vast majority of microcontrollers are 32 bit or less and even in a number of 64 bit applications (which the computer industry went to mainly because 32bit could only address 4GB) where custom buses can be designed, address buses are often 48 or 56 bits (and even those can't be fully utilised, just that the architects over-provision) even in very large compute systems the arithmetic also rarely uses 64 bit directly (float in C is 32 bits; to get to 64 you need to use double; it is useful but relatively infrequently), so data bus width of 64 is useful because more parallel wires allow faster bandwidth for packetized data on Network-on-chip (or fabric), and as far as the data bus width is concerned, that is in fact very often larger than 64 bits ( even if the address space is less than 64 bits) in other words, vast majority of computers aren't really 64bits even today in any meaningful sense. just that people don't know about those computers https://lnkd.in/dhFrhVb2

21

8 Comments

https://lnkd.in/gbCMzhxK this article may sound bullish but it isn't "Our forecast shows that the combined revenues of: 1) companies that supply manufacturing equipment, components and software to build fabrication facilities; 2) chip manufacturers; and 3) chip and AI software designers will approach $1T this decade." in fact by the end of this decade chip supplier revenues (without including revenues from software, equipment manufacturers etc.) will top a trillion dollars the article singles out NVIDIA, TSMC, Qualcomm and Broadcom as the major beneficiaries. But I think TSMC will be dominant while the others will compete amongst each other and may even find themselves in situation where TSMC keeps eating their lunch

11

12 techniques to reduce your LLM API bill and launch blazingly fast products Cost-efficiency and minimal latency are critical for deploying production-ready LLM applications.   Sahar Mor's article sheds light on 12 innovative techniques to slash your LLM API bills, with gems like Semantic Caching and Summarizing Conversations leading the charge. This isn't just about saving tokens; it's a strategic playbook for scaling smartly in an AI-dominated landscape.  #MachineLearning #AI #NLP #CostOptimization #Innovation #TechTrends https://lnkd.in/ec3qst-M

19

1 Comment

Arm has been present in PC space since Apple launched it's M1 based Macbook But with Qualcomm launching it's PC grade snapdragon chips the fight has come to Windows PCs https://lnkd.in/gQc_74c3 Intel Corporation and AMD both will feel the heat, loose marketshare and, even if somehow figure out how to compete, will now have a third major entrant in the long standing duopoly (which has been a near monopoly) Third major competitor usually breaks any monopoly and benefits consumers (which should be read to mean it hurts incumbents) And now ARM is saying even this isn't the end, multiple other vendors are about to launch their PC grade products ! one wonders whether the Apple Microsoft duopoly may come under attack soon

27

Thomas Coughlin has outlined the impact of AI advancements on storage needs in his current article published by Forbes. He notes that as AI applications grow, there's an increasing demand for memory and storage solutions to manage both the extensive data used in AI training and the results utilized in inference engines. This demand benefits a variety of storage technologies including DRAM, NAND flash, HDDs, even magnetic tape (#lto), and likely, emerging non-volatile memories, particularly in endpoint AI inference applications. Quantum Corporation's #ActiveScaleColdStore provides a comprehensive storage solution tailored for AI workloads. This system integrates LTO technology with SSD, HDD into a S3 Object storage system using an innovative #tape optimized Two-Dimensional-Erasure-Code technology #2DEC , ensuring efficient handling and scalability of AI-driven data demands for low cost, high performance green computing #quantum #lto #activescalecoldstore

8

1 Comment

There’s a seismic shift happening in the semiconductor industry, driven by the rapid rise of AI. Companies like NVIDIA and AMD are pushing the boundaries of AI, but they also dominate the market with a combined share that threatens to shut out smaller, innovative AI hardware companies and startups. Hope is on the horizon due to a growing movement toward a more open semiconductor economy; a pivot that aims to give smaller companies greater design flexibility and a faster time to market. However, achieving this requires industry partners to develop the necessary underlying technologies. Two such technologies are chiplets—sub-sections of chips purpose-built by specialized companies—and advanced interconnects. Amkor Technology, Inc. leader in this field, is at the forefront of these advancements. Founded in 1968, Amkor has become one of the industry’s leading Outsourced Semiconductor Assembly and Test (OSAT) providers and recently, they announced a $2 billion investment in a new facility right here in Arizona, which will be the country’s largest OSAT. As AI continues to advance, heterogeneous designs with disaggregated memory and compute require innovative solutions. Some companies are leveraging chiplets and advanced interconnect to meet those needs. However, making all components work together in the physical realm presents significant engineering challenges. This is where Amkor excels. Amkor has spent decades developing a portfolio of technology such as chip-on-chip, package-on-package, and differentiated methods of interconnecting dies and chips that unlocks the potential of these designs and will help bring new types of technology to market. Part of Amkor’s success lies in collaborating with chip design teams to ensure manufacturability and at the ISES Summit in Taiwan, a panel of top engineers and researchers including Amkor’s Walter Chen discussed the challenges with interconnect standardization and advanced packaging technology. They highlighted the tradeoffs between traditional single-die and chiplet architectures, showcasing Amkor’s dedication to this sector. I’m encouraged by this news of Amkor’s new facility because it will position them to be better aligned with their design ecosystem partners here in the US, and strengthen the supply chain with their expertise. Their ongoing investment in packaging and interconnect technologies will empower smaller companies to create the next breakthrough AI designs. With large players like NVIDIA dominating the market, the democratization of AI may just hang in the balance. #semiconductorindustry #OSAT#chiplets

146

2 Comments