Satadal Bhattacharjee

Synopsys Inc was a spinout of GE exiting the semiconductor business, much like ASML was a spinout of PHILIPS A lot of the great semiconductor companies were started this way (I don’t know any others) 1. When using AI to design chips, they emphasize absolute correctness, which is different from generative AI that looks like 🪄 inside a SaaS app 2. TSMC and Synopsys were founded within three months of each other, marking the start of an era where fabless giants like NVIDIA AND Broadcom (not you AMD) could flourish. Even Jensen says Synopsys is “mission critical” to their success. 3. In order to stay on an exponential (Moore’s Law) you need to race with people who are crazier than you. Customers like Jensen Huang or Andy Grove who are always paranoid of going out of business. Even though i’m the biggest Intel Corporation supporter, it’s not a surprise that the extremely nonparanoid Brian Krzanich (who was paranoid about the wrong things cough cough) was not able to lockdown the HPC market even when Intel had 99% market share in data center at times. 4. Synopsys and TSMC are so hard to disrupt because they are selling an entire ecosystem of support and decades of experience, where companies like NVIDIA, Intel, and AMD are selling compute at the end of the day. Obviously this is changing as NVIDIA is going full stack with networking as well as end user AI services like inference and model training. 5. Transistor density per area is tapering off, the next 100x comes from 3D packaging like CoWoS, or the best in class Foveros from Intel Corporation, software optimizations, and more specialized hardware, not to mention further density and PPAC from future nodes like Intel Corporation 18A, 14A. These improvements stack, so 2-3x improvement in each area can get 100x or more from here. What does this mean? 1. Samsung has terrible 3nm yields and their product teams have switched from samsung processors and nodes. We can count them out of the foundry race, especially with them needing to focus on their memory division which SK hynix is winning. 2. Use Intel Foundry Services From Acquired #ai #nvidia #hardware #intel #amd #openai #broadcom #samsung #gpu #compute #tsmc

3

1 Comment

As part of POLITICO’s “Future in Five Questions” series, Sr. Director, Product Management for AI & ML Dr. Vinesh Sukumar discussed Qualcomm’s AI leadership, his passion for machine learning, and how anyone — even his own mother — can use #AI tools. Take a look:

87

3 Comments

No this is not Intel Vs nVIDIA stock market cap comparison chart it is Congress vs BJP MP comparison chart Working under the radar on your core competency without much happening for decades and then scaling it up at the right time and opportunity leaving everyone far behind. BJP is perhaps one of the greatest management stories which is never taught in Business schools #Modi #GeneralElection2024 #BJP

65

2 Comments

This year's Hot Chips conference was interesting, with many companies showcasing the advances in their hardware - GPUs, TPUs, and domain-specific memories - using GPT models as benchmarks. It was not a surprise… GenerativeAI has taken the world by storm. The chip/memory companies are racing ahead with new offerings or justifying why their existing hardware is the best for these workloads. I liked the keynote speech by Dr. Bill Dally, chief scientist at Nvidia. He gave insights on how the single GPU inference performance increased by 1000x in the last decade. His breakdown is shown below: ✔ 2.5x for process (28nm -> 5nm). This took me by surprise, as I expected at least ~3-4x improvement for that range. ✔ 16x for going to lower precision floating point number representations (FP32 -> FP8). FP16 is more commonly used when doing mixed precision training/inference. Nvidia claims FP8 could give ~4x speed up in inference with less than 1% accuracy loss. ✔ 12x due to complex instructions that can do matrix multiplication and accumulation, etc. ✔ 2x - for sparsity. In sparse models, many parameters (weights) are set to zero (pruned out) after training, and the model is retrained again. Pruning reduces the computational effort involved in inference. Nvidia seems to be doing 2:4 structural sparsity. ✔ Transformer engine in H100 ✔ Better software. There is a 2.7X improvement seen in A100 GPU FLOP utilization between the first paper submission and the latest one. Dr. Dally envisioned that the future improvements in single GPU inference would be from even smaller number representations, exploring log formats (where multiplication is essentially an addition of exponents), better quantization techniques, process node improvements, and reducing data movement latencies with better memory tiling and 3D packages. Check out Hotchips archives for more details on this and other presentations. GPT-3 and other LLM models require a cluster of GPUs for inference. Looking at the inference performance of the GPU clusters will be more relevant going forward. That's when the fabric used for GPU interconnects and how the model is partitioned also start playing a role in overall inference performance. #sharadayeluriblogs #hotchips35 #gpus #llms #gpucomputing

190

3 Comments

Really cool NVIDIA blog that talks about the Cadence & NVIDIA collaboration resulting in substantial improvements in the energy efficiency of scientific compute for semiconductor design, system analysis and optimization. Together, we can develop #AI systems to address the challenges facing our world. Check out the blog here ⤵️ https://ow.ly/uZg950S0VEs

17

All in on #Transformer, it is beating all other architectures in end-to-end applications. Like #Etched, #Skymizer is betting on Transformer only accelerator with compiler-centric approach. Skymizer’s first IP product, EdgeThought, is a transformer based on-device LLM inferencing only accelerator.

11

My sense is that NVIDIA's dominance in AI compute is likely to be disrupted 1st in inference (if and when it happens). Mark Zuckerberg made a very interesting point in his conversation with Dwarkesh Patel recently. He said - Meta's ratio of inference to training compute is much higher than most other companies leveraging AI. This is because Meta serves extremely large user communities across Facebook, Instagram, WhatsApp, etc., resulting in huge volumes of inference requests. In contrast, companies focused mainly on research and development may have lower ratios of inference to training compute. The scale of Meta's user base means inference optimization is more important for them in order to efficiently serve real-time requests from billions of users. This was one factor in Meta's decision to develop custom silicon optimized for inference to reduce GPU consumption and free up more expensive GPU resources for training models. I suspect that all the large cos will reduce their dependencies on NVIDIA as they scale their AI inference compute. As of now, the four cos - Amazon Web Services (AWS), Meta, Google, and Microsoft - account for >1/3rd of NVIDIA’s data centre business, and all of them have their custom AI chip initiatives. I don't know what opportunities will be created for early-stage companies in this area, but I will be doing a deep dive here. Let me know who I should be speaking with for these initiatives.

556

35 Comments

A couple of recent college graduates from India against AI finetuning giants like Anyscale and Run:ai? I met Arko C at a NASSCOM event last October. He was building Xylem AI (YC S24) , a platform that allows you to train and deploy LLMs in production. So naturally most Indian VCs rejected him. Because how could a team of three recent college grads compete with the likes SV startups like Anyscale that have raised hundreds of millions of dollars, have teams of experienced AI researchers and a stash of H100 GPUs? Arko hustled and closed two Fortune 500 companies for paid contracts. I personally know unicorns that haven't managed to do this. He then got into YC in the last round in May. There's a lot to still be proved out, but Arko exemplifies how valuable grit is in building startups. I’m doing a webinar with him tomorrow on challenges of using LLMs in production. Link to register in the comments. Disclaimer: I'm a small investor in Xylem.

148

4 Comments

Hallucinations in LLMs are inevitable, no seriously! Josh Fritzsche and I have worked hard at Voze to find a way to tune, prompt and even plead 🙏 with LLMs to prevent them from the occasional 🍄 hallucination. It seemed like the better we have done the harder the edge cases have been to solve and eventually we realized that they tend to frequently happen when LLMs predict or make anticipations on "nothing". I remember at one point nearly going insane with frustration because when LLMs had a hallucination, they were EPIC! 😱 Then we pushed through it and benefited for the effort! How did we solve this common problem? We didn't (Not completely), we got better at identifying the issues as they happened. Once we identified in the data we were able to start looking for patterns. The same catalysts that cause it to happen once if left unchanged caused it over and over again. Our solution is astoundingly simple. Once you know that it's likely to happen..... let it! Accept it will 🍄, but also be preemptive in not focusing on it and use alternative data sources, generators or providers. Change the conditions that cause it but continue to monitor it as it happens. As Dan Caffee is fond of saying to me on a regular bases. "Dusty we need the system to be adaptive and provide mechanisms to learn from it's past." Well he is right! All we needed was data and the people that contribute to it's quality. Huge props 👏 to strong support from Kathy and Janelles audit teams. We were armed with a knowledge and a desire to learn from past mistakes. Armed with confidence that we could correct these problems we could focus on the data to prevent their impact going forward. When doing millions of LLM (Generative AI) calls regularly Josh and I have learned that hallucinations will happen. We also know exactly how we will handle and prevent it going forward! Failure is not doing anything and accepting defeat.

4

2 Comments

Thanks for hosting me Prakash. Loved discussing how Intel Corporation #TiberEdgePlatform enables Edge AI and operationalizes it for enterprises in brownfield IoT systems.

38

As #AI scales across industries and use cases, understanding how to measure AI performance becomes key in developing solutions with sufficient head room. Like we say, "not all TOPs are created equal". At Qualcomm, we focus on running AI inferencing workloads on our super-efficient and powerful Neural Processing Units (NPUs). Highly recommend reading our latest article on how to measure AI workloads, performance and the right place to run them.  https://lnkd.in/gzV33PCa

75

Did you know that Synopsys Inc is the driving force behind the new era of pervasive intelligence - AI, silicon proliferation, and software-defined systems? So, how do you land a job at this market leading company? We connected with Kristy Calcagno, VP of HR, to give you the inside scoop. In this video, Kirsty shares: details on Synopsys; What they look for in a candidate; Interview process; and how to stand out!

8

Exciting times ahead for the AI industry! Cerebras Systems, a leading AI chip developer, has announced plans for an IPO. This bold move positions them to take on industry giants like Nvidia, shaking up the competitive landscape. Cerebras has already made waves with their innovative chip architecture, designed specifically to accelerate AI workloads. Their Wafer-Scale Engine, the largest chip ever built, offers unprecedented performance for AI models, and their IPO could further fuel their ambitious roadmap. With AI applications rapidly expanding across industries, the demand for high-performance hardware is skyrocketing. Cerebras aims to meet this demand by providing cutting-edge solutions that push the boundaries of what's possible in AI computation. As they gear up for this significant milestone, Cerebras' vision and technology could reshape the AI hardware market. The IPO will not only bolster their financial strength but also enhance their ability to innovate and scale. Kudos to the Cerebras team for their pioneering efforts! It will be thrilling to see how their journey unfolds and how they stack up against Nvidia and other established players. #AI #TechInnovation #Cerebras #IPO #AIChips #Nvidia #AIHardware #FutureOfAI

3

2 Comments

It appears the semiconductor industry, that bastion of modern tech innovation and economic stability, is grappling with a dojo-sized talent gap; a significant hurdle to growth and continued prosperity. According to respected authority Deloitte, the sector will require in excess of a million skill-ready employees by 2030 to keep pace with increasing demand. • A situation like this underscores the urgent need for strategic talent acquisition, workforce training, and a profound revamping of the outlook on career modalities in the 21st century. The clock is ticking. Here's my prediction: • The sheer demand for expertise is going to trigger an unprecedented era of focused technological education and expedited learning pathways, the ripples of which will be felt not merely in the semiconductor sector, but across the tech industry at large. • Private and public sectors will each play pivot roles in shaping this radical new landscape - an arena truly fertile for innovative partnerships, resourceful strategies, and visionary leadership. Everybody has a stake at this table. • The economic impact of cinching this talent gap can't be overstated, particularly considering the semiconductor industry's centric position in today's technology-reliant landscape. Plugging the talent gap is equivalent to maintaining global prosperity and advancing our society – as sweet a deal as that might sound! A critical outlook: • The challenges are mammoth and the solutions will require a much-needed degree of complexity. However, this isn't a doom and gloom narrative. Instead, the story of securing our future in this high-stakes game requires a vision that forecasts far-reaching market dynamics, nascent technological evolutions, and the indispensible role of human skills and ingenuity in this awe-inspiring narrative. Taking an optimistic and professional stance underlines my belief that the future remains bright despite the challenges ahead. When the stakes are high, so too is the potential for groundbreaking innovations and inspiring collaboration. To the semiconductor industry, I say: Onwards and upwards! #SemiconductorIndustry #WorkforceDevelopment #EconomicGrowth #FutureOfWork

2

1 Comment

Looks like Qualcomm's business is stabilizing and the company has managed to maintain its strong position in the premium handset market delivering 1% YoY growth across their businesses. QCT (Handsets + IoT) was up 1% YoT and QTL (Licensing) was up 2% YoY - Non-GAAP. The company's diversification strategy is being led by automotive which has experienced consistent growth QoQ versus competitors who have exhibited declining or lumpy at best. The company has increased its design-win pipeline to $45B from $30B back in September of last year. My impression is that Qualcomm is doing a good job securing sockets rather than providing simulation services and infrastructure. Akash Palkhiwala mentioned progress on the ADAS side of the automotive business which makes up 1/3 of their design-win pipeline. This is a welcome nod to the SDV trend and a signal that Qualcomm customers are looking to digitally rearchitect their vehicles. AI PC should start to provide Qualcomm's Internet of Things segment a new revenue curve as X Series (Snapdragon X Elite and X Plus) beginning next quarter. It will be interesting to see how this semi-new (Qualcomm has been in the Arm-based PC game for years) will play out. It should be an inflection moment for Qualcomm as it comes to market with what looks like a formidable AI PC processor family in H2 of this year. I'm glad that Qualcomm included Snapdragon X80 in their presentation materials. It is the unsung hero of #5G which has largely been a back-burner item for the better part of a year. 5G will matter most when we don't think it matters. Qualcomm is an essential U.S. champion and global leader in my view. We shouldn't forget. It does seem that Qualcomm's One Technology Roadmap (which includes AI) is fostering resiliency across its portfolio of businesses against tough geopolitical and competitive environments. I'm looking forward to see how the proliferation of Oryon across product lines will fortify Qualcomm's business and solution competitiveness especially for handsets. There is some very interesting edge infrastructure and data center potential as well. 😉 All eyes on Snapdragon X Series in H2 2024! It's bigger deal than AI PC.

136

🌟 Exploring the Intersection of Business and Technology re AI! 🌟 Before my decade-long focus on AI, connecting technology and business in various domains has always been my passion; This is why I enjoyed this conversation with David Luan - No BS, No Twitter exaggerations, No clickbate - Just a super well informed, super smart David sharing his insights from years of hands-on experience: #Startegy, #Software, #Hardware, #Platforms, #ModelSpace and everything in between - LMK what you think of his projections. In This Episode: 1. The Biggest Lessons from OpenAI and Google Brain: What did OpenAI realize that no one else did that allowed them to steal the show with ChatGPT? Why did it take 6 years after the introduction of transformers for ChatGPT to be released? What are 1-2 of David’s biggest lessons from his time leading teams at OpenAI and Google Brain? 2. Foundation Models: The Hard Truths: Why does David strongly disagree that the performance of foundation models is at a stage of diminishing returns? Why does David believe there will only be 5-7 foundation model providers? What will separate those who win vs those who do not? Does David believe we are seeing the commoditization of foundation models? How and when will we solve core problems of both reasoning and memory for foundation models? 3. Bunding vs Unbundling: Why Chips Are Coming for Models: Why does David believe that Jensen and Nvidia have to move into the model layer to sustain their competitive advantage? Why does David believe that the largest model providers have to make their own chips to make their business model sustainable? What does David believe is the future of the chip and infrastructure layer? 4. The Application Layer: Why Everyone Will Have an Agent: What is the difference between traditional RPA vs agents? Why are agents a 1,000x larger business than RPA? In a world where everyone has an agent, what does the future of work look like? Why does David disagree with the notion of “selling the work” and not the tool? What is the business model for the next generation of application-layer AI companies? https://lnkd.in/dUpWWzvN

4

Another award for Synopsys Inc! Synopsys has been named a winner by Comparably for Best Company Leadership for 2024! This award criteria is based on millions of anonymous sentiment ratings from employees who submitted feedback over the past year on their CEOs, executive leadership, and direct managers. I see this great leadership in action every day around the world at Synopsys Inc - from executives to directors to managers. Not only are they some of the smartest people I have worked with in my career, but some of the kindest too. Heck - they even laugh at some of my cringiest dad jokes :). Do you love where you work? You should. We are hiring - www.synopsys.com/careers Full list of winners here: https://lnkd.in/emvYTWSq #hiring #jobs #leadership #management #culture #people #synopsys #lifeatsynopsys #engineeringjobs #developerjobs #AI #ASIC #semiconductors #global #hybrid #greatplacetowork

112

4 Comments