BRADLEY QUADROS

This is also the XPU using scenario In my mind too. “Whisper on NPU, LLMs on iGPU, document embeddings on CPU! Full use of #AIPC through #OpenVINO to implement scenarios like chat with documents and voice guided assistant that generates code. Everything on our new Lunar Lake platform that was just announced.”

9

Ok, so I get a lot of people asking how to setup custom, secure, safe, “ChatGPT Like” tools with access to proprietary information. In the procurement world this could be Spending and Transaction Policies, Delegations of authority, spend thresholds or any other or a myriad SOP’s. It can even be highly detailed category specific knowledge that you you want to surface to specific individuals. Now you can with #AmazonQ. Whether it’s straight for q&a or more advanced clause extraxtion, it’s all available right now. It’s easy to setup, super safe and secure and empowers procurement teams like nothing else! If you want to know how to get the most out of Generative AI tools in procurement (whether it’s a RAG like solution like this or something else) ping me and I will be glad to share my experiences! The world is changing procurement folk… let’s make sure we change with it! #aws, #amazon, #llm, #chatgpt, #innovation, #procurement, #generativeAI

29

1 Comment

Should you move to Silicon Valley as an engineer? Here's the perspective I gained after 20+ years in the Bay Area that I wish I could tell my younger self: When I moved to the Bay Area in the late '90s, it was amazing. Great weather, lots of opportunities, and a community full of passionate tech people. It was a great decision for me. But for you? I don’t think there’s a simple Yes or No. Because here’s the thing… It’s not just about the allure of the location. It’s not about the weather, or the opportunities. Yes, they’re factors to consider, but what matters most is you. What helped me decide: “What’s this location like on a bad day? Am I okay with that?” Smaller cities have their appeal – lower cost of living, being close to family and friends. ...But on a bad day, a smaller city is not a tech hub. And I’m a tech guy! If I had to look for a new tech job in a smaller city on a bad day, it’d be tough. But in Silicon valley I can throw a stone and find 10 companies. That’s where I want to be. Living in an innovation hub matters to me, but it might not make sense for you. It’s on you to self-reflect and find answers for yourself. Journal, and ask yourself: • What do I want? • Which roles do I want to try out? • What are my goals for the next three years? The perspective you gain will help you choose.

26

3 Comments

Writing is such valuable skill! ✏️💰 This is a great post that highlights the importance of that. #growthmindset #mastery #hacking

2

With a solution like Amazon Web Services (AWS) Q, Gen AI governance, ethical AI, and Gen AI Enablement across the company become necessary. I have already spoken to leaders who are thinking about organizational adoption, change management, and workforce readiness in this space and are already trying to "organize the chaos" with shadow-generative AI solutions. New challenges bring new room for innovation, not just on the technology side but on the people and corporate innovation side as well. Connect to learn more. #genai #generativeAI #chiefaiofficer #cdao #ai #artificialintelligence

8

🚀 Join our June 20th webinar to learn how to generate CUDA code directly from MATLAB and Simulink to run on NVIDIA GPUs using GPU Coder. 📅 Register today! 🔗 #MATLAB #Simulink #NVIDIAGPU #CUDA

2

Check Out How Amazon is Using Generative AI for Contract Reviews! Amazon is leveraging generative AI to streamline and enhance its contract review process. This cutting-edge technology is helping the company save time, increase accuracy, and boost efficiency. Dive into the article to learn more about this innovative approach and its impact on corporate finance. https://lnkd.in/gdca2RDY

2

I think this is the right step as Google is one of the first hyperscale that uses custom stack from hardware to applications. Having a custom chip delivers more performance per watt, gives a chance to optimise for specific Google products and services, and ensures a tighter security posture throughout the Cloud offerings. #google #googlecloud #googlecloudnext

4

Ever wondered how Amazon scales its resources?  Imagine air traffic control but for performance metrics: Here's a peek into this fascinating process: Manual Resource Scaling In terms of resource scaling, Amazon has traditionally relied on manual interventions. Picture a team of air traffic controllers monitoring the skies, ready to guide planes as needed. Similarly, Amazon’s developers and operations teams keep an eye on performance metrics using tools like Amazon CloudWatch, which monitors CPU usage, memory, and request rates. When a spike in user requests is detected, the team springs into action. They analyze the data to identify trends and bottlenecks, just as air traffic controllers assess weather patterns and air traffic flow. Based on this analysis, scaling decisions are made — perhaps adding more EC2 instances to handle the increased load. Once the decision is made, developers or operations teams manually implement the changes. This could involve launching new instances or adjusting load balancer settings, akin to redirecting air traffic to ensure smooth operations. Automated resource scaling with Artificial Intelligence/AI is another topic. Appreciate your time reading this post! If you found it insightful, please follow me for more insights like this!

1

Another significant development in the field of AI is the emergence of GPT-4o, as highlighted in this link: https://lnkd.in/dy6Ra2ud. Teaching mathematics https://lnkd.in/dCnDjrRv

1

Have you ever wondered whether the order of fields within a Go struct affects memory consumption or application performance? I did, and it turns out that it might indeed have an impact. The reason? Memory alignment requirements in modern CPU architectures. I documented my understanding of the topic at https://lnkd.in/ddDsG8MU . Take a look if you're interested, feedback is appreciated!

16

2 Comments

I joined Hyperline since I’ve seen what it takes to build and maintain world-class software. In places like Meta, Flipkart, and Block I’ve been lucky enough to work on AMAZING product engineering teams and bring new features to users. If you ask most engineers too, the consensus is that building something new is the most exciting and intellectually stimulating part of their job, but inevitably, you must maintain the software you’ve built. Whether you’re working on Back End, Front-End, Infra, or AI/ML, it is hard to fully avoid dreaded “maintenance”. Yet this phase is crucial – albeit a bit boring – and most engineers must take on this work at some point in their career. More than that, the people who built the software eventually move on and new teammates must build up the technical and business context needed to maintain it. In short, it can become quite expensive to maintain. I’ve put together this list to help other engineers think about the often hidden costs I’ve observed in the maintenance phase of work: 🔬 Performance Monitoring: Keeping an eye on performance metrics, error rates, and other key indicators to identify potential issues before they become major problems.  ☎ On Call Responsiveness: Ensuring that someone is always available to respond to issues and outages, even outside of regular working hours. 📦 Dependency Management: Regularly updating frameworks, libraries, and other dependencies to prevent breakages and security vulnerabilities. 💾 Data Management Costs: Managing data acquisition, storage and processing costs, as well as renewing contracts with vendors and service providers. ☁ Cloud and Server Infrastructure: Collaborating with teams responsible for managing cloud infrastructure, server hosting, and other critical services. 🔄 Tech Debt: As teams turnover, technical shortcuts catch up to you, you need to almost rebuild both the institutional knowledge, and at times, technology itself, in order to keep the lights on. All told, this takes time. The estimates vary widely, but can take anywhere between 10-50% of your team’s time, depending on the complexity and robustness of your stack. The teams I’ve been a part of do this well; they keep the maintenance cost low either by investing in high-leverage tooling (internal) or buying industry-leading platforms that help you scale with ease. At Hyperline, we’re trying to radically reduce the maintenance cost that goes into running modern data platforms by consolidating a lot of the pain I listed above. What do you think? Has your experience matched this? If not, let me know.

40

4 Comments

🌟 I am thrilled to announce that I recently presented my first paper, "Demystifying AI Advances in Explainable AI (XAI)" at the renowned IEEE Conference! 🚀 📄 Paper Overview: This paper explores the cutting-edge developments in Explainable AI (XAI), highlighting the importance of making AI systems more transparent, interpretable, and trustworthy. As AI continues to shape our world, ensuring that these systems are understandable and reliable is crucial for ethical and effective integration. 🔗 Check out more: IEEE Conference: https://iccmso.com I am grateful for this opportunity and excited to continue contributing to the field of AI and Digital Transformation. Stay tuned for more updates on my research and future endeavors! #AI #ExplainableAI #XAI #Research #IEEEConference #ArtificialIntelligence #EthicalAI #Innovation #FutureOfAI

21

Enabling the swapping of #EV batteries is a crucial element for the #OEM ecosystem. Zeliot's Condense can be the go-to solution for OEMs and #enterprises to build a solid #ecosystem.

9

🔧 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗦𝗵𝗼𝘄𝗰𝗮𝘀𝗲: ⚡ Reproducing GPT-2 ⚡ in C/CUDA with Karpathy's llm.c 💡 𝗖𝗿𝗲𝗱𝗶𝘁𝘀: Full credit to Andrej Karpathy for his innovative work on llm.c. This post is based on his GitHub repository, demonstrating how GPT-2 can be implemented directly in C/CUDA. 🧠 𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻: we are reproducing GPT-2 in llm.c, a project by Andrej Karpathy. This is "the GPT-2", the full, 1558M parameter version introduced in OpenAI's blog post Better Language Models and their Implications on February 14, 2019. llm.c achieves this directly in C/CUDA (~5,000 lines of code), bypassing the need for Python or complex deep learning libraries like PyTorch or JAX. 🚀 𝗞𝗲𝘆 𝗔𝗰𝗵𝗶𝗲𝘃𝗲𝗺𝗲𝗻𝘁𝘀: Training Efficiency: In 2019, training GPT-2 was an involved project requiring a team and was considered a big model run. Now, with advancements in compute (H100 GPUs), software (CUDA, cuBLAS, cuDNN, FlashAttention), and data (e.g., the FineWeb-Edu dataset), we can reproduce this model on a single 8XH100 node in 24 hours for $672. Performance Caveats: While impressive, llm.c is not perfectly tuned or stabilized yet. Loss spikes and bad activation ranges still occur occasionally, and comprehensive evaluations (e.g., multilingual, code, math) are still pending. 🦄 𝗠𝗼𝗱𝗲𝗹 𝗢𝘂𝘁𝗽𝘂𝘁 𝗘𝘅𝗮𝗺𝗽𝗹𝗲: Using the unicorn prompt from the original GPT-2 blog post, here's what our model generated: P͟r͟o͟m͟p͟t͟: In a shocking finding, scientists discovered a herd of unicorns living in a remote, previously unexplored valley, in the Andes Mountains. Even more surprising to the researchers was the fact that the unicorns spoke perfect English. ⚙️ 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗗𝗲𝘁𝗮𝗶𝗹𝘀: T͟r͟a͟i͟n͟i͟n͟g͟ ͟S͟e͟t͟u͟p͟:͟ Training GPT-2 with llm.c is straightforward due to its C/CUDA implementation: C͟o͟m͟p͟u͟t͟e͟ ͟R͟e͟q͟u͟i͟r͟e͟m͟e͟n͟t͟s͟: Ideally, an 8XH100 GPU setup is recommended, but llm.c is flexible. Training on 1 GPU takes 8 days, while 16 GPUs reduce the time to 12 hours. D͟e͟p͟e͟n͟d͟e͟n͟c͟i͟e͟s͟:͟ Install cudnn for FlashAttention. Optional: Install MPI for multiple GPU setups. 🔍 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻: V͟a͟l͟i͟d͟a͟t͟i͟o͟n͟ ͟L͟o͟s͟s͟: Tracks loss on FineWeb-EDU validation data, quickly outperforming the original GPT-2. H͟e͟l͟l͟a͟S͟w͟a͟g͟ ͟E͟v͟a͟l͟:͟  Compares common sense scenarios, showing llm.c surpassing GPT-2 around step ~25K. 💻 𝗠𝗲𝗺𝗼𝗿𝘆 𝗠𝗮𝗻𝗮𝗴𝗲𝗺𝗲𝗻𝘁: M͟i͟c͟r͟o͟ ͟B͟a͟t͟c͟h͟ ͟S͟i͟z͟e͟:͟ Adjust to fit your GPU memory. Recompute Setting: Trade off compute and memory (-r 0 for fast, 1 for memory saving, 2 for smallest memory saving). F͟P͟3͟2͟ ͟M͟a͟s͟t͟e͟r͟ ͟W͟e͟i͟g͟h͟t͟s͟: Disable with -w 0 if memory is constrained. Sequence Length: Reduce if necessary, though it impacts model performance. 🔗 G͟i͟t͟H͟u͟b͟ ͟R͟e͟p͟o͟s͟i͟t͟o͟r͟y͟:͟ For the full code and detailed instructions, visit - https://lnkd.in/gtd56hM2 #LLM #GenAI #OpenAI #GPT-2 #CUDA #H100

5

How to Trust Your Hardware Rick Altherr describes techniques for detecting whether hardware has been tampered with, specifically focusing on Google's Titan chip, which serves as a hardware Root of Trust (RoT) by interposing the Serial Peripheral Interface (SPI) bus between privileged components and boot firmware flash. Extended Clip: https://lnkd.in/ekHrBppa Full Episode: https://lnkd.in/etnXWARC #rootoftrust #firstinstructionintegrity #secureboot

43

Elon Musk Shares Glimpse Into The Near Future Of FSD: One Intervention Per Year I've been using Tesla's FSD version 12.3.3 to 12.3.6 for the last couple of months now and most of the time, it drives very well. According to Elon Musk, a future version of FSD, which I think means 12.6, will have only one intervention per year. The wording here is VERY interesting. An intervention, as what Elon described here, is doing something like hitting the accelerator pedal to encourage the car to move forward. This is different from a disengagement, which shuts off FSD completely and has the driver taking over all driving. I am going to say that Elon Musk means an intervention here and if that is true, is it also true that a disengagement, where the driver has to take over, will only happen once per year - or will it be even less? I think it's going to come down to the data of usage of FSD. Right now, I disengage, but it usually isn't for safety critical reasons. It's for semantics, such as the car not getting in the far left lane soon enough to avoid having to cut cars off to get in that lane down the road to make a turn half a mile away. The semantics of driving haven't been talked about much, but the only way to know if there is an improvement there is to stay subscribed to FSD and to keep using it in my daily life and in new places. I'll be able to see for myself if it improves in its route planning. Full Article: https://lnkd.in/gk-Fwr9r $TSLA #tesla #fsd #autonomy #software #driverless #robotaxi

7

1 Comment

Nvidia Talks About Tesla's Use Of 35,000 H100 GPU's, Helping The Breakthrough Performance Of FSD Version 12 Nvidia talked about Tesla and its FSD software during its earnings call. On the call, the following was said: Nvidia supported Tesla's expansion of their training cluster to 35,000 H100 GPU's. Their use of Nvidia AI infrastructure paved the way for breakthrough performance of FSD version12, their latest autonomous driving software based on vision. Video transformers consumed significantly more computing, are able to create dramatically more capable autonomous driving capabilities and propelling significant growth for Nvidia AI infrastructure. Nvidia expects a multi-billion revenue opportunity for AI infrastructure for automotive companies as more and more of them delve into autonomous software. Full Article: https://lnkd.in/gpBA--rg $TSLA @Tesla @torquenewsauto #nvidia #fsd #autonomoy #ai #software #aitraining

37

2 Comments