Keith Weng

"Advanced computing is starting to serve utilities..." As the rise of AI is contributing to a near-doubling of five-year energy load forecasts (from 2.6% to 4.7% growth), AI is also at the core of some of the innovation to solve the biggest problems in climate and energy. A couple of weeks ago I was in a room of investors who were asked if it's "too early" to be investing at the nexus of AI and climate. I was surprised to even hear the question, because we've been doing it for years. This article from Utility Dive highlights three companies in which MUUS Climate Partners was an early investor: - BrightNight, which created an advanced simulation tool to optimize the design and operations of clean energy projects; - Amperon, which has built the world's most accurate energy demand forecasting system; - Utilidata, which has partnered with NVIDIA to deliver the first distribution system AI platform. If you're interested in the AI-climate nexus (and not just the buzzwords, but the actual solutions being built and deployed), this article might be of interest. https://lnkd.in/evkkY4nX

12

1 Comment

Highlighting: "Nature has created this virtuous cycle of seeing and doing, powered by spatial intelligence," she said." (Fei Fei Li) ----- Spatial Intelligence sounds 1000x more useful than 'generative' AI. Our world is 3D, not 2D. Understanding 3D requires simulations that are spatially rendered in 3D. 3D is the measure of our world. We use 3D for games, robotics, drones, autonomous vehicles, digital twins and more. Not 'gen' AI word salad bullshitting. Gen AI is either a dead end branch or the tiniest part of a more complex 3D future. Bring on spatial intelligence. No bubble, just work. ----- Reuters: "Exclusive: Stanford AI leader Fei-Fei Li building 'spatial intelligence' startup." (Katie Paul, Anna Tong, & Krystal Hu) (May 3, 2024) "Prominent computer scientist Fei-Fei Li is building a startup that uses human-like processing of visual data to make artificial intelligence (AI) capable of advanced reasoning, sources told Reuters, in what would be a leap forward for the technology. Li is widely known as the "godmother of AI," a title derived from the "godfathers" moniker often used to refer to a trio of researchers who won the computing world's top prize, the Turing Award, in 2018 for their breakthroughs in AI technology. One source pointed to a talk Li gave at the TED conference in Vancouver last month, in which she said the cutting edge of research involved algorithms that could plausibly extrapolate what images and text would look like in three-dimensional environments and act upon those predictions, using a concept called "spatial intelligence." To illustrate the idea, she showed a picture of a cat with its paw outstretched, pushing a glass toward the edge of a table. In a split second, she said, the human brain could assess "the geometry of this glass, its place in 3D space, its relationship with the table, the cat and everything else," then predict what would happen and take action to prevent it. "Nature has created this virtuous cycle of seeing and doing, powered by spatial intelligence," she said. Her own lab at Stanford University was trying to teach computers "how to act in the 3D world," she added, for example by using a large language model to get a robotic arm to perform tasks like opening a door and making a sandwich in response to verbal instructions. Li made her name in the AI field by developing a large-scale image dataset called ImageNet that helped usher in a generation of computer vision technologies that could identify objects reliably for the first time. By jumping into the startup world, Li is joining a race among the hottest AI companies to teach their algorithms common sense in order to overcome the limitations of current technologies like large language models, which have a tendency to spit out nonsensical falsehoods in the midst of otherwise dazzling human-like responses." (keep reading) Reuters: https://lnkd.in/gmU8Ga2B #ai #notgenai #3d #spatialintelligence #spatialcomputing #ourworldis3d

12

4 Comments

For all of you space enthusiasts, excited to share the results of some work over the last 5 months between my friends at B612 and Google Cloud. Using GKE, BigQuery and Cloud Storage, Edward Lu and his team, researchers from the University of Washington along with Massimo Mascaro and some folks from Google, were able to comb through existing archived astronomical surveys and discover 27,500 new high confidence asteroid discovery candidates, including more than 100 that are classified as "Near Earth" 😳 And with the data pipelines established, we can now move to using AI to automate classification and risk based analysis at a rate that will allow us to match the pace of cosmological data growth with insights and actions. As Ed put it, being able to match the photonic capabilities of telescopes with the electrons in data centers at massive scale promises to unlock more discoveries like this, including supernovae explosions, minor planet discoveries and maybe even help us avoid the wrong kind of impacts ;) https://lnkd.in/gJa72Zn2 https://lnkd.in/gJRnH5pB

138

9 Comments

"AI beats humans on some tasks, but not on all." In this week's #RadicalReads, I share key takeaways from the 2024 AI Index Report:

16

Solitary LLMs are impressive, but several can work together as a Multi-Agent System to astounding effect and overcome solo LLM (GPT-4o, Claude 3, Gemini) limitations. Hear all about it in today's "Five-Minute Friday" episode. In a bit more detail: • In recent weeks, OpenAI's GPT-4o and Google's Project Astra showcased A.I. agents that engage in humanlike conversations and analyze real-time video. • Despite such impressive capabilities, single A.I. agents still struggle with many categories of complex tasks. • Thoughtfully linking individual LLMs together into a Multi-Agent System (MAS) enables them to work together, assign tasks, build upon each other's work, and deliberate to find solutions, in aggregate exceeding the capabilities of a single LLM on its own... often without any decrease in compute efficiency. • As an example, a DARPA experiment demonstrated 3 agents (named Alpha, Bravo, Charlie) collaborating to defuse (virtual!) bombs efficiently. • In an Massachusetts Institute of Technology study, two chatbots solving math problems outperformed a single agent through dialogue and updating each other's answers. • Microsoft developed a software-writing MAS with specialized roles (Commander, Writer and Safeguard; see the image for this post) wrote code 3x faster without sacrificing accuracy. • As always, there are potential risks, e.g.: illogical solutions cascading through the team, agents getting stuck in loops, and malicious exploitation. • Commercial interest in MAS is accelerating, with frameworks like AutoGen and Camel making it straightforward for you to get started with MAS today yourself. What in your industry could be automated or improved by MAS where single-LLM approaches aren’t sufficiently effective? 🤔 For more detail on all of the above, check out today's episode (#788)! The Super Data Science Podcast is available on all major podcasting platforms and a video version is on YouTube. #superdatascience #machinelearning #ai #llms #multiagentsystems

52

4 Comments

Multi-agent systems have been steadily gaining adoption, but it’s important to understand that they aren’t a blanket solution.  In fact, they introduce a range of complexities around coordination, consistency, and coherence that single-agent setups avoid. In my conversation with Chi Wang, the creator of AutoGen, he underscored the importance of matching your product architecture to the problem you’re trying to solve. If your use case involves narrowly defined tasks, a lone agent may be the simpler, more efficient choice. Other factors to consider before making the single agent versus multi agent decision? Response speed, decision-making frequency, inter-agent communication needs, latency, and bandwidth. Check out my full conversation with Chi here:

39

9 Comments

I remember learning category theory in undergrad and wondering: when would this ever be useful? Turns out it's very useful for program synthesis and software modeling -- thanks Kristopher for the great talk!

9

Andrew Ng has once again put together a great summary of an important #AI topic - I highly recommend reading this!

25

4 Comments

Can’t agree more on this as we are seeing more early stage startups building more useful AI-native agents instead of copilots on existing workflow.

4

This is worth a watch. A very insightful interview with Eric Schmidt on potential directions that AI can take. Thought provoking for sure. #ai #ml #generativeai #technology #ethics #policy #artificialintelligence #innovation #cybersecurity #skynet https://lnkd.in/e_bmbC4w

52

22 Comments

2 months ago, Viam asked our community to use Viam to bring AI to a real world device. It was really cool to see new use cases for Viam brought to life, most of which were practical solutions to everyday problems faced by participants. Here is a look at the winners: DEB-Forge: A Viam Powered Smart Desk With Contextual Lighting System. Made using Viam's camera component, data management service, vision service and ML model service to create a smart desk intuitive to his light sensitivities. Betsy: A  personal assistant robot designed to streamline your daily tasks and elevate your workspace experience. Made using Viam’s base remote control, ML and vision services along with Viam’s speech module to create Betsy. Assisted Smart Trash: A way to accurately and efficiently sort waste. Made using Viam’s camera component, ML model service, vision service and YOLOv5 module. Check out the full blog for tutorials on each + some really cool additional entries (including a companion robot for distance runners and a companion robot). https://lnkd.in/gtWemC4S

77

7 Comments

Excellent article honing in on second order effects of #AI from #security perspective

5

inception llms: experimenting with a new LLM design pattern where I prompt Gemini to write code on my behalf which, in turn, uses Gemini to perform tasks - So meta, isn't it? One prompt for one task. in less than two hours created a full pipeline which would have taken days otherwise. Gemini wrote more than 97% of my code, discoverying libraries I was not aware of - overall, saving me tons of time as swe. Where do you think you can use this power in your product? try it yourself next time you develop a feature. I named it "inception llms"

76

10 Comments

One of the first large scale web projects I did included ActiveX controls from Microsoft (for those of you that know, don't judge me...I was still basically a teenager). The problem at the time is that we knew the web was the future, but the tools were terrible. Short of static content, there wasn't really any way to build real applications for the web. ActiveX was Microsoft's attempt to bring desktop like software to the web. Only problem was it was hard to build, hard to deploy, and hard to maintain. Other than that, it was great. It took a decade of open-source development on JavaScript libraries to finally match the potential functionality with the portability of the web. We're seeing the same pattern repeat in AI. The tools are still primitive. Deployment and maintenance are going to be difficult. Early adopters will be faced with technical debt or rip and replace projects within the next 5 years. But that's OK. These early projects will help get the company AI-ready just like those early ActiveX apps got our customers web ready. https://buff.ly/3UWegLU

5

4 Comments

California's proposed SB-1047 law aims to regulate AI by holding developers responsible if their models are misused. While this is presented as well-intentioned, it could actually slow down AI innovation. - Only large companies might afford the legal risks, sidelining smaller innovators and startups. Those big corpos could use it against competition and get further advantage and monopoly. - Open-source projects could diminish as developers fear liability, reducing collaborative advancements. - Innovation could become centralized in big corporations, stifling diverse ideas and slowing overall progress and leaving the same possible misuse of the technology. SB-1047 targets the technology itself, not just its uses. This could discourage developers from creating new AI models or sharing their work. Imagine inventing something useful but being held responsible for any misuse—that’s the problem with this law. AI is a versatile tool. Regulating the entire technology because it can be misused is like banning all knives because they can cause harm. Instead, we should regulate specific harmful uses while encouraging the beneficial ones. Which is not easy but critical. We need to focus on specific high-risk uses of AI, not the technology itself. This way, we can address safety concerns without stopping innovation. Creating rules for how AI is used in critical areas while allowing development in other fields is a smarter strategy. Where do you stand on this issue?

9

5 Comments

#SynthAI - a great conceptual way to look at what we do Structize AI for customers. Thanks Zeya Yang for these insights!

15

I've been thinking a lot about how Tesla's vehicle infrastructure can leverage inference compute, distributing computational load and reducing data center energy usage. Dr. John Gibb does a nice job of summarizing these concepts in his video (Starting around 8:10). Inference Arbitrage: A Game Changer for AI Compute As AI continues to evolve, the demand for computational power is skyrocketing. Giants like OpenAI, Meta, and Google are pouring billions into training and inference compute, facing looming shortages of transformers and electricity. Here's where inference arbitrage, a novel concept leveraging Tesla's infrastructure, comes into play. The Challenge: Soaring AI Compute Costs Leading AI companies are investing billions to power their AI programs. However, the energy and computational needs are outpacing infrastructure capabilities. This bottleneck isn't just financial; it's physical. The growing demand for AI services could soon exceed available power and hardware. The Solution: Inference on the Edge xAI, in collaboration with Tesla, offers a groundbreaking solution: using Tesla vehicles for AI inference tasks. Tesla's fleet, equipped with powerful hardware and large batteries, can perform AI inference during idle times (about 90% of the time), reducing strain on central data centers How Inference Arbitrage Works Distributed Computing: Instead of relying solely on massive, centralized compute farms, inference requests (e.g., generating an email or image) are processed by idle Tesla vehicles. This distributed approach taps into the existing infrastructure efficiently. Energy Optimization: Tesla vehicles can perform inference tasks using their batteries during peak electricity demand times, avoiding high energy costs. At off-peak times, they recharge at lower costs, ensuring energy-efficient operations. Commercial Benefits: This model allows Tesla owners to earn money by utilizing their vehicle's idle time for AI computations. Tesla benefits from additional revenue streams with minimal overhead, and xAI gains a scalable, cost-effective solution for AI service delivery. Impact on the AI Industry By integrating AI inference with Tesla's distributed compute network, xAI can deliver services faster and more economically than competitors. This innovative use of existing resources not only addresses the infrastructure bottleneck but also offers a sustainable path forward for AI growth. Conclusion Inference arbitrage provides a strategic edge for xAI and Tesla, combining AI innovation with smart energy use. As the demand for AI compute continues to rise, this approach could redefine the landscape, positioning xAI and Tesla as leaders in efficient, scalable AI service delivery. Stay tuned as this exciting development unfolds and reshapes the future of AI. #ElectricVehicles #InferenceCompute #DataCenters #Electrification #AI #FrontierTechnology #FutureFrontier

4

1 Comment