Scott Montgomerie

💚🔮 The future of healthcare: Where computational biology (and other healthcare AI) models can be deployed as easily as iPhone apps 🔮💚 For those of you who could not make it to #NVIDIA #GTC24 I wanted to dive a bit deeper into what I saw there as some of the amazing things already possible today, e.g. with the newly announced #NVIDIA microservices. These pre-packaged models allow developers and researchers to focus on what the AI can do for their use cases rather than spending their time getting the AI to run. "The new suite of NVIDIA healthcare microservices includes optimized NVIDIA NIM™ AI models and workflows with industry-standard APIs, or application programming interfaces, to serve as building blocks for creating and deploying cloud-native applications. They offer advanced imaging, natural language and speech recognition, and digital biology generation, prediction and simulation." "Researchers, developers and practitioners can use the microservices to easily integrate AI into new and existing applications and run them anywhere — from the cloud to on premises — equipping them with copilot capabilities to enhance their life-saving work."

10

The 20th edition of The Vision Debugged is LIVE now! https://lnkd.in/gZxS5mPN In this edition, Sandra Anil & I dive deep into the HOTTEST new AI video generator: Luma Labs' 𝐃𝐫𝐞𝐚𝐦 𝐌𝐚𝐜𝐡𝐢𝐧𝐞. 𝐖𝐡𝐲 𝐬𝐡𝐨𝐮𝐥𝐝 𝐲𝐨𝐮 𝐜𝐚𝐫𝐞? ✅ Turn text into hyper-realistic video clips in MINUTES (think: dev demos, product mockups, crazy marketing ideas) ✅ Free option available! (who doesn't love that?) ✅ MVP alert! Learn how Luma Labs is using this to gather user feedback & build a killer product (perfect for product managers ) Plus, we compare Dream Machine to other AI video tools & tell you which one is best for YOUR needs. Don't miss out! Click the link to snag this edition. ➡️ https://lnkd.in/gZxS5mPN PS: We crossed 600 readers this week 🙏 🎉Thanks to all our amazing subscribers for their consistent support, feedback & curiosity! If you find our newsletter useful, please share it with a friend and consider subscribing if you haven’t already 👉 https://lnkd.in/gwUNCBdJ

99

2 Comments

In the #Metaverse with Rich Destiny Rafael J. Grossmann, MD, MHDS, FACS At our Volumetric Studio in Montreal with the latest in XR 🎥🎥🚀🚀 #Apple #VisionPro #Meta #Quest3 #QuestPro Apple GeniusXR Meta #XR #AR #VR #Spatial #AugmentedReality #VirtualReality #MixedReality #SpatialComputing

38

2 Comments

You know that one pic of Mr. Beast? Influencers are instantly recognizable from their profile pictures. As Social Media moves into game engines, 3D Avatars will become the new PFP. It's time to get ready for the 3D internet. Motion Capture brings lifelike realism to avatars and NPCs. It's easier than ever to add Mocap data to avatars with the Sony Mocopi. More info below ↘

7

A few, short video clips made with Luma from Luma Labs. As with all AI, the outcomes are highly dependent upon the prompt. Here are the results of 3 different prompts in an attempt to get a group of superheros working together at a conference table. Perhaps this is some hint of an upcoming meeting? You know it is.... While the content results need work, and there are improvements to be made in my prompting, the ability of AI to conceptualize a business meeting with people around a table, movement, conversation and the context of superheroes is something that was not possible 6 months ago with most genAI without rendering wildly distorted body parts/movement and wildly different outcomes from similar prompts. Tremendous progress in a very short amount of time. This is more than a panda riding a skateboard, or a gen ai beauty shot or landscape. It's complex scenario... And I'm here for it. Teaser: Wait for the final frames of the 3rd video. You gotta see it.... #genai #filmai #contentai #videocontent #promptengineering

9

2 Comments

💥Google's new Med-Gemini for healthcare outperforming GPT-4 Med-Gemini is a family of multimodal AI models built upon their Gemini models, specifically designed for the healthcare industry. From what I've read, these models are built on Google's existing Gemini AI architecture, but they've been specifically designed and trained for medical and clinical applications. It can process long-form medical records and research papers, and it can output text, images and videos to help explain things. Google is claiming that Med-Gemini has even outperformed OpenAI's powerful GPT-4 model on certain medical reasoning tasks, which is pretty impressive. On standard medical benchmarks like the MedQA exam, the Med-Gemini-L 1.0 model has scored really high, beating out even Google's own previous Med-PaLM 2 model. One cool feature is that Med-Gemini integrates web search, so it can pull in reliable information from the internet to provide more nuanced and factual responses. The models aren't publicly available yet - Google is still working on improving them and making sure they adhere to responsible AI principles around privacy and fairness before they release them. But from what I've seen, Med-Gemini is shaping up to be a really powerful set of medical AI tools.

1

This is a highly requested tutorial. Learn a fun way to utilize all the powerful features to create this interactive 3d scene. Give it a try!

12

This is the second year of Meta Quest Mixed Reality Hackathon. I am participating with Cai Plank to build a multiplayer mixed reality game for the casual social gaming category. We are using Meta XR Scene SDK, Hand tracking SDK, Interaction SDK, Netcode, Relay and Lobby. Thank you so much Quentin Valembois and Justin P Barnett for the comprehensive tutorials! I am inspired by Jason Jerald, PhD, The VR Book in 2022 and still working towards it. Can't wait to see other entrant's work! #Meta #Mixed #Reality #Hackathon

13

🤯 Luma AI dropped their Dream Machine & it's insane! ▶ You can create a short-form video from a prompt or image ! 💡 Luma AI is OpenAI Sora and Google Veo competitor to create short-form video from a prompt! 🤩 The best part? They are actually letting people use it. ➡ Try it now : https://lnkd.in/eU9n5Ywh #️⃣ #luma #lumaai #dreammachine #ai #openai #sora #google #veo

2

☠ ALL things AR/VR/Spatial/Meta are DEAD! Maybe? With the negative Apple Vision Pro headlines this week (cutting production) and the constant attack on Meta (they lost almost $4 billion on XR last Q alone), there was a timely post from Jack Soslow (partner at Andreessen Horowitz). "We believe ARVR is among the most underrated markets today. Most people fixate on past (and current) missteps, leading to assumptions that are simply wrong. With ~30m Meta Quest headsets sold and a favorable industry outlook, we think the time is now." ➡ The VR market is bigger than most think (Meta sells over 10m headsets annually) ➡ Several ventures scale outcomes already (Beat Saber, RecRoom etc) ➡ Enterprise is still in its infancy (training, education etc are all best delivered in 3D) ➡ Undersupply of content (classic chicken and egg with hardware/software imbalance) While I do think comparing the XR hardware trajectory to iPhones is appropriate (image below), the biggest difference today is XR has nowhere near the user retention levels. BUT it does serve as a very important reminder that the iPhone took years to develop a robust app ecosystem. It didn't happen overnight. There was NO app store on the first iPhone lol. It feels like Meta Quest 3 and Apple Vision Pro mark DAY 1 of the next chapter in XR, and we're going to see a new batch of industry-redefining companies emerge that will only look obvious in hindsight. How are other VC's thinking about the sector? #spatialcomputing #metaquest3 #applevisionpro #vr #ar #virtualreality #mixedreality #innovation #technology #ai

168

30 Comments

Ok, Roblox released a long-awaited feature, Avatar Auto Setup, as a beta, and it seems very promising. This tool offers a streamlined approach to rigging, segmenting, and skinning 3D models with minimal effort. Leveraging AI to drastically reduce the time spent on avatar creation is undoubtedly appealing to creators looking to optimize their workflow. However, while exploring the Avatar Auto Setup feature, I encountered a few issues that may hinder its effectiveness, especially for more complex models that GIANTY normally produces. Firstly, the requirement for models to be in an A-pose with a single mesh and optimized Tris count could pose limitations, particularly for those working with intricate designs or existing models with multiple meshes. Additionally, the challenge of ensuring UV and texture compliance, especially when dealing with separate textures, could result in a lack of textures in the final output, as observed in my tests. Furthermore, the auto setup process faced difficulties in certain areas such as hips, shoulders, neck, and facial rigging, leading to instability in emotional expressions and potentially compromising the overall quality of the avatar animation. Lastly, the issue of clothing compatibility post-auto setup is worth noting, particularly if the character already wears clothing during the rigging process. This may require additional manual adjustments to ensure seamless integration. Overall, while Avatar Auto Setup shows promise in expediting the avatar creation process, especially for simpler models, addressing these issues will be crucial to fully realizing its potential and ensuring its effectiveness across a wider range of scenarios and models. Check out the video of the play-through!

19

🏮 Short-form Narrative Pipelines for Social Platforms, Part 3 🏮 To help us streamline our workflow, we worked with Chaos, Epic Games, NVIDIA, and Lenovo to push the real time rendering envelope by fully ray tracing live using Unreal Engine. For those unfamiliar with the nuances of lighting and rendering, here's a high level breakdown along with an explanation as to why FULLY ray tracing is a big deal: Games use a rendering technique called rasterization to light environments. Basically, individual pixels in a 3D mesh are shaded based on how lights in the environment hit each pixel. While computationally intensive, game engines are able to handle the workload in real time. That's why when you walk down a street in a video game it feels like you're in a 3D world, even though you're viewing that world on a 2D screen -- the horizon/environment is being rendered in real time as you walk, so it feels like you're in it. Movies and TV shows on the other hand use ray tracing to light environments. Working backwards from the POV of the viewer, ray tracing is a physics-based rendering technique that follows (traces) light paths (photons) as they bounce off objects. This technique is more computationally intensive than rasterized rendering and is done offline on servers after a scene has been filmed. Harnessing the power of modern GPUs, ray tracing produces images that are indistinguishable from reality, and that's why films look more "real" than games. The holy grail for games is to achieve film quality rendered environments, while the holy grail for 3D films is to be able to see a final frame image in real time. Certain game engines, like Unreal Engine, come equipped with a ray traced effect added on top of the rasterized image. This gives the game an incredible look right out of the box, but to get to that film quality bar, you need to go that extra little mile. You need to FULLY ray trace. That means film teams will film once in engine and then re-render everything they just filmed offline to reach that film quality bar. That means more time and more $$. Working with the team at Chaos, they live linked Chaos Vantage directly into Unreal Engine. This gave us two screens when filming. One that showed the scene in Unreal and one that showed the same scene in Vantage. Both were capturing in real time. The scene in Vantage was fully ray traced live at about 30fps (see example below). This allowed us to get really close to a final frame image in real time, which we presented at NVIDIA GTC22. Why is this a big deal? Because the director can CHOOSE what shot to finish rendering offline instead of re-rendering everything. It's a much more agile workflow that reduces the time between concept and market. If you want to learn more about what an interoperable 3D asset future looks like, check out my 'Interoperable Playbook' on my profile page or download it for free here --> https://lnkd.in/gGHDMZ4p Cheers! 🖖

1

cool workflow here shared by a customer: use (private beta) 3D illustration tool Project Neo to create an asset that you can use as a “structure reference” for a prompt in Firefly. when we play with new tools, new possibilities emerge

239

9 Comments

More evidence of Luma's 3D tracking and movement consistency when given a complex 3D form, beyond impressive - and indicative of Luma's core biz...

2

Another AI this week! Dream Machine from Luma AI. a text-to-video model design likes of OpenAI's Sora and Kuaishou Technology's Kling but quite different. It generates high quality, realistic clips from simple text prompts in minutes. "Dream Machine is our first step towards building a universal imagination engine and to creating general intelligence in its most beautiful and useful form." - Amit Jain co-founder of Luma AI. Luma ai also able to create video with cinematic and naturalistic camera motions. This looks like a movie clip. The process! - you have to enter a text PROMPT. - it will be in DREAMING. - and CREATION is done. Though the sever is busy your prompt will be queued and takes some minutes to generate. The user interface is quite amazing! It currently generates 5-second video and has some limitations. I’ve been playing with this! Do try this out its for free now… #lumaai #dreammachine

15

2 Comments

Yann LeCun , Chief AI Scientist at Meta, says in 10 years we won't have smartphones anymore. We will have augmented reality glasses and bracelets to interact with our intelligent assistants. I think LeCun's prediction align with the growing trend towards wearable technology and augmented reality. Smart glasses and bracelets could revolutionize how we interact with #AI, offering a better and immersive experience. Have you seen the Ray-Ban/Meta glasses yet? This shift could redefine personal computing, integrating AI into our daily lives in a more intuitive way. It's a vision that's both exciting and challenging, requiring advancements in AI, hardware, and user experience design. What do you think, are we getting rid of smartphones any time soon?

23

27 Comments

🚀 Exciting times ahead for XR! According to Jack Soslow of Andreessen Horowitz (A16Z GAMES), #AR+#VR technology is one of the most underrated markets today. While it’s easy to dwell on past challenges, current data tells a different story. With millions of headsets sold, the #XR market is showing signs of robust growth, similar to the early days of another revolutionary device—the iPhone. Jack notes, “We believe ARVR is among the most underrated markets today. Most people fixate on past (and current) missteps, leading to assumptions that are simply wrong. With ~30m Meta Quest headsets sold and a favorable industry outlook, we think the time is now.” I personally view VR headsets as more akin to computers than mobile phones due to their complex functionality and usage. However, comparing their adoption to that of the iconic iPhone underscores the significant traction and potential of XR technology. ⁉️What are your thoughts on the future of AR/VR? Are we looking at the next big shift in how we interact with technology? #ARVR #TechTrends #Innovation #Future #AndreessenHorowitz #VirtualReality There are some more data points to verify and further reading linked in the articles below.

54

16 Comments