Ronjon Nag

A design pattern in LLM workflow is to use LLMs to augment each step along the build workflow so that human efforts can be statistically amplified. Areas where we can use LLMs for amplification: human-in-the-loop synthetic training data generation, LLM as a judge, eval data generation, RLAIF, etc. In this pattern, the role of human shift from creators to reviewers. And with ongoing human review, the LLMs are self-improving overtime (using in-context learning) to become better human amplifiers. Who are building self-improving AI systems to support this pattern of ongoing human intent clarification & amplification?

21

GENE EDITOR CREATED BY #GENERATIVEAI 𝗙𝗿𝗼𝗺 𝗧𝗲𝘅𝘁 𝘁𝗼 𝗩𝗶𝗱𝗲𝗼 𝗮𝗻𝗱 𝗻𝗼𝘄 𝗗𝗡𝗔, 𝗚𝗲���𝗔𝗜 𝗰𝗼𝗻𝘁𝗶𝗻𝘂𝗲𝘀 𝘁𝗼 𝗰𝗿𝗲𝗮𝘁𝗲... Generative AI, known for creating poetry, programs, and imagery, is now used to generate blueprints for biological mechanisms to edit DNA, paving the way for more precise treatments for illnesses and diseases. Profluent, a Berkeley startup, introduced new AI technology to create gene editors by analyzing extensive biological data, including the microscopic mechanisms scientists use to edit human DNA. These gene editors leverage Nobel Prize-winning CRISPR technology, which has revolutionized the study and treatment of hereditary conditions like sickle cell anemia and blindness. Profluent's AI-generated gene editor, OpenCRISPR-1, is open-sourced, allowing individuals, academic labs and companies to experiment with the technology for free. Though Profluent is open sourcing the gene editors generated by its A.I. technology, it is not open sourcing the A.I. technology itself. While these innovations could transform healthcare, they haven't undergone clinical trials, and ethical and safety concerns, particularly about human gene editing, remain. If we remember, already the emergence of "home scientists" using CRISPR has raised concerns about unsupervised gene editing. 🔗 Check full article on The New York Times, link in the comments. 📽 Video Credit: Joseph Gallagher / Profluent Bio : A time-lapse of human cells edited by OpenCRISPR; source: The New York Times. #ArtificialIntelligence #transformers

8

1 Comment

Study Finds LLM Errors in Clinical Coding "Unacceptably Large" A new study from the Icahn School of Medicine at Mount Sinai has found that currently available (LLMs) like GPT-3.5, GPT-4, Gemini Pro and Llama2-70b are not sufficiently accurate to automate the assignment of medical codes for billing and research purposes. The researchers compared these state-of-the-art LLMs on their ability to match over 27,000 diagnosis and procedure codes to their official text descriptions from Mount Sinai patient records. They found that all models scored below 50% accuracy, with GPT-4 performing best at around 45% for diagnosis codes and 50% for procedure codes. The researchers determined the extent of errors was still "unacceptably large" for using LLMs to fully automate medical coding without human oversight. What does this mean? General-purpose LLMs have limitations for specialized healthcare tasks like medical coding. There is a need for more specialized, tailored AI models and tools specifically designed and refined for healthcare use cases. Integrating LLMs with expert clinical knowledge could improve their accuracy. Rigorous evaluation and iterative refinement will be critical before deploying AI The implications point toward a future where AI will likely play an increasingly important role in healthcare, but via specialized models that are thoroughly validated. General AI may augment human expertise, but is unlikely to fully automate complex clinical tasks anytime soon without significant refinement tailored to the healthcare domain. More like this? I invite you to my LinkedIn group, “Artificial Intelligence in Mental Health”—a vetted, science-based forum focused on the intersection of AI and mental health care, free from promotions or marketing. Link in comments. #ai #aihealthcare #healthcare #coding #LLM The Study: https://lnkd.in/gmaf_4Z8

3

1 Comment

🚀 Revolutionizing Infant Health: UChicago's AI Breakthrough! 🧬Exciting news from the University of Chicago! Researchers have developed a groundbreaking AI tool that models the infant microbiome, dubbed a "digital twin." This revolutionary technology predicts how microbial interactions within the gut can affect neurodevelopment. Using data from NICU preterm infants, this model (Q-net) has impressively predicted cognitive risks with 76% accuracy. This AI isn't just a faster way to study complex microbial dynamics—it's a transformative approach that may significantly advance our understanding and treatment of neurodevelopmental disorders. Dive into how AI is paving the way for major healthcare innovations! #HealthTech #AI #MicrobiomeResearch #UChicago

11

If you are involved in developing LLM applications, you may find this resource useful. It highlights key strategies for addressing the challenges in LLM application development. #LLMDevelopment #DataQuality #AIApplications 

3

https://buff.ly/3USEnmB The FDA is exploring the use of AI technology to predict and detect symptoms of Parkinson's disease. By utilizing artificial intelligence and machine learning models, they aim to establish effective methods for measuring Parkinson's symptoms using devices like wearables and smartphones. As the field of digital health advances, the FDA is examining how these technologies can gather health data beyond traditional clinical settings. Several wearable technologies have already received clearance to monitor Parkinson's symptoms, highlighting the growing importance of remote health monitoring. #FDA #ParkinsonsDisease

2

2 Comments

Andrew Huberman, a distinguished professor of neurobiology and ophthalmology at Stanford School of Medicine, delves into the fascinating world of improving focus and attention. Let’s explore the key takeaways from his discussion. #Brain #Focus #Neurodegenerative #Scitke

2

AI is often described as a 'black box'. Could some of the new research from Anthropic lift the lid and allow us to peer inside and understand what is going on? What does this mean for interpretability, ensuring safety, and what does any of this have to do with the Golden Gate Bridge?? Earlier this week I attended an event in London held by Anthropic, where they revealed some interesting new internal research that unveiled a little about how LLMs 'think'. By using a process called 'Dictionary Learning', the scentists studied the activation of groups of neurons and were able to identify features that correspond to concepts, which in turn are clustered into regions that are semantically similar. They'd done this on small models before, but now they have succeeded in using it on large production models. "So what?" I hear you say. Well, other than showing how LLMs represent real world concepts, and how they might link to each other, it turns out that you can activate or inhibit these regions to guide the model to generate outputs that either include or avoid the concepts. In the video I've linked you can see this in action: https://lnkd.in/eT2a6YFb As the work continues, the researchers hope to better understand how LLMs generate their output, but also how they can use these findings to help control models and steer them away from harmful or dangerous content. They've even added a temporary feature which allows you to speak with Golden Gate Claude! (an AI that is clamped to obsess about this famous San Francisco landmark) Just login and click the small red bridge in the top right. WARNING: It gets very repetitive. Why does this matter for healthcare? Two reasons: 1) Interpretability - this is seen as critical if we are to trust the output of models in clinical uses. Although current AI models will provide explanations, these are not reasoned answers in the sense we understand it. If we can open up the black box and peer inside, it will be easier to bring this technology into regulated devices. 2) Alignment - LLM developers may be able to identify features that signify harm, or bias, or unintended side effects, and use these to restrict how a model generates output. You can read more about their work on the blog: https://lnkd.in/e5AjmUAx You can also log into Claude and give it a try! https://claude.ai/chats #AI #LLM #GenerativeAI #Interpretability #Healthcare

5

1 Comment

LLMs are designed to make stuff up. In most applications this an acceptable risk, after all, people make stuff up too. But in regulated, risk averse industries (finance, healthcare, insurance, legal) even a small mistake can cause big problems. Manual evaluation and monitoring is a crucial yet often overlooked component to catching errors and mitigating risk. Automated oversight can help, but there's no such thing as "Goodbye Hallucinations" unless humans are firmly in the loop.

10

1 Comment

Exciting Opportunity for AI Researchers: Agora Compute Grants We are thrilled to announce the launch of Agora Compute Grants, an initiative designed to empower AI researchers with the computational resources they need to push the boundaries of multi-modal research. https://buff.ly/4clNtQF What is Agora Compute Grants? Agora Compute Grants is an open-source AI research lab initiative aimed at providing AI researchers with access to powerful GPU clusters. These resources are essential for conducting advanced multi-modal research, which integrates diverse data types such as text, images, and audio to create more sophisticated and capable AI models. Why Focus on Multi-Modal Research? Multi-modal research represents the future of AI. By combining various data types, researchers can develop more robust and versatile models, leading to innovations that can revolutionize industries and improve lives. At Agora, we are committed to supporting this cutting-edge research and helping you achieve breakthroughs. How to Get Started: -> Join our Discord community: https://buff.ly/4clNtQF -> Sign up and introduce yourself to the community. -> Share your project in the Creator Showcase to be considered for the compute grants. We invite all passionate AI researchers to take advantage of this incredible opportunity. By joining Agora Compute Grants, you'll gain access to the computational power needed to drive your research forward and make significant contributions to the field of AI. Let's shape the future of AI together. Join our Discord today and start your journey with Agora Compute Grants. For more details and to sign up, visit our Discord community: https://buff.ly/4clNtQF We look forward to seeing the innovative projects you bring to life!

8

1 Comment

https://lnkd.in/dSQGZg_T "They warned that such AI systems could perpetuate the racism and sexism found in the huge troves of written data they've been trained on. "The problem with that kind of misinformation is that we're swimming in it," Bender said. "And so people are likely to get their biases confirmed. And it's harder to spot misinformation when it's confirming your biases." Another concern was a deeper one — that ceding information retrieval to chatbots was degrading the serendipity of human search for knowledge, literacy about what we see online, and the value of connecting in online forums with other people who are going through the same thing. Those forums and other websites count on Google sending people to them, but Google's new AI overviews threaten to disrupt the flow of money-making internet traffic. Google's rivals have also been closely following the reaction. The search giant has faced pressure for more than a year to deliver more AI features as it competes with ChatGPT-maker OpenAI and upstarts such as Perplexity AI, which aspires to take on Google with its own AI question-and-answer app."

2

1 Comment

🌟 BREAKTHROUGH DISCOVERY IN CANCER GENETICS 🌟 UCLA researchers have published a groundbreaking study in Nature Communications that revolutionizes our understanding of how noncoding DNA influences cancer progression. Their discovery of mutations in these "silent" regions significantly impacting mRNA levels, leading to abnormal protein production which can accelerate or hinder cell proliferation, challenges previous assumptions about noncoding DNA's role. The study also introduces a new prognostic tool, the untranslated tumor mutation burden (uTMB), which could predict patient survival in certain cancers. This pioneering research paves the way for developing targeted treatments and personalized medicine strategies, offering new hope in the fight against cancer. #CancerResearch #Genetics #InnovativeScience

16

Google presents Capabilities of Gemini Models in Medicine https://lnkd.in/dG3mZamb "Gemini models, with strong general capabilities in multimodal and long-context reasoning, offer exciting possibilities in medicine. Building on these core strengths of Gemini, we introduce Med-Gemini, a family of highly capable multimodal models that are specialized in medicine with the ability to seamlessly use web search, and that can be efficiently tailored to novel modalities using custom encoders. We evaluate Med-Gemini on 14 medical benchmarks, establishing new state-of-the-art (SoTA) performance on 10 of them, and surpass the GPT-4 model family on every benchmark where a direct comparison is viable, often by a wide margin. On the popular MedQA (USMLE) benchmark, our best-performing Med-Gemini model achieves SoTA performance of 91.1% accuracy, using a novel uncertainty-guided search strategy. On 7 multimodal benchmarks including NEJM Image Challenges and MMMU (health & medicine), Med-Gemini improves over GPT-4V by an average relative margin of 44.5%. We demonstrate the effectiveness of Med-Gemini's long-context capabilities through SoTA performance on a needle-in-a-haystack retrieval task from long de-identified health records and medical video question answering, surpassing prior bespoke methods using only in-context learning. Finally, Med-Gemini's performance suggests real-world utility by surpassing human experts on tasks such as medical text summarization, alongside demonstrations of promising potential for multimodal medical dialogue, medical research and education. Taken together, our results offer compelling evidence for Med-Gemini's potential, although further rigorous evaluation will be crucial before real-world deployment in this safety-critical domain."

5

I'm currently putting together a newsletter (vivianchan.substack.com) to share the latest startup and VC jobs in the Deeptech space. Quite a few people have asked me for intros or to tap into my network, so I thought I’d collate everything and share it with everyone. If you or your portfolio companies are hiring, let me know and I'll include it in the next edition! 🙌 Examples of available roles: AI Research Engineer, FutureHouse (AI, Non-profit research org backed by Eric Schmidt), San Francisco, US Head of Operations - AimHi Earth (Sustainability, Seed), Remote Clinical Laboratory Scientist - Juno Bio (Bioinformatics / Microbiomes, Seed), Oakland, California, US Deeptech/Climate Investor - Founders Factory (VC), London, UK or NY, US Bioinformatics Consultant - Deep Science Ventures (VC), London, UK VP Finance - Space Capital (VC), NY, US 📩 More in the first edition: vivianchan.substack.com #Deeptech #Startups #VentureCapital #JobOpportunities #Networking #TechJobs #Innovation #Entrepreneurship #Hiring #Jobs #CareerGrowth

25

5 Comments

Thought-provoking perspective from Cheryl Cui, Co-Founder & CEO of Bota Biosciences, on how AI can accelerate frontier innovation: "What we see, AI, because of the knowledge structure is so different from human education, it become a generalist in many disciplines. You would enable some of the specialized researchers, I use biologist as example, have a cohort of chemists and programmers working for it to enable its ideas. I think that's the beauty of AI, that would open up something other than just the labor efficiency aspect of things, but bring novelties and innovations to the world." Beyond productivity gains, word-crunching AI can serve as a translation layer, enabling highly specialized experts from different fields to communicate and collaborate more efficiently. Executives across industries should consider not only how these tools might make their teams more efficient at what they're already down, but also how AI might supercharge diverse, cross-functional teams of deep experts to drive outsized progress on the frontier. https://lnkd.in/gehEdjTf

4

This new research matches what we're seeing when testing #LLMs for #ResponsibleAI in #Healthcare: Fragility and Bias are the two more common issues today. Robustness and bias mitigation currently seem to be more impactful than other issues that receive more attention like hallucinations, toxicity, and prompt injection. I wonder how our benchmarks compare to what others find. https://lnkd.in/gHuxi2Py #generativeai #ai #datascience #ethicalai #responsibleai #medicalai #healthcareai

49

2 Comments

Confidence in model output (veracity and eliminating hallucination) is a huge need in the market to accelerate adoption of GenAI! Timely launch of Cleanlab's TLM solution....

20

3 Comments

America may run out of power with AI's growing energy appetite 🔌🏭 The bottleneck for AI progress will shift from NVIDIA/TSMC's chip production to energy companies' capacity to power AI data centers. For example: ⚡️ Amazon's 1GW data center next to a nuclear plant https://lnkd.in/g4zqTwhG ⚡️ Microsoft & OpenAI planning $100B, multi-gigawatt "Stargate" AI supercluster https://lnkd.in/gWvnhZms To feed the AI beast, we'll need 💰💰💰 of investments in new power plants, massive data centers, and chip fabs that can churn millions of GPUs/year. Leopold Aschenbrenner's back-of-the-envelope analysis highlights the staggering power needs (https://lnkd.in/gfBc5xhb) 💡 Rumored $100B cluster = 10GW = small US State's power draw The intersection of AI and energy markets is 🔥🔥🔥 Think beyond chips and models. What's the NVIDIA or TSMC of the energy space? Nuclear? Solar? Exotic energy source? #ai #ml #llm #largelanguagemodels #gpu #cloud #datacenter #aicompute #energy #datacenterenergy #nuclear #greenenergy #startups #venturecapital

34

24 Comments

Join C10 Labs and their AI Venture Studio for a roundtable discussion on building AI-first ventures. This is an opportunity to meet founders from the 1st cohort, network with other applicants from the 2nd cohort, and connect with early-stage entrepreneurs in the AI community. Don't miss out on the chance to learn from the experts and build your own AI venture. Register now! #AI #entrepreneurship #C10Labs #CIC

48

1 Comment