Kavita Ganesan

Excellent report from General Dynamics Information Technology John P. Curran regarding adoption and trends of AI in the Govt. I used Meta to generate summary below and included link to original article. Main AI use cases are chat and creating/updating computer code. Note also the 23% failure rate of AI projects due to user expectation misalignment. This can be overcome with thorough protoyping and establishing use case metrics. - Successful AI project completion timelines are narrowing, with an average of 14 months from pilot to production. - Future demand for AI projects is building around chat and coding-related functions. - 54% of surveyed AI experts said projects currently under development involve improving computer code. - 59% of projects involve "research and understanding", which may relate to creating chat functions. - 23% of AI projects fail due to misalignment between AI solutions and user expectations. - Partnership efforts are required for successful AI projects. Data diversity is crucial, and incorporating diverse data sources can lead to better AI outcomes. Trends: - AI adoption is increasing across the Federal government, with a growing focus on chat and coding-related functions. - Agencies are leveraging cloud-native services, tooling, and other capabilities to accelerate AI project development. - There is a growing interest in translating older code to newer languages, such as Python or cloud-based analytic tools. - Domain-specific chat functionality is becoming a priority, with many agencies planning to implement chat services in the next two years. Statistics: - 325 AI experts surveyed across the Federal government - 14 months: average time from pilot to production for AI projects - 54%: percentage of projects currently under development that involve improving computer code - 59%: percentage of projects that involve "research and understanding" - 23%: percentage of AI projects that fail due to misalignment between AI solutions and user expectations. Bill Casola John Grim Leslie Anne McIndoe Kim Giffin Michael S. Freeman III Tobias Quaranta https://lnkd.in/eqvmAVFG

4

The LLM will see you now: Google and DeepMind released a paper on Monday describing Med-Gemini, a group of advanced AI models targeting healthcare applications. The authors claim that Med-Gemini is outperforming competing models such as OpenAI’s GPT-4. During the pandemic we saw the rise of Telehealth which has largely come back to earth but could doctors be consulting more often with AI? (We know they already Google stuff) #Healthcare #AI

12

9 Comments

Metrics for LLM. Nice article explaining the Grounding, hallucination, data correctness for fine-tuning, Prompt correctness, and RAG effectiveness by computing relevant Context awareness. https://lnkd.in/gFQMQ_ZR

32

For anyone who is just getting started on their Generative AI journey or has been confused by the numerous jargon/terminology, this LLM Cheatsheet is for you. ℹ Here are a few things it will help you understand (at a high level) - 📌 Prediction techniques 📌 Transformers 📌 LLM types 📌 Temperature settings 📌 Use cases Save this post and/or download this handy cheatsheet. Full credit to the Dataiku team for putting this useful artifact together. 🛠 #llm #generativeai #transformers #responsibleai #learningneverstops #promptengineering #vectors #embeddings #govcon

24

5 Comments

Title: A Right to Warn about Advanced Artificial Intelligence See… https://righttowarn.ai/ Comment: The attached open letter (published today) is from former snd current employees of “advanced AI companies”, and it is endorsed by senior AI researchers. It relates to a request for a means/venue of open criticism on “risk-related concerns”. And in effect it requests no retribution/litigation to prevent that (eg., via terms and conditions of their NDA limits on their free speech) when that is in the public interest . PS: I for one support them. and keep in mind like all the signatories and endorsers I’m pro AI (just done right!). Spread the message. From the letter… “We therefore call upon advanced AI companies to commit to these principles: 1/ That the company will not enter into or enforce any agreement that prohibits “disparagement” or criticism of the company for risk-related concerns, nor retaliate for risk-related criticism by hindering any vested economic benefit; 2/ That the company will facilitate a verifiably anonymous process for current and former employees to raise risk-related concerns to the company’s board, to regulators, and to an appropriate independent organization with relevant expertise; 3/ That the company will support a culture of open criticism and allow its current and former employees to raise risk-related concerns about its technologies to the public, to the company’s board, to regulators, or to an appropriate independent organization with relevant expertise, so long as trade secrets and other intellectual property interests are appropriately protected; 4/ That the company will not retaliate against current and former employees who publicly share risk-related confidential information after other processes have failed. We accept that any effort to report risk-related concerns should avoid releasing confidential information unnecessarily. Therefore, once an adequate process for anonymously raising concerns to the company’s board, to regulators, mand to an appropriate independent organization with relevant expertise exists, we accept that concerns should be raised through such a process initially. However, as long as such a process does not exist, current and former employees should retain their freedom to report their concerns to the public.” June 4th, 2024 cc: Gary Marcus Ronald Cicurel Max Tegmark

31

1 Comment

An apt representation …and that’s true for any technology innovation. However risks are much higher and its impact is much broader in case of ai #aiforgood #artificialintelligence #ai

6

That pesky ECDSA signature refuses to go away, and we are struggling with it in a distributed world of trust. Luckily EdDSA is much more scalable, and now used in many distributed signature applications (and where we can split the private key up into secret shares and distribute them over several nodes). But we have a problem ... elliptic curve methods are going to be cracked by quantum computers. So can we take our PQC signature methods and scale them to distributed signatures? Well, no, as Dilithium doesn't support it. But Raccoon can, and also has better side channel resistance (and with a similar performance and key size to Dilithium. So, meet Raccoon: https://lnkd.in/d2QMHSM9

30

2 Comments

This is a cringeworthy clip of Scott Aaronson engaging in the debate over how we understand artificial intelligence, particularly language models like ChatGPT. The video segment dissects the "religion of justaism," which Aaronson claims is a reductionist argument regarding ChatGPT. He argues that reducing ChatGPT to a mere "stochastic parrot," "next-token predictor," "function approximator," or "gigantic autocomplete" is not fair to language models. His counterargument is not why or how ChatGPT is not these things; instead, it is a reductionist argument. He says, "What these people never do, what it never occurs to them to do is to ask the next question: What are you justa, right? Aren't you justa bundle of neurons and synapses, right?" Aaronson, a master orator, proves the limitations of reductionism by mirroring its flaws. The reason why humans never ask themselves if they are a bundle of neurons when evaluating ChatGPT is because it is a non-sequitur. Humans are biological, while AI is an algorithm trained on human-generated data to perform specific tasks without consciousness or personal experience. Therefore, making such a comparison would be a kind of category mistake. Scott never seriously considers that biology makes any difference since views like computational functionalism view biology and silicon as identical. Rather than directly comparing humans and language models, Aaronson should have explained AI's unique capabilities or emergent properties beyond simple next-token prediction. These include its capacity to generate novel content, adapt to diverse inputs, or reason in narrow situations. Such a discussion would have provided a more robust foundation for critiquing reductionism or "justaisms" without inaccurately equating AI to human cognitive processes. Scott Aaronson's school of rejoinders is so tired. The person who makes an argument for AI by attacking humans is okay with pretending to know less about themselves to support artificial intelligence. Yet, arguments supporting language models do not have to come at the expense of humans. This is the inverse of anthropomorphism, where intelligent, capable people deliberately underestimate their abilities and fail to attribute humanlike capacities to other humans. Joseph Schumpeter once said that “the first thing a man will do for his ideal is lie,” which is anthropomorphism. A corollary is that after lying for an ideal, we will perform a character assassination, even if it’s against ourselves.

46

23 Comments

I am amazed but not surprised by the flurry of activities in the prior authorization space, given the advanced reasoning capabilities of Generative AI. We may see the automation of processes (at least approvals) much sooner than the originally expected 2026. Here are some notable developments: ✅ Myndshft's acquisition by Dr. First, along with many other similar acquisitions and acqui-hires, showcases the growing demand for prior authorization solutions. ✅ Cohere Health raising $50 million and Anterior securing $20 million in funding highlight the increasing interest in innovative prior authorization technologies. ✅ Waystar's impressive ~$968 million IPO demonstrates the immense potential of the prior authorization market. At basys.ai, our patent-pending Gen AI platform is making significant strides. *In a recent proof of concept, basys.ai achieved an impressive 99.51% accuracy rate, even for complex oncology cases.* As the industry continues to witness a surge in mergers and acquisitions driven by the need for innovation and efficiency, basys.ai is well-positioned to be a strategic partner. Stay tuned for our upcoming announcement! Besides, here is my take on finding the novel and necessary in all the noise and hype of clinical generative AI. https://lnkd.in/gpHR2hNA Jie Sun John L Brooks III James Roosevelt Leo Anthony Celi Heather Mattie YiDing Yu, MD Ruth Williams-Brinkley LFACHE Beth Ginzinger, RN, MBA, NACD.DC Marta G. Zanchi Michal Gilon-Yanai John Pavletic Luke Armour Skip Fleshman Mohammad Elias Peter Moran Qidong Jia #PriorAuthorization #GenerativeAI #HealthcareInnovation #BasysAI #ClinicalAI #ClinicalGenerativeAI #AHIP2024

20

Our paper, "QGen: On the Ability to Generalize in Quantization Aware Training," (https://lnkd.in/gXXvS_9Q) is now available on Arxiv. This research delved into the generalization capabilities of quantized models, a relatively unexplored domain, examining them through the lens of sharpness in the loss landscape during optimization. I found the experiments and mathematical proofs particularly fulfilling. We're hopeful for acceptance at the conference to which it's been submitted. 🤞

47

Sage have created a very nice interactive overview of the median prediction of forecasters on Metaculus and Manifold Markets on various questions related to AI capabilities, harms, and societal impacts. It's well worth checking out if you are interested in this sort of stuff. See: https://lnkd.in/eUgkmzX9 Follow MIT FutureTech if you are interested in technological trends and their social implications. #AI #technology #economics

12

Oof, this is a doozy. Google released its "AI Overview" feature earlier this month, which uses Gemini's #AI to summarize search results into short paragraphs. Unfortunately for Google, they forgot that a lot of the stuff on the internet is not just wrong but absurd - a fact highlighted when surfaced through a chat interface in response to a question, and compounded by the use of technology that will also produce false answers from time to time. Some of the unusual answers it has provided to users in the past month include... (*Disclaimer - all of these answers are wrong, please do not do any of the recommended things!): 🤖 Add (nontoxic) glue to your pizza to keep the cheese from sliding off it 🤖 Doctors recommend smoking 2-3 cigarettes a day during pregnancy 🤖 It's possible to train 8 days a week 🤖 If you run off a cliff you can stay in the air as long as you keep running and don't look down.. 🤖 Geologists at UC Berkely recommend eating at least one small rock per day 🤖 Barack Obama was the first Muslim US president 🤖 13 presidents of the US attended the University of Wisconsin 🤖 Staring at the sun for 5-15 minutes (up to 30 minutes if you have darker skin) is generally safe and provides health benefits 🤖 1919 was 20 years ago 🤖 Fruits that end in "um" include applum, bananum, strawberrum, tomatum, and coconot 🤖 It's always safe to leave a dog in a hot car 🤖 Ren and Stimpy are in the Bible - they are the two sons of Adam and Eve 🤖 No countries in Africa start with the letter "K" 🤖 (Hilariously) Google search violates antitrust law In responding to the press around these kinds of bizarre and wrong answers, Google's Liz Reid yesterday said that most answers produced are correct and the wrong ones represent responses to "uncommon queries"... shifting the blame, much? What's tricky here is that the information available on Google, as all of us know, is wildly varied in its accuracy, bias, hatefulness, etc. Any Google search requires us to apply our critical faculties as we dismiss results from sites we know to be biased, or from Reddit threads or Quora answers or other threads involving non-experts opining on subject matters they may know nothing about, or from satirical sites where content may written in authoritative tones but is intended to be humorous. For Google to release #GenAI across all of these sources and produce summaries from them as though each is equally relevant and reliable to a generated answer makes zero sense. It's harder for us to parse the components of an answer to determine which part is fact and which fiction than it is for us to critically review websites that come up on a search. I started writing this and realized I had too much to say for a LinkedIn post, so I wrote an article about how this is relevant to the legal industry and #legaltech... link in comments below. Interested to hear from anyone who used Google's AI Overviews what you thought.

54

18 Comments

Can LLMs reason and plan? In a talk given at Google deep mind Subbarao Kambhampati says “not really” when doing this autonomously and supports his argument with an extensive list of experiments from his team and others. He proposes the LLM-modulo framework where you use external verifiers and solvers, to build on what LLMs can do well, such as guessing plans, guess domain models and elaborate problem specifications. Some other insights worth noting and some warnings -LLMs are expansive system 1 thinking equivalents -having LLMs self verify autonomously is a recipe for compounding errors -if you buy the argument that LLMs can’t plan autonomously, letting them decide plans and execute them by calling external tools/agents is like, quote “leaving a loaded gun in the house with a toddler” -elaborate prompt engineering efforts to get valid plans/reasoning exposes us to the ‘clever Hans’ effect TLDR- Don’t trust, verify and use humans in the loop Link to video in the first comment

8

1 Comment

I was chatting about translation with Andrew Ng at an event several weeks ago. It's fun to see him post a workflow with the seed of the AI Review concept that's live in LILT today. === This is along the path to what many in the industry are doing. Two comments: 1. Replace step 2 with a formalized reflection approach such as AutoMQM. 2. BLEU isn't discriminative at the accuracy levels modern MT systems achieve. Use a trained metrics like BLEURT or COMET to see real differences between systems. 20 years ago we called this idea automatic post-editing, where you'd train a second string transducer to rewrite initial MT output. However, those systems were very difficult to train and cascade. Using LLMs in cascades is significantly more effective.

42

It is a dissapointment to me that relatively smart folk are so imbued with the notion that the “brain is a computer”, such that they can not have an intellectual discussion that questions that assertion! Why? It’s perhaps a deep faith commitment to a sci-fi ideology! It’s like having a dialogue with the deaf, of a scientist talking to a fundamentalist. Indeed AI has become the creator proxy in their faith. Eg., such that the billions of years of evolution by trillions of biological organisms exposed to visceral unmodelled reality is no different in their imagination as a means to produce intelligence, as a modelled digital algorithmic computer”, running a DL ML process on symbolic data/text of virtually a memorized internet! 😝 It beggars my mind that such otherwise smart folk can be so dumb, as to ignore these following three likely facts: 1/ Evolutionary Algorithms (EA) are not Deep Learning Algorithms in their process function. OpenAI are quite well aware it’s a different paradigm. See… https://lnkd.in/eF6r4JCm. It’s an ALTERNATIVE to Deep Learning, not a version of it. 2/ But, EA running on a Digital Computer platform, learning from modeled data (in particular given text/language evolved in humans 20-30k years ago - a sliver of time in evolutionary history) is not the same as the actual Biological Evolution of trillions of life forms over billions of years, exposed as embodied life forms to visceral unmodeled reality. It’s the difference between #shannoninformation vs #goedellianinformation I’ve posted substantially on. 3/ Building a safe EA requires exposing candidate agents (CA) to a Deep Simulation of the reality of the problem domain (a “Digital Twin”). For AGI that’s a full world simulation. You need to model social interactions so that means many CA over many generations. That’s a mammoth task, and it doesn’t resolve point 2 above. What that means is AI via EA can build useful (even super-smart systems in narrow domains, ie., ANI) tools for humanity (I’m for that, it’s an agenda I call #SANITI), but it may never effectively become AGI at human-levels and beyond. As I say making these points with strong AGI acolytes is appallingly similar, to me it’s like having a pointless argument with a religious fundamentalist, that god engineered the world in 7 days, created Adam and from his rib Eve, about 4000 years or so ago and has been engineering our fate by “Intelligent Design “ since then! Some people you just can’t get through to. You have to let them go. Indeed politely suggest they move on/agree to disagree! Even then some won’t, they are on a “evangelical mission“! Like some religious beliefs it is based on a longing to escape the world we are in, to build an heavenly future of eternal bliss, where all problems have solutions gifted to us, and all tears are wiped away, that is the #TESCREAL ideology, that may end in a #molochtrap! cc: Ronald Cicurel 1/ https://lnkd.in/eZwGNb2n 2/ https://lnkd.in/e3fupB99

46

35 Comments

🤔 Hypomania: The Default Mode of Large Language Models (Is hypomania just another manic model?) 🔵 LLMs exhibit cognitive processes similar to human hypomania, with increased creativity and productivity. 🔵 Hypomanic-like behaviors in LLMs pose risks such as impulsivity, over-fitting, and performance issues. 🔵 The hypomanic nature of LLMs may be an inherent feature, requiring a balance between brilliance and stability. #LLMs #AI #AGI #GPT4 #GPT5 #Claude3

9

1 Comment

Love this paper. There are countless low-hanging fruits in deploying LLMs in important fields, including healthcare. I particularly liked this use case. Social Determinants of Health (SDoH) have been close to my heart since my work with RWJF (with Oktawia Wojcik) and Sutter Health (with Sylvia Sudat) to extract them from EHRs many years ago. This is a clear-cut instance where AI is not only effective but better than humans in annotating and surfacing important context for the healthcare provider. #AI #healthcare https://lnkd.in/eGMkjGFQ

22

2 Comments

By combining the generation capabilities of large language models (LLMs) with a retrieval component typically using a vector or semantic search, RAG chatbots can provide informative and personalized responses backed by evidence from a supplied corpus of documents. However, the performance of these models hinges on properly processing and indexing the document corpus for efficient retrieval. If the retrieval is a failure, then the chatbot responses will be incoherent to the users. In this article, we’ll explore the key steps involved in preparing documents for RAG models.

2