Wenqi Glantz

Excellent report from General Dynamics Information Technology John P. Curran regarding adoption and trends of AI in the Govt. I used Meta to generate summary below and included link to original article. Main AI use cases are chat and creating/updating computer code. Note also the 23% failure rate of AI projects due to user expectation misalignment. This can be overcome with thorough protoyping and establishing use case metrics. - Successful AI project completion timelines are narrowing, with an average of 14 months from pilot to production. - Future demand for AI projects is building around chat and coding-related functions. - 54% of surveyed AI experts said projects currently under development involve improving computer code. - 59% of projects involve "research and understanding", which may relate to creating chat functions. - 23% of AI projects fail due to misalignment between AI solutions and user expectations. - Partnership efforts are required for successful AI projects. Data diversity is crucial, and incorporating diverse data sources can lead to better AI outcomes. Trends: - AI adoption is increasing across the Federal government, with a growing focus on chat and coding-related functions. - Agencies are leveraging cloud-native services, tooling, and other capabilities to accelerate AI project development. - There is a growing interest in translating older code to newer languages, such as Python or cloud-based analytic tools. - Domain-specific chat functionality is becoming a priority, with many agencies planning to implement chat services in the next two years. Statistics: - 325 AI experts surveyed across the Federal government - 14 months: average time from pilot to production for AI projects - 54%: percentage of projects currently under development that involve improving computer code - 59%: percentage of projects that involve "research and understanding" - 23%: percentage of AI projects that fail due to misalignment between AI solutions and user expectations. Bill Casola John Grim Leslie Anne McIndoe Kim Giffin Michael S. Freeman III Tobias Quaranta https://lnkd.in/eqvmAVFG

4

Totally agree with you, Anthony. The name of the game is to get the right context as tight as possible to the question or task on hand. Graphs are one good way to do this, but you should use all the tools in your belt. With our solution, we use graphs, tables, key value pairs, geospatial information, regular search, vectors, and so on. When you combine all these things, the probability of getting the right context as much higher.

5

That pesky ECDSA signature refuses to go away, and we are struggling with it in a distributed world of trust. Luckily EdDSA is much more scalable, and now used in many distributed signature applications (and where we can split the private key up into secret shares and distribute them over several nodes). But we have a problem ... elliptic curve methods are going to be cracked by quantum computers. So can we take our PQC signature methods and scale them to distributed signatures? Well, no, as Dilithium doesn't support it. But Raccoon can, and also has better side channel resistance (and with a similar performance and key size to Dilithium. So, meet Raccoon: https://lnkd.in/d2QMHSM9

30

2 Comments

Thanks for sharing Baris Yuksel this great site on machine learning trends. https://lnkd.in/d3DTuRBp Next generation AI Systems will need to innovate across different domains at once: Systems - How do we build and run these large systems (cost to train, cost to run) Algorithms - How do we change the architecture such that we can get much better scaling laws.

9

Metrics for LLM. Nice article explaining the Grounding, hallucination, data correctness for fine-tuning, Prompt correctness, and RAG effectiveness by computing relevant Context awareness. https://lnkd.in/gFQMQ_ZR

32

27,500 new asteroids discovered! This was achieved by mining historical data sets rather than new observations. The open source Asteroid Discovery Analysis and Mapping (ADAM) platform used Kubernetes on Google Cloud and consumed 8.5 million CPU-hours of compute. Data analysis was done using BigQuery, The moral of the story is make sure you've looked twice at your data before the near-Earth asteroid impacts us! #Cloud #GoogleCloud #stories #news #data #ai #bigdata #tech #science

10

An apt representation …and that’s true for any technology innovation. However risks are much higher and its impact is much broader in case of ai #aiforgood #artificialintelligence #ai

6

Very insightful work by Anthropic’s interpretability team. And an amazing paper, with outstanding writing and figures. The idea is very simple: interpret LLMs by leveraging sparse autoencoders as surrogate models of the MLP of transformer blocks, which allow one to disambiguate the superposition of features captured by a single neuron. A simple idea, but a very careful and complex execution, as it is often the case in our line of work. The paper goes into many details and provide a large array of insights, although the gist of the implementation remains obfuscated due to the closed source nature of Claude. Too bad, because this is the kind of work that we need to better understand and eventually trust LLMs. This is demonstrated by the authors in the section ‘Influence on Behavior’, where they show that clamping some features to either high or low value during inference is “remarkably effective at modifying model outputs in specific, interpretable ways”. Hopefully this kind of work is going to be replicated and generalized to open-weights models, such that we have new ways to steer their behavior. https://lnkd.in/eVym7f_f #interpretability #xai #explainableai #steerableai #anthropic #claude #anthropic

3

😊Want to jazz up your terminal? TTE has you covered! What is TTE? TerminalTextEffects (TTE) is a terminal visual effects engine. TTE can be installed as a system application to produce effects in your terminal, or as a Python library to enable effects within your Python ⏬⏬⏬⏬⏬⏬⏬ https://lnkd.in/gcMnZb6v #cli

1

1 Comment

Interesting paper; if the interaction between layers is linear, there are possibly new ways of optimizing them. We have just started with these.

3

Sage have created a very nice interactive overview of the median prediction of forecasters on Metaculus and Manifold Markets on various questions related to AI capabilities, harms, and societal impacts. It's well worth checking out if you are interested in this sort of stuff. See: https://lnkd.in/eUgkmzX9 Follow MIT FutureTech if you are interested in technological trends and their social implications. #AI #technology #economics

11

📊🛍️ Enhancing Retail Product Recommendations with Trillion-Edge Graphs through Hierarchical Agglomerative Clustering 🤖💡 Google Research has unveiled exciting advancements in expanding Hierarchical Agglomerative Clustering (HAC) to manage vast trillion-edge graphs! 🌐🚀 This advanced method enables retailers to transform their recommendation engines, achieving unprecedented levels of personalization. 🎯 The retail challenge is to recommend products from extensive catalogs to a diverse customer base efficiently and in real time. 🌐⏰ While traditional techniques falter under scalability demands, combining HAC with large language model (LLM) pre-training presents a robust solution. 💡 As detailed in the Google Research blog post [1], the new TeraHAC algorithm uses a MapReduce-style framework to break down graphs into manageable subgraphs and executes merging based on localized data. 🧩 This process allows HAC to handle trillion-edge graphs, creating new opportunities for retailers. 📈 Pre-training LLMs with vast amounts of product data captures deep semantic insights, identifying crucial characteristics and interrelationships. 🧠 These embeddings are then utilized by HAC to cluster effectively based on semantic similarities. 🎛️ The synergy of TeraHAC and LLM pre-training enables retailers to offer highly personalized and accurate product recommendations swiftly, even when dealing with billions of items. ⚡💯 This enhances customer engagement, satisfaction, and loyalty, elevating them to unprecedented levels. 📈🌟 Leverage these innovative technologies to maximize your product catalog's potential and stay competitive in the retail market. 🏆 #RetailInnovation #PersonalizedRecommendations #MachineLearning #CustomerExperience [1] https://lnkd.in/edQ_EJA7 #gonnect #llm #kg

10

With constant LLM's compute performance and cost concerns, here is the phi-3 https://lnkd.in/eZRn56cP  Phi-3-mini can not only run on traditional computing devices, but also on edge devices such as mobile devices and IoT devices.

2

In the last two years, we have seen the emergence of Large Language Models(LLMs), specifically in enterprises, there is a trend of either building or leveraging small open source language models to handle targeted use-cases with necessary optimization both in terms of system resource usage & accuracy. We are working on some exciting stuff here & our technical disclosure on ‘multi-stage fine-tuning process for optimizing small LLMs in RAG applications’ just got published! Happy reading!!! #SLMs #GenAI https://lnkd.in/gBvHZvHt  

18

6 Comments

"Smaller is better." There are now Small Language Models (SLMs) that are “smaller” in size compared to LLMs. SLMs are trained on 10s of billions of parameters, while LLMs are trained on 100s of billions of parameters. More importantly, SLMs are trained on data pertaining to a specific domain. They might not have broad contextual information, but they perform very well in their chosen domain -- write Iyengar Ashok and Praneet Adusumilli in this IBM blog. One area that has not seen much innovation is at the far edge and on constrained devices. However, there are smaller models that have the potential to innovate gen AI capabilities on mobile devices. Enterprises that operate in specialized domains, like telcos or healthcare or oil and gas companies, have a laser focus. While they can and do benefit from typical gen AI scenarios and use cases, they would be better served with smaller models. Because of their smaller size, these models can be hosted in an enterprise’s data center instead of the cloud. SLMs might even run on a single GPU chip at scale, saving thousands of dollars in annual computing costs. While running SLMs on-premises seems practical and tiny LLMs on mobile edge devices are enticing, what if the model requires a larger corpus of data to respond to some prompts? Hybrid cloud computing offers the best of both worlds. Might the same be applied to AI models? Click below to read the article and find out! #LLM #SLM #LargeLanguageModel #SmallLanguageModel   https://lnkd.in/gFzyHhfH

13

1 Comment

DeepMind Researchers Propose Naturalized Execution Tuning (NExT): A Self-Training Machine Learning Method that Drastically Improves the LLM's Ability to Reason about Code Execution https://lnkd.in/gg9pvjpE

3

Why Low-rank Adaptation(#LoRA) of #LLMs seems more pragmatic for #enterpriseai, some early thoughts (pls share your views) First, how does it work? By freezing the #pretrained model weights and injecting trainable rank decomposition matrices into each layer of the #transformerarchitecture, this reduces the number of trainable parameters for downstream tasks. So, why is this better? Efficiency: Low rank models require fewer parameters and computations compared to their full-rank counterparts ((say for #GPT3, LoRA can reduce trainable params by 10000 times and computation reqs by almost 3 times) . This efficiency leads to faster inference times and reduced computational costs, making them more suitable for large-scale deployments in enterprise settings. Scalability: #Enterpriseapplications often deal with massive datasets and require models that can scale effectively. #Lowrankmodels can handle large volumes of data more efficiently, making them a practical choice for enterprise-scale applications. Memory and storage: With reduced parameter sizes, low rank models consume less memory and storage. This is advantageous for deploying models on resource-constrained devices or when managing a large number of models simultaneously. #Generalization: Low rank models tend to generalize better on unseen data due to their ability to capture the essential patterns in the data while avoiding overfitting. This is crucial in enterprise applications where the model's performance on new data is of paramount importance. #TransferLearning and #Finetuning: Low rank models can serve as a good starting point for transfer learning and fine-tuning on specific enterprise tasks. Their compact representations enable faster adaptation to new domains or tasks with limited labeled data. Privacy and security: In some enterprise settings, there might be concerns about sharing sensitive data with third-party cloud providers. Low rank models can be more easily adapted and deployed locally, reducing the need to send raw data to external servers. Most of all, Cost-effectiveness: Low rank models can offer competitive performance at a fraction of the cost of more complex models. This cost-effectiveness is an attractive feature for enterprises looking to optimize their resources and budgets. These apart from Robustness to Noise, Ease of Interpretability (simpler structure) makes it all the more a good candidate for many firms to experiment for fitment, what do you think? #saxonai #generativeai #languagemodels #qlora #finetuning #llms

7

1 Comment