Jason Tang’s Post

Interesting article. With the ever increasing size of context windows, massive prompts could potentially outperform fine-tuning for LLMs, based on a study from CMU. That means depending on situations, one can choose to front load their cost to fine-tune LLMs, or to have larger prompts and pay more per transaction to get to similar performance. https://lnkd.in/eUYpK9w2

As we start to experiment with customers interacting with our data via LLMs, there is a high probability that they will be inexperienced with how to write effective prompts and questions.   This paper https://lnkd.in/e9D_sSrx proposes if we use the LLM to regenerate the input context to include only relevant portions of information to dramatically increase the likelihood of a successful/accurate answer. For example: If they ask: "Which laptop has performance issues with Windows 11? I think the answer is Laptop1235 but I’m really not sure." Should be reframed as: "S2A: Context: Which laptop has performance issues with Windows 11? Question: Which laptop has performance issues?"   Furthermore, it suggests that we could take a multi-shot approach (aka regenerate the content and then provide an alternate prompt) to further improve the value customer can achieve with a simple GenAI/Chatbot interface. Clever and worth the read.

Some interesting counter-intuitive properties of longer context windows in LLMs. https://lnkd.in/gKSq7JfU (btw still working on getting my product to mvp - got sick/delayed!)

Some interesting A.I. concepts on how to achieve longer context windows for LLMs >> https://lnkd.in/eADu_yR2

Discover the rapid evolution of context windows in large language models (LLMs) from GPT's initial 512 tokens to the groundbreaking 1M+ tokens in today's advanced models. This article by Krzysztof K. Zdeb in Towards Data Science highlights the exponential growth in LLM capabilities, enabling more nuanced and contextual interactions than ever before. From GPT-3’s 2K tokens to Google’s Gemini and Anthropic's Claude models at 128K-200K tokens, explore how these advancements allow LLMs to handle complex tasks like analyzing entire codebases in a single go. Yet, with great power comes great cost—understand the financial and computational implications of these extended contexts and consider innovative solutions like RAG (Retrieval-Augmented Generation) for more efficient data handling. Dive into the future of AI where the possibilities—and costs—of vast context windows open new avenues for personalized AI interactions. https://lnkd.in/edFSqa35

Using server-side Swift for machine learning processing. Learn how to use a machine learning model on a Vapor server using Swift. https://lnkd.in/gMisC9Bf #ios #iosdevelopment #swift #swiftui #vapor 

https://lnkd.in/gkhWCgXw Some challenges : LLM Tokenizer, Context Windows, Nature of the Training, Static Knowledge Base, Version Specificity

The larger context windows of LLMs are really quite nice because the task of filtering the right context is an art in itself. However, giving it that much context can make it difficult for the LLM to know which parts are the most important. This research confirms other research I've dug into around the problem with these larger context windows: Less is more. https://lnkd.in/d8YiEKGZ

AIOS: Operating System for LLM Agents - Over the past six decades, operating systems have evolved progressively, advancing from basic systems to the complex and interactive operating systems that power today's devices. Initially, operating systems served as a bridge between the binary functionality of computer hardware, such as gate manipulation, and user-level tasks. Over the years, however, they have developed from simple batch job processing systems to more sophisticated process management techniques, including multitasking and time-sharing. These advancements have enabled modern operating systems to manage a wide array of complex tasks. The introduction of graphical user interfaces (GUIs) like Windows and MacOS has made modern operating systems […] - https://lnkd.in/gv7PJFQ5

Awesomeness from Marcelo Gennari do Nascimento and others on SliceGPT! Techniques like these and other optimizations is how we are continuing to bring AI-powered goodness to Windows. https://lnkd.in/g47r4ZGH