I've been experimenting with a variety of fine-tuned large language models for writing fiction interactively. LoRA seems to be a very successful technique for capturing tone and turn of phrase, but I keep running into context limits even with models with fairly large windows like Mistral. I made a library for ChatGPT that deals with this by automatically asking the model to summarize the chat history into a dense recap that preserves key information about the chat history while discarding the verbatim chat history to reduce context size. It seems to work pretty well. I'm going to try to adapt this for open source models as well. Has anyone else tried a technique like this? It seems to map to human experience - we don't remember every word of a long conversation, but we do remember key points and unique turns of phrase. For anyone who is interested, here's the demo code: https://lnkd.in/gBtZUbzM
Dan Ratner’s Post
More Relevant Posts
-
Interesting finding of the day...I ran content classification requests on 15,000 pieces of content where I had reliable ground truth (multiple concurring human labels). There were three distinct tests to identify if the content was or was not part of a specified topic. On average, GPT-3.5-Turbo outperformed GPT-4 significantly across the test set. Latency was comparable. Cost is not. While it's still not enough to generalize, it certainly seems like you should save yourself $$$ and use 3.5-Turbo if you are doing classification. Now to try these same tests on Anthropic...
-
Some weekend thoughts about how to write tests for AI projects. It's tricky testing non-deterministic tools... https://lnkd.in/gDA9AQND
Shadowlands Update
danratner.com
-
I made an AI-based Halloween-themed game. Check it out and let me know what you think! #AI #halloween
An AI Halloween Present
Dan Ratner on LinkedIn
-
Your Canada correspondent's thoughts about Canada's crucial Online News Act
The Urgency of Canada's Online News Act (C18)
Dan Ratner on LinkedIn
-
Since I've been working on AI for a long time, I get a lot of questions from investors about where I think the money is. While it's everywhere at the moment, here are some thoughts on who I think will win.
How to Invest in AI
Dan Ratner on LinkedIn
-
With the discussion of regulating AI on Capitol Hill today, I've written a partial list of the questions I wish they'd talk about...
AI Regulation
Dan Ratner on LinkedIn
-
I've been meaning to write some thoughts on the rapid evolution of AI and finally got pen to proverbial paper. Hoping to write more on this subject soon.
The Near/Long Future of AI
Dan Ratner on LinkedIn
-
Dan Ratner reposted this
Lina Khan has taken the gloves off. An independent auditor found that Meta failed to meet the privacy program requirements outlined in their settlement with the FTC. The FTC is now asking a court to approve modifications, which include a ban on monetizing minors' data, and a pause on all new products and services until they can prove their compliance. The craziest part: Meta has spent $5 Billion on their privacy program since 2019. GoodRx and BetterHelp both recently signed settlements with the FTC that included similar privacy program mandates. It's time to make those settlements a top priority. #compliance #FTC #privacy #digitalhealth
FTC Proposes Blanket Prohibition Preventing Facebook from Monetizing Youth Data
ftc.gov
