Dan Ratner’s Post

I've been experimenting with a variety of fine-tuned large language models for writing fiction interactively. LoRA seems to be a very successful technique for capturing tone and turn of phrase, but I keep running into context limits even with models with fairly large windows like Mistral. I made a library for ChatGPT that deals with this by automatically asking the model to summarize the chat history into a dense recap that preserves key information about the chat history while discarding the verbatim chat history to reduce context size. It seems to work pretty well. I'm going to try to adapt this for open source models as well. Has anyone else tried a technique like this? It seems to map to human experience - we don't remember every word of a long conversation, but we do remember key points and unique turns of phrase. For anyone who is interested, here's the demo code: https://lnkd.in/gBtZUbzM

Interesting finding of the day...I ran content classification requests on 15,000 pieces of content where I had reliable ground truth (multiple concurring human labels). There were three distinct tests to identify if the content was or was not part of a specified topic. On average, GPT-3.5-Turbo outperformed GPT-4 significantly across the test set. Latency was comparable. Cost is not. While it's still not enough to generalize, it certainly seems like you should save yourself $$$ and use 3.5-Turbo if you are doing classification. Now to try these same tests on Anthropic...

I made an AI-based Halloween-themed game. Check it out and let me know what you think! #AI #halloween

Your Canada correspondent's thoughts about Canada's crucial Online News Act

Since I've been working on AI for a long time, I get a lot of questions from investors about where I think the money is. While it's everywhere at the moment, here are some thoughts on who I think will win.

With the discussion of regulating AI on Capitol Hill today, I've written a partial list of the questions I wish they'd talk about...

I've been meaning to write some thoughts on the rapid evolution of AI and finally got pen to proverbial paper. Hoping to write more on this subject soon.