From the course: Hands-On AI: Build a Generative Language Model from Scratch

What is a language model? - Python Tutorial

From the course: Hands-On AI: Build a Generative Language Model from Scratch

Start my 1-month free trial Buy for my team

What is a language model?

“

- In this chapter we're going to create a language model similar to the one that helps you pick the next word on your phone's keyboard. Our language model is going to take a prompt and return a completion, to explain how such a model would work. My family and I got a little crafty and made this box with this prompt. "I try to learn something new every" Notice that's not a complete sentence. And our language model is going to help us come up with the next word by running a raffle. And if you have seen a lot of English text you may know that a very likely word to come up next would be the word day. So let's add the word day once and twice and three times. There's also a small likelihood of the word single coming up, as in "I try to learn something new every single day." So let's add single once. So now there's a three out of four chance of the word day coming up and a one out of four chance of the word single coming up. So let's run our raffle. I try to learn something new every day so now we have our completion. I try to learn something new every day. So perhaps the magic of a language model is its ability to come up with these raffles over and over again. So put on your coder hat because it's time for text completion from scratch.

Contents