From the course: Hands-On AI: Build a Generative Language Model from Scratch
Join today to access over 23,200 courses taught by industry experts.
Markov chains and Python - Python Tutorial
From the course: Hands-On AI: Build a Generative Language Model from Scratch
Markov chains and Python
- [Instructor] Large language models are able to look at multiple tokens as their input and produce pretty remarkable results. Our model is only going to concern itself with one word, that is the last word that it's on. If it receives the input, "I try to learn something new," our model is only going to concern itself with the word or token, new. During its training, our model is going to obtain this chart that maps words to possibilities of the next words. So if we have the word, new, it's going to hold those raffle possibilities every, every, and each. So if we run that raffle, we have a two-third chance of having every and a one-third chance of having each. If the word that comes up is every, then the chart is going to equip us with the possibilities day, day, and single. And then we have a two-third chance of day coming up and a one-third chance of single. Now, in order to represent this chart in code,…
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.