From the course: Applied AI: Getting Started with Hugging Face Transformers
Join today to access over 23,200 courses taught by industry experts.
Pretrained Transformer models
From the course: Applied AI: Getting Started with Hugging Face Transformers
Pretrained Transformer models
- [Instructor] Pre-trained transformer models are becoming popular these days, as they make building use cases quicker and easier. Pre-trained transformer models are general purpose models built based on the transformer architecture. They're also called foundational models and language models. They're built using a large corpora of data and shared by the community. They can be used to solve a wide variety of NLP tasks. What is a pre-trained transformer model? Pre-trained models have a specific architecture along with a trained set of parameters and hyper-parameters. When you download a pre-trained model, it comes with its model architecture definition, as well as its parameters like weights. Pre-trained transformers are usually created for tasks like masked language modeling, and next sentence prediction. Masked language modeling is done by randomly hiding a given percent of tokens from a sentence, and building a model to…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.