From the course: Applied AI: Getting Started with Hugging Face Transformers
Join today to access over 23,200 courses taught by industry experts.
Positional encoding
From the course: Applied AI: Getting Started with Hugging Face Transformers
Positional encoding
- [Instructor] Positional encoding is an important feature that is proposed in the transformer architecture. Positional encoding is the process of deriving a vector for each token in a sentence to represent its position in the sentence. The same token in different positions in a sentence will have different positional encoding vectors. These vectors help transformers to provide positional and sequencing context for machine learning. Recurrent neural networks or RNNs, capture positional information by processing one token at a time and using the previous hidden states in their computations, so they do not need explicit positional input. On the other hand, transformers process each token in parallel, hence, it needs additional input about the position of a token in the sentence. The positional encoding vector is of the same dimension as the embedding vector that would be used for the model. For example, let's consider the…
Practice while you learn with exercise files
Download the files the instructor uses to teach the course. Follow along and learn by watching, listening and practicing.
Download courses and learn on the go
Watch courses on your mobile device without an internet connection. Download courses using your iOS or Android LinkedIn Learning app.