Self-supervised learning is a training method where the model learns from unlabeled data by generating its own labels, such as predicting the next word in a sentence or filling in missing parts of an input.
A key driver behind the success of modern NLP and computer vision models.
Enables training on internet-scale datasets without the need for expensive human annotation.