Meta announces data2vec
Big update from Meta on an advancement to a general purpose learning algorithm:
Meta AI is excited to announce data2vec, the first high-performance self-supervised algorithm that works for multiple modalities. We apply data2vec separately to speech, images and text and it outperformed the previous best single-purpose algorithms for computer vision and speech and it is competitive on NLP tasks.
In a previous post from March of last year, then Facebook AI asserted:
We believe that self-supervised learning (SSL) is one of the most promising ways to build such background knowledge and approximate a form of common sense in AI systems.
This feels like a big leap going from an idea of how to generalize self-supervised learning to releasing working models, in less than a year.
We’re encouraged by the progress of self-supervision in recent years, though there’s still a long way to go until this method can help us uncover the dark matter of AI intelligence. Self-supervision is one step on the path to human-level intelligence, but there are surely many steps that lie behind this one. Long-term progress will be cumulative. That’s why we’re committed to working collaboratively with the broader AI community to achieve our goal of, one day, building machines with human-level intelligence.
Cumulative, but with big leaps along the way.