Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning

Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other.From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods intrinsically rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches 74.3% top-1 classification accuracy on ImageNet using the standard linear evaluation protocol with a standard ResNet-50 architecture and 79.6% with a larger ResNet. We also show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks.

引导自己的潜能-自我监督学习的新方法

我们介绍了Bootstrap Your Own Latent（BYOL），这是一种用于自我监督的图像表示学习的新方法。BYOL依赖于两个相互交互并相互学习的神经网络，称为在线和目标网络。.. 从图像的增强视图中，我们训练在线网络以预测不同增强视图下同一图像的目标网络表示。同时，我们以平均水平的在线网络更新目标网络。虽然最先进的方法本质上依赖于否定对，但BYOL在没有否定对的情况下实现了新的技术水平。使用具有标准ResNet-50体系结构的标准线性评估协议，BYOL在ImageNet上可达到74.3％的top-1分类准确性，而在具有较大ResNet的情况下，BYOL可以达到79.6％。我们还表明，BYOL在传输基准和半监督基准上的表现均与当前技术水平相当或更好。 （阅读更多）