Contrastive Predictive Coding

• Representation Learning with Contrastive Predictive Coding
• Contrastive Predictive Coding
• framework for extracting compact latent representations to encode predictions over future observations
• learn such representations by predicting the future in latent space by using powerful Autoregressive models
• probabilistic Contrastive Loss based on NCE, which both the encoder and Autoregressive model are trained to jointly optimize, which they call InfoNCE
• InfoNCE induces the latent space to capture information that is maximally useful to predict future samples
• combines Autoregressive modeling and noise-contrastive estimation with intuitions from predictive coding to learn abstract representations in an unsupervised fashion
• negative sampling