toc: true title: NCE
tags: [‘temp’]
NCE
- Conditional Negative Sampling for Contrastive Learning of Visual Representations
- Contrastive Learning
- noise-contrastive estimation
- bound on mutual information between two views of an image
- randomly sampled negative examples to normalize the objective
- choosing difficult negatives, or those more similar to the current instance, can yield stronger representation
- Conditional Noise Contrastive Estimator
- sample negatives conditionally
- in a “ring” around each positive, by approximating the partition function using samples from a class of conditional Distributions
- hese estimators lower-bound mutual information
- higher bias but lower variance than NCE Bias Vs Variance
- Applying these estimators as objectives in contrastive representation learning
- transferring Features to a variety of new image Distributions from the meta-dataset collection
- Contrastive Loss