Autoregressive Flows

• 
• use them as normalizing flows
• a model that sequentially samples each pixel on an image, wrt the previous pixel in a fixed order
• consider the conditional densities as gaussians
  • From MADE - Masked autoencoder for distribution estimation and Masked autoregressive flow for density estimation,
    • the sampling noise at each step can be used as a latent variable
    • $p_{\theta }(x_1,x_2,\dots ,x_n)={\Pi }_i^n{p}_{\theta }(x_i|x_{<i})$
    • ${\mu }_i=f_{\mu }(x_{<i})$ and $log{\sigma }_i=f_{\sigma }(x_{<i})$
    • the latent factor = $z_i\sim N(0,1)$
    • observed → $x_i={\mu }_i+z_i\exp (\log {\sigma }_i)$
  • Since these are sequentially sampled, we can’t parallelize them. Instead from Improved variational inference with inverse autoregressive flows we can use inverse autoregressive flows