5 ms·
Defer to other experts, but (briefly) normalizing flows are a method for constructing complex distributions by transforming a probability density through a seri
by sarosh 2y ago
Defer to other experts, but (briefly) normalizing flows are a method for constructing complex distributions by transforming a probability density through a series of invertible transformations. Normalizing flows are trained using
a plain log-likelihood function, and they are capable of exact density evaluation and efficient sampling. See:
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In ICML, 2015. Link: https://bigdata.duke.edu/wp-content/uploads/2022/08/1505.05770.pdf?x84468&x64183 https://bigdata.duke.edu/wp-content/uploads/2022/08/1505.057...
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation. In ICLR Workshop, 2015. Link: https://arxiv.org/pdf/1410.8516 https://arxiv.org/pdf/1410.8516
And for your direct question, the following paper "Efficient Bayesian Sampling Using Normalizing Flows to Assist Markov Chain Monte Carlo Methods" appears upon a superficial glance to be relevant. Link: https://arxiv.org/pdf/2107.08001 https://arxiv.org/pdf/2107.08001
- JHonaker 2y agoThanks! I've read the first one before. I'll take a look at the other two!
- 1980phipsi 2y agoSo it's like converting a normal distribution to log normal (and then back). But a more general way of thinking about it. Where does the name "normalizing flows" come from?
- hotstickyballs 2y agoIt comes from the Jacobian which you can get from auto diff. It measures how much distortion the function created and normalizes it so that you can integrate correctly without blowing up gradients
- theGnuMe 2y agoI mean the whole thing sounds like a deep neural network…