benanne

33 ms·

1.
by benanne 17d ago
Glad to hear it! My stubbornness about this makes me feel like a luddite sometimes, but the additional effort required is probably still worth it, for the time being.
2.
by benanne 17d ago
I suppose what I was trying to say is that in the research community, people were still a lot more willing to entertain alternative modelling paradigms for language at that point, there was much less of a monoculture than there is today. It
3.
Continuous Diffusion Language Models(sander.ai)
7 points by benanne 24d ago|0 comments
4.
by benanne 4mo ago
Thanks for pointing this out! I'm not sure why, the files are still on my Dropbox, they must have changed the link format at some point? I've gone ahead and fixed them.
5.
by benanne 4mo ago
It kind of does! In the modern era of generative modelling, it seems like we rely on pre-training to capture the data distribution, and then on post-training (and various other tricks) to carve out a sliver of that distribution that we actu
6.
by benanne 4mo ago
I briefly covered that connection in an earlier blog post: https://sander.ai/2023/07/20/perspectives.html#flow ... but it's definitely something that might deserve a longer-form treatment at some point :
7.
Learning the Integral of a Diffusion Model(sander.ai)
161 points by benanne 4mo ago|23 comments
8.
by benanne 2y ago
I actually wrote down some thoughts about audio phase in a previous blog post: https://sander.ai/2020/03/24/audio-generation.html#motivatio... I have an example audio clip in there where the phase information
9.
by benanne 2y ago
Sorry to hear that. My blog posts are intended to build intuition. I also write academic papers, which of course involves a different standard of rigour. Perhaps you'd prefer those, only one of those is about diffusion models though.
10.
by benanne 2y ago
I'm not sure if frequency decomposition makes sense for anything that's not grid-structured, but there is certainly evidence that there is positive "transfer" between generative modelling tasks in vastly different domain
11.
by benanne 2y ago
Oof, you're not going to like this other blog post I wrote then :D https://sander.ai/2023/07/20/perspectives.html
12.
by benanne 2y ago
Thanks for reading! Absolutely, I included a few references that explore that approach at the bottom of section 4 (last two paragraphs).
13.
by benanne 2y ago
Thanks for reading! The paper that directly inspired this blog post actually investigates the latter (blurring as the corruption process): https://arxiv.org/abs/2206.13397
14.
by benanne 2y ago
Thanks for reading! Check out subspace diffusion: https://arxiv.org/abs/2205.01490
15.
Diffusion Is Spectral Autoregression(sander.ai)
2 points by benanne 2y ago|0 comments
16.
by benanne 2y ago
I've since moved on to work primarily on diffusion models, so I have a series of blog posts about that topic as well! - https://sander.ai/2022/01/31/diffusion.html is about the link between diffusion mod
17.
Generating Music in the Waveform Domain(benanne.github.io)
2 points by benanne 6y ago|0 comments
18.
Piano Genie: An Intelligent Musical Interface(magenta.tensorflow.org)
3 points by benanne 8y ago|2 comments
19.
by benanne 10y ago
In Georgian, mother is "deda" and father is "mama", which lends further evidence to this :)
20.
by benanne 10y ago
A nice property of the model is that it is easy to compute exact log-likelihoods for both training data and unseen data, so one can actually measure the degree of overfitting (which is not true for many other types of generative models). An
21.
WaveNet: A Generative Model for Raw Audio(deepmind.com)
627 points by benanne 10y ago|145 comments
22.
Diagnosing Heart Diseases with Deep Neural Networks(irakorshunova.github.io)
2 points by benanne 11y ago|0 comments
23.
by benanne 11y ago
Agreed. Theano has trouble dealing efficiently with very deeply nested graphs.
24.
by benanne 11y ago
Why wouldn't this work in Theano? >>> import theano >>> import theano.tensor as T >>> state = theano.shared(1.0) >>> states = [] >>> for step in range(10): >>
25.
by benanne 11y ago
Nobody forced us to open-source Lasagne, so I think that remark was a bit unfair. If we really didn't care about anything but graduating, why would we bother going through the trouble of sharing the code in the first place? But I do se
26.
by benanne 11y ago
I have some doubts about this. Deep learning moves fast and DBNs are pretty much outdated models, even for unsupervised pre-training. It doesn't make much sense to me that unsupervised pre-training would help for this problem to begin
27.
by benanne 11y ago
I feel like this guide comes about 5 years too late - RBMs as density models have been shown to be relatively weak, except in the case of binary data. For continuous data, you can often do better even with a simple Gaussian mixture model. O
28.
by benanne 11y ago
Not to my knowledge. For speech it should be sort of doable to get enough data to train a large net and then do inceptionism-style stuff with it. For music, gathering enough training data is a huge challenge due to licensing issues, among o
29.
by benanne 11y ago
That has been there since the start :) it randomly picks from the chat suggestions every minute and acknowledges the one it picked. I'm one of Jonas's co authors, feel free to ask me anything as well!
30.
by benanne 11y ago
Not yet, I start in about a week! :)

More ›