5 ms·
Consistency models is a special case of IMM where you do moment matching with 1 sample from each distribution (i.e., you cannot match distributions properly). S
by bearobear 2y ago
Consistency models is a special case of IMM where you do moment matching with 1 sample from each distribution (i.e., you cannot match distributions properly). See Fig 5 for an ablation study, of course, adding more samples when you are doing moment matching makes it more stable during training :)
- throwaway2562 2y agoI’m trying to understand what the ‘spectral’ interpretation of IMM is: but perhaps I shouldn’t https://sander.ai/2024/09/02/spectral-autoregression.html https://sander.ai/2024/09/02/spectral-autoregression.html
- bbminner 2y agoMakes sense. How can you even approximately estimate higher order differences in conditional moments in such a high dim space? Seems statistically impossible to get a reasonable estimate for a gradient. Moment matching in sample space has always been very hard.