7 ms·

I really want to be impressed, but I've been reading papers about breakthroughs in deblurring and upscaling for two decades now, and the state of the art in com

I really want to be impressed, but I've been reading papers about breakthroughs in deblurring and upscaling for two decades now, and the state of the art in commercial and open-source tools is still pretty underwhelming. Chances are, if you have a low-res keepsake photo, or take a blurry nature shot, you're gonna be stuck with that. Video, where the result needs to be temporally coherent and make sense in 3D, can't be the easier one.

tomp 2y ago
this work won't solve that. it requires a video (sequence of images)
IshKebab 2y ago
> Video, where the result needs to be temporally coherent and make sense in 3D, can't be the easier one. Why not? Video is a much more tractable problem because you have much more information to go on.
contravariant 2y ago
At this stage there's really only a couple of options. More than there used to be, but still. When you want to stay faithful to the actual data then your options are limited, for quite a large part of the image a simple convolution is about as good as it gets, except for the edges. Basically the only problem we couldn't solve 20 years ago was excessive ringing (which is why softer scaling algorithms were preferred). You can put quite a lot of effort into getting clearer edges, especially for thin lines, but for most content you can't expect too much more sharpness than what the basic methods get you. And then there is the generative approach where you just make stuff up. It's quite effective but a bit iffy. It's fine for entertainment but it's debatable if the result is actually a true rescale of the image (and if you average over the distribution of possible images the result is too soft again). In theory video can do better by merging several frames of the same content.
echelon 2y ago
If your brain can imagine it de-blurred or de-scuffed, we'll get there with techniques eventually.
deleted 2y ago
[deleted]
smoothbran 2y ago
Video is absolutely the easier case - there's a lot more information to go on. A single blurry photo has lost information compared to the original, but you can theoretically recover that information in a video where you get to see the subject with a variety of different blurs/distortions applied. Note that a limitation of this result is that it assumes a static scene, but that's already a typical limitation of most gaussian splat applications anyway, so it kind of doesn't matter?
phh 2y ago
Hi, I'm trying to make an opensource NN camera pipeline (objective is to be able to run on smartphones but offline, not real-time), and I'm still barely managing the demosaicing part... would you be open to discussing with me?

tomp 2y ago

this work won't solve that. it requires a video (sequence of images)

IshKebab 2y ago

> Video, where the result needs to be temporally coherent and make sense in 3D, can't be the easier one. Why not? Video is a much more tractable problem because you have much more information to go on.

contravariant 2y ago

At this stage there's really only a couple of options. More than there used to be, but still. When you want to stay faithful to the actual data then your options are limited, for quite a large part of the image a simple convolution is about as good as it gets, except for the edges. Basically the only problem we couldn't solve 20 years ago was excessive ringing (which is why softer scaling algorithms were preferred). You can put quite a lot of effort into getting clearer edges, especially for thin lines, but for most content you can't expect too much more sharpness than what the basic methods get you. And then there is the generative approach where you just make stuff up. It's quite effective but a bit iffy. It's fine for entertainment but it's debatable if the result is actually a true rescale of the image (and if you average over the distribution of possible images the result is too soft again). In theory video can do better by merging several frames of the same content.

echelon 2y ago

If your brain can imagine it de-blurred or de-scuffed, we'll get there with techniques eventually.

deleted 2y ago

[deleted]

smoothbran 2y ago

Video is absolutely the easier case - there's a lot more information to go on. A single blurry photo has lost information compared to the original, but you can theoretically recover that information in a video where you get to see the subject with a variety of different blurs/distortions applied. Note that a limitation of this result is that it assumes a static scene, but that's already a typical limitation of most gaussian splat applications anyway, so it kind of doesn't matter?

phh 2y ago

Hi, I'm trying to make an opensource NN camera pipeline (objective is to be able to run on smartphones but offline, not real-time), and I'm still barely managing the demosaicing part... would you be open to discussing with me?