5 ms·

Anyone know of open models useful (and good quality) for going the other way? I.e., Input is a 800x600 jpg and output is 4k version.

by jfdi 3y ago

cuuupid 3y ago
There are a bunch of great upscaler models although they tend to hallucinate a bit, I personally use magic-image-refiner: https://replicate.com/collections/super-resolution https://replicate.com/collections/super-resolution
lsb 3y ago
You’re looking for what’s called upscaling, like with Stable Diffusion: https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler https://huggingface.co/stabilityai/stable-diffusion-x4-upsca...
hansvm 3y ago
I haven't explored the current SOTA recently, but super-resolution has been pretty good for a lot of tasks for few years at least. Probably just start with hugging-face [0] and try a few out, especially diffusion-based models. [0] https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/upscale https://huggingface.co/docs/diffusers/api/pipelines/stable_d...
godelski 3y ago
Look for SuperResolution. These models will typically come as a GAN, Normalizing Flow (or Score, NODE), or more recently Diffusion (or SNODE) (or some combination!). The one you want will depend on your computational resources, how lossy you are willing to be, and your image domain (if you're unwilling to tune). Real time (>60fps) is typically going to be a GAN or flow. Make sure to test the models before you deploy. Nothing will be lossless doing superresolution but flows can get you lossless in compression.
- sitkack 3y ago
  Or else you get Ryan Gosling https://news.ycombinator.com/item?id=24196650 https://news.ycombinator.com/item?id=24196650
davidbarker 3y ago
Magnific.ai (https://magnific.ai https://magnific.ai) is a paid tool that works well, but it is expensive. However, this weekend someone released an open-source version which has a similar output. (https://replicate.com/philipp1337x/clarity-upscaler https://replicate.com/philipp1337x/clarity-upscaler) I'd recommend trying it. It takes a few tries to get the correct input parameters, and I've noticed anything approaching 4× scale tends to add unwanted hallucinations. For example, I had a picture of a bear I made with Midjourney. At a scale of 2×, it looked great. At a scale of 4×, it adds bear faces into the fur. It also tends to turn human faces into completely different people if they start too small. When it works, though, it really works. The detail it adds can be incredibly realistic. Example bear images: 1. The original from Midjourney: https://i.imgur.com/HNlofCw.jpeg https://i.imgur.com/HNlofCw.jpeg 2. Upscaled 2×: https://i.imgur.com/wvcG6j3.jpeg https://i.imgur.com/wvcG6j3.jpeg 3. Upscaled 4×: https://i.imgur.com/Et9Gfgj.jpeg https://i.imgur.com/Et9Gfgj.jpeg ---------- The same person also released a lower-level version with more parameters to tinker with. (https://replicate.com/philipp1337x/multidiffusion-upscaler https://replicate.com/philipp1337x/multidiffusion-upscaler)
- quaintdev 3y ago
  Here's free and open source alternative that works pretty well https://www.upscayl.org/ https://www.upscayl.org/
- aspyct 3y ago
  That magnific.ai thingy is taking a lot of liberty on the images, and denaturing it. Their example with the cake is the most obvious. To me, the original image shows a delicious cake, and the modified one shows a cake that I would rather not eat...
  - hug 3y ago
    Every single one of their before & after photos looks worse in the after. The cartoons & illustrations lose all of their gradations in feeling & tone with every outline a harsh edge. The landscapes lose any sense of lushness and atmosphere, instead taking a high-clarity HDR look. Faces have blemishes inserted the original actor never had. Fruit is replaced with wax imitation. As an artist, I would never run any of my art through anything like this.
jfdi 3y ago
thank you! will enjoy reviewing each of these
codercowmoo 3y ago
Current SOTA open source is I believe SUPIR (Example - https://replicate.com/p/okgiybdbnlcpu23suvqq6lufze https://replicate.com/p/okgiybdbnlcpu23suvqq6lufze), but it needs a lot of VRAM, or you can run it through replicate, or here's the repo (https://github.com/Fanghua-Yu/SUPIR https://github.com/Fanghua-Yu/SUPIR)
physPop 2y ago
This is called super resolution (SR). 2x SR is pretty safe and easy (so every pixel in becomes 2x2 out, in your example 800x600->1600x1200). Higher scalings are a lot harder and prone to hallucination, weird texturing, etc.

Anyone know of open models useful (and good quality) for going the other way? I.e., Input is a 800x600 jpg and output is 4k version.

by jfdi 3y ago

Anyone know of open models useful (and good quality) for going the other way? I.e., Input is a 800x600 jpg and output is 4k version.

cuuupid 3y ago

There are a bunch of great upscaler models although they tend to hallucinate a bit, I personally use magic-image-refiner: https://replicate.com/collections/super-resolution https://replicate.com/collections/super-resolution

lsb 3y ago

You’re looking for what’s called upscaling, like with Stable Diffusion: https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler https://huggingface.co/stabilityai/stable-diffusion-x4-upsca...

hansvm 3y ago

I haven't explored the current SOTA recently, but super-resolution has been pretty good for a lot of tasks for few years at least. Probably just start with hugging-face [0] and try a few out, especially diffusion-based models. [0] https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/upscale https://huggingface.co/docs/diffusers/api/pipelines/stable_d...

godelski 3y ago

Look for SuperResolution. These models will typically come as a GAN, Normalizing Flow (or Score, NODE), or more recently Diffusion (or SNODE) (or some combination!). The one you want will depend on your computational resources, how lossy you are willing to be, and your image domain (if you're unwilling to tune). Real time (>60fps) is typically going to be a GAN or flow. Make sure to test the models before you deploy. Nothing will be lossless doing superresolution but flows can get you lossless in compression.

sitkack 3y ago
Or else you get Ryan Gosling https://news.ycombinator.com/item?id=24196650 https://news.ycombinator.com/item?id=24196650

davidbarker 3y ago

Magnific.ai (https://magnific.ai https://magnific.ai) is a paid tool that works well, but it is expensive. However, this weekend someone released an open-source version which has a similar output. (https://replicate.com/philipp1337x/clarity-upscaler https://replicate.com/philipp1337x/clarity-upscaler) I'd recommend trying it. It takes a few tries to get the correct input parameters, and I've noticed anything approaching 4× scale tends to add unwanted hallucinations. For example, I had a picture of a bear I made with Midjourney. At a scale of 2×, it looked great. At a scale of 4×, it adds bear faces into the fur. It also tends to turn human faces into completely different people if they start too small. When it works, though, it really works. The detail it adds can be incredibly realistic. Example bear images: 1. The original from Midjourney: https://i.imgur.com/HNlofCw.jpeg https://i.imgur.com/HNlofCw.jpeg 2. Upscaled 2×: https://i.imgur.com/wvcG6j3.jpeg https://i.imgur.com/wvcG6j3.jpeg 3. Upscaled 4×: https://i.imgur.com/Et9Gfgj.jpeg https://i.imgur.com/Et9Gfgj.jpeg ---------- The same person also released a lower-level version with more parameters to tinker with. (https://replicate.com/philipp1337x/multidiffusion-upscaler https://replicate.com/philipp1337x/multidiffusion-upscaler)

quaintdev 3y ago
Here's free and open source alternative that works pretty well https://www.upscayl.org/ https://www.upscayl.org/
aspyct 3y ago
That magnific.ai thingy is taking a lot of liberty on the images, and denaturing it. Their example with the cake is the most obvious. To me, the original image shows a delicious cake, and the modified one shows a cake that I would rather not eat...
- hug 3y ago
  Every single one of their before & after photos looks worse in the after. The cartoons & illustrations lose all of their gradations in feeling & tone with every outline a harsh edge. The landscapes lose any sense of lushness and atmosphere, instead taking a high-clarity HDR look. Faces have blemishes inserted the original actor never had. Fruit is replaced with wax imitation. As an artist, I would never run any of my art through anything like this.

jfdi 3y ago

thank you! will enjoy reviewing each of these

codercowmoo 3y ago

Current SOTA open source is I believe SUPIR (Example - https://replicate.com/p/okgiybdbnlcpu23suvqq6lufze https://replicate.com/p/okgiybdbnlcpu23suvqq6lufze), but it needs a lot of VRAM, or you can run it through replicate, or here's the repo (https://github.com/Fanghua-Yu/SUPIR https://github.com/Fanghua-Yu/SUPIR)

physPop 2y ago

This is called super resolution (SR). 2x SR is pretty safe and easy (so every pixel in becomes 2x2 out, in your example 800x600->1600x1200). Higher scalings are a lot harder and prone to hallucination, weird texturing, etc.