10 ms·
AI-Powered Nvidia RTX Video HDR Transforms Standard Video into HDR Video
- kwanbix 3y agoThe HDR transformation was really impresive. The Upscale not so much. At least not in my monitor.
- cm2187 3y agoSo now we need to stop making fun of cops pressing the "enhance" button in films...
- BlueTemplar 3y agoWe're going to have at least one episode of those lawyer shows where they pressed enhance, and the neural network hallucinated something that wasn't there.
- zamadatix 3y agoA bit of a let down that the video demoing SDR->HDR conversion is itself only published in SDR. Makes as much sense as demoing a colorization tool in a grayscale video!
- sharperguy 3y agoAt this point, with any new model I think it makes sense to wait until you can run the model on your own input before making any assumptions based on cherry picked examples.
- rado 3y agoRidiculous. Like when James Cameron promoted Avatar HDR with an SDR YouTube video, while YT is perfectly capable of HDR playback.
- babypuncher 3y agoAt least as of a couple of years ago, HDR support on YouTube has been pretty bad[1]. I know they've been working to improve things since, but I kind of don't blame people for walking away from that mess. 1. https://www.youtube.com/watch?v=DwDWQyBF9II https://www.youtube.com/watch?v=DwDWQyBF9II
- rado 3y agoThanks, will check out LTT's gripes, but I've been watching the following HDR channels forever and they look great: https://www.youtube.com/@SeoulWalker https://www.youtube.com/@SeoulWalker https://www.youtube.com/@Rambalac https://www.youtube.com/@Rambalac https://www.youtube.com/@Relaxing.Scenes.Driving https://www.youtube.com/@Relaxing.Scenes.Driving
- m4rtink 3y agoCan´t recommend Rambalac enough - I have pretty much re-traced his steps multiple times during our Japan trip a couple times & it really helped with orientation. :) Also some of the walks are really interesting & really gives you the context of various places in Japan. :)
- eurekin 3y agoThe real issue is it's either HDR or good SDR, but not both at the same time
- dansky 3y agoI'm also glad that Rambalac is back as he quit a few months ago. I've recently also started uploading 4K60 HDR content to YouTube [1] and it takes up to one week more time for them to encode than the SDR version. You can include your own LUT instead of YouTube conversion which seems to help. Here's an article and LUT [2] + a video [3] with valuable info. They allowed me getting DJI Pocket 3 HLG recordings to HDR10. [1] https://youtu.be/0S8hw8Lrvlk https://youtu.be/0S8hw8Lrvlk [2] https://www.wesleyknapp.com/blog/hdr https://www.wesleyknapp.com/blog/hdr [3] https://youtu.be/4izJfgRtkZE https://youtu.be/4izJfgRtkZE
- kelseyfrog 3y agoI guess. There's a lot of details we don't know that would change the calculus on this. To use a analogous workflow, it could be like saying, "It's pointless to shoot video in 10-bit log if it's going to be displayed on Rec.709 at 8-bits." It completely leaves out available transforms and manipulations in HDR that do have a noticeable impact even when SDR is the target. Again, we can't know if it's important given the information that's available, but we can't know if it's pointless either.
- deleted 3y ago[deleted]
- CamperBob2 3y agoYouTube tends to post a downscaled SD version first, then they encode and post the higher-res versions when they get around to it. This can take days in some cases. Meanwhile the creator catches the flak...
- unsane 3y agoCreators don't publish videos until the high-res versions are done processing.
- zamadatix 3y agoYou don't need high res for HDR on YouTube (144p HDR is a thing there oddly enough) and the 4k version had already processed when I posted that comment (with no change since in HDR availability). Usually media announcements/large channels pre-upload the video so it's ready when they want it to actually publish to avoid that kind of issue though.
- dannyw 3y agoThey probably use 144p for automated systems like ML recommendations, moderation and content ID.
- jiggawatts 3y ago4K processing takes just minutes, but HDR processing can take over a month to… never. There is no indication of this at all, no progress bar or eta. Just check manually every few days! This is why everyone is giving up on HDR, it’s just too painful because the content distributors are all so bad at it, with Netflix being the sole exception.
- BlueTemplar 3y agoSounds more like you should be giving up on YouTube.
- kevingadd 3y agoHDR video playback in the browser is pretty unreliable unless you're on a Mac.
- deleted 3y ago[deleted]
- ffsm8 3y agoIt's also pretty unreliable on Mac too... It's more reliable then on linux though, and windows has been doing "auto HDR" for videos for years, so kinda hard to tell when something is HDR or not there.
- devwastaken 3y agoHDR through YouTube appears to work fine even on my non HDR certified HDR monitor.
- zamadatix 3y agoIn what way? I've been doing it without issue on PC longer than I've even owned a Mac.
- jiggawatts 3y agoFirefox has zero HDR support, as an example.
- blackoil 3y agoThat would be true on Mac also?
- zamadatix 3y agoFirefox supports HDR video output on Mac since release 100.
- jiggawatts 3y ago
- mysteria 3y agoIf they were serious about showing this tech off they should've provided a video file download. Also indicate that it's a HDR file and should only be viewed on a HDR display. Youtube is just making this look bad as people won't see a difference.
- zamadatix 3y agoYouTube supports HDR video, no need for a separate download.
- Sparkyte 3y agoI am frequently disappointed by such videos.
- zokier 3y ago> Using the power of Tensor Cores on GeForce RTX GPUs, RTX Video HDR allows gamers and creators to maximize their HDR panel’s ability to display vivid, dynamic colors, preserving intricate details that may be inadvertently lost due to video compression. There is so much marketing BS in one small paragraph. For starters, generating(/hallucinating) data is imho the opposite of preserving anything. Then HDR is less associated with "intricate details" and more to do with color reproduction. Finally, video compression is the one thing that usually does not have problems with HDR, even the now venerable x264 can handle HDR content, generally it's almost everything else that struggles. Of course in a true marketing tradition, none of the things are also strictly false. I'm sure there are many ways to weasel the claims.
- nwellnhof 3y agoThey claim to preserve color detail that was lost due to compression of the dynamic range. What's wrong with that?
- dbspin 3y agoNot the OP, but you have to understand that 'compression of the dynamic range' is an artistic tool. Literally choosing the lighting ratio of an image is how you build out lighting for a scene. With AI overwriting these choices, you're looking at something more akin to colorization than upscaling.
- Grafikenjo 3y agoI think you need to understand that it is not always a 'artistic tool' but a money or knowledge limitation.
- dbspin 3y agoI'm a filmmaker... I don't know a single DOP or director who wishes they could work in HD but is limited by finances or knowledge. Again, shaping light is the essence of cinematography. Modern DSLRs far surpass the dynamic range (although not the effective resolution) of 35mm film. And yet the image they produce isn't comparable. When it comes to image quality, bit depth is enormously more important than dynamic range. When it comes to creating an artistic image, dynamic range hasn't been a limit for many decades.
- skottenborg 3y agoI could see a future where this works really well. It doesn't seem to be the case right now though. The "super resolution" showcased in the video seemed almost identical to adjusting the "sharpness" in any basic photo editing software. That is to say, perceived sharpness goes up, but actual conveyed details stays identical.
- brucethemoose2 3y agoNote that YouTube is really bad for these demos due to the re-compression, even in zoomed in stills.
- thefourthchime 3y agoAllegedly the new one plus phone does this trick in real time as well as up sampling and interframe motion interpretation. Mrwhostheboss seems impressed, but I don't really trust his yet judgment on these things. https://www.youtube.com/watch?v=J9-9fP_pcEc&t=1107s https://www.youtube.com/watch?v=J9-9fP_pcEc&t=1107s
- nomel 3y agoThe iPhone has also done this, for a few years now. It was, surprisingly, a one sentence mention in the keynote/release notes.
- moondev 3y agoWhatever special sauce the Nvidia shield uses is honestly incredible. Real time upscaling of any stream, and not just optimized for low res source, its like a force multiplier on content that is already HD. Supposedly the windows drivers do it as well but the effect seems less noticeable to me in my tests
- varispeed 3y agoThat seems like a gimmick and I actually prefer SDR video that is not upscaled. There is something ugly about those AI treated videos. They look fake.
- deergomoo 3y agoThey are fake. Ultimately it’s not recovering lost detail, it’s making shit up
- nomel 3y agoI don't think making things up is the problem, it's if it's believable. If it's indistinguishable to a viewer, then who cares. I never would have thought the HDR of the clouds was "made up".
- SirMaster 3y agoMaybe I'm odd, but a big part of art to me is seeing things how the creator intended it to be seen. So I calibrate all my media consumption displays etc. I could never see myself using some automated SDR -> HDR conversion like this. Even if it looks natural, it doesn't look like it was supposed to, and I want to see it how it was supposed to look.
- nomel 3y agoI would use it on every single video I've ever made myself, because intent had nothing to do with how my videos look. They were made with then best camera I had available, and HDR has only been available relatively recently. This is a tool that I want to use. If nobody can tell I used it, that's a good thing for me. If you don't want to use it, then nobody is making you.
- UberFly 3y ago"I don't think making things up is the problem, it's if it's believable" Depends of course if it's being passed off as reality. Slippery slope and all.
- aantix 3y agoI'm curious - what's the best open-source video upscaling library out there? I looked back about a year ago, and it didn't seem like there were any good open-source solutions.
- cf100clunk 3y agoAn HN search of ''Deep Space Nine'' and ''Topaz'' will show some great discussions here covering the dearth of such upscaling solutions, as well as some huge efforts before commonplace AI.
- Dylan16807 3y agoI found this single discussion? https://news.ycombinator.com/item?id=19453745 https://news.ycombinator.com/item?id=19453745 And that's by avoiding the word "topaz", where I see no story results with discussions and not much of comment results.
- cf100clunk 3y agoI can help (I added ''remaster''): https://hn.algolia.com/?dateRange=all&page=0&prefix=true&query=%22Deep%20Space%20Nine%22%20Remaster&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... Links to the artciles found in those discussions are to some very enlightening efforts by Joel Hruska to find an upscaling solution related to what the person asked about. As somebody else mentioned, Topaz is out there and Hruska gave it a good shot, but it is not open source.
- Dylan16807 3y agoRight, I did find those, but only one had a discussion. Just making sure I'm not missing anything, so thanks for the clarification.
- adzm 3y agoTopaz is light years ahead of any open source solution unfortunately.
- manmtstream 3y agoNow it will be absolutely impossible to accurately convey the artistic intent, when there's no way to know how it will look on consumer devices.
- cmcconomy 3y agoI think we lost that battle with motion interpolation on consumer TVs
- WithinReason 3y agoAlready happened with brightness and contrast controls
- cm2187 3y agoI can't get any two computer monitors that are not the same model to give me the same color.
- empiricus 3y agoVery easy solution: start filming in 120fps and then the TV motion interpolation does nothing.
- luma 3y agoConsumer devices have never been known for color accuracy and goes back a very long ways. The running joke in broadcast was that NTSC stood for "Never Twice the Same Color".
- aaroninsf 3y agoThe work I am interested in this broader domain is conversion (say, via some NeRF) of existing standard video into spatial video e.g. MV-HEVC for immersive experience on the Vision Pro etc.
- rixrax 3y agoI recently had some old super8 films shot by my parents scanned into 1080p resolution in ProresHQ. Because of the poor optics of the original camera, imperfect focus when shooting, poor lightning conditions, and general deterioration of the film stock, most of the footage won't get anywhere near what 1080p could deliver. What I'd like to try at some point is to let some AI/ML model process the frames, and instead of necessarily scaling it up to 4k etc., 'just' add (aka magic) missing detail into 1080p version and generally unblur it. Is there anything out there, even in research phase that can take existing video stock, and then hallucinate into it detail that never was there to begin with? What NVidia is demoing here seem like steps to that direction... I did test out Topaz Video and DaVinci's built-in super resolution feature, both of which gave me a 4k video with some changes to the original. But not the magic I am after.
- baq 3y agothere is AI tech to do this already. it has a slight problem, though: it adds detail to faces (this is marketing speak for completely changes how people look).
- UberFly 3y agoSomething like this will always change the original as it's guessing what should be there as it up scales. Only time will improve the guessing.
- actionfromafar 3y agoAn interesting thing about Super8: the resolution is generally very poor, but it can have quite the dynamic range. Also, with film in general (and video, but it's easier with film because you have global shutter) you can compensate motion blur and get more detail out which isn't visible when you look at the film frame by frame. And none of this needs AI. Regarding hallucination, I agree with the sibling comment, the problem is that faces change. And with video, I'm not even sure the same person would have the same face in various parts of the video...
- anjc 3y agoI also restored some Super 8 footage recently and had great success. The biggest win I had wasn't resolution, but slowing down the speed to be correct in DaVinci, and interpolating frames to make it 60fps using the RIFE algorithm in FlowFrames. I then used Film9 to remove shake, colour-correct, sharpen and so on. Correcting the speed and interpolating frames added an amazing amount of detail that wasn't perceptible to me in the originals (albeit it was there). All of this processing does remove some of the charm of the medium, so I'll be keeping the original scans in any case.
- bendergarcia 3y agoI think they should rephrase. It makes SDR appear HDR. It’s just making up information no? It’s not actually making it HDR just it appears to be HDR?
- Alghranokk 3y agoMaking up information? The same can be said for most commonly used modern compressed video formats. Just low bitrate streams of data that gets interpolated and predicted into producing what looks like high resolution video. AV1 even has entire systems for synthesizing film grain. The way i see it, if the ai generated HDR looks good, why not? It wouldn't be more fake or made up than the rest of the video.
- renewiltord 3y agoThis stuff is sick. If we had a real-time upscaler on a zoom telescope it would be a fantastic tool while traveling. I'd get a kick out of that.
- genman 3y agoAnd what would fake detail in the real world give to you?
- arcticfox 3y agoit doesn't have to be "fake" detail, an AI can use multiple frames to gather much more information than is available in a single frame and composite them into a much more detailed image
- genman 3y agoThis makes sense, it is called super resolution, but you don't need any AI for that, but I see that AI companies are trying to hijack the term.
- renewiltord 3y agoI can pretty easily distinguish useful LLM output from non-useful LLM output though others on this website seem to have lots of trouble. I think I can pretty easily do the same for things in the visual field. To be honest, part of why I'm successful is that I can draw use out of imperfect tools.
- genman 3y agoOh boy. Fake detail is well, fake. It doesn't allow you to learn anything about the world beyond self-deception. But of course, if this is what rocks your boat then be my guest.
- renewiltord 3y agoSkill issue.
- DrNosferatu 3y agoSpeaking of which, Nvidia has built-in live AI upscaling on the Shield TV android box. - Is there any stand-alone live AI upscaling / 'enhance' alternative for android or any other platform?
- lagadu 3y agoThe Shield is kind of an extreme outlier in today's environment. A device from 2015 that 9 years later is still one of the top tier choices in its (consumer) market is almost unheard of. In fact it's reportedly the currently supported Android device out there with the longest support[0], it's crazy that mine still gets updates. [0]https://www.androidcentral.com/android-longest-support-life-isnt-phone-its-nvidia-shield-tv https://www.androidcentral.com/android-longest-support-life-...
- moondev 3y agoIt really is awesome. I also enjoy the UI that allows you to side by side compare a stream and the difference is insane. I have been meaning to see how well it handles streaming a desktop via moonlight to the shield to real time upscale a second monitor's content. I assume it's trained for video footage and not static UI components. The RTX windows drivers don't seem to upscale as well as the shield.
- DrNosferatu 3y agoSo, should one buy a Shield TV today? It’s pricey, and being so old, I fear it will soon be obsoleted…
- maxglute 3y agoInterested in this too. I replaced my shield with a steamlink to desktop that does upscaling which is very clunky.
- fsiefken 3y agoI wonder if AI can be used to extrapolate 4:3 to 16:9 format or to create stereoscopic video (for use in VR or 3D TV's)
- LeoPanthera 3y agoDuring the brief moment that 3DTV was popular, almost all 3DTVs had a mode that could "convert" 2D to 3D, based on movement in the scene and other pre-learned cues. "Things that look like people should be in front of things that look like scenery", and so on. I miss 3D. I loved it, and I was sad that it didn't catch on. It enjoyed a longer life in Europe, where 3D blu-rays were produced for a few more years after they stopped selling them in the US, and I imported and enjoyed several. Maybe Apple's VR headset will be a 3D renaissance.
- jedberg 3y agoThe main reason at home 3D failed is because most people don't watch at home like they do at a theater. At a theater you sit down knowing that you can't get up and leave until it's over. At home you are doing other things: eating, folding laundry, going to the bathroom, taking phone calls, answering the door, and so on. It's not conducive to wearing glasses. Vision will have the same problem (as does any at home headset). I don't think it will lead to a 3D renaissance, at least not for a long time, until it becomes acceptable (and feasible) to walk around with it on all the time. Otherwise we need to wait for holographic projectors that can make a 3D image without having to wear glasses that make it hard or impossible to look at other 3D objects.
- 4d4m 3y agoFeels like a misnomer, its really "HDR style" video. The source material does not have the dynamic range embedded, this is an effect filter.
- squarefoot 3y agoThe upscaling doesn't seem particularly convincing, however, the HL2 RTX video on the same page definitely is.
- tpreetham 3y agoHL2 RTX also has newer textures and other assets