6 ms·
AI video you can watch and interact with, in real-time
- bkmeneguello 1y agoThis is amazing! I think the AI will completely replace the way we create and consume media currently. A well written story, with an amazing graphics generation AI can be both interactive and surprising every time you watch it again.
- lwo32k 1y agoWait till the bills arrive.
- qingcharles 1y agoNote that it isn't being created from whole cloth, it is trained on videos of the places and then it is generating the frames: "To improve autoregressive stability for this research preview, what we’re sharing today can be considered a narrow distribution model: it's pre-trained on video of the world, and post-trained on video from a smaller set of places with dense coverage. The tradeoff of this post-training is that we lose some generality, but gain more stable, long-running autoregressive generation." https://odyssey.world/introducing-interactive-video https://odyssey.world/introducing-interactive-video
- thetoon 1y agoCould probably be (semi?)automated to run on 3d models of places that doesn't exist. Even ai-built 3d models.
- doug_durham 1y agoI recognized the Santa Cruz Beach Boardwalk channel. It was exactly as I remember.
- netsharc 1y agoWell, that felt like entering a dream on my phone. Fuzzy virtual environments generated by "a mind" based on its memory of real environments... I wonder if it'd break our brains more if the environment changes as the viewpoint changes, but doesn't change back (e.g. if there's a horse, you pan left, pan back right, and the horse is now a tiger).
- afro88 1y agoOur minds are used to that: dreams
- mortenjorck 1y agoI kept expecting that to happen, but it apparently has some mechanism to persist context outside the user’s FOV. In a way, that almost makes it more dreamlike, in that you have what feels like high local coherence (just enough not to immediately tip you off that it’s a dream) that de-coheres over time as you move through it. Fascinatingly strange demo.
- deleted 1y ago[deleted]
- mensetmanusman 1y agoExploring babel’s library!
- Daisywh 1y agoit’s super cool. I keep thinking it kind of feels like dream logic. It looks amazing at first but I’m not sure I’d want to stay in a world like that for too long. I actually like when things have limits. When the world pushes back a bit and gives you rules to work with.
- aswegs8 1y agoDoesn't it have rules? I couldn't move past a certain point and hitting a wall made you teleport. Maybe I was just rationalizing random events, though.
- Tade0 1y agoI kept getting teleported to the start when I picked the world channel that showed some sort of well-lit catacombs. Eventually managed to leave the first room, but then got teleported somewhere else.
- booleandilemma 1y agoSounds like something a deity would say before creating a universe such as ours.
- spzb 1y agoThis seems like a staggeringly inefficient way to develop what is essentially a FPS engine.
- xpl 1y agoOnly at first glance. It can easily render things that would be very hard to implement in an FPS engine. What AI can dream up in milliseconds could take hundreds of human hours to encode using traditional tech (meshes, shaders, ray tracing, animation, logic scripts, etc.), and it still wouldn't look as natural and smooth as AI renderings — I refer to the latest developments in video synthesis like Google's Veo 3. Imagine it as a game engine running in real time.
- rxtexit 1y agoThis thing looks like a complete joke next to the Unreal Engine.
- ivape 1y agoWhy do you think this is so hard, even for technical people here, to make the inductive leap on this one? Is it that close to magic? The AI is rendering pillars and also determining collision detection on it. As in, no one went in there and selected a bunch of pillars and marked it as a barrier. That means in the long run, I'll be able to take some video or pictures of the real world and have it be game level.
- ffsm8 1y agoBecause that's been a thing for years already - and works way better then this research does. Unreal engine 5 has been demoing these features for a while now, I heard about it early 2020 iirc, but the techniques like gaussian splattering predate it. I have no experience in either of these, but I believe MegaScans and RealityCapture are two examples doing this. And the last nanite demo touched on it, too.
- ivape 1y ago
- scudsworth 1y ago[flagged]
- Sohcahtoa82 1y agoFor sure, but consider it a "first draft" of what this type of generative AI can do. The resolution is extremely low. The website doesn't specify, but I'd guess it's only 160x120. Such a low resolution was necessary to render it in real time and maintain a reasonable frame rate. To try to hide the blurring a bit, they apply some filters to add scan lines and other effects to make it look like an old TV. That said, I'd be surprised if anybody could gather the hardware to work well enough to get it to a useable resolution, let alone even something like 1080p. It's literally over 100x the pixels of 160x120.
- qwerty59 1y agoThis is cool. I think there is good chance that this is the future of videogames.
- yieldcrv 1y agoNow this is an Assassin’s Creed memory machine that I can get behind
- Traubenfuchs 1y agoThank you for this experience. Feels like you are exploring a dream. I LOVE dreamy AI content. That stuff where everything turned into dogs for example. As AI is maturing, we are slowly losing that im favor of boring realism and coherence.
- akomtu 1y agoInteractive ads and interactive porn are the AI killer apps we miss so much.
- gcanyon 1y agoAm I the only one stuck with a black screen as the audio plays?
- hx8 1y agoI found an interesting glitch where you could never actually reach a parked car, as you move forward the car also moved. It looked a lot like traffic moving through Google Street View.
- Hobadee 1y agoYeah. I found the same thing. Cars would disappear in front of me, then I reached the end of the world and it reset me. I'm not sure I believe this is AI, and instead some crappy street view interface.
- vouaobrasil 1y agoI think this step towards a more immerse virtual reality can actually be dangerous. A lot of intellectual types might disagree but I do think that creating such immersion is a dangerous thing because it will reduce the value people place on the real world and especially the natural world, making them even less likely to care if big corporations screw it up with biospheric degradation. It seems like it has a high chance of leading to even more narcissism as well because we are reducing our dependence on others to such a degree that we will care about others less and less, which is something that has already started happening with increasingly advanced interactive technology like AI.
- TaupeRanger 1y agoThis is why we never see any alien life. When they reach a sufficient level of technology, they realize the virtual/mental universe is much more compelling and fun than the boring rule-bound physical one.
- vouaobrasil 1y agoActually, it's that the aliens eventually get completely immersed in technology to the point where they just self-destruct due to the meaninglessness of the life they have created.
- mvdtnz 1y agoPeople were saying literally the exact same thing when those crappy VR headsets were all the rage. I think we're ok.
- vouaobrasil 1y agoSo your argument is that because it didn't work once, that it won't work now? At some point, a breakthrough is made and in this case the breakthrough is a bad one.
- dragonwriter 1y ago> I think this step towards a more immerse virtual reality can actually be dangerous I don't think its a step toward that; I think this is literally trained using techniques to generate more immersive virtual reality that already exists and takes less compute, to produce a more computationally expensive and less accurate AI version. At least, that's what every other demo of a real-time interactive AI world model has been, and they aren't trumpeting any clear new distinction.
- Morizero 1y agoI feel like we're so close to remaking the classic Rob Schneider full motion video game "A Fork in the Tale" https://m.youtube.com/watch?v=YXPIv7pS59o https://m.youtube.com/watch?v=YXPIv7pS59o
- OfficeChad 1y ago[dead]
- godelski 1y agoThat felt so wrong AND someone is cheating here. This felt really suspicious... I got to the graffiti world and there were some stairs right next to me. So I started going up them. It felt like I was walking forward and the stairs were pushing under me until I just got stuck. So I turned to go back down and half way around everything morphed and I ended up back down at the ground level where I originally was. I was teleported. That's why I feel like something is cheating here. If we had mode collapse I'm not sure how we should be able to completely recover our entire environment. Not unless the model is building mini worlds with boundaries. It was like the out of bond teleportation you get in some games but way more fever dream like. That's not what we want from these systems, we don't want to just build a giant poorly compressed videogame, we want continuous generation. If you have mode collapse and recover, it should recover to somewhere new, now where you've been. At least this is what makes me highly suspicious.
- AnotherGoodName 1y agoYes the thing that got me was i went through the channels multiple times (multiple browser sessions). The channels are the same everytime (the numbers don't align to any navigation though - flip back and forth between two numbers and you'll just hit a random channel everytime - don't be fooled by that). Every object is in the same position and the layout is the same. What makes this AI generated over just rendering a generated 3D scene? Like it may seem impressive to have no glitches (often in AI generated works you can turn around a full rotation and you're what's in front of you isn't what was there originally) but here it just acts as a fully modelled 3D scene rendering at low resolution? I can't even walk outside of certain bounds which doesn't make sense if this really is generated on the fly. This needs a lot of skepticism and i'm surprised you're the first commenting on the lack of actual generation here. It's a series of static scenes rendered at low fidelity with limited bounds.
- AnotherGoodName 1y agoOk playing with this more there's very subtle differences between sessions. As in there is some hallucination here with certain small differences. I think what's happening is this is AI generated but it is very very overfitted to real world 3D scenes. The AI is almost rendering exactly a real world scene and not much more. They can't travel out of bounds or the model stops working since it's so overfitted to these scenes. The overfitting solves hallucinations but it also makes it almost indistinguishable from pre modelled 3D scenes.
- coldcache 1y agorelated (and quite cool) -- minecraft generated on-the-fly which you can interact with: https://news.ycombinator.com/item?id=42014650 https://news.ycombinator.com/item?id=42014650
- lerp-io 1y agogoing outside breaks the ai lol
- deadbabe 1y agoIt’s pointless to do this with real world places. Why not do it for TV shows or a photograph? You could walk around inside and explore the scenes.
- abe94 1y agovery cool - what was the hardest part of building this?
- olivercameron 1y agoIf I had to choose one, I'd easily say maintaining video coherence over long periods of time. The typical failure case of world models that's attempting to generate diverse pixels (i.e. beyond a single video game) is that they degrade to a mush of incoherent pixels after 10-20 seconds of video. We talk about this challenge in our blog post here (https://odyssey.world/introducing-interactive-video https://odyssey.world/introducing-interactive-video). There's specifics in there on how we improved coherence for this production model, and our work to improve this further with our next-gen model. I'm really proud of our work here! > Compared to language, image, or video models, world models are still nascent—especially those that run in real-time. One of the biggest challenges is that world models require autoregressive modeling, predicting future state based on previous state. This means the generated outputs are fed back into the context of the model. In language, this is less of an issue due to its more bounded state space. But in world models—with a far higher-dimensional state—it can lead to instability, as the model drifts outside the support of its training distribution. This is particularly true of real-time models, which have less capacity to model complex latent dynamics. Improving this is an area of research we're deeply invested in. In second place would absolutely be model optimization to hit real-time. That's a gnarly problem, where you're delicately balancing model intelligence, resolution, and frame-rate.
- deleted 1y ago[deleted]
- jowday 1y agoThis is pretty much the same thing as those models that baked dust2 into a diffusion model then used the last few frames as context to continue generating - same failure modes and everything. https://diamond-wm.github.io/ https://diamond-wm.github.io/
- rehaan0807 1y ago[dead]
- amelius 1y agoWould be more interesting with people in it.
- olivercameron 1y agoI agree! Check out outputs from our next-gen world model here(https://odyssey.world/introducing-interactive-video https://odyssey.world/introducing-interactive-video), featuring richer pixels and dynamics.
- chenxi9649 1y agodo u personally feel like scaling this approach is going to be the end game for generating navigatable worlds? ie. as opposed to first generating a 3d env then doing some sorts of img2img on top of it?
- andrewstuart 1y agoLove the atmosphere.
- fortran77 1y agoI'm unable to navigate anywhere. I'm on a laptop with a touchscreen and a trackpad. I clicked, double clicked, scrolled, and tried everything I could think of and the views just hovered around the same spot.
- lolinder 1y agoThis is similar to the Minecraft version of this from a few months back [0], but it does seem to have a better time keeping a memory of what you've already seen, at least for a bit. Spinning in circles doesn't lose your position quite as easily, but I did find that exiting a room and then turning back around and re-entering leaves you with a totally different room than you exited. [0] Minecraft with object impermanence (229 points, 146 comments) https://news.ycombinator.com/item?id=42762426 https://news.ycombinator.com/item?id=42762426
- jedberg 1y agoIn playing with this it was unclear to me how this differs from a pre-programmed 3d world with bit mapped walls. What is the AI adding that I wouldn't get otherwise?
- exe34 1y agoThis reminds me of the scapes in Diaspora (by Greg Egan).
- squiffy 1y agoCan you say where the underground cellar with the red painting is? It's compelling.
- olivercameron 1y agoHi HN, I hope you enjoy our research preview of interactive video! We think it's a glimpse of a totally new medium of entertainment, where models imagine compelling experiences in real-time and stream them to any screen. Once you've taken the research preview for a whirl, you can learn a lot more about our technical work behind this here (https://odyssey.world/introducing-interactive-video https://odyssey.world/introducing-interactive-video).
- givemeethekeys 1y agoFeels like the Mist (Myst?) game.
- tiahura 1y agoTo me, this is evidence we're not in a simulation. Even with a gazillion H100's the model runs out of memory just (very roughly) simulating a 50'x50' space over just a few seconds.
- butz 1y agoI do not get the "interactive" part. I expect to be able to manipulate objects or at least move them, you know, "interact" with the "video". Now it is some cheap walking simulator, without narration or any plot. Disappearing lamp posts when you get near them also should not be considered an interaction. Maybe you should take a bit different approach to interactive videos, and let's say build a tech review video for some gadget or device, where viewer could interrupt host, using voice, and ask them questions, skip to some part or repeat something in more detail, explain some concept, even compare to other devices.
- Euphorbium 1y agoSeems like it ingested google street view