8 ms·

The HTTP of VR

53 points by rvkennedy 5y ago

unwind 5y ago
Oh how this reminded me of the Verse protocol and Uni-Verse! To be young again, and so on. :) [1]: https://en.m.wikipedia.org/wiki/Verse_protocol https://en.m.wikipedia.org/wiki/Verse_protocol
jayd16 5y ago
This is pretty silly. We can't throw away http because http solves problems that VR does not alleviate. >A real-time, dynamic, stateful two-way client-server protocol. As such, it will be if not fully RTP then close to it. Why didn't we always have this if all we needed to do was ask? So...realizing we still have the internet of today, what we actually need to rethink is html and the concept of the web as documents alone. I would be interested to see some work on hyper-objects. As in, hypertext beyond text. The article should be "HTML for VR" and we should be musing about how to find, load, interact and link web based virtual objects.
sxp 5y ago
tl;dr: "So at Simul, for the past few years we’ve been building this protocol: it’s called Teleport VR. Let’s see what we can make with it!" An alternative view would be that HTTP(S) would be "the HTTP of VR". With WebXR and standard JS APIs for HTTPS, async fetching, WebRTC, etc, all the items listed in "Imagine an application-layer protocol for VR with the following characteristics..." are satisfied. And the stack can use battle-tested web technologies so that it can leverage standard CDNs, cloud servers, etc. VR has some extra constraints over 2D webpages due to tighter frames per second and latency tolerances, but most of the web protocols can get you 90% of the way there.
- jayd16 5y ago
  I wouldn't even say performance is all that different. Something that is unique is the idea that a website is a single document where as a virtual website might take the form of an interactive object and/or an interactive space. I would say it's an open question how we want these web based virtual objects to interact with each other. Would we want to physically pull a video object off the Google Drive shelf and drop it into the YouTube workstation? How would such an interaction be possible? Even if, as today, they just never speak directly, could those objects live in the same space or would each website fully immerse the user?
bborud 5y ago
Nothing in the blog posting suggests to me you can't use HTTP and Websockets for VR. The understanding of HTTP in the blog posting seems to be rooted in the early 2000s. I don't think the author has much experience in protocol design (it is harder than it looks). It would be more productive to define a layer on top of HTTP/2 so we can leverage a lot of code that already works, rather than having to spend 10-15 years creating a new spec and codebases that need maturing. And if you're not happy with websockets for low latency bidirectional communication: it would make more sense to improve websockets rather than reinvent the wheel.
- andrewmcwatters 5y ago
  WebSockets are not an appropriate medium for VR in the most common use case today: real-time video games. Even if you're not building a video game today and want to do something other-VR esque, I guarantee that you will inevitably end up recreating something the video game industry has done in the last 22-23 years. Everything real-time the author of this article suggests is in the realm of something you would want for VR. It makes sense to just take what both industries of web development and game development currently understand and build on that. So let's take that to its logical conclusion. Let's say you wanted to navigate virtual worlds. You're going to end up having some sort of "navigator" or "explorer" or end up going on some sort of virtual "safari" of sorts. You'll do that probably initially with HTTPS or talk to some HTTPS-based server. WebSockets are not sufficient for real-time VR based work, so you'll probably end up with some sort of, let's say, WebUDP or WebSockets with UDP functionality. Everyone will end up wanting to build their own layers of abstraction over and over again and again so entry-level web and game developers have something to do, so it'll look like some incarnations that build on or supersede Three.js. Why? Because everyone will have a different interpretation of what they want their camera, or user entity or actor to be able to do. So, you'll need HTTPS, web-based UDP, some sort of localplayer series of libraries or framework, then you'll need levels or maps, because to do any sort of VR, you need a world, or worlds you can navigate. Huh, weird. All of this just sounds like someone porting Quake to the web with VR. How boring. Of course, if someone says "OK, yep, let's do it then," it won't be anything like that, or it will, but only superficially, because that's what happens when you live long enough to see people take technologies the broader population already knows about and cram them together. The alternative is that Zuckerberg is an old person and no one was asking for a metaverse, just video games that don't suck eggs, and that Meta is just Mark's way of graduating Facebook to an Alphabet-type conglomerate in order to keep growth moving forward. I seriously feel bad for kids today. You have what, Fortnite, Minecraft, and Roblox to play and that's it? Too many micro transactions and low quality games. No 30-something is asking to put on a headset and go to the VR equivalent of Something Awful, which would probably look like a back alley with players farting on each other Jaykin' Bacon-style. You want a metaverse? That's what it would look like. There's always going to be some Something Awful/4chan/Facepunch equivalent. No one is like "oh yeah I wanna go to work and sit in a virtual cubical with my Meta® Quest 3," and yet there are some people really disconnected from reality who think people are actually asking for that instead of, like, affordable housing or something. Weird.
douglaswlance 5y ago
Latency is incredibly important in VR. If everything is streaming from a remote server, even if it's a straight fiber connection, it'll still be too much latency.
Mizza 5y ago
I don't want to have to strap a fucking telephone to my face to go to some shitty fake job. Please don't build this world.
Ono-Sendai 5y ago
I'm building something similar for metaverses, although with less emphasis on VR currently. See https://substrata.info/about_substrata https://substrata.info/about_substrata Currently it's a relatively simple bidirectional protocol over TLS. It's not fully documented yet but you can get an idea of it by looking at an example bot client in python: https://github.com/glaretechnologies/substrata-example-bot-python/blob/main/substrata_chatbot_demo.py https://github.com/glaretechnologies/substrata-example-bot-p...
binarynate 5y ago
> By far the greatest reason to look beyond HTML and HTTP for spatial computing is simply this: these technologies will continue to develop, and will always be driven by their primary purpose: to deliver webpages, websites and static, or marginally dynamic content. This is a valid point, but I believe there's still enormous potential to innovate on top of WebXR. Since browser engines are open source, it's possible for upstart XR browser apps to add additional features to Gecko or Chromium that push WebXR forward.
- binarynate 5y ago
  On a related note, I develop libraries for embedding web browsers in Unity 3D (https://vuplex.com https://vuplex.com), including a library for embedding the Mozilla GeckoView library used by FireFox Reality. I plan to develop a WebXR driver for it, but haven't prioritized it yet. If you're interested in developing a WebXR driver for use with GeckoView (for example, to use with Oculus Quest), you can contact me, and I'll send you my notes from my research: https://support.vuplex.com/contact https://support.vuplex.com/contact
  - jimmySixDOF 5y ago
    I'm interested in how you might be able to get through the CORS problem in WebXR/Browser standards ? In a VR Unity app with embedded browsers you can click through hyperlinks no problem but in WebXR the same hyperlink will trigger an origin mismatch and break out of immersion. Not having good 2D web content integration is a major blocker in WebXR for so many applications.
    - binarynate 5y ago
      Hi, Jimmy . I think what you're saying is that when a new page is loaded, you want to avoid getting kicked out to the 2D view of the browser first (similar to switching between apps on Oculus Quest). I think there are opportunities to achieve that both at the web page level and at the browser level. • At the web page level, an alternative to loading a new page entirely could be to load new content into the existing page, similar to a SPA. However, like you mentioned, that wouldn't really work if you want to seamlessly transition to a WebXR experience from a 3rd party. That's where browser-level innovation could come in: • At the browser-level, it could be possible to come up with an innovative solution to this. For example, perhaps the user could grant a site permission to load the second site in a second browser instance and then transition from the first to the second once the second has loaded. This is kind of like allowing popups in a traditional 2D browser, where a page can create a new tab with window.open(), but with the additional capability of allowing the page to programmatically navigate to the new window once it has loaded.
raidicy 5y ago
Aframe comes to mind. You can have full VR experiences that link just like a Link in HTML to other VR experiences. https://aframe.io/examples/ https://aframe.io/examples/
- edoceo 5y ago
  I thought this post was gonna be about aframe. It's super cool and the docs are good enough that a fool like me could get something neat in a day. Made an aframe HTML with PHP reading from my DB. It's rad.
Animats 5y ago
Since I'm writing a new client, in Rust, for Second Life/Open Simulator, I'm very aware of these issues. A metaverse client for a high-detail virtual world has most of the problems of an MMO client plus many of the problems of a web browser. First, much of what you're doing is time-sensitive. You have a stream of high-priority events in each direction that have to be dealt with quickly but don't have a high data volume. Then you have a lot of stuff that's less time critical. The event stream is usually over UDP in the game world. Since you might lose a packet, that's a problem. Most games have "unreliable" packets, which, if lost, are superseded by later packets. ("Where is avatar now" is a typical use.) You'd like to have that stream on a higher quality of service than the others, if only ISPs and routers actually paid attention to that. Then you have the less-critical stuff, which needs reliability. ("Object X enters world" is a typical use.) I'd use TCP for that, but Second Life has its own not very good UDP-based protocol, with a fixed retransmit timer. Reliable delivery, in-order delivery, no head of line blocking - pick two. TCP chooses the first two, SL's protocol chooses the first and third ones. Out of order delivery after a retransmit can cause avatars to lose clothing items, because the child item arrived before the parent item. Then you have asset fetching. In Second Life/Open Simulator this is straight HTTP/1. But there are some unusual tricks. Textures are stored in progressive JPEG 2000. It's possible to open a connection and just read a few hundred bytes to get a low-rez version. Then, the client can stop reading for a while, put the low-rez version on screen, and wait to see if there's a need to keep reading, or just close the connection because a higher-rez version is not needed. The poor server has to tolerate a large number of stalled connections. Worse, the actual asset servers on AWS are front-ended by Akamai, which is optimized for browser-type behavior. Requesting an asset from an Akamai cache results in fetching the entire asset from AWS, even if only part of it is needed. There's a suspicion that large numbers of partial reads and stalled reads from clients sometimes causes Akamai's anti-DDOS detection to trip and throttle the data flow. So those are just some of the issues "the HTTP of VR" must handle. Most are known to MMO designers. The big difference in virtual worlds is there's far more dynamic asset loading. How well that's managed has a strong influence on how consistent the world looks. It has to be constantly re-prioritized as the viewpoint moves. (Demo, from my own work: https://vimeo.com/user28693218 https://vimeo.com/user28693218 This shows the client frantically trying to load the textures from the network before the camera gets close. Not all the tricks to make that look good are in this demo.) It's not an overwhelmingly hard problem, but botch it and you will be laughed off Steam.
gfxgirl 5y ago
I think there is a different problem that needs to be solved and it's probably impossible. I've dreamed of the metaverse since Snow Crash and maybe before (Tron?) but ... when it comes to actually making it, lets assume unlimited CPU/GPU power and unlimited memory. Ideally, I want the Metaverse to allow people to run their own code. Whether its VR or AR it's a shared 3D space. So I want my Nintendo "Nintendogs" to be able to run around my "Ikea furniture" with my "Google/Apple/OSM maps" showing me navigation directions and my "FB Messenger/Discord/iOS Messenger" letting me connect to people inside. In a webpage, each of these things runs in an IFRAME isolated from the other and browsers go to great lengths to disllow one spying on another. But in this 3D space my Nitendogs can't run through the space unless they can "sense the space". They need to know where the fire hydrants are, where the side walk is, what things they're allowed to climb/chew etc. But to do that effectively means they need enough info to spy on me. Same for all the other apps. I can use messaging apps on my phone with GPS off and full network access off so that the app can't know my location, but order for different apps in the Metaverse to do similar they'll need to know at least the virtual location of themselves and the stuff around them which is enough to track/fignerprint You can maybe get around some of this with a massive walled garden but that arguably is not the metaverse.
- berkes 5y ago
  You presume a push model. And compare it to a pull model (iframes). I think that is where the solutions are. The messages could be delivered as a simple XML feed. Your virtual home, or HUD knows where to place them. Through hyperlinks they know where to subscribe, or refresh, or get details. The messages don't need to know anything about placement and usage.
- jayd16 5y ago
  Seems like you could share collision meshes without much risk of spying, no?
usrbinbash 5y ago
What exactly is the "metaverse" supposed to be, other than a marketing term to sell a more expensive class of IO devices? People will not switch over in droves to do their text/image/video editing in VR all of a sudden, because other than a few special design applications, there is no point in doing so...it's slower, clumsier and the input devices are much less precise than mouse&keyboard. Another supposed target demographic, people in IT won't switch either. I see no point in virtually grabbing a glowing code-ball and throing it into the "deploy-tube", or navigate a codebase using haptic gestures with the huge meat-styluses at the end of my arms, when I can simply type `git push` or `/myAwesomeStruct` I also have a hard time imagining management sitting in meetings while wearing a 400g headset for 3h. Or companies being willing to cough up 350+$ for every employee just so they can join meetings, when Zoom is basically free. So, what else is there? Gaming and maybe some "recreational apps" (aka. alsogaming, only less interactive). And since not all games will take place in the same unified MMORPG-ish permanent universe (yes, people want to play in sessions, and people want to play single player, and people want to play while not connected to the internet), this will not be a paradigm-shift, but rather a new toy in an already large collection of other toys.
- charcircuit 5y ago
  Nothing is stopping you from using your keyboard in VR.
  - usrbinbash 5y ago
    Nothing is stopping me from writing a google-ASR powered program hooked into my terminal, and start yelling my code into the computer either. But as long as this isn't faster, easier or more reliable than typing it on a keyboard, I won't. People don't work a certain way because it's possible. People work a certain way because it saves time, money, sanity, or simply because it's convenient. If wearing a VR headset while coding isn't providing substantial benefits over what my current system provides, why would I do it?
    - charcircuit 5y ago
      The difference is you can already use a keyboard in VR where your system sounds like you would have to go and mild it. >If wearing a VR headset while coding isn't providing substantial benefits over what my current system provides, why would I do it? You wouldn't.
Dirak 5y ago
Networking for multiplayer games is a super interesting problem space since games tend to be more sensitive to latency, packet loss, and the accuracy of game states between clients. The problems are even more pronounced in VR where noticeable latency or artifacts can cause motion sickness. In modern fighter games, the industry seems to be tending toward predictive lockstep networking. This is a type of networking where if the client doesn't receive the inputs of other clients from the server, it will "predict" those inputs (usually by replaying the last received input) to give the illusion of zero latency gameplay. The drawback being that you need to implement rollback in the case where the predicted input doesn't match the real received input. When poorly executed, this could look like jittery player movement with entities rubber banding and teleporting and cause artifacts, but when done properly is mostly unnoticeable. If you're interested in this domain, I recommend checking out https://www.ggpo.net/ https://www.ggpo.net/ which is the library used in many of the modern fighter games (notably Skullgirls). It also comes with an in depth explanation of how to implement predictive networking with rollback on your own https://drive.google.com/file/d/1cV0fY8e_SC1hIFF5E1rT8XRVRzPjU8W9/view https://drive.google.com/file/d/1cV0fY8e_SC1hIFF5E1rT8XRVRzP...
bullen 5y ago
The HTTP of VR is HTTP! http://fuse.rupy.se/about.html http://fuse.rupy.se/about.html You also need a P2P protocol (probably some binary UDP thing) for tick based data like limb positions if you want body language. But really VR is much less important for immersion than action MMO = Mario/Zelda with 1000+ players.
schmorptron 5y ago
Kinda Off-topic, but if anyone is looking to play around with building vr spaces or games, i recently found out about LÖVR[0] which is a sipmle lua-based open source VR "framework". Haven't had a chance to play with it but it seems other people like it! [0] https://lovr.org/ https://lovr.org/

53 points by rvkennedy 5y ago

Oh how this reminded me of the Verse protocol and Uni-Verse! To be young again, and so on. :) [1]: https://en.m.wikipedia.org/wiki/Verse_protocol https://en.m.wikipedia.org/wiki/Verse_protocol

jayd16 5y ago

This is pretty silly. We can't throw away http because http solves problems that VR does not alleviate. >A real-time, dynamic, stateful two-way client-server protocol. As such, it will be if not fully RTP then close to it. Why didn't we always have this if all we needed to do was ask? So...realizing we still have the internet of today, what we actually need to rethink is html and the concept of the web as documents alone. I would be interested to see some work on hyper-objects. As in, hypertext beyond text. The article should be "HTML for VR" and we should be musing about how to find, load, interact and link web based virtual objects.

sxp 5y ago

tl;dr: "So at Simul, for the past few years we’ve been building this protocol: it’s called Teleport VR. Let’s see what we can make with it!" An alternative view would be that HTTP(S) would be "the HTTP of VR". With WebXR and standard JS APIs for HTTPS, async fetching, WebRTC, etc, all the items listed in "Imagine an application-layer protocol for VR with the following characteristics..." are satisfied. And the stack can use battle-tested web technologies so that it can leverage standard CDNs, cloud servers, etc. VR has some extra constraints over 2D webpages due to tighter frames per second and latency tolerances, but most of the web protocols can get you 90% of the way there.

jayd16 5y ago
I wouldn't even say performance is all that different. Something that is unique is the idea that a website is a single document where as a virtual website might take the form of an interactive object and/or an interactive space. I would say it's an open question how we want these web based virtual objects to interact with each other. Would we want to physically pull a video object off the Google Drive shelf and drop it into the YouTube workstation? How would such an interaction be possible? Even if, as today, they just never speak directly, could those objects live in the same space or would each website fully immerse the user?

bborud 5y ago

Nothing in the blog posting suggests to me you can't use HTTP and Websockets for VR. The understanding of HTTP in the blog posting seems to be rooted in the early 2000s. I don't think the author has much experience in protocol design (it is harder than it looks). It would be more productive to define a layer on top of HTTP/2 so we can leverage a lot of code that already works, rather than having to spend 10-15 years creating a new spec and codebases that need maturing. And if you're not happy with websockets for low latency bidirectional communication: it would make more sense to improve websockets rather than reinvent the wheel.

andrewmcwatters 5y ago
WebSockets are not an appropriate medium for VR in the most common use case today: real-time video games. Even if you're not building a video game today and want to do something other-VR esque, I guarantee that you will inevitably end up recreating something the video game industry has done in the last 22-23 years. Everything real-time the author of this article suggests is in the realm of something you would want for VR. It makes sense to just take what both industries of web development and game development currently understand and build on that. So let's take that to its logical conclusion. Let's say you wanted to navigate virtual worlds. You're going to end up having some sort of "navigator" or "explorer" or end up going on some sort of virtual "safari" of sorts. You'll do that probably initially with HTTPS or talk to some HTTPS-based server. WebSockets are not sufficient for real-time VR based work, so you'll probably end up with some sort of, let's say, WebUDP or WebSockets with UDP functionality. Everyone will end up wanting to build their own layers of abstraction over and over again and again so entry-level web and game developers have something to do, so it'll look like some incarnations that build on or supersede Three.js. Why? Because everyone will have a different interpretation of what they want their camera, or user entity or actor to be able to do. So, you'll need HTTPS, web-based UDP, some sort of localplayer series of libraries or framework, then you'll need levels or maps, because to do any sort of VR, you need a world, or worlds you can navigate. Huh, weird. All of this just sounds like someone porting Quake to the web with VR. How boring. Of course, if someone says "OK, yep, let's do it then," it won't be anything like that, or it will, but only superficially, because that's what happens when you live long enough to see people take technologies the broader population already knows about and cram them together. The alternative is that Zuckerberg is an old person and no one was asking for a metaverse, just video games that don't suck eggs, and that Meta is just Mark's way of graduating Facebook to an Alphabet-type conglomerate in order to keep growth moving forward. I seriously feel bad for kids today. You have what, Fortnite, Minecraft, and Roblox to play and that's it? Too many micro transactions and low quality games. No 30-something is asking to put on a headset and go to the VR equivalent of Something Awful, which would probably look like a back alley with players farting on each other Jaykin' Bacon-style. You want a metaverse? That's what it would look like. There's always going to be some Something Awful/4chan/Facepunch equivalent. No one is like "oh yeah I wanna go to work and sit in a virtual cubical with my Meta® Quest 3," and yet there are some people really disconnected from reality who think people are actually asking for that instead of, like, affordable housing or something. Weird.

douglaswlance 5y ago

Latency is incredibly important in VR. If everything is streaming from a remote server, even if it's a straight fiber connection, it'll still be too much latency.

Mizza 5y ago

I don't want to have to strap a fucking telephone to my face to go to some shitty fake job. Please don't build this world.

Ono-Sendai 5y ago

I'm building something similar for metaverses, although with less emphasis on VR currently. See https://substrata.info/about_substrata https://substrata.info/about_substrata Currently it's a relatively simple bidirectional protocol over TLS. It's not fully documented yet but you can get an idea of it by looking at an example bot client in python: https://github.com/glaretechnologies/substrata-example-bot-python/blob/main/substrata_chatbot_demo.py https://github.com/glaretechnologies/substrata-example-bot-p...

binarynate 5y ago

> By far the greatest reason to look beyond HTML and HTTP for spatial computing is simply this: these technologies will continue to develop, and will always be driven by their primary purpose: to deliver webpages, websites and static, or marginally dynamic content. This is a valid point, but I believe there's still enormous potential to innovate on top of WebXR. Since browser engines are open source, it's possible for upstart XR browser apps to add additional features to Gecko or Chromium that push WebXR forward.

binarynate 5y ago
On a related note, I develop libraries for embedding web browsers in Unity 3D (https://vuplex.com https://vuplex.com), including a library for embedding the Mozilla GeckoView library used by FireFox Reality. I plan to develop a WebXR driver for it, but haven't prioritized it yet. If you're interested in developing a WebXR driver for use with GeckoView (for example, to use with Oculus Quest), you can contact me, and I'll send you my notes from my research: https://support.vuplex.com/contact https://support.vuplex.com/contact
- jimmySixDOF 5y ago
  I'm interested in how you might be able to get through the CORS problem in WebXR/Browser standards ? In a VR Unity app with embedded browsers you can click through hyperlinks no problem but in WebXR the same hyperlink will trigger an origin mismatch and break out of immersion. Not having good 2D web content integration is a major blocker in WebXR for so many applications.
  - binarynate 5y ago
    Hi, Jimmy . I think what you're saying is that when a new page is loaded, you want to avoid getting kicked out to the 2D view of the browser first (similar to switching between apps on Oculus Quest). I think there are opportunities to achieve that both at the web page level and at the browser level. • At the web page level, an alternative to loading a new page entirely could be to load new content into the existing page, similar to a SPA. However, like you mentioned, that wouldn't really work if you want to seamlessly transition to a WebXR experience from a 3rd party. That's where browser-level innovation could come in: • At the browser-level, it could be possible to come up with an innovative solution to this. For example, perhaps the user could grant a site permission to load the second site in a second browser instance and then transition from the first to the second once the second has loaded. This is kind of like allowing popups in a traditional 2D browser, where a page can create a new tab with window.open(), but with the additional capability of allowing the page to programmatically navigate to the new window once it has loaded.

raidicy 5y ago

Aframe comes to mind. You can have full VR experiences that link just like a Link in HTML to other VR experiences. https://aframe.io/examples/ https://aframe.io/examples/

edoceo 5y ago
I thought this post was gonna be about aframe. It's super cool and the docs are good enough that a fool like me could get something neat in a day. Made an aframe HTML with PHP reading from my DB. It's rad.

Animats 5y ago

Since I'm writing a new client, in Rust, for Second Life/Open Simulator, I'm very aware of these issues. A metaverse client for a high-detail virtual world has most of the problems of an MMO client plus many of the problems of a web browser. First, much of what you're doing is time-sensitive. You have a stream of high-priority events in each direction that have to be dealt with quickly but don't have a high data volume. Then you have a lot of stuff that's less time critical. The event stream is usually over UDP in the game world. Since you might lose a packet, that's a problem. Most games have "unreliable" packets, which, if lost, are superseded by later packets. ("Where is avatar now" is a typical use.) You'd like to have that stream on a higher quality of service than the others, if only ISPs and routers actually paid attention to that. Then you have the less-critical stuff, which needs reliability. ("Object X enters world" is a typical use.) I'd use TCP for that, but Second Life has its own not very good UDP-based protocol, with a fixed retransmit timer. Reliable delivery, in-order delivery, no head of line blocking - pick two. TCP chooses the first two, SL's protocol chooses the first and third ones. Out of order delivery after a retransmit can cause avatars to lose clothing items, because the child item arrived before the parent item. Then you have asset fetching. In Second Life/Open Simulator this is straight HTTP/1. But there are some unusual tricks. Textures are stored in progressive JPEG 2000. It's possible to open a connection and just read a few hundred bytes to get a low-rez version. Then, the client can stop reading for a while, put the low-rez version on screen, and wait to see if there's a need to keep reading, or just close the connection because a higher-rez version is not needed. The poor server has to tolerate a large number of stalled connections. Worse, the actual asset servers on AWS are front-ended by Akamai, which is optimized for browser-type behavior. Requesting an asset from an Akamai cache results in fetching the entire asset from AWS, even if only part of it is needed. There's a suspicion that large numbers of partial reads and stalled reads from clients sometimes causes Akamai's anti-DDOS detection to trip and throttle the data flow. So those are just some of the issues "the HTTP of VR" must handle. Most are known to MMO designers. The big difference in virtual worlds is there's far more dynamic asset loading. How well that's managed has a strong influence on how consistent the world looks. It has to be constantly re-prioritized as the viewpoint moves. (Demo, from my own work: https://vimeo.com/user28693218 https://vimeo.com/user28693218 This shows the client frantically trying to load the textures from the network before the camera gets close. Not all the tricks to make that look good are in this demo.) It's not an overwhelmingly hard problem, but botch it and you will be laughed off Steam.

gfxgirl 5y ago

I think there is a different problem that needs to be solved and it's probably impossible. I've dreamed of the metaverse since Snow Crash and maybe before (Tron?) but ... when it comes to actually making it, lets assume unlimited CPU/GPU power and unlimited memory. Ideally, I want the Metaverse to allow people to run their own code. Whether its VR or AR it's a shared 3D space. So I want my Nintendo "Nintendogs" to be able to run around my "Ikea furniture" with my "Google/Apple/OSM maps" showing me navigation directions and my "FB Messenger/Discord/iOS Messenger" letting me connect to people inside. In a webpage, each of these things runs in an IFRAME isolated from the other and browsers go to great lengths to disllow one spying on another. But in this 3D space my Nitendogs can't run through the space unless they can "sense the space". They need to know where the fire hydrants are, where the side walk is, what things they're allowed to climb/chew etc. But to do that effectively means they need enough info to spy on me. Same for all the other apps. I can use messaging apps on my phone with GPS off and full network access off so that the app can't know my location, but order for different apps in the Metaverse to do similar they'll need to know at least the virtual location of themselves and the stuff around them which is enough to track/fignerprint You can maybe get around some of this with a massive walled garden but that arguably is not the metaverse.

berkes 5y ago
You presume a push model. And compare it to a pull model (iframes). I think that is where the solutions are. The messages could be delivered as a simple XML feed. Your virtual home, or HUD knows where to place them. Through hyperlinks they know where to subscribe, or refresh, or get details. The messages don't need to know anything about placement and usage.
jayd16 5y ago
Seems like you could share collision meshes without much risk of spying, no?

usrbinbash 5y ago

What exactly is the "metaverse" supposed to be, other than a marketing term to sell a more expensive class of IO devices? People will not switch over in droves to do their text/image/video editing in VR all of a sudden, because other than a few special design applications, there is no point in doing so...it's slower, clumsier and the input devices are much less precise than mouse&keyboard. Another supposed target demographic, people in IT won't switch either. I see no point in virtually grabbing a glowing code-ball and throing it into the "deploy-tube", or navigate a codebase using haptic gestures with the huge meat-styluses at the end of my arms, when I can simply type `git push` or `/myAwesomeStruct` I also have a hard time imagining management sitting in meetings while wearing a 400g headset for 3h. Or companies being willing to cough up 350+$ for every employee just so they can join meetings, when Zoom is basically free. So, what else is there? Gaming and maybe some "recreational apps" (aka. alsogaming, only less interactive). And since not all games will take place in the same unified MMORPG-ish permanent universe (yes, people want to play in sessions, and people want to play single player, and people want to play while not connected to the internet), this will not be a paradigm-shift, but rather a new toy in an already large collection of other toys.

charcircuit 5y ago
Nothing is stopping you from using your keyboard in VR.
- usrbinbash 5y ago
  Nothing is stopping me from writing a google-ASR powered program hooked into my terminal, and start yelling my code into the computer either. But as long as this isn't faster, easier or more reliable than typing it on a keyboard, I won't. People don't work a certain way because it's possible. People work a certain way because it saves time, money, sanity, or simply because it's convenient. If wearing a VR headset while coding isn't providing substantial benefits over what my current system provides, why would I do it?
  - charcircuit 5y ago
    The difference is you can already use a keyboard in VR where your system sounds like you would have to go and mild it. >If wearing a VR headset while coding isn't providing substantial benefits over what my current system provides, why would I do it? You wouldn't.

Dirak 5y ago

Networking for multiplayer games is a super interesting problem space since games tend to be more sensitive to latency, packet loss, and the accuracy of game states between clients. The problems are even more pronounced in VR where noticeable latency or artifacts can cause motion sickness. In modern fighter games, the industry seems to be tending toward predictive lockstep networking. This is a type of networking where if the client doesn't receive the inputs of other clients from the server, it will "predict" those inputs (usually by replaying the last received input) to give the illusion of zero latency gameplay. The drawback being that you need to implement rollback in the case where the predicted input doesn't match the real received input. When poorly executed, this could look like jittery player movement with entities rubber banding and teleporting and cause artifacts, but when done properly is mostly unnoticeable. If you're interested in this domain, I recommend checking out https://www.ggpo.net/ https://www.ggpo.net/ which is the library used in many of the modern fighter games (notably Skullgirls). It also comes with an in depth explanation of how to implement predictive networking with rollback on your own https://drive.google.com/file/d/1cV0fY8e_SC1hIFF5E1rT8XRVRzPjU8W9/view https://drive.google.com/file/d/1cV0fY8e_SC1hIFF5E1rT8XRVRzP...

bullen 5y ago

The HTTP of VR is HTTP! http://fuse.rupy.se/about.html http://fuse.rupy.se/about.html You also need a P2P protocol (probably some binary UDP thing) for tick based data like limb positions if you want body language. But really VR is much less important for immersion than action MMO = Mario/Zelda with 1000+ players.

schmorptron 5y ago

Kinda Off-topic, but if anyone is looking to play around with building vr spaces or games, i recently found out about LÖVR[0] which is a sipmle lua-based open source VR "framework". Haven't had a chance to play with it but it seems other people like it! [0] https://lovr.org/ https://lovr.org/