6 ms·

A browser plugin for Unity

154 points by imoldfella 6y ago

nwah1 6y ago
You can also render Unity inside of Firefox. Infinite recursion potential.
- samanator 6y ago
  Do it please
- agumonkey 6y ago
  w3c webdroste RFC
- wtmt 6y ago
  Firefox Unity Browser Addon Renderer?
  - high_density 6y ago
    with web-assembly, we might get: Firefox in Unreal in Firefox in Unity in Firefox...
bzb3 6y ago
More like Servo in Unity... And from what I recall, Servo is not even an alpha at this point
- tashoecraft 6y ago
  I don’t believe servo is ever intended to be more then a testbed for integrating components into Firefox core. It’s modular, and pieces of it have already been merged and are used in Firefox. So my interpretation of this is that they are using their more modern code to test out this feature and if it’s solid enough and useful enough it can be pulled into core.
  - sitkack 6y ago
    Given the complexity of a browser, having a test bed to vet ideas and techniques before trying to renovate the Tower of Babel while folks are using it makes a ton of sense. So much of what the folks around Rust have done is really the systematic and pervasive application of science. A shining example of what humans are capable of.
  - mook 6y ago
    I thought they _originally_ intended for servo to perhaps someday be its own thing; I had this impression specifically because I think they had at some point aspirations for servo to have a stable embedding API (to let other applications use it as a rendering engine). Of course I don't think that's the case anymore.
    - chrismorgan 6y ago
      There was explicitly never any plan for Servo to be a product on its own. Note that this does not mean that there was an explicit plan for Servo never to be a product.
twoodfin 6y ago
I’m a little disappointed the WebGL demo didn’t show what happens when the “outer” viewport moves.
meesterdude 6y ago
This is something I've wanted for years, for numerous uses like collaboration, Method of loci, and documentation. Overlaying our built-in spatial awareness with focused data is an area of information architecture with a lot of untapped potential. there is also this which allows multiple windows within unity (there's a demo you can try): https://zenfulcrum.com/browser https://zenfulcrum.com/browser
- chrisweekly 6y ago
  "Overlaying our built-in spatial awareness with focused data is an area of information architecture with a lot of untapped potential." +1 for this; couldn't agree more
Guest81 6y ago
This seems similar to ZFBrowser which implements chromium in unity, and is used by Tabletop Simulator. I wonder how the performance compares?
chrismorgan 6y ago
The current title here on HN of “Firefox in Unity” is incorrect. This is based on Servo, not Firefox. From what I understand as a very casual observer that hasn’t been keeping up with things, Servo’s architecture is rather more conducive to this application both in embedding convenience and rendering, being able to render pages in a way that will make them perfect quality regardless of your 3D perspective, whereas I imagine Firefox might still be at the “rasterise the viewport and use it as a tile” level, though maybe WebRender (applying Servo’s renderer to Firefox) has resolved that? As I say, I haven’t been keeping up with things. Anyway, the correct title for the article is “A browser plugin for Unity”.
- c-smile 6y ago
  > "rasterise the viewport and use it as a tile" Yes, rendering HTML/CSS using solely Unity's backend is problematic at best. For the Unreal Engine though it is perfectly feasible. Here is Sciter Engine (HTML/CSS/script) rendered directly into DirectX scene: https://sciter.com/see-sciter-lite-unreal-engine-in-theaters-near-you/ https://sciter.com/see-sciter-lite-unreal-engine-in-theaters... using Sciter's DX rendering backend.
- dang 6y ago
  Ok, we've changed the title to what the article says. "Please use the original title, unless it is misleading or linkbait; don't editorialize." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
  - andybak 6y ago
    Whilst I understand the motivation behind this guideline, it seems to be applied with little regards for it's usefulness. Post titles need more context than the original article title because posting to HN strips out most of that original context.
    - leeoniya 6y ago
      agreed. this policy applied as dogma does more harm than good on a frequent basis here. especially given the clear nature of this article where there's no debatable nuance.
dleslie 6y ago
What are the security guarantees about this?
- devwastaken 6y ago
  Exactly this. Vrchat had chromium for a while, but exploits got it removed. Itd be great if it was just a static browser engine, no Js, with a strong sandbox. So at most all that can be done is messing with the browsers runtime memory itself, not the system it's on.
  - mhh__ 6y ago
    What kind of exploits? Was the browser not isolated at all or was it (say) sandbox-escaping via the JS?
    - deleted 6y ago
      [deleted]
    - dleslie 6y ago
      We posted this: https://medium.com/@vrchat/security-update-web-panels-4699fa0be103 https://medium.com/@vrchat/security-update-web-panels-4699fa... It was severe enough that I ripped them out, though we were unable to find any evidence of the vulnerability in use.
      - mmmkkaaayy 6y ago
        But what was the vulnerability? It's not mentioned in the post.
        dleslie 6y ago
        It's been a few years, and I can't recall the specifics with certainty. It was a serious Chromium SVE that was unpatched in the middleware we were using.
  - dleslie 6y ago
ourcat 6y ago
I remember when this sort of functionality arrived in Second Life. It enabled a lot of easy 'shared experiences'. eg: Watching a live video stream of an event with people, or group presentations etc.
- Animats 6y ago
  Yes. SL launches an instance of Chromium as a subprocess in the viewer for each object face that's an HTTP browser. I wonder if Servo could do that with less overhead.
- skavi 6y ago
  Per callumprentice, Second Life uses Dullahan[0]. [0]: https://bitbucket.org/lindenlab/dullahan/src/master/ https://bitbucket.org/lindenlab/dullahan/src/master/
arendtio 6y ago
What is wrong when something cool like this triggers a response along the lines of 'finally I can get tracked while being in-game'?
robterrell 6y ago
As the author of the first web browser plugin for Unity (HTMLTexture) I must say I'm very impressed by this. Kudos. Sounds like Unity has changed enough under the hood that the technique I used (calling gl_bind() behind Unity's back, using the texture ID you pass in, to basically write to the texture Unity is using for your mesh) no longer works.
callumprentice 6y ago
For anyone else looking to embed a modern browser in their Windows/macOS/Linux* C++ application, you might consider Dullahan: source: https://bitbucket.org/lindenlab/dullahan/src/master/ https://bitbucket.org/lindenlab/dullahan/src/master/ and example screenshot: https://bitbucket.org/lindenlab/dullahan/raw/master/docs/webcube.png https://bitbucket.org/lindenlab/dullahan/raw/master/docs/web... I recently updated it to use Chromium 81 and would have updated it to the most recent version (84) if there weren't some non-trivial changes needed to make video/audio streaming work again. It's what we use in Second Life and would welcome other users' comments to help improve it.
gfxgirl 6y ago
I'm sure having a browser in game as a million uses but ... Oculus Rift has in game browser built into their OS. In any game and any time I can pull up a browser in VR (or any desktop app). Pin a walkthru up. Pin up video chat with a friend. It seems like browser in VR would be better served at an OS level. The OS could provide a way for the app to request VR be placed in the scene. Ideally it happens in a way the app can't look at the screen since a game shouldn't be able to look at the contents of your browser window.
jayd16 6y ago
>As we do not have control over the binding of the texture and the Unity context, the current design for updating this texture uses a blit (copy) via Servo’s surfman-chains API I don't think this is necessary. You should be able to make Texture objects that ref existing native textures in Unity and I've been able to use OES_EGL_image_external extensions to sample Android decoded video textures sampled from Unity shaders. Its not the most widely available option but the perf is much better.
- phil_moz 6y ago
  The issue is not access to an external texture, it's access to an external texture on the right thread in the right GL context. We did look at writing a new backend for surfman to integrate directly, but it's a bit of work and not at all cross-platform, so this was not a goal of the project at this stage. I'd like to find a way to avoid the blit though, so I'm continuing to experiment in this direction.

154 points by imoldfella 6y ago

nwah1 6y ago

You can also render Unity inside of Firefox. Infinite recursion potential.

samanator 6y ago
Do it please
agumonkey 6y ago
w3c webdroste RFC
wtmt 6y ago
Firefox Unity Browser Addon Renderer?
- high_density 6y ago
  with web-assembly, we might get: Firefox in Unreal in Firefox in Unity in Firefox...

bzb3 6y ago

More like Servo in Unity... And from what I recall, Servo is not even an alpha at this point

tashoecraft 6y ago
I don’t believe servo is ever intended to be more then a testbed for integrating components into Firefox core. It’s modular, and pieces of it have already been merged and are used in Firefox. So my interpretation of this is that they are using their more modern code to test out this feature and if it’s solid enough and useful enough it can be pulled into core.
- sitkack 6y ago
  Given the complexity of a browser, having a test bed to vet ideas and techniques before trying to renovate the Tower of Babel while folks are using it makes a ton of sense. So much of what the folks around Rust have done is really the systematic and pervasive application of science. A shining example of what humans are capable of.
- mook 6y ago
  I thought they _originally_ intended for servo to perhaps someday be its own thing; I had this impression specifically because I think they had at some point aspirations for servo to have a stable embedding API (to let other applications use it as a rendering engine). Of course I don't think that's the case anymore.
  - chrismorgan 6y ago
    There was explicitly never any plan for Servo to be a product on its own. Note that this does not mean that there was an explicit plan for Servo never to be a product.

twoodfin 6y ago

I’m a little disappointed the WebGL demo didn’t show what happens when the “outer” viewport moves.

meesterdude 6y ago

This is something I've wanted for years, for numerous uses like collaboration, Method of loci, and documentation. Overlaying our built-in spatial awareness with focused data is an area of information architecture with a lot of untapped potential. there is also this which allows multiple windows within unity (there's a demo you can try): https://zenfulcrum.com/browser https://zenfulcrum.com/browser

chrisweekly 6y ago
"Overlaying our built-in spatial awareness with focused data is an area of information architecture with a lot of untapped potential." +1 for this; couldn't agree more

Guest81 6y ago

This seems similar to ZFBrowser which implements chromium in unity, and is used by Tabletop Simulator. I wonder how the performance compares?

chrismorgan 6y ago

The current title here on HN of “Firefox in Unity” is incorrect. This is based on Servo, not Firefox. From what I understand as a very casual observer that hasn’t been keeping up with things, Servo’s architecture is rather more conducive to this application both in embedding convenience and rendering, being able to render pages in a way that will make them perfect quality regardless of your 3D perspective, whereas I imagine Firefox might still be at the “rasterise the viewport and use it as a tile” level, though maybe WebRender (applying Servo’s renderer to Firefox) has resolved that? As I say, I haven’t been keeping up with things. Anyway, the correct title for the article is “A browser plugin for Unity”.

c-smile 6y ago
> "rasterise the viewport and use it as a tile" Yes, rendering HTML/CSS using solely Unity's backend is problematic at best. For the Unreal Engine though it is perfectly feasible. Here is Sciter Engine (HTML/CSS/script) rendered directly into DirectX scene: https://sciter.com/see-sciter-lite-unreal-engine-in-theaters-near-you/ https://sciter.com/see-sciter-lite-unreal-engine-in-theaters... using Sciter's DX rendering backend.
dang 6y ago
Ok, we've changed the title to what the article says. "Please use the original title, unless it is misleading or linkbait; don't editorialize." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- andybak 6y ago
  Whilst I understand the motivation behind this guideline, it seems to be applied with little regards for it's usefulness. Post titles need more context than the original article title because posting to HN strips out most of that original context.
  - leeoniya 6y ago
    agreed. this policy applied as dogma does more harm than good on a frequent basis here. especially given the clear nature of this article where there's no debatable nuance.

dleslie 6y ago

What are the security guarantees about this?

devwastaken 6y ago
Exactly this. Vrchat had chromium for a while, but exploits got it removed. Itd be great if it was just a static browser engine, no Js, with a strong sandbox. So at most all that can be done is messing with the browsers runtime memory itself, not the system it's on.
- mhh__ 6y ago
  What kind of exploits? Was the browser not isolated at all or was it (say) sandbox-escaping via the JS?
  - deleted 6y ago
    [deleted]
  - dleslie 6y ago
    We posted this: https://medium.com/@vrchat/security-update-web-panels-4699fa0be103 https://medium.com/@vrchat/security-update-web-panels-4699fa... It was severe enough that I ripped them out, though we were unable to find any evidence of the vulnerability in use.
    - mmmkkaaayy 6y ago
      But what was the vulnerability? It's not mentioned in the post.
      - dleslie 6y ago
        It's been a few years, and I can't recall the specifics with certainty. It was a serious Chromium SVE that was unpatched in the middleware we were using.
- dleslie 6y ago

ourcat 6y ago

I remember when this sort of functionality arrived in Second Life. It enabled a lot of easy 'shared experiences'. eg: Watching a live video stream of an event with people, or group presentations etc.

Animats 6y ago
Yes. SL launches an instance of Chromium as a subprocess in the viewer for each object face that's an HTTP browser. I wonder if Servo could do that with less overhead.
skavi 6y ago
Per callumprentice, Second Life uses Dullahan[0]. [0]: https://bitbucket.org/lindenlab/dullahan/src/master/ https://bitbucket.org/lindenlab/dullahan/src/master/

arendtio 6y ago

What is wrong when something cool like this triggers a response along the lines of 'finally I can get tracked while being in-game'?

robterrell 6y ago

As the author of the first web browser plugin for Unity (HTMLTexture) I must say I'm very impressed by this. Kudos. Sounds like Unity has changed enough under the hood that the technique I used (calling gl_bind() behind Unity's back, using the texture ID you pass in, to basically write to the texture Unity is using for your mesh) no longer works.

callumprentice 6y ago

For anyone else looking to embed a modern browser in their Windows/macOS/Linux* C++ application, you might consider Dullahan: source: https://bitbucket.org/lindenlab/dullahan/src/master/ https://bitbucket.org/lindenlab/dullahan/src/master/ and example screenshot: https://bitbucket.org/lindenlab/dullahan/raw/master/docs/webcube.png https://bitbucket.org/lindenlab/dullahan/raw/master/docs/web... I recently updated it to use Chromium 81 and would have updated it to the most recent version (84) if there weren't some non-trivial changes needed to make video/audio streaming work again. It's what we use in Second Life and would welcome other users' comments to help improve it.

gfxgirl 6y ago

I'm sure having a browser in game as a million uses but ... Oculus Rift has in game browser built into their OS. In any game and any time I can pull up a browser in VR (or any desktop app). Pin a walkthru up. Pin up video chat with a friend. It seems like browser in VR would be better served at an OS level. The OS could provide a way for the app to request VR be placed in the scene. Ideally it happens in a way the app can't look at the screen since a game shouldn't be able to look at the contents of your browser window.

jayd16 6y ago

>As we do not have control over the binding of the texture and the Unity context, the current design for updating this texture uses a blit (copy) via Servo’s surfman-chains API I don't think this is necessary. You should be able to make Texture objects that ref existing native textures in Unity and I've been able to use OES_EGL_image_external extensions to sample Android decoded video textures sampled from Unity shaders. Its not the most widely available option but the perf is much better.

phil_moz 6y ago
The issue is not access to an external texture, it's access to an external texture on the right thread in the right GL context. We did look at writing a new backend for surfman to integrate directly, but it's a bit of work and not at all cross-platform, so this was not a goal of the project at this stage. I'd like to find a way to avoid the blit though, so I'm continuing to experiment in this direction.