40 ms·

React Renderer for Three.js

213 points by oleksiitwork 5y ago

beebeepka 5y ago
One of those things that make zero sense but use react, and that makes them cool enough to blog about.
- mlsarecmg 5y ago
  It has similar benefits as react-dom has for the dom. Less code, faster performance, less memory consumption, real interop with a growing eco system.
throw_m239339 5y ago
VRML anyone?
- mlsarecmg 5y ago
  you can't express a 3d scene with markup alone, that is why VRML didn't succeed. the JSX you see just masks function calls, it is not XML or HTML, but the true power is in components and hooks (useFrame, etc). a r3f component is self-contained, it will even subscribe to the render-loop. click into these two examples to see the difference: https://twitter.com/0xca0a/status/1426924274527477764 https://twitter.com/0xca0a/status/1426924274527477764
  - e12e 5y ago
    > you can't express a 3d scene with markup alone What?
    - mlsarecmg 5y ago
      you want your view to do something, to participate in the render loop, to animate, user interaction etc. look at the example on the main page: https://github.com/pmndrs/react-three-fiber#what-does-it-look-like https://github.com/pmndrs/react-three-fiber#what-does-it-loo...
      - dheera 5y ago
        So just do it with VRML5 like HTML5. <cube id="myCube" x="0" y="0" z="0" width="10px" height="10px" length="10px" onClick="someFoo()" onMouseOver="someOtherFoo()" onTouchStart="someBar()"> <video src="blah.mp4" rotateX="45deg" rotateY="30deg" autoplay="true" loop="true" onClick="document.querySelector('#myCube').rotateX(45);"> ... No reason the above can't be done. But many people will come up with lame excuses about why we shouldn't have nice things.
        mlsarecmg 5y ago
        i can't say why webgl took over, as well as threejs, but i like how close to the metal they are. i would also much rather have a lower level representation underneath instead of starting outright with a markup language. i think that's why vrml eventually faded. i generally don't see why 3d should be represented in html. as for react, it merely expresses threejs, and adds something which three doesn't have: self contained components that are now sharable. something like this: https://twitter.com/0xca0a/status/1394697847556149250 https://twitter.com/0xca0a/status/1394697847556149250 just didn't exist in the web previously.
      - e12e 5y ago
        I mean, this looks like a nice, more modern approach - but vrml did allow full interaction? For something that works in current browsers, see eg: https://github.com/create3000/x_ite/wiki/Sensing-viewer-actions https://github.com/create3000/x_ite/wiki/Sensing-viewer-acti...
- moron4hire 5y ago
  There's always one of you. It's not funny.
gentleman11 5y ago
So react builds a tree of react elements that react dom turns into html. So this must replace react dom? Reusable components are easy in a 3d engine like 3js. You can still program declaratively if you liked. It’s claim to outperform raw threejs is surely untrue. React is also bad for animations, and they recommend you sort of use reacts “back door” to do complex animations. 3d engines are all about animations. You could use redux easily without react if you liked. I bet you still have to learn a new API. This doesn’t seem that useful. Am I mistaken? (Honestly curious) That said, I bet it was interesting and pleasantly challenging to write
- runawaybottle 5y ago
  The interesting thing about switching renderers is that you are now free of the DOM. I just think web developers only know one api really well (the DOM and it’s offspring frameworks) that when you give us a blank canvas (no pun intended), we resort to the same data structure and api of what we’ve always known. We’re truly free to make a <Modal /> however we like. Perhaps not even in those tags. We just don’t know it yet.
  - eyelidlessness 5y ago
    JSX has very little to do with the DOM. It’s always been a language extension to provide render-agnostic expressions. Non-DOM renderers have existed for years (React Native, smart TV, CLI, PDF, PowerPoint, the list keeps going…). The reality is it’s a good abstraction because it’s so decoupled from the DOM, but it’s associated with the DOM and web mainly for historical reasons. It’s also worth noting that there are similar extensions to other languages that are similarly render-agnostic.
    - runawaybottle 5y ago
      So I have a to contest that a bit. It parallels the DOM in structure entirely. You can see how this is the truth when you want to have components outside of a parent-child hierarchy belong to the same ‘component’, you have to use this very abstract concept of a Portal, as in, nothing is natural once you start thinking out of the tree structure. Certainly you can map this api to a variety of other APis, which I’m sure is what they did with React Native, but it was built with the DOM tree structure in mind. I think it’s one of the most brilliant things frontend has ever created along with Jquery, but they are both slaves to the DOM.
miklosme 5y ago
When AR/VR finally happens, UI developers will have to deal with complexity from a completely different paradigm. For me, React's biggest strength has always been its ability to organize complexity into a manageable order. Combine this with the large pool of developers and extensive ecosystem, I think React will be the go-to tool for AR/VR apps. For this reason, I'm super hyped for R3F.
- deleted 5y ago
  [deleted]
- nobbis 5y ago
  As a counterpoint, I've enjoyed building VR on the web using C++ (with WebAssembly/WebGL/WebXR) and not having to touch the DOM or JS, see: https://twitter.com/nobbis/status/1425266634982248451 https://twitter.com/nobbis/status/1425266634982248451 Benefits include complete control at frame and pixel level, being cross-platform (same code runs on web, iOS, macOS, Linux), and having access to third-party C/C++ libraries for 3D graphics.
  - brailsafe 5y ago
    This looks really neat
  - nightski 5y ago
    I have not put a lot of time into learning WebAssembly. But isn't WebGL a JavaScript API? Meaning wouldn't you be going from WebAssembly -> JavaScript engine -> WebGL? I was under the impression WebAssembly had no access to the outside world and could only access the relevant JavaScript APIs. But if it is true you can basically do WebAssembly -> native GL then that would be amazing.
    - nobbis 5y ago
      You're correct. WebGL does require extra validation compared to native GL, but it's effectively the same API as OpenGL ES 2.0/3.0 and Emscripten handles the translation from C/C++ for you. There's some overhead but it's negligible (assuming you're not making overly redundant API calls.)
deleted 5y ago
[deleted]
deleted 5y ago
[deleted]
franciscop 5y ago
We're using these at my job, and it's been a pretty amazing experience. We built a wrapper[1] so that every front-end dev can build components even easier (without needing to know Three.js), since in the end of the day they are just React components that can be easily composed together: <View3D> <Box position={[1, 0, 0]} color="green" /> <Box position={[-1, 0, 0]} color="red" /> </View3D> [1] https://www.npmjs.com/package/@standard/view https://www.npmjs.com/package/@standard/view
tills13 5y ago
Is there an error in the examples? You have const mesh = useRef() ... <mesh ref={mesh} ... You'll be rendering an undefined element (before the ref has a chance to attach). Also, the TypeScript example makes me head hurt. const ref = useRef(null!) Ah, yes. A non-null null literal.
- shallowthought 5y ago
  I have to wonder: Have we gone too far? Is TypeScript becoming more tedious than helpful? Has React lost its way in its blind zeal for pure functions? ...no, no, definitely not. Must've gone crazy for a second there.
  - tills13 5y ago
    I think I blame JavaScript for this mess, not the other way around.
- CraftThatBlock 5y ago
  I'm guessing they are overloading the ref's usage with the custom renderer. I believe that a renderer can accept non-function/class components in theory, which is why this works. I don't really like this though, although interesting, it feels like a hack and might not integrate well with tooling (e.g. the Typescript example). How I would have implemented it was exposition a Mesh component which wraps this, and a useMesh hook to wrap and type the ref.
  - yashap 5y ago
    Yeah, I don’t have experience with this lib, but the example code looks overly magic. But I’m also more of a backend dev, and find the useRef API to be inherently confusing, so that’s probably part of it.
  - mlsarecmg 5y ago
    r3f is a custom renderer, there is no difference in useRef between a div or a mesh, refs give you the underlying object. lowercase elements are native elements (div, span, mesh, view, box), they are defined by the renderer. uppercase is for components.
nojvek 5y ago
How are they making custom Tags with lowercase names. I assumed react would render that as native tags.
- mlsarecmg 5y ago
  this is a custom renderer. "react" does not know what a "div" is, this comes from "react-dom", that is why they have split these two packages apart. but they also allow you to make your own renderer (https://github.com/facebook/react/tree/main/packages/react-reconciler https://github.com/facebook/react/tree/main/packages/react-r...) which then defines its own elements. you can try a mini threejs renderer here: https://codesandbox.io/s/reurope-reconciler-hd16y https://codesandbox.io/s/reurope-reconciler-hd16y
arcanon 5y ago
The age of WebGPU is almost upon us. Chrome 94 comes out this quarter. I doubt React will be the major player in this next epoch. It will be something new that can pull off an instant load shared world game.
- Jasper_ 5y ago
  In my experience ths issue with this has always been asset download speeds, not web frameworks. The lack of a storage solution means we can't ship game quality assets and have things be instant load.. I run a large WebGL application and the Chrome network cache is still my biggest issue ( e.g. this unsolved bug despite me having an 100% repro https://bugs.chromium.org/p/chromium/issues/detail?id=770694 https://bugs.chromium.org/p/chromium/issues/detail?id=770694 )
  - rock_hard 5y ago
    Agree that the framework issnt the bottleneck here but asset loading is! We have been investing in building ourselves a asset pipeline backend and it’s been a dramatic improvement. Simply scripting Blender and other tools on a EC2 instance to optimize assets and then deliver them via cloudfront. Works really well
    - runawaybottle 5y ago
      I think we are not verbalizing the elephant in the room. I know game developers have their ship tight and will use the gpu to it’s fullest (the proof is in the pudding). The thing I’ve been trying say all along is, I want applications using the same renderer at a primitive level.
  - dakom 5y ago
    Yes, I agree... case in point - Unity's Wasm/Webgl exporter works really well for the most part, especially if you test and develop specifically for that target - but we don't see it taking off the way Flash did because nobody wants to wait 10 mins for their page to load.
- 5y ago
shadowgovt 5y ago
This is very cool. Sounds like there's room for performance improvements, but those can happen now that the framework exists.
deleted 5y ago
[deleted]
irevdev 5y ago
For what its worth I've been very impressed and happy with what I've been able to achieve with R3F [1] Its an online IDE for a number of CodeCAD packages to help lower the barrier to this paradigm.[2] Going from knowing nothing about 3d, just hacking up example code it's been easy to put something respectable together without much dedicated learning. Im super grateful to the pmndrs team [1] https://cadhub.xyz/dev-ide/cadquery https://cadhub.xyz/dev-ide/cadquery [2] https://news.ycombinator.com/item?id=27649270 https://news.ycombinator.com/item?id=27649270

213 points by oleksiitwork 5y ago

beebeepka 5y ago

One of those things that make zero sense but use react, and that makes them cool enough to blog about.

mlsarecmg 5y ago
It has similar benefits as react-dom has for the dom. Less code, faster performance, less memory consumption, real interop with a growing eco system.

throw_m239339 5y ago

VRML anyone?

mlsarecmg 5y ago
you can't express a 3d scene with markup alone, that is why VRML didn't succeed. the JSX you see just masks function calls, it is not XML or HTML, but the true power is in components and hooks (useFrame, etc). a r3f component is self-contained, it will even subscribe to the render-loop. click into these two examples to see the difference: https://twitter.com/0xca0a/status/1426924274527477764 https://twitter.com/0xca0a/status/1426924274527477764
- e12e 5y ago
  > you can't express a 3d scene with markup alone What?
  - mlsarecmg 5y ago
    you want your view to do something, to participate in the render loop, to animate, user interaction etc. look at the example on the main page: https://github.com/pmndrs/react-three-fiber#what-does-it-look-like https://github.com/pmndrs/react-three-fiber#what-does-it-loo...
    - dheera 5y ago
      So just do it with VRML5 like HTML5. <cube id="myCube" x="0" y="0" z="0" width="10px" height="10px" length="10px" onClick="someFoo()" onMouseOver="someOtherFoo()" onTouchStart="someBar()"> <video src="blah.mp4" rotateX="45deg" rotateY="30deg" autoplay="true" loop="true" onClick="document.querySelector('#myCube').rotateX(45);"> ... No reason the above can't be done. But many people will come up with lame excuses about why we shouldn't have nice things.
      - mlsarecmg 5y ago
        i can't say why webgl took over, as well as threejs, but i like how close to the metal they are. i would also much rather have a lower level representation underneath instead of starting outright with a markup language. i think that's why vrml eventually faded. i generally don't see why 3d should be represented in html. as for react, it merely expresses threejs, and adds something which three doesn't have: self contained components that are now sharable. something like this: https://twitter.com/0xca0a/status/1394697847556149250 https://twitter.com/0xca0a/status/1394697847556149250 just didn't exist in the web previously.
    - e12e 5y ago
      I mean, this looks like a nice, more modern approach - but vrml did allow full interaction? For something that works in current browsers, see eg: https://github.com/create3000/x_ite/wiki/Sensing-viewer-actions https://github.com/create3000/x_ite/wiki/Sensing-viewer-acti...
moron4hire 5y ago
There's always one of you. It's not funny.

gentleman11 5y ago

So react builds a tree of react elements that react dom turns into html. So this must replace react dom? Reusable components are easy in a 3d engine like 3js. You can still program declaratively if you liked. It’s claim to outperform raw threejs is surely untrue. React is also bad for animations, and they recommend you sort of use reacts “back door” to do complex animations. 3d engines are all about animations. You could use redux easily without react if you liked. I bet you still have to learn a new API. This doesn’t seem that useful. Am I mistaken? (Honestly curious) That said, I bet it was interesting and pleasantly challenging to write

runawaybottle 5y ago
The interesting thing about switching renderers is that you are now free of the DOM. I just think web developers only know one api really well (the DOM and it’s offspring frameworks) that when you give us a blank canvas (no pun intended), we resort to the same data structure and api of what we’ve always known. We’re truly free to make a <Modal /> however we like. Perhaps not even in those tags. We just don’t know it yet.
- eyelidlessness 5y ago
  JSX has very little to do with the DOM. It’s always been a language extension to provide render-agnostic expressions. Non-DOM renderers have existed for years (React Native, smart TV, CLI, PDF, PowerPoint, the list keeps going…). The reality is it’s a good abstraction because it’s so decoupled from the DOM, but it’s associated with the DOM and web mainly for historical reasons. It’s also worth noting that there are similar extensions to other languages that are similarly render-agnostic.
  - runawaybottle 5y ago
    So I have a to contest that a bit. It parallels the DOM in structure entirely. You can see how this is the truth when you want to have components outside of a parent-child hierarchy belong to the same ‘component’, you have to use this very abstract concept of a Portal, as in, nothing is natural once you start thinking out of the tree structure. Certainly you can map this api to a variety of other APis, which I’m sure is what they did with React Native, but it was built with the DOM tree structure in mind. I think it’s one of the most brilliant things frontend has ever created along with Jquery, but they are both slaves to the DOM.

miklosme 5y ago

When AR/VR finally happens, UI developers will have to deal with complexity from a completely different paradigm. For me, React's biggest strength has always been its ability to organize complexity into a manageable order. Combine this with the large pool of developers and extensive ecosystem, I think React will be the go-to tool for AR/VR apps. For this reason, I'm super hyped for R3F.

deleted 5y ago
[deleted]
nobbis 5y ago
As a counterpoint, I've enjoyed building VR on the web using C++ (with WebAssembly/WebGL/WebXR) and not having to touch the DOM or JS, see: https://twitter.com/nobbis/status/1425266634982248451 https://twitter.com/nobbis/status/1425266634982248451 Benefits include complete control at frame and pixel level, being cross-platform (same code runs on web, iOS, macOS, Linux), and having access to third-party C/C++ libraries for 3D graphics.
- brailsafe 5y ago
  This looks really neat
- nightski 5y ago
  I have not put a lot of time into learning WebAssembly. But isn't WebGL a JavaScript API? Meaning wouldn't you be going from WebAssembly -> JavaScript engine -> WebGL? I was under the impression WebAssembly had no access to the outside world and could only access the relevant JavaScript APIs. But if it is true you can basically do WebAssembly -> native GL then that would be amazing.
  - nobbis 5y ago
    You're correct. WebGL does require extra validation compared to native GL, but it's effectively the same API as OpenGL ES 2.0/3.0 and Emscripten handles the translation from C/C++ for you. There's some overhead but it's negligible (assuming you're not making overly redundant API calls.)

deleted 5y ago

[deleted]

deleted 5y ago

[deleted]

franciscop 5y ago

We're using these at my job, and it's been a pretty amazing experience. We built a wrapper[1] so that every front-end dev can build components even easier (without needing to know Three.js), since in the end of the day they are just React components that can be easily composed together: <View3D> <Box position={[1, 0, 0]} color="green" /> <Box position={[-1, 0, 0]} color="red" /> </View3D> [1] https://www.npmjs.com/package/@standard/view https://www.npmjs.com/package/@standard/view

tills13 5y ago

Is there an error in the examples? You have const mesh = useRef() ... <mesh ref={mesh} ... You'll be rendering an undefined element (before the ref has a chance to attach). Also, the TypeScript example makes me head hurt. const ref = useRef(null!) Ah, yes. A non-null null literal.

shallowthought 5y ago
I have to wonder: Have we gone too far? Is TypeScript becoming more tedious than helpful? Has React lost its way in its blind zeal for pure functions? ...no, no, definitely not. Must've gone crazy for a second there.
- tills13 5y ago
  I think I blame JavaScript for this mess, not the other way around.
CraftThatBlock 5y ago
I'm guessing they are overloading the ref's usage with the custom renderer. I believe that a renderer can accept non-function/class components in theory, which is why this works. I don't really like this though, although interesting, it feels like a hack and might not integrate well with tooling (e.g. the Typescript example). How I would have implemented it was exposition a Mesh component which wraps this, and a useMesh hook to wrap and type the ref.
- yashap 5y ago
  Yeah, I don’t have experience with this lib, but the example code looks overly magic. But I’m also more of a backend dev, and find the useRef API to be inherently confusing, so that’s probably part of it.
- mlsarecmg 5y ago
  r3f is a custom renderer, there is no difference in useRef between a div or a mesh, refs give you the underlying object. lowercase elements are native elements (div, span, mesh, view, box), they are defined by the renderer. uppercase is for components.

nojvek 5y ago

How are they making custom Tags with lowercase names. I assumed react would render that as native tags.

mlsarecmg 5y ago
this is a custom renderer. "react" does not know what a "div" is, this comes from "react-dom", that is why they have split these two packages apart. but they also allow you to make your own renderer (https://github.com/facebook/react/tree/main/packages/react-reconciler https://github.com/facebook/react/tree/main/packages/react-r...) which then defines its own elements. you can try a mini threejs renderer here: https://codesandbox.io/s/reurope-reconciler-hd16y https://codesandbox.io/s/reurope-reconciler-hd16y

arcanon 5y ago

The age of WebGPU is almost upon us. Chrome 94 comes out this quarter. I doubt React will be the major player in this next epoch. It will be something new that can pull off an instant load shared world game.

Jasper_ 5y ago
In my experience ths issue with this has always been asset download speeds, not web frameworks. The lack of a storage solution means we can't ship game quality assets and have things be instant load.. I run a large WebGL application and the Chrome network cache is still my biggest issue ( e.g. this unsolved bug despite me having an 100% repro https://bugs.chromium.org/p/chromium/issues/detail?id=770694 https://bugs.chromium.org/p/chromium/issues/detail?id=770694 )
- rock_hard 5y ago
  Agree that the framework issnt the bottleneck here but asset loading is! We have been investing in building ourselves a asset pipeline backend and it’s been a dramatic improvement. Simply scripting Blender and other tools on a EC2 instance to optimize assets and then deliver them via cloudfront. Works really well
  - runawaybottle 5y ago
    I think we are not verbalizing the elephant in the room. I know game developers have their ship tight and will use the gpu to it’s fullest (the proof is in the pudding). The thing I’ve been trying say all along is, I want applications using the same renderer at a primitive level.
- dakom 5y ago
  Yes, I agree... case in point - Unity's Wasm/Webgl exporter works really well for the most part, especially if you test and develop specifically for that target - but we don't see it taking off the way Flash did because nobody wants to wait 10 mins for their page to load.
5y ago

shadowgovt 5y ago

This is very cool. Sounds like there's room for performance improvements, but those can happen now that the framework exists.

deleted 5y ago

[deleted]

irevdev 5y ago

For what its worth I've been very impressed and happy with what I've been able to achieve with R3F [1] Its an online IDE for a number of CodeCAD packages to help lower the barrier to this paradigm.[2] Going from knowing nothing about 3d, just hacking up example code it's been easy to put something respectable together without much dedicated learning. Im super grateful to the pmndrs team [1] https://cadhub.xyz/dev-ide/cadquery https://cadhub.xyz/dev-ide/cadquery [2] https://news.ycombinator.com/item?id=27649270 https://news.ycombinator.com/item?id=27649270