6 ms·
What’s the status on companies building AI models to build actual 3D backend behind these generative videos. Anyone working on something similar? Imagine that’d
by lvl102 2y ago
What’s the status on companies building AI models to build actual 3D backend behind these generative videos. Anyone working on something similar? Imagine that’d be far more productive. For example, lookdev mlop is pretty low hanging fruit. Not sure why we don’t already have models from Autodesk, Epic or even Adobe (with resources ie A100/H100) where you upload an image/video and the model spits out workable 3D scaffolds.
- huytersd 2y agoIf I’m not mistaken, Stability just released something like that a few days ago.
- lvl102 2y agoLooks like I completely overlooked threestudio released last year. Thank you for pointing it out.
- huytersd 2y agoIt’s something I’ve been interested in too. I do a bunch of CNC woodworking so would love the ability to atleast generate close enough 3D models I can then refine.
- Legend2440 2y agoYes, but it works by generating a video first and doing photogrammetry on it to produce a 3D model.
- stale2002 2y agoThis is a good question, and the answer is that from a tech side it is surprisingly easier to solve the problem in the reverse direction. As in, making workable 3d models is harder than making video. And it is easier to make a 3d model by generating a video of the object instead. Why is that? I don't know. But that's the current state of the industry. 3D model generation is simply harder.
- lvl102 2y agoI am thinking reinforcement on top of Blender would be straight forward with unlimited synthetic data potential. I’ve come across people incorporating SD into rendering workflow so tools are all there.
- euW3EeBe 2y agoProbably also helps that there's way more image/video data to train on than 3D data.
- htrp 2y agoNerfs and splatting