20 ms·

I've spent a lot of time thinking about this (i.e. taking a video and creating a 3D scene) and I don't think that it is feasible in most cases to have good accu

by c4wrd 4y ago

I've spent a lot of time thinking about this (i.e. taking a video and creating a 3D scene) and I don't think that it is feasible in most cases to have good accuracy. If you need to infer the angle, you need make a lot of biased assumptions about things like velocity, position, etc., of the camera and even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time. Now I'm not saying it's not possible, but I'd believe that if you want an accurate 3D scene, you'd rather be spending your computation budget on things other than determining those angles when it can be simply be provided by hardware.

krasin 4y ago
https://github.com/NVLabs/instant-ngp https://github.com/NVLabs/instant-ngp has a script that converts a video into frames and then uses COLMAP ([1]) to compute camera poses. You can then train a NeRF model within a few seconds. It all works pretty well. Trying it on your own video is pretty straightforward. 1. https://colmap.github.io/ https://colmap.github.io/
riotnrrd 4y ago
You're far too pessimistic (or maybe you don't know the field well). The problem of estimating the relative poses of the cameras responsible for a set of photos is a long standing and essentially "solved" problem in computer vision. I say "solved" because there is still active research (increasing accuracy, faster, more robust, etc.) but there are decades-old, well known techniques that any dedicated programmer could implement in a week. If you're genuinely curious, look into structure from motion, visual odometry, or SLAM.
doliveira 4y ago
> even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time Not really, with SLAM there are various algorithms to keep inaccuracy in check. Basically it works by a feedback loop of guessing an estimate for position and then updating it using landmarks.

by c4wrd 4y ago

krasin 4y ago

https://github.com/NVLabs/instant-ngp https://github.com/NVLabs/instant-ngp has a script that converts a video into frames and then uses COLMAP ([1]) to compute camera poses. You can then train a NeRF model within a few seconds. It all works pretty well. Trying it on your own video is pretty straightforward. 1. https://colmap.github.io/ https://colmap.github.io/

riotnrrd 4y ago

You're far too pessimistic (or maybe you don't know the field well). The problem of estimating the relative poses of the cameras responsible for a set of photos is a long standing and essentially "solved" problem in computer vision. I say "solved" because there is still active research (increasing accuracy, faster, more robust, etc.) but there are decades-old, well known techniques that any dedicated programmer could implement in a week. If you're genuinely curious, look into structure from motion, visual odometry, or SLAM.

doliveira 4y ago

> even if you were 99.9% accurate, that 0.1% inaccuracy is compounded over time Not really, with SLAM there are various algorithms to keep inaccuracy in check. Basically it works by a feedback loop of guessing an estimate for position and then updating it using landmarks.