6 ms·

Hey HN, this project started out when I was exploring rendering radiance fields in real-time for standalone VR and AR headsets. I was frustrated by the lack of

by 0x02A 3y ago

Hey HN, this project started out when I was exploring rendering radiance fields in real-time for standalone VR and AR headsets. I was frustrated by the lack of performant, yet non-CUDA, implementations. Also, this would be a good excuse to learn about compute pipelines in Vulkan. Right now, the renderer runs on Windows, Linux, macOS, iOS, and visionOS (as an iPad app). OpenXR support and an immersive visionOS app are coming soon. Training is also WIP. As we're seeing the industry adopt research in Gaussian Splatting at a fast pace, I hope this makes it easier for folks implementing Gaussian Splatting or variants in their products. Would love to hear your feedback! For more context, see previous HN discussions on Gaussian Splatting: https://news.ycombinator.com/item?id=39120016 https://news.ycombinator.com/item?id=39120016 https://news.ycombinator.com/item?id=38576974 https://news.ycombinator.com/item?id=38576974

heliophobicdude 3y ago
Thank you so much for open sourcing this! I've started a project to initialize each pixel in an image into a mesh of GS. I used a depth map to unproject them into space. I've been very curious what a few training iterations would do to optimize my scenes. The original 3DGS implementation is not accessible on my hardware at the moment! I really look forward to your training implementation!
- 0x02A 3y ago
  That's super interesting. Are you trying to do single-view image to 3D?
  - heliophobicdude 3y ago
    Kinda! My ultimate goal is to be able to join multiple 3D reconstructions from a set of iPhones all in different positions! Like hemisphere. The single-view image to 3D is pretty accurate but across lots of iPhones.
    - 0x02A 3y ago
      Does colmap fail on your dataset?
      - heliophobicdude 2y ago
        I honestly haven't tried! I'm trying to do this per frame of a live video. I had not considered COLMAP for performance reasons.
- pierotofy 3y ago
  For training you can also try https://github.com/pierotofy/opensplat https://github.com/pierotofy/opensplat

by 0x02A 3y ago

heliophobicdude 3y ago

Thank you so much for open sourcing this! I've started a project to initialize each pixel in an image into a mesh of GS. I used a depth map to unproject them into space. I've been very curious what a few training iterations would do to optimize my scenes. The original 3DGS implementation is not accessible on my hardware at the moment! I really look forward to your training implementation!

0x02A 3y ago
That's super interesting. Are you trying to do single-view image to 3D?
- heliophobicdude 3y ago
  Kinda! My ultimate goal is to be able to join multiple 3D reconstructions from a set of iPhones all in different positions! Like hemisphere. The single-view image to 3D is pretty accurate but across lots of iPhones.
  - 0x02A 3y ago
    Does colmap fail on your dataset?
    - heliophobicdude 2y ago
      I honestly haven't tried! I'm trying to do this per frame of a live video. I had not considered COLMAP for performance reasons.
pierotofy 3y ago
For training you can also try https://github.com/pierotofy/opensplat https://github.com/pierotofy/opensplat