5 ms·

I'm a cofounder at World Labs - happy to answer questions about Atlas!

by jcjohns 17d ago

pj_mukh 17d ago
Please give us access :) We are doing super useful things for museums and travelers and this would be a huge unlock! Also, big fan of Dr.Li!
lurker137 17d ago
How does it compare to Google's Genie 3 or other similar models? At first glance it does look much more impressive visually
- jcjohns 17d ago
  One of the biggest differences is the conditioning signal. Genie 3 and similar input raw keyboard commands (WASD + arrow keys), while Atlas inputs camera poses. This small difference means that Genie 3 has no 3D whatsoever; the model needs to learn an internal mapping between keyboard commands, world states, and pixels; and with Genie 3 there is no clear way to control the generated world aside from the input image and text prompt. Since Atlas makes camera pose explicit it can use posed input frames to shape the generated world, giving you a lot more creative control. Another big differentiator is multimodality. Genie 3 only outputs pixels. Atlas also outputs pixels, but it can also output explicit 3D for the cases where you need it (such as plugging into game engines, simulators, or VFX workflows)
  - lurker137 16d ago
    Thanks, those are great points, now I'm definitely excited for the full release to try it out
hi_hi 17d ago
How long does it take to process from input images to model output?
- jcjohns 17d ago
  Unfortunately that's a complex question... this depends on the number of diffusion steps, the size of the context, the image resolution, and the type and number of inference devices we use. There are lots of knobs to trade off speed, quality, latency, throughput, and cost. An ideal workflow would be something semi-interactive that you can use to quickly iterate on an idea, followed by a longer offline bake-out to generate final production-quality assets.
E-Reverance 17d ago
Are y'all using any sort of self-distillation similar to https://self-evo.github.io/ https://self-evo.github.io/ to sharpen representations?
edg5000 16d ago
Could this be used to replace photogrammtry when accuracy is needed? Photogrammetry requires lots of images and can be brittle, and is slow to compute.
kkotak 16d ago
Maybe a dumb question, but when navigating the worlds on the site, I notice that the space is 3D, but the objects (the drum set, or the vending machine) are flat 2D layers that one cannot walk around or examine from a different angle. Is this the inherit limitation of the approach or future features? Thanks.
liaobh 15d ago
[dead]
stranded-man 17d ago
spacial context feature is cool - what are the limitations, if any? What would it take to geo and rotation tag every photo ever taken , combine it into a mass spatial context, run it through atlas and build an entire 3D model of the world?
- keunhong 17d ago
  Atlas project lead here. Atlas is an auto-regressive diffusion model, so context length limitations apply similar to LLMs and video models. Where Atlas has an edge is that its context comprised of an arbitrary sequence of images with camera poses, which lends itself to managing the context in creative ways (we called this "context juggling" in our RTFM blog, https://www.worldlabs.ai/blog/rtfm https://www.worldlabs.ai/blog/rtfm). So yes through clever context management you could potentially build an entire 3D model of the world.
  - cman1444 17d ago
    Would it be more reasonable to take images from movies and create worlds of various IPs? My first thought is a detailed Hogwarts that is fully explorable using scenes from the movies (or even descriptions from the books?)
  - stranded-man 17d ago
    can atlas also generate 3D without pose information attached to the input images?
    - jcjohns 17d ago
      Yes, as long as the input images are "poseable" -- if they were taken in the same space they need to have some overlap, where the same object or part of the scene is visible in multiple views so the pose can be predicted. You can also manually position the input images in 3D space to create scenes generatively; we show examples of this in the "generating with spatial context" section
- deleted 17d ago
  [deleted]

I'm a cofounder at World Labs - happy to answer questions about Atlas!

by jcjohns 17d ago

I'm a cofounder at World Labs - happy to answer questions about Atlas!

Please give us access :) We are doing super useful things for museums and travelers and this would be a huge unlock! Also, big fan of Dr.Li!

lurker137 17d ago

How does it compare to Google's Genie 3 or other similar models? At first glance it does look much more impressive visually

jcjohns 17d ago
One of the biggest differences is the conditioning signal. Genie 3 and similar input raw keyboard commands (WASD + arrow keys), while Atlas inputs camera poses. This small difference means that Genie 3 has no 3D whatsoever; the model needs to learn an internal mapping between keyboard commands, world states, and pixels; and with Genie 3 there is no clear way to control the generated world aside from the input image and text prompt. Since Atlas makes camera pose explicit it can use posed input frames to shape the generated world, giving you a lot more creative control. Another big differentiator is multimodality. Genie 3 only outputs pixels. Atlas also outputs pixels, but it can also output explicit 3D for the cases where you need it (such as plugging into game engines, simulators, or VFX workflows)
- lurker137 16d ago
  Thanks, those are great points, now I'm definitely excited for the full release to try it out

hi_hi 17d ago

How long does it take to process from input images to model output?

jcjohns 17d ago
Unfortunately that's a complex question... this depends on the number of diffusion steps, the size of the context, the image resolution, and the type and number of inference devices we use. There are lots of knobs to trade off speed, quality, latency, throughput, and cost. An ideal workflow would be something semi-interactive that you can use to quickly iterate on an idea, followed by a longer offline bake-out to generate final production-quality assets.

E-Reverance 17d ago

Are y'all using any sort of self-distillation similar to https://self-evo.github.io/ https://self-evo.github.io/ to sharpen representations?

edg5000 16d ago

Could this be used to replace photogrammtry when accuracy is needed? Photogrammetry requires lots of images and can be brittle, and is slow to compute.

kkotak 16d ago

Maybe a dumb question, but when navigating the worlds on the site, I notice that the space is 3D, but the objects (the drum set, or the vending machine) are flat 2D layers that one cannot walk around or examine from a different angle. Is this the inherit limitation of the approach or future features? Thanks.

liaobh 15d ago

[dead]

stranded-man 17d ago

spacial context feature is cool - what are the limitations, if any? What would it take to geo and rotation tag every photo ever taken , combine it into a mass spatial context, run it through atlas and build an entire 3D model of the world?

keunhong 17d ago
Atlas project lead here. Atlas is an auto-regressive diffusion model, so context length limitations apply similar to LLMs and video models. Where Atlas has an edge is that its context comprised of an arbitrary sequence of images with camera poses, which lends itself to managing the context in creative ways (we called this "context juggling" in our RTFM blog, https://www.worldlabs.ai/blog/rtfm https://www.worldlabs.ai/blog/rtfm). So yes through clever context management you could potentially build an entire 3D model of the world.
- cman1444 17d ago
  Would it be more reasonable to take images from movies and create worlds of various IPs? My first thought is a detailed Hogwarts that is fully explorable using scenes from the movies (or even descriptions from the books?)
- stranded-man 17d ago
  can atlas also generate 3D without pose information attached to the input images?
  - jcjohns 17d ago
    Yes, as long as the input images are "poseable" -- if they were taken in the same space they need to have some overlap, where the same object or part of the scene is visible in multiple views so the pose can be predicted. You can also manually position the input images in 3D space to create scenes generatively; we show examples of this in the "generating with spatial context" section
deleted 17d ago
[deleted]