jcjohns

6 ms·

1.
by jcjohns 15d ago
Unfortunately that's a complex question... this depends on the number of diffusion steps, the size of the context, the image resolution, and the type and number of inference devices we use. There are lots of knobs to trade off speed, q
2.
by jcjohns 15d ago
One of the biggest differences is the conditioning signal. Genie 3 and similar input raw keyboard commands (WASD + arrow keys), while Atlas inputs camera poses. This small difference means that Genie 3 has no 3D whatsoever; the model needs
3.
by jcjohns 15d ago
Yes, as long as the input images are "poseable" -- if they were taken in the same space they need to have some overlap, where the same object or part of the scene is visible in multiple views so the pose can be predicted. You can
4.
by jcjohns 15d ago
It can operate in different modes for different applications. If you only want to reconstruct the content in the input images without any imagination, Atlas can do sparse 3D reconstruction where it predicts a depth map for each input pixel,
5.
by jcjohns 15d ago
Yes, one of the main goals of this model was to keep 3D consistency as the camera moves around; it can do this even without relying on an explicit 3D point cloud or Gaussian splat representation of the scene.
6.
by jcjohns 15d ago
Small correction, World Labs isn't betting exclusively on explicit 3D -- many of the Atlas results in this post are generated frames without any explicit underlying 3D representation. But Atlas can (optionally) output explicit 3D for t
7.
by jcjohns 15d ago
It's an overloaded term that different people use for different things. At World Labs we think about three major categories of world models: - Renderers output pixels or views of a world, either the real world or a generated world - Si
8.
by jcjohns 15d ago
In addition to frozen time, Atlas can handle some scene motion - see for example the candy New York City in the Camera-Controlled Generation section where cars are moving, or some subtle motion of the waves toward the end of the 1-minute vi
9.
by jcjohns 15d ago
I'm a cofounder at World Labs - happy to answer questions about Atlas!
10.
by jcjohns 11mo ago
Make sure to try the live demo here: https://rtfm.worldlabs.ai/
11.
by jcjohns 2y ago
Arrow keys also work now, thanks for the feedback!
12.
by jcjohns 2y ago
That's weird, what device are you using? (I'm part of World Labs)
13.
by jcjohns 3y ago
The usual narrative around this skill is that it was banned because it was too dangerous, but that doesn't ring true to me -- many release moves performed both by women on the uneven bars and men on the high bar have similar motions wh
14.
by jcjohns 3y ago
This makes sense in theory, but is hard to get working in practice. We tried using nvjpeg to do JPEG decoding on GPU as a additional baseline, but using it as a drop-in replacement to a standard training pipeline gives huge slowdowns for a
15.
by jcjohns 3y ago
I'm one of the authors of this CVPR paper -- cool to see our work mentioned on HN! The Uber paper from 2018 is one that has been floating around in the back of my head for a while. Decoding DCT to RGB is essentially an 8x8 stride 8 con
16.
by jcjohns 6y ago
I don't think Google has ever released models trained on JFT. But if you're interested in large-scale vision models, you can check out these models from Facebook trained on 940M Instagram images (several times bigger than JFT!) h
17.
by jcjohns 10y ago
Yes, I think that is a likely explanation. Also note that Vincent Dumoulin is an author of both the deconv-checkerboard blog post and the new paper from Google, and that the new Google paper uses the upsample+convolution technique suggested
18.
by jcjohns 10y ago
I've found that instance normalization usually gives better results so I prefer it over batch normalization. With batch norm you learn four scalars per convolutional feature map: mu (mean), sigma (stddev), alpha (scale) and beta (shift
19.
by jcjohns 10y ago
Real-time neural style transfer is not new; in the past year there have been several academic papers [1-4] on this topic and several open-source code releases: https://github.com/jcjohnson/fast-neural-style https:&#x2
20.
by jcjohns 10y ago
Author here. I'm not a lawyer so I can't write anything too official myself, and after some searching it seemed like none of the standard open-source licenses apply to this use-case.
21.
by jcjohns 10y ago
Code is here: https://github.com/jcjohnson/densecap