nee1r

15 ms·

1. 1.▲
   by nee1r 1mo ago

   are you specifically focused on non-invasive methods/why? seems like you reach a noise barrier which limits things like this
2. 2.▲
   by nee1r 2mo ago

   how do you pick good goal conditioning images/do you have to hand pick a dataset of good goal images? seems hard if you don't have full context. really cool though!
3. 3.▲
   by nee1r 7mo ago

   thanks! a lot of credit to the people who helped write/edit
4. 4.▲
   by nee1r 7mo ago

   giving back to the research community! releasing and talking about research helps everyone
5. 5.▲
   by nee1r 7mo ago

   thanks! i definitely love diffusion + pushed for it, as a non-causal generative method i think its pretty unique
6. 6.▲
   by nee1r 7mo ago

   thanks! got a lot of inspiration from VPT https://arxiv.org/abs/2206.11795 is a great paper, would recommend a read we all have various backgrounds, me particularly i did a lot of material science x ai research and jus
7. 7.▲
   by nee1r 7mo ago

   we have an alignment blog post dropping soon! scaling up in the next couple of months, then hopefully opening up an API or licensing it. Benchmarks are really fun—lots of secret ones. Our main thesis is that you should be using the same ben
8. 8.▲
   by nee1r 7mo ago

   planning on instruct tuning soon!
9. 9.▲
   by nee1r 7mo ago

   safety was important for the demo, the model didn't have access to the brake or accelerator.
10. 10.▲
    by nee1r 7mo ago

    thanks! the math and architecture of the FDM (no video encoder) is pretty simple, its a regular transformer with next-token predictions but with frames interleaved.
11. 11.▲
    by nee1r 7mo ago

    yeah! i love the BCO paper, i think its extremely intuitive and these methods are really interesting in a time where data without labels is abundant. i especially like the idea of iteratively making the inverse dynamics better—might lean cl
12. 12.▲
    by nee1r 7mo ago

    cool thanks for the title idea!! hopefully when we scale up in the next month/two we can update the community
13. 13.▲
    by nee1r 7mo ago

    collected! no synthetic
14. 14.▲
    by nee1r 7mo ago

    thanks! the inverse dynamics model is trained first on 40k hours of data and then frozen to label all 11 million hours. yup! the idea is that it should take a small amount of data to generalize environment dynamics, then you can use a lot o
15. 15.▲
    by nee1r 7mo ago

    real
16. 16.▲
    by nee1r 7mo ago

    this is honestly an issue for the inverse dynamics (for app specific shortcuts etc.) but for general UI learning we still see promising eval trends
17. 17.▲
    by nee1r 7mo ago

    no finetuning data for the blender task! we actually think its the opposite, there are a lot of video tutorials for complex tasks like onshape/blender/fusion360 but not as much of people idly browsing. but also at the 11M hour sca
18. 18.▲
    by nee1r 7mo ago

    i actually drove the car (with arrow keys) around south park for around ~45 minutes as finetuning data, no extra labelling other than that. think the car line graph is super cool because you actually see the videegame prior working
19. 19.▲
    by nee1r 7mo ago

    the main chain of experiments was trying causal => non-causal => non-causal with ctc and CE. i think a good intuition here is that you need a generative approach fundamentally because there definitely are multiple correct IDM labels.
20. 20.▲
    by nee1r 7mo ago

    good question! we use exponential binning (map the mouse movements onto a plane with exponentially increasing tick marks https://si.inc/fdm1/exponential_binning.webp ) but tried a bunch of other methods (linear creates
21. 21.▲
    by nee1r 7mo ago

    Hey guys! I’m Neel, been holed up in our south park office for the past year working on model training. excited to share our research! This is a preview of a very different type of computer use model—we train on the internet. Specifically w
22. 22.▲
    The First Fully General Computer Action Model(si.inc)

    345 points by nee1r 7mo ago|80 comments
23. 23.▲
    by nee1r 9mo ago

    glad the timelines are short and hope its user friendly
24. 24.▲
    by nee1r 9mo ago

    how are you planning on getting the robots to learn a base policy from scratch? seems hard without a base model
25. 25.▲
    by nee1r 9mo ago

    being able to search through the files when they're purposefully so obfuscated is great
26. 26.▲
    by nee1r 9mo ago

    hmmm wonder if decosting is actually linear vs. discrete jumps in ability (ie. we might just nail fusion or boosts in efficiency)
27. 27.▲
    We collected 10k hours of neuro-language data in our basement(condu.it)

    117 points by nee1r 9mo ago|60 comments
28. 28.▲
    by nee1r 11mo ago

    Love MOFs! Did research about MOFs <=> language modeling a couple years ago and I'm excited to see them getting more coverage https://arxiv.org/abs/2311.07617
29. 29.▲
    by nee1r 1y ago

    We have a custom barebones solution that uses a hashring to route the files!
30. 30.▲
    by nee1r 1y ago

    We use the same nginx rust server to do file writes, it's done via web requests