srush

7 ms·

1.
by srush 11mo ago
Awesome to hear, I will share with the team.
2.
by srush 11mo ago
Roughly, we had Cursor software engineers record real questions they were asking models, and then had them record the PR that they made that contained the result. We then cleaned these up. That is the benchmark.
3.
by srush 11mo ago
A lot of people use it! It scores very well on our benchmarks, significantly better than Composer-1.
4.
by srush 11mo ago
We use Ray data for our map-style processing jobs. For example one tool have runs over all the rollouts from the RL system and collects qualitative statistics to understand which type of agent trajectories are being reward, and what types o
5.
by srush 11mo ago
Oh good question. Actually speaking at the Ray Summit next week in SF so we will talk more about it. We used Ray throughout the pipeline for running evals, for the RL controller, for data collation, and for visualizations. One tool we found
6.
by srush 11mo ago
We train with a single agent. is that the question?
7.
by srush 11mo ago
neat!
8.
by srush 11mo ago
Our view is that there is a now a minimal amount of intelligence that is necessary to be productive, and that if you can pair that with speed that is awesome.
9.
by srush 11mo ago
Let us know how you like it.
10.
by srush 11mo ago
Straight up untrue.
11.
by srush 11mo ago
There are lots of good models we like here. But we agree that getting the right point on the smart+fast graph can make agentic coding feel really good. (Cursor researcher)
12.
by srush 11mo ago
Thanks! Yeah, been working here for 9 months now. Fascinated byt agentic coding both as a researcher and user. Totally agree that "smart model" is the table stakes for usefulness these days.
13.
by srush 11mo ago
We will do our best. Luckily I don't think there are major telecom companies called Composer-2.
14.
by srush 11mo ago
Unfortunately not, as we used our own internal code for the benchmark. We would also like to see more benchmarks that reflect the day-to-day agentic coding use.
15.
by srush 11mo ago
The blog talks about the training process. Specifically we trained with RL post-training on coding examples.
16.
by srush 11mo ago
Thanks! I do like the labs blog posts as well though, OpenAI and Anthropic have some classics.
17.
by srush 11mo ago
Our primary focus is on RL post-training. We think that is the best way to get the model to be a strong interactive agent.
18.
by srush 11mo ago
We did a lot of internal testing and thought this model was already quite useful for release.
19.
by srush 11mo ago
Cheetah was an earlier (and dumber) version of this model that we used to test production speed. They are both developed in-house. If you liked Cheetah, give this model a try.
20.
by srush 11mo ago
We like the name Composer and were sad to see it go. Excited to bring it back. (Agree Cheetah is a cool name too.)
21.
by srush 11mo ago
There is a footnote that should help with the models. Training is a harder thing to report on, but roughly our finding here is that RL scales.
22.
by srush 11mo ago
We don't wear shoes [1]. [1] https://www.businessinsider.com/no-shoes-policy-in-office-cu...
23.
by srush 11mo ago
Agree that Sonnet 4.5 is an excellent model. Would be curious to hear your experience using Composer though, it's quite good.
24.
by srush 11mo ago
We also are big Tab users here at Cursor. In the blog we talk about the motivation for this project came from thinking about a Tab-like agent.
25.
by srush 11mo ago
Hi everyone, I am an ML researcher at Cursor, and worked on this project. Would love to hear any feedback you may have on the model, and can answer question about the blog post.
26.
by srush 1y ago
A recent tutorial video from one of the authors featured in this article: Evaluating AI's World Models ( https://www.youtube.com/watch?v=hguIUmMsvA4 ) Goes into details about several of the challenges discussed.
27.
by srush 1y ago
Actually realize this is to the modern version not the original. So props to Austin Huang, Suraj Subramanian, Jonathan Sum, Khalid Almubarak, and Stella Biderman who rewrote this one.
28.
by srush 1y ago
Oh hey, I wrote this. Been a long time. I had the lucky of break of working in machine translation / parsing when the most important invention of the century happened in my niche field. I'm pretty interested in the intersection of
29.
by srush 2y ago
I've tried to build sort of model several times, but could never get it to work. The challenge is that small perturbations in encoder space lead to removing semantically important details (e.g. dates). You really want these to mess up
30.
by srush 2y ago
For problems that require multi-step reasoning, standard LLMs seem to be stuck. The field is increasingly interested in models like o1 that output many "guesses" to find the right one. Currently open-source does not know how to do

More ›