veselin

7 ms·

1.
by veselin 3mo ago
Here, it appears they compare a single prompt "find IDOR", against a multi-agent system. However, one can also start far more sophisticated skills that spin up subagents and mostly do the same in Claude Code, Codex, OpenCode, Pi,
2.
by veselin 3mo ago
The more I think on the problem, the more I believe this will be solved with US interventions. And the interventions will increase inflation by a lot, so prices will not go down. The other alternatives with LLMs becoming more expensive in a
3.
by veselin 4mo ago
Claude code itself spins a lot of its subagents with Haiku. The model has low hallucination rate, so it is great for exploration tasks. I guess this is what the best purpose of this model here will be as well. Which is a lot of tokens - man
4.
by veselin 4mo ago
I would argue that with AI, this becomes less of an issue. Connect N services, deploy to bare metal. Granted, AI is an additional cost now local or remote. But so is the MacBook people use to develop their software.
5.
by veselin 4mo ago
Exactly our experience too. Effectively we catch these and on these status codes, we send to OpenAI. Retrying the same query in Gemini has high chance to give kind-of the same status code.
6.
by veselin 5mo ago
I think this is a promise, probably also for spec driven development. You write the spec, the whole thing can be reimplemented in rust tomorrow. Make small modules or libraries. One colleague describes monolith vs microservices as "the
7.
by veselin 6mo ago
This is an experiment that, although not to this extreme, was tested by OpenAI. Their responses API allow you to control verbosity: https://developers.openai.com/api/reference/resources/respon... I don't
8.
by veselin 6mo ago
I guess we will have a lot more benefits if we can get this to work on something like llama.cpp - since it really has a lot of kernels for different quantizations, a lot of home users, high hardware diversity - so it is a likely place with
9.
by veselin 7mo ago
I think they put two things: * Likely they will seek regulation that would ban some models. Not sure this can work, but they will certainly try. * Likely they will not release some of their next models in the API.
10.
by veselin 7mo ago
I am actually going to complain about this: that neither of the Gemini models are not preview ones. Anthropic seems the best in this. Everything is in the API on day one. OpenAI tend to want to ask you for subscription, but the API gets the
11.
by veselin 8mo ago
What is the state of using quants? For chat models, a few errors or lost intelligence may matter a little. But what is happening to tool calling in coding agents? Does it fail catastrophically after a few steps in the agent? I am interestin
12.
by veselin 8mo ago
I am taking for SWE bench style problems where Todo doesn't help, except for more parallelism.
13.
by veselin 8mo ago
I run evals and the Todo tool doesn't help most of the time. Usually models on high thinking would maintain Todo/state in their thinking tokens. What Todo helps is for cases like Anthropic models to run more parallel tool calls. I
14.
by veselin 10mo ago
I work a lot on testing also SWE bench verified. This benchmark in my opinion now is good to catch if you got some regression on the agent side. However, going above 75%, it is likely about the same. The remaining instances are likely under
15.
by veselin 1y ago
Anybody knows if one can find an inference provider that offers input token caching? It should be almost required for agentic use - first speed, but also almost all conversations start where the previous ended, so cost may end up quite high
16.
by veselin 1y ago
I think that people are just too quick to assume this is amazing, before it is there. Which doesn't mean it won't get there. Somehow if I take the best models and agents, most hard coding benchmarks are at below 50% and even swe b
17.
by veselin 1y ago
I noticed a similar trends in selling on X. Put a claim, peg on some product A with good sales - Cursor, Claude, Gemini, etc. Then say, the best way to use A is with our best product, guide, being MCP or something else. For some of these I
18.
by veselin 2y ago
Yes. The article is click bait. With such a title I would have expected majority of the area to be dummy, but it is just structurally more silicon, exactly like a picture may be majority of its mass wood.
19.
by veselin 2y ago
I used them both. I ended up disabling copilot. The reason is that the completions do not always integrate with the rest of the code, in particular with non-matching brackets. Often it just repeats some other part of the code. I had much fe
20.
by veselin 2y ago
It seems recent years give us a lot of licenses (for core infra software) and now for LLMs. They all say in very legalese basically: these top 5-10 tech companies will not compete fairly with us, thus they are banned from using the software
21.
by veselin 2y ago
When I saw the name, I knew immediately this is Jyrki's work.
22.
by veselin 2y ago
Exactly. The whole thing reads like some propaganda. It pits interesting topics ahead then to move on and push some agenda that sounds super political to me. Yes, some languages are underrepresented and there are some thresholds. But exactl
23.
by veselin 3y ago
I think this is simply the default of lm-evaluation-harness. They said they ran every single benchmark they could out of the box.
24.
by veselin 3y ago
The product they often presented as started in 20% time is Google news. I don't know the actual details, just this is what I remember from my time at Google (2006-2012).
25.
by veselin 3y ago
It is true that nobody competes in the low power high efficiency workstation market or maybe such a market does not exist yet and Apple is creating it. But also as users, some were expecting the M series are so good that they are going to t
26.
by veselin 3y ago
I find the article quite informative. Yes, M2 and the other chips are completely different products with different goals. If one wants to say that something completely trumps the other, it will be wrong. But here is what is visible: The M2
27.
by veselin 3y ago
What speed should we expect from the model on consumer hardware? I tried a 8 bit quantized version on 4090 and got it to generate 100 tokens for 13 second, which seems a bit slow to me.
28.
by veselin 3y ago
One of main reasons somebody may want to use such a library is to constrain the output of a LLM. The language is designed to make this easy and abstract this part of the querying away. There are trivial cases when some value is coming from
29.
by veselin 4y ago
The interesting part of that prediction is that depending on how you read it, you may say it failed embarrassingly, or you may say it predicted the current reality fairy well. The next software as a differentiable thing that is the program
30.
by veselin 4y ago
Contrary to what many say here, this is a real threat and Google are right to consider to move fast in that area. There are a few technical improvements needed that can come in the next months where ChatGPT may be tuned to rewrite queries f

More ›