5 ms·
Also tried it on a small project, it did ok finding issues but completely failed doing rather basic edits, like it lost closing brackets or used wrong syntax an
by Staross 9mo ago
Also tried it on a small project, it did ok finding issues but completely failed doing rather basic edits, like it lost closing brackets or used wrong syntax and couldn't recover. The CLI was easy to setup and use though.
- InsideOutSanta 9mo agoIn my experience, the messed up closing brackets are a surprisingly common issue for LLMs. Both Sonnet 4.5 and Gemini 3 also do this regularly. Seems like something that should be relatively easy to fix, though.
- marco_craveiro 9mo agoI use both Sonnet 4.5 and Opus 4.5 to edit lisp (emacs lisp to be be precise) and run into this issue extremely infrequently. Not sure if they have some special handling for this but seems to work ok. I have this problem with Gemini, and less frequently, with Qwen.
- embedding-shape 9mo agoDid you try it via OpenRouter? If so, what provider? I've noticed some providers seems to not exactly be upfront about what quantization they're using, you can see that the responses from some providers who supposedly run the exact same model and weights give vastly different responses. Back when Devstral 1 released, this was made very noticeable to me because the ones who used the smaller quantizations were unable to actually properly format the code, just as you noticed, that's why this sounded so similar to what I've seen before.