9 ms·

I switched from OpenCode to Pi and there was a big improvement in terms of tool calling performance and I find the experience less buggy. OpenCode has also see

by volf_ 2mo ago

I switched from OpenCode to Pi and there was a big improvement in terms of tool calling performance and I find the experience less buggy. OpenCode has also seemed to have disappeared from https://openrouter.ai/apps/category/coding https://openrouter.ai/apps/category/coding

awinter-py 2mo ago
yeah I noticed it missing on the openrouter leaderboard -- very interested to know what that's about
- dv_dt 2mo ago
  probably about their own model routing service zen?
  - awinter-py 2mo ago
    I can imagine the builtin router stealing traffic, but not removing opencode from the ranking completely? Like the 'top apps' in my private openrouter.ai activity usage is opencode with 55.1M tokens The bottom of the public apps leaderboard is 'kern agent' with 150 million tokens if there are 2 other people like me, opencode should be on here somewhere (edit: user shiggity points out https://github.com/anomalyco/opencode/issues/11926#issuecomment-3848024835 https://github.com/anomalyco/opencode/issues/11926#issuecomm..., they asked to be removed from the ranking)
nthypes 2mo ago
How you deal with LSP? One of the best things in OpenCode is LSP integration.
- 638d127673 2mo ago
  I haven't tried it myself but there seems to be an extension for this: https://pi.dev/packages/pi-lsp https://pi.dev/packages/pi-lsp
- the_mitsuhiko 2mo ago
  > How you deal with LSP? One of the best things in OpenCode is LSP integration. I would love to learn how LSP helps you. I did various basic evals with LSP and I did not find it to improve the agent performance at all. Maybe on newer models this gets better because they are now RLing with LSP in the context, but at least in the past having a commit hook that runs lints and typecheck at the end, is more token efficient than having every edit be injected extra LSP results. You are almost guaranteed to be wasting tokens since partial edits are unlikely going to result in type checks passing. (Disclaimer: I work on Pi)
  - mosura 2mo ago
    LSP helps massively with some models. It sounds a lot like you are focused on using Pi as a OpenAI/Anthropic wrapper when that isn’t your audience. (Hint: raw token usage is less of a concern).
  - lkjdsklf 2mo ago
    lsp integration isn't for checks like that. It's for using lsp to do things like symbol renames. It's cheaper to call an LSP server to do those kind of simple refactors than to spend tokens on the model doing it. It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it. How helpful all that is is heavily dependent on the quality of the language server so usefulness is varies wildly between languages
    - the_mitsuhiko 2mo ago
      > It's for using lsp to do things like symbol renames. It's cheaper to call an LSP server to do those kind of simple refactors than to spend tokens on the model doing it. Is OpenCode pushing the LLM into doing these refactors via LSP? I found that making these refactors with ast-grep is quite efficient. > It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it. I guess I was not particularly successful with having the agent actually do that. I think what fff is attempting to do (with fuzzy expansion of slightly inaccurate greps) results in better results to me, but even that is debatable. To be fair, we do not have evals for this today, but we're generally looking into token efficiency and how likely models are at using tools the right way, and we did not find a ton of evidence of LSP helping, even for finding data. But I would love to see some sessions from people where they have success with LSP data for either refactoring or looking up, because this would be super useful to better understand out blind spots.
- lkjdsklf 2mo ago
  I use a forked version of the pi-lsp. It works pretty well out of the box, but I just tend to fork extensions by default and then hack them up how i see fit.
- resonious 2mo ago
  Oh My Pi has it. I'm a big OMP shill right now. Seems not very popular, but it has the stability of Pi with the features Opencode (and more I think; OMP has web browsing and a more advanced edit system too). OMP often outperforms Claude Code and Codex for me.
- melodyogonna 2mo ago
  I disliked OpenCode's LSP integration, as Pyright kept distracting the model from what I wanted done in a Django project. I prefer to run MyPy at my own convenience
jcul 2mo ago
I tried opencode and pi recently. Having come from Claude code, I was a bit shocked that both seem to allow edits by default with no confirmation dialog. One IIRC allows you to enable it in config and the other needs a plugin.
- InsideOutSanta 2mo ago
  > Having come from Claude code, I was a bit shocked that both seem to allow edits by default with no confirmation dialog. There are confirmation dialogs in OpenCode. You can configure them using the permission config, same as Claude Code. Having said that, as a general rule, none of these harnesses are safe to run on your local file system.
  - BeetleB 2mo ago
    Which is why he said "by default".
    - InsideOutSanta 2mo ago
      There are confirmation dialogs by default, though.
      - BeetleB 2mo ago
        I wonder if that's a recent change. When I first started using OpenCode, it didn't ask permissions. I had to manually go and set that config to get it to ask. There may be differences based on whether the command is under the directory you're running OpenCode from vs some other directory...
        InsideOutSanta 2mo ago
        Yeah. I installed OpenCode in a container about a month ago, and I had a hell of a time figuring out how to get it into complete yolo mode.
shiggity 2mo ago
OpenCode requested to be removed from OpenRouter rankings: https://github.com/anomalyco/opencode/issues/11926#issuecomment-3848024835 https://github.com/anomalyco/opencode/issues/11926#issuecomm...
- kristianp 2mo ago
  Why is stated: > a lot of apps run 100% of traffic through openrouter while for us it's a small subset > but companies we were trying to work with used openrouter as a source of truth on popularity so they thought products that were much smaller were bigger than us
  - ethbr1 2mo ago
    "Our ranking on some metric wasn't as high as we would have liked" seems like a bad reason to self-exclude.
ffamac 2mo ago
I purged OpenCode when I noticed it downloads npm packages in the background without asking the users first. This exposes higher risk of supply chain attach risk.
BeetleB 2mo ago
> I switched from OpenCode to Pi and there was a big improvement in terms of tool calling performance and I find the experience less buggy. That's not because OpenCode is bad, but because Pi is good. I can say the same about Claude vs pi.
fallingbananna 2mo ago
Kilo is the top 3 option there, and that's a fork of OpenCode.

by volf_ 2mo ago

awinter-py 2mo ago

yeah I noticed it missing on the openrouter leaderboard -- very interested to know what that's about

dv_dt 2mo ago
probably about their own model routing service zen?
- awinter-py 2mo ago
  I can imagine the builtin router stealing traffic, but not removing opencode from the ranking completely? Like the 'top apps' in my private openrouter.ai activity usage is opencode with 55.1M tokens The bottom of the public apps leaderboard is 'kern agent' with 150 million tokens if there are 2 other people like me, opencode should be on here somewhere (edit: user shiggity points out https://github.com/anomalyco/opencode/issues/11926#issuecomment-3848024835 https://github.com/anomalyco/opencode/issues/11926#issuecomm..., they asked to be removed from the ranking)

nthypes 2mo ago

How you deal with LSP? One of the best things in OpenCode is LSP integration.

638d127673 2mo ago
I haven't tried it myself but there seems to be an extension for this: https://pi.dev/packages/pi-lsp https://pi.dev/packages/pi-lsp
the_mitsuhiko 2mo ago
> How you deal with LSP? One of the best things in OpenCode is LSP integration. I would love to learn how LSP helps you. I did various basic evals with LSP and I did not find it to improve the agent performance at all. Maybe on newer models this gets better because they are now RLing with LSP in the context, but at least in the past having a commit hook that runs lints and typecheck at the end, is more token efficient than having every edit be injected extra LSP results. You are almost guaranteed to be wasting tokens since partial edits are unlikely going to result in type checks passing. (Disclaimer: I work on Pi)
- mosura 2mo ago
  LSP helps massively with some models. It sounds a lot like you are focused on using Pi as a OpenAI/Anthropic wrapper when that isn’t your audience. (Hint: raw token usage is less of a concern).
- lkjdsklf 2mo ago
  lsp integration isn't for checks like that. It's for using lsp to do things like symbol renames. It's cheaper to call an LSP server to do those kind of simple refactors than to spend tokens on the model doing it. It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it. How helpful all that is is heavily dependent on the quality of the language server so usefulness is varies wildly between languages
  - the_mitsuhiko 2mo ago
    > It's for using lsp to do things like symbol renames. It's cheaper to call an LSP server to do those kind of simple refactors than to spend tokens on the model doing it. Is OpenCode pushing the LLM into doing these refactors via LSP? I found that making these refactors with ast-grep is quite efficient. > It can also use LSP to build context without having to grep around a ton. You don't have to grep and hten pull lines and make guesses for a function. You just ask the LSP for it. I guess I was not particularly successful with having the agent actually do that. I think what fff is attempting to do (with fuzzy expansion of slightly inaccurate greps) results in better results to me, but even that is debatable. To be fair, we do not have evals for this today, but we're generally looking into token efficiency and how likely models are at using tools the right way, and we did not find a ton of evidence of LSP helping, even for finding data. But I would love to see some sessions from people where they have success with LSP data for either refactoring or looking up, because this would be super useful to better understand out blind spots.
lkjdsklf 2mo ago
I use a forked version of the pi-lsp. It works pretty well out of the box, but I just tend to fork extensions by default and then hack them up how i see fit.
resonious 2mo ago
Oh My Pi has it. I'm a big OMP shill right now. Seems not very popular, but it has the stability of Pi with the features Opencode (and more I think; OMP has web browsing and a more advanced edit system too). OMP often outperforms Claude Code and Codex for me.
melodyogonna 2mo ago
I disliked OpenCode's LSP integration, as Pyright kept distracting the model from what I wanted done in a Django project. I prefer to run MyPy at my own convenience

jcul 2mo ago

I tried opencode and pi recently. Having come from Claude code, I was a bit shocked that both seem to allow edits by default with no confirmation dialog. One IIRC allows you to enable it in config and the other needs a plugin.

InsideOutSanta 2mo ago
> Having come from Claude code, I was a bit shocked that both seem to allow edits by default with no confirmation dialog. There are confirmation dialogs in OpenCode. You can configure them using the permission config, same as Claude Code. Having said that, as a general rule, none of these harnesses are safe to run on your local file system.
- BeetleB 2mo ago
  Which is why he said "by default".
  - InsideOutSanta 2mo ago
    There are confirmation dialogs by default, though.
    - BeetleB 2mo ago
      I wonder if that's a recent change. When I first started using OpenCode, it didn't ask permissions. I had to manually go and set that config to get it to ask. There may be differences based on whether the command is under the directory you're running OpenCode from vs some other directory...
      - InsideOutSanta 2mo ago
        Yeah. I installed OpenCode in a container about a month ago, and I had a hell of a time figuring out how to get it into complete yolo mode.

shiggity 2mo ago

OpenCode requested to be removed from OpenRouter rankings: https://github.com/anomalyco/opencode/issues/11926#issuecomment-3848024835 https://github.com/anomalyco/opencode/issues/11926#issuecomm...

kristianp 2mo ago
Why is stated: > a lot of apps run 100% of traffic through openrouter while for us it's a small subset > but companies we were trying to work with used openrouter as a source of truth on popularity so they thought products that were much smaller were bigger than us
- ethbr1 2mo ago
  "Our ranking on some metric wasn't as high as we would have liked" seems like a bad reason to self-exclude.

ffamac 2mo ago

I purged OpenCode when I noticed it downloads npm packages in the background without asking the users first. This exposes higher risk of supply chain attach risk.

BeetleB 2mo ago

> I switched from OpenCode to Pi and there was a big improvement in terms of tool calling performance and I find the experience less buggy. That's not because OpenCode is bad, but because Pi is good. I can say the same about Claude vs pi.

fallingbananna 2mo ago

Kilo is the top 3 option there, and that's a fork of OpenCode.