7 ms·

Terra has not been able to do any of the technical tasks I've asked of it correctly. I'm surprised others get use out of it. Anything below Sol high tends to gi

Terra has not been able to do any of the technical tasks I've asked of it correctly. I'm surprised others get use out of it. Anything below Sol high tends to give me mostly unreliable results. I'm using codex as my main harness but maybe it performs better with a different one.

freakynit 1mo ago
Depends on project complexity. For one of my more complex projects, I exclusively use sol-high ... nothing below that works correctly. For this however, a comparatively much simpler task, tarra-high works fine.
- Foobar8568 1mo ago
  Right now, sol-xhigh is my favorite model. I feel that Opus 5 is dumber than 4.8. Fable is too expensive to do anything (limit of $50, started a prompt at $25, ended up at $75, is bullshit, but at least it's "free credits"). DeepSeek is okay for random API-based stuff, as it's cheap. Local open models running on a 5090 are hit or miss. I feel that most GGUFs/quants are awful...
  - miohtama 1mo ago
    Opus 5 degrades to word salad. I wonder if it is because of watermarking.
    - SwellJoe 1mo ago
      Opus 5 doesn't really even speak coherent English. I'm not sure what's going on, but it can't explain anything. It still does an excellent job with code and writing tests and code review and creating and completing a plan, and it seems to be able to understand English instructions, but it sure as hell can't explain what it did or how to use the code it wrote. That was true before they announced the watermarking, I'd already started to back off of using Opus as much because I like to understand what the model is doing and have it write documentation I can use to reproduce its results, but maybe watermarking was already in there unannounced.
      - sscaryterry 1mo ago
        This 100%. The incoherent drivel has literally made me move away from CC completely.
    - isoprophlex 1mo ago
      [dead]
    - mlrtime 1mo ago
      Agreed, I'll get a terminal full of text and I just reply "I don't understand this" And it retypes it for a human, I'm doing this more and more lately.
    - pulkitsh1234 1mo ago
      Ah, so it's not only me :-D I think it could be the watermarking, but at this point they might be deliberately complicating the prose so that we ask clarifying questions and that leads to more token spend.
  - ericfr11 1mo ago
    I am still on Opus 4.8, with a custom built harness and it works very well even on multi-repos, across stack, deep changes. I also have a very solid test suite which is helping the coding agent a lot
  - deleted 1mo ago
    [deleted]
cyanydeez 1mo ago
the breadth and width of the universe of oneshot challenges are all arbitrary. It's unsurprising different workflows oneshot better than others. All the more reason to favor local models under your control, as once you find that sweet spot model, no one can change it, upgrade it, align it, take it down or otherwise harm the time investment you made it making it your own. I can't really believe no one understands, after decades, how valueable a rock solid development environment is.
- Phemist 1mo ago
  Exactly! I am not opposed to cloud-based models, but I do only stick to open-weight models because I know I can move my whole stack to local (given enough hardware) and continue development without any of the LLM interaction contracts being broken. I would like to see some development where proof of authenticity certs are generated alongside the actual output of the model. Prove to me (or at least claim to me liable to breach of contract) that this output was generated by FP8 DeepSeek V4 Pro 0813. Not some cheaper quantization of the model.
- f6v 1mo ago
  But I don’t want to manage a local model…
  - cyanydeez 1mo ago
    Likr sll infrastructure, once its setup and your flow is set, the only thingin the way is FOMO.
bob1029 1mo ago
I am seeing essentially deterministic results with Terra running a custom browser automation agent across >100 interaction events. The harness is everything. If I just threw something like Codex at this and said "good luck" I wouldn't make it beyond 5-10 interactions. I tried that already. Carefully designing the views and tools over the environment is where you can go from 50% to 99.9999%.
- tanishqkanc 1mo ago
  Curious about what you found. I agree harness for browser automation is vital - I work on https://libretto.sh https://libretto.sh
  - bob1029 1mo ago
    Hand-crafted adapters that sit between playwright primitives and the agent loop are the secret sauce. The goal is to insulate the agent from the raw DOM without any loss in fidelity regarding the logical business information and available actions.
    - drewnick 1mo ago
      +1 I have found extremely reliable systems require a mix of deterministic "adapters" is a good word for agents to actually get through the workflows I've created. I'm still amazed that it can work with both those and some "intuition" to bend the rules around the adapters if prompted.
mixedCase 1mo ago
With Pi as a harness I've been using OpenAI models as a worker with an Opus 5 (in Claude Code) planner. I've only had a few issues with Terra High/Medium and absolutely none with Sol Medium+ on a fairly complex Rust project that targets Linux, Mac, Windows and Web, with plenty of nasty FFI, VMs, remotely debugging systems, among some other things within a monorepo. I think the key is to give them a nice assortment of self-verification tools, an AGENTS.md or reference document that they're encouraged to routinely check, and asking the planner to be thorough with the ACs but give the model some space. The planner routinely finds issues with the worker's output, but that's what it is for.
- sejje 1mo ago
  Plan with sol-med, implement with luna-high. Rarely a problem.
  - ericfr11 1mo ago
    Same for me, with Claude Opus/Sonnet. All the models are almost equivalent if well steered
- ericfr11 1mo ago
  Harness is the key. I built my own to "talk" our institutional knowledge and it's working great
- thesneakykoala 1mo ago
  [dead]
zeven7 1mo ago
I bounce between Sol high/medium and Luna max. I don't know why you'd use anything between Luna max and Sol medium. Luna is so extremely cheap and cranked up to max it does anything I'd want Terra to do for a fraction of the cost. What is Terra for?
- Juvination 1mo ago
  One thing I've really noticed with Luna Max is its speed. I've got a review script setup on a custom Pi extension. Luna finds some issues/some false positives, while Sol finds issues but disregards false positives. The biggest thing is Sol finishes in about half the time.
Art9681 1mo ago
Terra is great. It's wild how different our experiences are. Install the Superpowers plugin. Behold.
- blackqueeriroh 1mo ago
  Man, Superpowers is sooooo good. I use it with Opus, Sonnet, and Fable, and its review cycles are truly clutch. I layer on CodeRabbit for PR Review and it’s just so solid.
- MuteXR 1mo ago
  In my experience, Superpowers has begun to massively slow down the capable models at this point. The skills they add are incredibly bloated and just get you worse results nowadays, tbh.
  - agrippanux 1mo ago
    I loved Superpowers and evangelized it heavily. This week I removed it because it now gets in the way of the frontier models.
deleted 1mo ago
[deleted]

freakynit 1mo ago

Depends on project complexity. For one of my more complex projects, I exclusively use sol-high ... nothing below that works correctly. For this however, a comparatively much simpler task, tarra-high works fine.

Foobar8568 1mo ago
Right now, sol-xhigh is my favorite model. I feel that Opus 5 is dumber than 4.8. Fable is too expensive to do anything (limit of $50, started a prompt at $25, ended up at $75, is bullshit, but at least it's "free credits"). DeepSeek is okay for random API-based stuff, as it's cheap. Local open models running on a 5090 are hit or miss. I feel that most GGUFs/quants are awful...
- miohtama 1mo ago
  Opus 5 degrades to word salad. I wonder if it is because of watermarking.
  - SwellJoe 1mo ago
    Opus 5 doesn't really even speak coherent English. I'm not sure what's going on, but it can't explain anything. It still does an excellent job with code and writing tests and code review and creating and completing a plan, and it seems to be able to understand English instructions, but it sure as hell can't explain what it did or how to use the code it wrote. That was true before they announced the watermarking, I'd already started to back off of using Opus as much because I like to understand what the model is doing and have it write documentation I can use to reproduce its results, but maybe watermarking was already in there unannounced.
    - sscaryterry 1mo ago
      This 100%. The incoherent drivel has literally made me move away from CC completely.
  - isoprophlex 1mo ago
    [dead]
  - mlrtime 1mo ago
    Agreed, I'll get a terminal full of text and I just reply "I don't understand this" And it retypes it for a human, I'm doing this more and more lately.
  - pulkitsh1234 1mo ago
    Ah, so it's not only me :-D I think it could be the watermarking, but at this point they might be deliberately complicating the prose so that we ask clarifying questions and that leads to more token spend.
- ericfr11 1mo ago
  I am still on Opus 4.8, with a custom built harness and it works very well even on multi-repos, across stack, deep changes. I also have a very solid test suite which is helping the coding agent a lot
- deleted 1mo ago
  [deleted]

cyanydeez 1mo ago

the breadth and width of the universe of oneshot challenges are all arbitrary. It's unsurprising different workflows oneshot better than others. All the more reason to favor local models under your control, as once you find that sweet spot model, no one can change it, upgrade it, align it, take it down or otherwise harm the time investment you made it making it your own. I can't really believe no one understands, after decades, how valueable a rock solid development environment is.

Phemist 1mo ago
Exactly! I am not opposed to cloud-based models, but I do only stick to open-weight models because I know I can move my whole stack to local (given enough hardware) and continue development without any of the LLM interaction contracts being broken. I would like to see some development where proof of authenticity certs are generated alongside the actual output of the model. Prove to me (or at least claim to me liable to breach of contract) that this output was generated by FP8 DeepSeek V4 Pro 0813. Not some cheaper quantization of the model.
f6v 1mo ago
But I don’t want to manage a local model…
- cyanydeez 1mo ago
  Likr sll infrastructure, once its setup and your flow is set, the only thingin the way is FOMO.

bob1029 1mo ago

I am seeing essentially deterministic results with Terra running a custom browser automation agent across >100 interaction events. The harness is everything. If I just threw something like Codex at this and said "good luck" I wouldn't make it beyond 5-10 interactions. I tried that already. Carefully designing the views and tools over the environment is where you can go from 50% to 99.9999%.

tanishqkanc 1mo ago
Curious about what you found. I agree harness for browser automation is vital - I work on https://libretto.sh https://libretto.sh
- bob1029 1mo ago
  Hand-crafted adapters that sit between playwright primitives and the agent loop are the secret sauce. The goal is to insulate the agent from the raw DOM without any loss in fidelity regarding the logical business information and available actions.
  - drewnick 1mo ago
    +1 I have found extremely reliable systems require a mix of deterministic "adapters" is a good word for agents to actually get through the workflows I've created. I'm still amazed that it can work with both those and some "intuition" to bend the rules around the adapters if prompted.

mixedCase 1mo ago

With Pi as a harness I've been using OpenAI models as a worker with an Opus 5 (in Claude Code) planner. I've only had a few issues with Terra High/Medium and absolutely none with Sol Medium+ on a fairly complex Rust project that targets Linux, Mac, Windows and Web, with plenty of nasty FFI, VMs, remotely debugging systems, among some other things within a monorepo. I think the key is to give them a nice assortment of self-verification tools, an AGENTS.md or reference document that they're encouraged to routinely check, and asking the planner to be thorough with the ACs but give the model some space. The planner routinely finds issues with the worker's output, but that's what it is for.

sejje 1mo ago
Plan with sol-med, implement with luna-high. Rarely a problem.
- ericfr11 1mo ago
  Same for me, with Claude Opus/Sonnet. All the models are almost equivalent if well steered
ericfr11 1mo ago
Harness is the key. I built my own to "talk" our institutional knowledge and it's working great
thesneakykoala 1mo ago
[dead]

zeven7 1mo ago

I bounce between Sol high/medium and Luna max. I don't know why you'd use anything between Luna max and Sol medium. Luna is so extremely cheap and cranked up to max it does anything I'd want Terra to do for a fraction of the cost. What is Terra for?

Juvination 1mo ago
One thing I've really noticed with Luna Max is its speed. I've got a review script setup on a custom Pi extension. Luna finds some issues/some false positives, while Sol finds issues but disregards false positives. The biggest thing is Sol finishes in about half the time.

Art9681 1mo ago

Terra is great. It's wild how different our experiences are. Install the Superpowers plugin. Behold.

blackqueeriroh 1mo ago
Man, Superpowers is sooooo good. I use it with Opus, Sonnet, and Fable, and its review cycles are truly clutch. I layer on CodeRabbit for PR Review and it’s just so solid.
MuteXR 1mo ago
In my experience, Superpowers has begun to massively slow down the capable models at this point. The skills they add are incredibly bloated and just get you worse results nowadays, tbh.
- agrippanux 1mo ago
  I loved Superpowers and evangelized it heavily. This week I removed it because it now gets in the way of the frontier models.

deleted 1mo ago

[deleted]