5 ms·

I asked Astra to try and fix an issue in my codebase I've been battling for a month. Day and half later on high and it's still circling the drain. Seems like wi

I asked Astra to try and fix an issue in my codebase I've been battling for a month. Day and half later on high and it's still circling the drain. Seems like with each new model we praise it based on toys, but then reality hits the ground soon and we wait for the next model. In the meantime, good ol' brain.

pluc 10d ago
As long as it keeps AI in the news
dwallin 10d ago
I’m just trying to understand, there’s an issue you’ve been unable to deal with for a month, and Astra not being able to handle it in 36 hours is disqualifying?
- Keyframe 10d ago
  On and off for a month, not straight month and not dedicated time. I am quite confident I will be able to solve it within a week or two of dedicated time. Here, on the other hand, we have this supposedly powerful thing that can do magic, unlike myself. We'll see. I'm not even confident, by looking at what it does, that it will even be able to solve it. Only thing I saw that I like is it's better in communication style compared to Fable and Opus.
- pllbnk 10d ago
  Long time ago I had one particular hardware issue, which Opus 4.6 found a workaround to fix. I don't remember the workaround and being careless (I thought I could ask an LLM again if I needed) I lost that solution. Some time passed and I needed it again - neither one of the newer models is able to come up with a nice solution I had back then. They can solve the issue and find a different workaround eventually, but not as nice. On a side note, I could use 4.6 again of course and try to reproduce it, somehow I only thought about it now writing this comment. Anyway, my point is that I think these frontier companies are advertising their one-shot model abilities, but underneath the models aren't getting so much better as they try to make you believe.
comboy 10d ago
Have you tried turning it on and off? More seriously try to remove agent memory or make it refactor it or go through the docs and look for inconsistencies. But I still don't know how people work with codex when it keeps resetting the context so often, I mean it's surprisingly good at making notes to itself and can follow a long task, but if there are multiple instructions it sometimes forgets some of them.
- Keyframe 10d ago
  I just did that and turned on max, let's see if that thing is more of a force!
  - comboy 9d ago
    I'm curious how it went.
    - Keyframe 9d ago
      26% weekly limit left, 21+ hours strong and still going. I can see the progress though, but I am not confident it will finish before weekly limit is done for (ChatGPT Pro 20x subscription). Since I don't really use LLMs for anything else, except pestering gemini for stupid questions instead of straight up googling it, that's ok. I just wish it would either finish or let me hand it over to it once again whence limit resets.
mywacaday 10d ago
[dead]
therealdrag0 10d ago
Can you say anymore about the issue it can’t solve?
- Keyframe 9d ago
  Yeah absolutely. I've been working on my rendering engine on and off for the past 20 odd years. It has been used in commercially in my gigs. However, it's what's considered an offline renderer (like 30+ minutes to render a frame). Last few years, on and off again, I've been slowly working towards a limited in scope realtime version of it. One of the components I've managed to kind of bring almost to the finish line is for it to work with a limited VRAM budget and streaming geometry, where camera has a (I'm simplifying here) bubble around you (around, not a frustum in-front because I need semi global visibility since I'm doing realtime raytracing). This bubble as it moves dictates what geometry and resources are loaded, sometimes even ahead of time because I also do a form of motion prediction. Far out things, outside of the bubble, and things on the border of the bubble go through a coarse to fine representation. That way I can render distant stuff which I don't need much detail for it. This part is a bit buggy, to say the least, but also viably measurable so it's a prime candidate for an LLM to loop/goal over it. Or at least I though so. With max I can see now the progress, but with max I'm now 21+ hours into it and not a clear solution ahead yet. I think I'm running out of weekly quota on ChatGPT Pro 20x for this. I actually bought subscription exactly to drive this home since I don't use LLMs much anymore except gemini in lieu of googling. I haven't told it how to do stuff, but what I want to do and it picked up my tests to see if it's making progress and I can see it's tuning the movement prediction and caching logic and it fixed few minor bugs. So, it seems to be doing the right approach.. it just takes an awful lot of time. Question is if it's a viable approach even though, but it didn't flag it as wrong. It actually agreed with me and unlike Claude I haven't gotten the impression it's much of a yesman.
  - therealdrag0 9d ago
    Interesting, that was a bit hard to follow because of my own lack of ‘context’ :). But it sounds tough. I imagine since rendering has a visual component that adds to the difficulty for LLM. Though you said it’s measurable, is the measurableness visual? And it sounds like you just sent it off with no direction; but good prompting and direction can go a long way. Is the architecture and design and constraints well documented in the codebase?

pluc 10d ago

As long as it keeps AI in the news

dwallin 10d ago

I’m just trying to understand, there’s an issue you’ve been unable to deal with for a month, and Astra not being able to handle it in 36 hours is disqualifying?

Keyframe 10d ago
On and off for a month, not straight month and not dedicated time. I am quite confident I will be able to solve it within a week or two of dedicated time. Here, on the other hand, we have this supposedly powerful thing that can do magic, unlike myself. We'll see. I'm not even confident, by looking at what it does, that it will even be able to solve it. Only thing I saw that I like is it's better in communication style compared to Fable and Opus.
pllbnk 10d ago
Long time ago I had one particular hardware issue, which Opus 4.6 found a workaround to fix. I don't remember the workaround and being careless (I thought I could ask an LLM again if I needed) I lost that solution. Some time passed and I needed it again - neither one of the newer models is able to come up with a nice solution I had back then. They can solve the issue and find a different workaround eventually, but not as nice. On a side note, I could use 4.6 again of course and try to reproduce it, somehow I only thought about it now writing this comment. Anyway, my point is that I think these frontier companies are advertising their one-shot model abilities, but underneath the models aren't getting so much better as they try to make you believe.

comboy 10d ago

Have you tried turning it on and off? More seriously try to remove agent memory or make it refactor it or go through the docs and look for inconsistencies. But I still don't know how people work with codex when it keeps resetting the context so often, I mean it's surprisingly good at making notes to itself and can follow a long task, but if there are multiple instructions it sometimes forgets some of them.

Keyframe 10d ago
I just did that and turned on max, let's see if that thing is more of a force!
- comboy 9d ago
  I'm curious how it went.
  - Keyframe 9d ago
    26% weekly limit left, 21+ hours strong and still going. I can see the progress though, but I am not confident it will finish before weekly limit is done for (ChatGPT Pro 20x subscription). Since I don't really use LLMs for anything else, except pestering gemini for stupid questions instead of straight up googling it, that's ok. I just wish it would either finish or let me hand it over to it once again whence limit resets.

mywacaday 10d ago

[dead]

therealdrag0 10d ago

Can you say anymore about the issue it can’t solve?

Keyframe 9d ago
Yeah absolutely. I've been working on my rendering engine on and off for the past 20 odd years. It has been used in commercially in my gigs. However, it's what's considered an offline renderer (like 30+ minutes to render a frame). Last few years, on and off again, I've been slowly working towards a limited in scope realtime version of it. One of the components I've managed to kind of bring almost to the finish line is for it to work with a limited VRAM budget and streaming geometry, where camera has a (I'm simplifying here) bubble around you (around, not a frustum in-front because I need semi global visibility since I'm doing realtime raytracing). This bubble as it moves dictates what geometry and resources are loaded, sometimes even ahead of time because I also do a form of motion prediction. Far out things, outside of the bubble, and things on the border of the bubble go through a coarse to fine representation. That way I can render distant stuff which I don't need much detail for it. This part is a bit buggy, to say the least, but also viably measurable so it's a prime candidate for an LLM to loop/goal over it. Or at least I though so. With max I can see now the progress, but with max I'm now 21+ hours into it and not a clear solution ahead yet. I think I'm running out of weekly quota on ChatGPT Pro 20x for this. I actually bought subscription exactly to drive this home since I don't use LLMs much anymore except gemini in lieu of googling. I haven't told it how to do stuff, but what I want to do and it picked up my tests to see if it's making progress and I can see it's tuning the movement prediction and caching logic and it fixed few minor bugs. So, it seems to be doing the right approach.. it just takes an awful lot of time. Question is if it's a viable approach even though, but it didn't flag it as wrong. It actually agreed with me and unlike Claude I haven't gotten the impression it's much of a yesman.
- therealdrag0 9d ago
  Interesting, that was a bit hard to follow because of my own lack of ‘context’ :). But it sounds tough. I imagine since rendering has a visual component that adds to the difficulty for LLM. Though you said it’s measurable, is the measurableness visual? And it sounds like you just sent it off with no direction; but good prompting and direction can go a long way. Is the architecture and design and constraints well documented in the codebase?