5 ms·

Co-designing a sparse music codec with ChatGPT o3

52 points by avaku 1y ago

jstanley 1y ago
For sure programming with AI is a game-changer, and o3 in particular is really quite good at maths. I've tried getting ChatGPT to write blog posts for me as well, but it seems to struggle with knowing which things are important and which things aren't. You need to apply a lot of editorial control. For example: > We swapped 3 × 3 windows for 5 × 5, removed global gains then re-introduced per-occurrence magnitudes, and replaced hard clamping with bilinear interpolation so gradients would flow What does this stuff mean? Why mention these very specific details in the middle of a very high-level overview? Obviously ChatGPT said this because there was a part in your conversation where you decided to make these changes, but without any of the context behind that it is meaningless to readers.
- ninininino 1y ago
  > Obviously ChatGPT said this because there was a part in your conversation where you decided to make these changes, but without any of the context behind that it is meaningless to readers. Have you considered you might not be the target audience the author wrote this for?
  - th0ma5 1y ago
    Do AI commentators only expect to be read by AI fans?
    - jstanley 1y ago
      I am an AI fan, I just thought the random specific irrelevant details sprinkled in the middle were out of place in such a high level overview. One technique I use is to paste a draft into a new conversation and let ChatGPT work on it some more without the context from your earlier chat. You can do this several times.
      - avaku 1y ago
        You're right, your comment is valid. I will make better blog posts in the future :)
- avaku 1y ago
  I thought that the full details might not be interesting for people, since the algorithm is just a mock up of the idea, but which already works! If I wrote the whole post about all the details, it would be very long. I can see how it's a bit out of context, because other parts of the algorithm are not described. But at least I uploaded it to github :)
cypherpunks01 1y ago
I've had similar breakthroughs in the pet-project department lately too. Normally I'd never write small projects in languages I'm unfamiliar with, or learn how to do Android app development just to fix a minor issue with an open-source app, but pair coding with claude code has made these things much more achievable. Vibe coding quickly goes off the rails towards garbage outputs of course, but I've found actually reviewing and guiding the quality of the work piece-by-piece can work with the right effort. There was a very cool small hardware project posted here some time ago, I forgot what it was, where the creator admitted that he had almost no prior experience doing hardware design/build, but pushed through it solo with LLM assistance far enough to get some pretty impressive outputs.
- cube2222 1y ago
  Yeah. I love the Zed AI assistant for the way it manages context (inline edits in the context of the chat) and how "raw" it is. However, I mostly use Goland for coding, so was getting annoyed at having to switch constantly. So, in a couple of evenings (with already having a working PoC evening one) I managed to basically replicate a Zed AI-like integration as a Goland plugin (much less polished and feature-rich, but covering exactly what I need). I've never written kotlin nor a jetbrains plugin before, and since what I wanted is quite complex, it would've easily taken me 1-4 weeks full-time work otherwise - which is to say I would've never done it. It required a ton of grepping around the intellij-community repo too, to see what is available, and what existing patterns are (all done by the AI of course). In this case I vibe coded it (I deem it fine for side-projects, not for production work of course) with Claude Code and it cost me ~$100 all in all, while I was mostly chilling / gaming. Later stages definitely required me to read the code and understand it, since as you say, vibe coding falls apart at a certain point pretty drastically. But having a working skeleton (and code I can modify instead of authoring from scratch) I could easily polish it to sufficient stability. All in all, a pretty magical experience, and I now use my new plugin all day. Not only amazing bang for buck, but just enabling me to do side projects that I otherwise just wouldn't have the time nor want to put in the effort to do.
jonplackett 1y ago
Very similar experiences coding with AI. Getting so much more done and enjoying it more. But people will still read this and say, nah AI is just hype...
- _bin_ 1y ago
  I don't know if it's a skill issue but there seems to be a ceiling to the problem complexity it can handle. It's alright for web but I haven't gotten great results from anything more. It also falls over when you need it done your way, not the model's preferred way (aka whatever is most likely). When you need a UI library besides shadcn. When you're using Svelte over React. Etc. It would be really constructive if people with other experiences could share their chats so we could see what they're doing differently.
  - th0ma5 1y ago
    I can't upvote ya enough. There's some kind of Clever Hans thing where people aren't honest with how much fudging they have to do, or how brittle the results are.
    - avaku 1y ago
      I made this blog post, because I did very little fudging and was shocked by how good the results are. However, this only happened with the latest model (o3). That's why I wanted to describe my approach. First, give AI the model / architecture description, and make sure it understands. Then proceed to the code.
      - _bin_ 1y ago
        Yep, I've heard many people have much stronger results with o3 and am glad it's working for them! I think it would help if you could dig up and publish at least a few of your chats through ideation/architecture, initial implementation, refinement, testing, etc. It might help those of us luddites who can't seem to make even o3 work the way we want it to.
  - 1y ago
cwmoore 1y ago
Mystifying work
- avaku 1y ago
  I am sorry for not making a long blog post with all the details, I just wanted to highlight how the cooperation with AI went to the next level (model design, then implementation). At least I posted the code :) It's just a mock up, and the model will evolve, so the architecture of it is not the important part.

52 points by avaku 1y ago

For sure programming with AI is a game-changer, and o3 in particular is really quite good at maths. I've tried getting ChatGPT to write blog posts for me as well, but it seems to struggle with knowing which things are important and which things aren't. You need to apply a lot of editorial control. For example: > We swapped 3 × 3 windows for 5 × 5, removed global gains then re-introduced per-occurrence magnitudes, and replaced hard clamping with bilinear interpolation so gradients would flow What does this stuff mean? Why mention these very specific details in the middle of a very high-level overview? Obviously ChatGPT said this because there was a part in your conversation where you decided to make these changes, but without any of the context behind that it is meaningless to readers.

ninininino 1y ago
> Obviously ChatGPT said this because there was a part in your conversation where you decided to make these changes, but without any of the context behind that it is meaningless to readers. Have you considered you might not be the target audience the author wrote this for?
- th0ma5 1y ago
  Do AI commentators only expect to be read by AI fans?
  - jstanley 1y ago
    I am an AI fan, I just thought the random specific irrelevant details sprinkled in the middle were out of place in such a high level overview. One technique I use is to paste a draft into a new conversation and let ChatGPT work on it some more without the context from your earlier chat. You can do this several times.
    - avaku 1y ago
      You're right, your comment is valid. I will make better blog posts in the future :)
avaku 1y ago
I thought that the full details might not be interesting for people, since the algorithm is just a mock up of the idea, but which already works! If I wrote the whole post about all the details, it would be very long. I can see how it's a bit out of context, because other parts of the algorithm are not described. But at least I uploaded it to github :)

cypherpunks01 1y ago

I've had similar breakthroughs in the pet-project department lately too. Normally I'd never write small projects in languages I'm unfamiliar with, or learn how to do Android app development just to fix a minor issue with an open-source app, but pair coding with claude code has made these things much more achievable. Vibe coding quickly goes off the rails towards garbage outputs of course, but I've found actually reviewing and guiding the quality of the work piece-by-piece can work with the right effort. There was a very cool small hardware project posted here some time ago, I forgot what it was, where the creator admitted that he had almost no prior experience doing hardware design/build, but pushed through it solo with LLM assistance far enough to get some pretty impressive outputs.

cube2222 1y ago
Yeah. I love the Zed AI assistant for the way it manages context (inline edits in the context of the chat) and how "raw" it is. However, I mostly use Goland for coding, so was getting annoyed at having to switch constantly. So, in a couple of evenings (with already having a working PoC evening one) I managed to basically replicate a Zed AI-like integration as a Goland plugin (much less polished and feature-rich, but covering exactly what I need). I've never written kotlin nor a jetbrains plugin before, and since what I wanted is quite complex, it would've easily taken me 1-4 weeks full-time work otherwise - which is to say I would've never done it. It required a ton of grepping around the intellij-community repo too, to see what is available, and what existing patterns are (all done by the AI of course). In this case I vibe coded it (I deem it fine for side-projects, not for production work of course) with Claude Code and it cost me ~$100 all in all, while I was mostly chilling / gaming. Later stages definitely required me to read the code and understand it, since as you say, vibe coding falls apart at a certain point pretty drastically. But having a working skeleton (and code I can modify instead of authoring from scratch) I could easily polish it to sufficient stability. All in all, a pretty magical experience, and I now use my new plugin all day. Not only amazing bang for buck, but just enabling me to do side projects that I otherwise just wouldn't have the time nor want to put in the effort to do.

jonplackett 1y ago

Very similar experiences coding with AI. Getting so much more done and enjoying it more. But people will still read this and say, nah AI is just hype...

_bin_ 1y ago
I don't know if it's a skill issue but there seems to be a ceiling to the problem complexity it can handle. It's alright for web but I haven't gotten great results from anything more. It also falls over when you need it done your way, not the model's preferred way (aka whatever is most likely). When you need a UI library besides shadcn. When you're using Svelte over React. Etc. It would be really constructive if people with other experiences could share their chats so we could see what they're doing differently.
- th0ma5 1y ago
  I can't upvote ya enough. There's some kind of Clever Hans thing where people aren't honest with how much fudging they have to do, or how brittle the results are.
  - avaku 1y ago
    I made this blog post, because I did very little fudging and was shocked by how good the results are. However, this only happened with the latest model (o3). That's why I wanted to describe my approach. First, give AI the model / architecture description, and make sure it understands. Then proceed to the code.
    - _bin_ 1y ago
      Yep, I've heard many people have much stronger results with o3 and am glad it's working for them! I think it would help if you could dig up and publish at least a few of your chats through ideation/architecture, initial implementation, refinement, testing, etc. It might help those of us luddites who can't seem to make even o3 work the way we want it to.
- 1y ago

cwmoore 1y ago

Mystifying work

avaku 1y ago
I am sorry for not making a long blog post with all the details, I just wanted to highlight how the cooperation with AI went to the next level (model design, then implementation). At least I posted the code :) It's just a mock up, and the model will evolve, so the architecture of it is not the important part.