7 ms·

The most interesting part about this for me is that they decided to create their own language or DSL for the task at hand. So it's not just a large language mod

The most interesting part about this for me is that they decided to create their own language or DSL for the task at hand. So it's not just a large language model; it's an LLM with its own language. I have a feeling that the best AI systems to come will, in fact, be a complete package like this: a harness, a DSL, and an entire package designed to produce certain outcomes cheaper and faster. And producing that complete package is why software engineering will not be obsolete.

WithinReason 8d ago
I agree. I'm waiting for someone to invent a programming language designed for LLMs where for a given partial program p and candidate token t it's possible to tell whether p+t can be the prefix of a correct program or not so that t can be excluded from the LLM's probability distribution at generation time, so the LLM can only generate correct programs. Or something like that.
- cpill 8d ago
  doesn't this imply it would solve the halting problem?
  - WithinReason 8d ago
    No, practical programs don't need full Turing completeness. Most programs you want to write you intend to either halt or run indefinitely, because you want to avoid crashes.
    - deleted 8d ago
      [deleted]
modemNoises 8d ago
Why can't an AI be trained to generate "the whole package"? It too is just software
- znnajdla 8d ago
  There is a well known problem with LLMs that if you feed it its own output, it gradually gets worse and worse. We haven't really understood what the limitations of LLMs are. And I do not know. But as a person who uses Fable and Sol regularly to design my own new programming language, they suck at the task of defining new systems coherently. So far no AI I have tried is good at defining new coherent systems well. I suspect it's because of the recursive problem of using it on its own output. So, a human is still needed to define and write the sort of first genesis of the system, and then AI can take it once the problem has been defined. But defining the problem and using it on itself is what AI is really bad at (for now).

WithinReason 8d ago

I agree. I'm waiting for someone to invent a programming language designed for LLMs where for a given partial program p and candidate token t it's possible to tell whether p+t can be the prefix of a correct program or not so that t can be excluded from the LLM's probability distribution at generation time, so the LLM can only generate correct programs. Or something like that.

cpill 8d ago
doesn't this imply it would solve the halting problem?
- WithinReason 8d ago
  No, practical programs don't need full Turing completeness. Most programs you want to write you intend to either halt or run indefinitely, because you want to avoid crashes.
  - deleted 8d ago
    [deleted]

modemNoises 8d ago

Why can't an AI be trained to generate "the whole package"? It too is just software

znnajdla 8d ago
There is a well known problem with LLMs that if you feed it its own output, it gradually gets worse and worse. We haven't really understood what the limitations of LLMs are. And I do not know. But as a person who uses Fable and Sol regularly to design my own new programming language, they suck at the task of defining new systems coherently. So far no AI I have tried is good at defining new coherent systems well. I suspect it's because of the recursive problem of using it on its own output. So, a human is still needed to define and write the sort of first genesis of the system, and then AI can take it once the problem has been defined. But defining the problem and using it on itself is what AI is really bad at (for now).