8 ms·

> Harnesses are the next frontier. If LLMs are electricity, harnesses are the “electronics.” I really though this comment was a satire ...

grey-area 27d ago
Sadly, many people have bought into the cult that LLMs will lead to AGI. I guess if that is your worldview then all this babbling about new frontiers makes more sense. They probably used an LLM to come up with this bizarre metaphor.
- theturtletalks 27d ago
  Did I even mention AGI? All I’m saying is that we’re hitting a plateau with how good models are while harnesses are untapped potential. And with Pi, you can swap models like electricity companies. Yes for now, the electricity is better with some companies but this will stabilize. And no I came up with the metaphor all on my own, send me the chat of you getting the LLM to come up with it. Why not argue based on merit instead of strawman and ad hominem attacks?
  - grey-area 27d ago
    I’m afraid it’s a terrible metaphor, starting with the fact that LLMs are nothing like electricity, and the relation of harnesses to them is nothing like that of electronics to electricity, save perhaps one is a prerequisite of the other. Harnesses (and the concept of agents before them) presuppose competence in LLMs which simply doesn’t exist.
    - theturtletalks 27d ago
      I didn’t come with the electricity idea, it was Sam Altman saying it will be like a utility down the line and metered[0]. What would the “electronics” be in your opinion? 0. https://www.businessinsider.com/sam-altman-ai-utility-electricity-water-openai-2026-3 https://www.businessinsider.com/sam-altman-ai-utility-electr...
      - grey-area 27d ago
        Altman is a salesman selling flimflam to people who should know better. His idea of metering is predicated on the thing he’s selling being AGI, it is not, and all his predictions have turned to dust. Also that isn’t how metaphors work - they illuminate by comparison, if the comparison is not close they are not useful.
        theturtletalks 27d ago
        If it is metered and like a utility, Sam Altman will not benefit alone. All the models will have plateaued and you can swap for any of them. Then the only differentiator is the harness. I don’t believe in AGI, but that doesn’t mean I don’t find AI useful. I just understand that the correct harness can take them to the next level.
    - rasputin243 27d ago
      “Just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years” - Andrew Ng
      - Avicebron 27d ago
        Is it just a coincidence that he works in the space and will directly benefit if this is true?
    - qarl2 27d ago
      > presuppose competence in LLMs which simply doesn’t exist. Then how do you explain the wild success at using them for development?
      - grey-area 26d ago
        Guided by humans, code generators which have ingested the worlds’ code and can recognise and generate patterns can be useful tools. I wouldn’t personally qualify it as a wild success as we are early and there are significant downsides. That doesn’t make them intelligent agents which think independently.
        qarl2 26d ago
        Would you like to see some of my own examples of wild success? I have a system that entirely reverse engineers old arcade games. Creates semantic symbol mappings that were considered impossible just a couple years ago. Granted, it took me a couple weeks to build the system. From impossible to a couple weeks in just a couple years. Would you like to see it or continue to pretend these things don't exist? Your call. (It's finding the coolest stuff - the anti-tampering hacks they put into the old machines is fascinating.)
        grey-area 26d ago
        I’ve made my own arcade games with an LLM. It was fun though the code wasn’t great. Mostly I felt they reproduced games from the training data, some with more examples available worked better than others. I use them most days for work, and for that reason don’t trust them that much and certainly don’t worry or fantasise about AGI.
        qarl2 26d ago
        You were making some pretty strong statements about the usefulness of harnesses that seemed to me to be entirely detached from reality. Maybe I misunderstood you? Because I can provide evidence to correct that misconception. Another of my projects is to incorporate Pixar's ideas from RenderMan into a 3d printer slicer. Displacement shaders, in a 3d printer, have never been done before. Would you like to see that? I had the idea 10 years ago but it was too tedious to implement. I have a working system now in just a couple weeks AGAIN. Anyone claiming agents aren't profoundly useful is WRONG. If you disagree, please let's discuss it.
- sph 27d ago
  > Sadly, many people have bought into the cult that LLMs will lead to AGI You can never tell if the goomba opinion of the forum will agree we have reached AGI (seen that happen on a few threads lately) or will readily call that a ludicrous proposition.
- jbstack 27d ago
  I find it difficult to understand people who are wildly skeptical about LLMs leading to AGI (assuming we can even agree on what that means). Consider: - They can already reason better than many humans and are still improving all the time - Harnesses are improving all the time - We're already exploring things like long term memory, long term goals, and other things that humans have which LLMs traditionally lack - An AI agent can read and reason about every piece of AI research ever published, including looking for insights that humans may have missed. A team of humans could never do this even if they dedicated their whole lives to it. - They can design and execute experiments on a mass scale to determine what does and doesn't work - Large AI labs have more than sufficient resources and motivation to throw at the problem, and are in fact doing this.
  - Topfi 27d ago
    So you believe LLMs (despite their inherent deficiencies vs EBMs [0], etc.) can lead to what you'd consider AGI, but you also admit that there is no agreement what AGI actually would be and you further don't provide your own definition? But you are surprised that some (like e.g. Yann LeCun (I am very convinced by his published works beyond his authority in the field, but am willing to admit his could be seen as a biased position)) are skeptical? If you provide what you'd consider AGI, we may not agree on that definition, but I and other skeptics could at least discuss with you whether A.) that seems reasonably achievable given LLMs inherent limitations and B.) whether any of what you'd listed is actually likely to get us there. As it stands, neither is possible without knowing what you believe AGI to be, but for what it's worth, coming from someone who both does see LLMs as valuable tools but whose definition for AGI also contains, among other things, reliable self-assessment of factual uncertainty [1] and basic counting and grade school maths [0][2] without tools or eternally scaling training data, I have yet to read any evidence that LLMs can achieve my, rather strict, metric for AGI. These models are amazing tools, their ability to leverage massive amounts of high quality training data to further sciences truly awe inspiring, but that does not mean intelligence, at least in my definition that requires some internals these models have never been proven to possess. It's nuts that solving Erdos problems can be done by a model which struggles to count or solve a sudoku without external tools, but that's where the technology has been for years now and no paper I have read has shown that LLMs can overcome that to any scalable degree. You can push further with training data, but the limitations remain, albeit less noticeable. Any externalities, be it tools (self-scripted or called by the model), external memory solutions of all shapes and sizes, etc. I personally also feel cannot be required for or lead towards AGI as intelligence may be better leveraged by such externalities, but should never require them, so much of your suggestion I feel shouldn't be considered even if one believes LLMs can yield intelligence. I will admit that I am very extreme here though, this is not a position held by everyone for good reason. At the end I will always point towards the "extraordinary claims require extraordinary proof" of it all and that LLMs, in the face of any doubt, should be viewed akin to how Stockfish can play better than any grandmaster, but that does not mean intelligence, at least in my world. If your definition for intelligence does not require basic arithmetics or an understanding of ones own knowledge gaps, then maybe LLMs can achieve that, but I'd push back on that truly rising to the AGI moniker. Maybe a more comprehensive or even my definition of AGI is possible whilst keeping the autoregressive nature after all, but there is no evidence supporting that by itself and quite a few things that haven't even begun to be overcome before something of that magnitude could be honestly considered. It's akin to "let's colonise Mars by 2020 or 2030 or 2040 for sure, then terraform it" proposals. If that were possible, wouldn't we see a lot of these methods applied on earth and in a moon base long before (as in, we'd have had a permanent moon base in the early 2000s)? Same with LLMs, if they can truly yield AGI, we'd see some of the major deficiencies dealt with long before. The fact that we neither are terraforming earth, nor have any permanent off world colonies, nor have solved some of the listed, inherent limitations with LLMs by their design, that's what informs my skepticism that both are reasonably achievable in the timelines some industry "experts" (read hype merchants) propose on the regular. You tend to see some progress, a path toward solving actionable problems long before full implementation, at least in the real world... [0] https://logicalintelligence.com/blog/energy-based-model-sudoku-demo https://logicalintelligence.com/blog/energy-based-model-sudo... [1] https://arxiv.org/html/2607.19367v1 https://arxiv.org/html/2607.19367v1 [2] https://arxiv.org/html/2605.02028v2 https://arxiv.org/html/2605.02028v2
PepegaRoach 27d ago
[dead]
_superposition_ 27d ago
In a sense they are the last frontier imo. At some point a harness will be built that can modify itself to fit the needs of the majority of people's workflows and evolve with them.
- layer8 27d ago
  Then people will want to share and exchange their evolved harnesses. Ways will be found to modularize certain aspects to enable mixing and matching. I’m thinking of how in cyberpunk, people are replacing their cybernetic enhancements all the time. You could alternatively bioengineer your own body towards the desired outcomes, but that’s more constrained by the trajectory your body has already taken, whereas the promise of cybernetic parts is that they are more independently replaceable. (Probably an illusion in practice, but I’m talking about the fictional ideal.) As another analogy, monolithic software tends to quickly become hard to change significantly, whereas a plugin architecture tends to be more flexible and modular, and people can share and combine their various plugins.
wwalexander 27d ago
E = mc^2 + AI
- tomrod 26d ago
  If we take this at face value, this means AI = 0 !
DarmokTanagra 27d ago
Its literally the same people who were making hyperbolic crypto claims a few years ago. This entire forum is infested with shameless hype chasers and biological linkedin bots.

grey-area 27d ago

Sadly, many people have bought into the cult that LLMs will lead to AGI. I guess if that is your worldview then all this babbling about new frontiers makes more sense. They probably used an LLM to come up with this bizarre metaphor.

theturtletalks 27d ago
Did I even mention AGI? All I’m saying is that we’re hitting a plateau with how good models are while harnesses are untapped potential. And with Pi, you can swap models like electricity companies. Yes for now, the electricity is better with some companies but this will stabilize. And no I came up with the metaphor all on my own, send me the chat of you getting the LLM to come up with it. Why not argue based on merit instead of strawman and ad hominem attacks?
- grey-area 27d ago
  I’m afraid it’s a terrible metaphor, starting with the fact that LLMs are nothing like electricity, and the relation of harnesses to them is nothing like that of electronics to electricity, save perhaps one is a prerequisite of the other. Harnesses (and the concept of agents before them) presuppose competence in LLMs which simply doesn’t exist.
  - theturtletalks 27d ago
    I didn’t come with the electricity idea, it was Sam Altman saying it will be like a utility down the line and metered[0]. What would the “electronics” be in your opinion? 0. https://www.businessinsider.com/sam-altman-ai-utility-electricity-water-openai-2026-3 https://www.businessinsider.com/sam-altman-ai-utility-electr...
    - grey-area 27d ago
      Altman is a salesman selling flimflam to people who should know better. His idea of metering is predicated on the thing he’s selling being AGI, it is not, and all his predictions have turned to dust. Also that isn’t how metaphors work - they illuminate by comparison, if the comparison is not close they are not useful.
      - theturtletalks 27d ago
        If it is metered and like a utility, Sam Altman will not benefit alone. All the models will have plateaued and you can swap for any of them. Then the only differentiator is the harness. I don’t believe in AGI, but that doesn’t mean I don’t find AI useful. I just understand that the correct harness can take them to the next level.
  - rasputin243 27d ago
    “Just as electricity transformed almost everything 100 years ago, today I actually have a hard time thinking of an industry that I don’t think AI will transform in the next several years” - Andrew Ng
    - Avicebron 27d ago
      Is it just a coincidence that he works in the space and will directly benefit if this is true?
  - qarl2 27d ago
    > presuppose competence in LLMs which simply doesn’t exist. Then how do you explain the wild success at using them for development?
    - grey-area 26d ago
      Guided by humans, code generators which have ingested the worlds’ code and can recognise and generate patterns can be useful tools. I wouldn’t personally qualify it as a wild success as we are early and there are significant downsides. That doesn’t make them intelligent agents which think independently.
      - qarl2 26d ago
        Would you like to see some of my own examples of wild success? I have a system that entirely reverse engineers old arcade games. Creates semantic symbol mappings that were considered impossible just a couple years ago. Granted, it took me a couple weeks to build the system. From impossible to a couple weeks in just a couple years. Would you like to see it or continue to pretend these things don't exist? Your call. (It's finding the coolest stuff - the anti-tampering hacks they put into the old machines is fascinating.)
        grey-area 26d ago
        I’ve made my own arcade games with an LLM. It was fun though the code wasn’t great. Mostly I felt they reproduced games from the training data, some with more examples available worked better than others. I use them most days for work, and for that reason don’t trust them that much and certainly don’t worry or fantasise about AGI.
        qarl2 26d ago
        You were making some pretty strong statements about the usefulness of harnesses that seemed to me to be entirely detached from reality. Maybe I misunderstood you? Because I can provide evidence to correct that misconception. Another of my projects is to incorporate Pixar's ideas from RenderMan into a 3d printer slicer. Displacement shaders, in a 3d printer, have never been done before. Would you like to see that? I had the idea 10 years ago but it was too tedious to implement. I have a working system now in just a couple weeks AGAIN. Anyone claiming agents aren't profoundly useful is WRONG. If you disagree, please let's discuss it.
sph 27d ago
> Sadly, many people have bought into the cult that LLMs will lead to AGI You can never tell if the goomba opinion of the forum will agree we have reached AGI (seen that happen on a few threads lately) or will readily call that a ludicrous proposition.
jbstack 27d ago
I find it difficult to understand people who are wildly skeptical about LLMs leading to AGI (assuming we can even agree on what that means). Consider: - They can already reason better than many humans and are still improving all the time - Harnesses are improving all the time - We're already exploring things like long term memory, long term goals, and other things that humans have which LLMs traditionally lack - An AI agent can read and reason about every piece of AI research ever published, including looking for insights that humans may have missed. A team of humans could never do this even if they dedicated their whole lives to it. - They can design and execute experiments on a mass scale to determine what does and doesn't work - Large AI labs have more than sufficient resources and motivation to throw at the problem, and are in fact doing this.
- Topfi 27d ago
  So you believe LLMs (despite their inherent deficiencies vs EBMs [0], etc.) can lead to what you'd consider AGI, but you also admit that there is no agreement what AGI actually would be and you further don't provide your own definition? But you are surprised that some (like e.g. Yann LeCun (I am very convinced by his published works beyond his authority in the field, but am willing to admit his could be seen as a biased position)) are skeptical? If you provide what you'd consider AGI, we may not agree on that definition, but I and other skeptics could at least discuss with you whether A.) that seems reasonably achievable given LLMs inherent limitations and B.) whether any of what you'd listed is actually likely to get us there. As it stands, neither is possible without knowing what you believe AGI to be, but for what it's worth, coming from someone who both does see LLMs as valuable tools but whose definition for AGI also contains, among other things, reliable self-assessment of factual uncertainty [1] and basic counting and grade school maths [0][2] without tools or eternally scaling training data, I have yet to read any evidence that LLMs can achieve my, rather strict, metric for AGI. These models are amazing tools, their ability to leverage massive amounts of high quality training data to further sciences truly awe inspiring, but that does not mean intelligence, at least in my definition that requires some internals these models have never been proven to possess. It's nuts that solving Erdos problems can be done by a model which struggles to count or solve a sudoku without external tools, but that's where the technology has been for years now and no paper I have read has shown that LLMs can overcome that to any scalable degree. You can push further with training data, but the limitations remain, albeit less noticeable. Any externalities, be it tools (self-scripted or called by the model), external memory solutions of all shapes and sizes, etc. I personally also feel cannot be required for or lead towards AGI as intelligence may be better leveraged by such externalities, but should never require them, so much of your suggestion I feel shouldn't be considered even if one believes LLMs can yield intelligence. I will admit that I am very extreme here though, this is not a position held by everyone for good reason. At the end I will always point towards the "extraordinary claims require extraordinary proof" of it all and that LLMs, in the face of any doubt, should be viewed akin to how Stockfish can play better than any grandmaster, but that does not mean intelligence, at least in my world. If your definition for intelligence does not require basic arithmetics or an understanding of ones own knowledge gaps, then maybe LLMs can achieve that, but I'd push back on that truly rising to the AGI moniker. Maybe a more comprehensive or even my definition of AGI is possible whilst keeping the autoregressive nature after all, but there is no evidence supporting that by itself and quite a few things that haven't even begun to be overcome before something of that magnitude could be honestly considered. It's akin to "let's colonise Mars by 2020 or 2030 or 2040 for sure, then terraform it" proposals. If that were possible, wouldn't we see a lot of these methods applied on earth and in a moon base long before (as in, we'd have had a permanent moon base in the early 2000s)? Same with LLMs, if they can truly yield AGI, we'd see some of the major deficiencies dealt with long before. The fact that we neither are terraforming earth, nor have any permanent off world colonies, nor have solved some of the listed, inherent limitations with LLMs by their design, that's what informs my skepticism that both are reasonably achievable in the timelines some industry "experts" (read hype merchants) propose on the regular. You tend to see some progress, a path toward solving actionable problems long before full implementation, at least in the real world... [0] https://logicalintelligence.com/blog/energy-based-model-sudoku-demo https://logicalintelligence.com/blog/energy-based-model-sudo... [1] https://arxiv.org/html/2607.19367v1 https://arxiv.org/html/2607.19367v1 [2] https://arxiv.org/html/2605.02028v2 https://arxiv.org/html/2605.02028v2

PepegaRoach 27d ago

[dead]

_superposition_ 27d ago

In a sense they are the last frontier imo. At some point a harness will be built that can modify itself to fit the needs of the majority of people's workflows and evolve with them.

layer8 27d ago
Then people will want to share and exchange their evolved harnesses. Ways will be found to modularize certain aspects to enable mixing and matching. I’m thinking of how in cyberpunk, people are replacing their cybernetic enhancements all the time. You could alternatively bioengineer your own body towards the desired outcomes, but that’s more constrained by the trajectory your body has already taken, whereas the promise of cybernetic parts is that they are more independently replaceable. (Probably an illusion in practice, but I’m talking about the fictional ideal.) As another analogy, monolithic software tends to quickly become hard to change significantly, whereas a plugin architecture tends to be more flexible and modular, and people can share and combine their various plugins.

wwalexander 27d ago

E = mc^2 + AI

tomrod 26d ago
If we take this at face value, this means AI = 0 !

DarmokTanagra 27d ago

Its literally the same people who were making hyperbolic crypto claims a few years ago. This entire forum is infested with shameless hype chasers and biological linkedin bots.