6 ms·

? What world are you living on.

aogaili 5d ago
Let me ask you, have you noticed that all the major labs are releasing models very similar in gains and performance? How do you explain that other than juicing the max out of the current architecture and processes.
- tadejpetric 5d ago
  They have been releasing similar gains/performance for years now... Pretty much never has one company been dominant for a long time (except the initial GPT3/4 release I suppose. That period took a while for others to catch up)
  - aogaili 5d ago
    Yeah - that's why I think they are basically squeezing the scaling laws and the current architecture with incremental innovations. I expect they will continue optimizing and improving for the current use cases/benchmarks. But the core capabilities will stagnant, that's why they will be forced to slow down until another major breakthrough happens. I personally don't think LLMs have any understanding of the world, nor any imagination or even agency. I think it got very good as recognizing patterns and shapes in human thinking, language and knowledge, but that's pretty much it.
- bpodgursky 5d ago
  ? "everyone is advancing but tied" is completely different than arguing we're at the top of an S curve
  - aogaili 5d ago
    I think it's narrowing at the top.
an0malous 5d ago
The one where it’s infeasible that a company that needs to live up to a valuation of like 5% of US GDP but is still burning cash and has no moat would willingly decide to slow down the pace of development of their core technology
- gfody 5d ago
  > has no moat I would love to see someone try to escape with a copy of openai's raw unaligned base model
  - naishoya 5d ago
    I expect they are slowing down in an attempt to keep from having those models improve to the point where they can effectively escape unassisted into the wild.
  - quikoa 5d ago
    There is no need for a copy. If another can do most of what OpenAI/Anthropic can do at a fraction of the cost (like the Chinese models) then the moat evaporates.
    - gfody 5d ago
      the capability gap is underestimated. the raw unaligned base model from a pretrain is like the telemetry recorded from a particle accelerator - it's hugely valuable and not just because of the cost sunk building a collider. the things are kept under air-gapped national weapons grade security measures not because it's literally going to escape and threaten the world but because if somebody walked out the door with a copy they would have everything. the capabilities we see at the surface are mostly the result of mining the great unknown space and attaching feeble control surfaces, ablating/lobotomizing dangerous areas, and fencing off illegal/secret/embarrassing ones.
      - bigbadfeline 5d ago
        > the capability gap is underestimated. The capability gap is in the minds of the users but the frontiers' closed business models run opposite to it. > the things are kept under air-gapped national weapons grade security measures That's an enhancement of closed but not of moat > because if somebody walked out the door with a copy they would have everything. Including you and me? I'm not sure this is comforting news, despite the "trust me bro" asurances from behind the door where we can't see or verify anything.
gyguu 5d ago
How long would it take for a team of 60 to replicate the software that operates a neobank? Serious question - I want serious responses pls ty.
manmal 5d ago
Unless you’re making 3D models of apartments and oneshotting trivial games, I‘m not convinced Astra writes better code than Sol. I‘m way more often disappointed about Astra forgetting some aspect that Sol just nails.

Let me ask you, have you noticed that all the major labs are releasing models very similar in gains and performance? How do you explain that other than juicing the max out of the current architecture and processes.

tadejpetric 5d ago
They have been releasing similar gains/performance for years now... Pretty much never has one company been dominant for a long time (except the initial GPT3/4 release I suppose. That period took a while for others to catch up)
- aogaili 5d ago
  Yeah - that's why I think they are basically squeezing the scaling laws and the current architecture with incremental innovations. I expect they will continue optimizing and improving for the current use cases/benchmarks. But the core capabilities will stagnant, that's why they will be forced to slow down until another major breakthrough happens. I personally don't think LLMs have any understanding of the world, nor any imagination or even agency. I think it got very good as recognizing patterns and shapes in human thinking, language and knowledge, but that's pretty much it.
bpodgursky 5d ago
? "everyone is advancing but tied" is completely different than arguing we're at the top of an S curve
- aogaili 5d ago
  I think it's narrowing at the top.

an0malous 5d ago

The one where it’s infeasible that a company that needs to live up to a valuation of like 5% of US GDP but is still burning cash and has no moat would willingly decide to slow down the pace of development of their core technology

gfody 5d ago
> has no moat I would love to see someone try to escape with a copy of openai's raw unaligned base model
- naishoya 5d ago
  I expect they are slowing down in an attempt to keep from having those models improve to the point where they can effectively escape unassisted into the wild.
- quikoa 5d ago
  There is no need for a copy. If another can do most of what OpenAI/Anthropic can do at a fraction of the cost (like the Chinese models) then the moat evaporates.
  - gfody 5d ago
    the capability gap is underestimated. the raw unaligned base model from a pretrain is like the telemetry recorded from a particle accelerator - it's hugely valuable and not just because of the cost sunk building a collider. the things are kept under air-gapped national weapons grade security measures not because it's literally going to escape and threaten the world but because if somebody walked out the door with a copy they would have everything. the capabilities we see at the surface are mostly the result of mining the great unknown space and attaching feeble control surfaces, ablating/lobotomizing dangerous areas, and fencing off illegal/secret/embarrassing ones.
    - bigbadfeline 5d ago
      > the capability gap is underestimated. The capability gap is in the minds of the users but the frontiers' closed business models run opposite to it. > the things are kept under air-gapped national weapons grade security measures That's an enhancement of closed but not of moat > because if somebody walked out the door with a copy they would have everything. Including you and me? I'm not sure this is comforting news, despite the "trust me bro" asurances from behind the door where we can't see or verify anything.

gyguu 5d ago

How long would it take for a team of 60 to replicate the software that operates a neobank? Serious question - I want serious responses pls ty.

manmal 5d ago

Unless you’re making 3D models of apartments and oneshotting trivial games, I‘m not convinced Astra writes better code than Sol. I‘m way more often disappointed about Astra forgetting some aspect that Sol just nails.