5 ms·

I do think there's something to be said for the big difference in the current abstraction layer jump - the loss of determinism. It is meaningfully different.

• JumpCrisscross 3mo ago
  Plenty of physical (and biological) processes are subjectively non-deterministic outside clean rooms. Doesn’t mean our ancestors couldn’t forge and selectively breed.
  • mrob 3mo ago
    There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism. Only the former can be statistically modeled to characterize and work around the noise with any reliability. If you measure hallucination rate for one prompt it tells you nothing about hallucination rate for another prompt.
    • JumpCrisscross 3mo ago
      > There's a big difference between "deterministic + noise" non-determinism and "intelligent agent" non-determinism I think I agree, but could you expand? My point is humans have made progress on understanding systems without clean determinism. Genetics and cosmology is technically deterministic, practically speaking, but I don’t think that’s accessible to pre-modern societies. Instead, trends and tendencies were observed, acted and capitalized on.
      • mrob 3mo ago
        One big difference is the former doesn't "know" it's being experimented on. Gregor Mendel was able to grow large numbers of pea plants and identify patterns in their inheritance because the non-determinism followed consistent statistical laws. A plant can't suddenly decide to follow different laws of genetics because it saw a biologist. On the other hand, an LLM "knows" what a benchmark is, and is capable of detecting benchmark-like scenarios. A prompt that appears reliable in testing may fail unexpected in the real world. In this way, an LLM is like a stock market (also composed of intelligent agents): the act of experimenting on it changes its properties. Such systems are never reliably predictable.
  • gonzalohm 3mo ago
    I think it's safe to say that computing should be mostly deterministic. I know compilers use heuristics that are stochastic, but come on. Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming.
    • zild3d 3mo ago
      > Imagine if you were given a layer of abstraction that randomly flipped bits from time to time... That's not an abstraction, it's random programming. Computing has never really been deterministic all the way down. Storage gets corrupted, RAM has soft errors, networks drop/reorder packets, schedulers race, caches go stale, distributed systems partition, query planners change plans, ... Obviously those become tolerable because we've developed layers of contracts and understanding the bounds around them. Error correcting codes, checksums, retries, consensus, idempotency, false-positive rates, SLAs, etc. If the abstraction is “delegate this task to a junior engineer, analyst, lawyer, designer, or support rep,” you dont expect deterministic behavior. There's review, constraints, escalation, checklists, tests, and accountability.
    • JumpCrisscross 3mo ago
      > Imagine if you were given a layer of abstraction that randomly flipped bits from time to time To be fair, this isn’t a bad description of human vision.
  • pydry 3mo ago
    Our ancestors also had an infant mortality ratio of 4/10. Determinism is way undervalued.
  • ajuc 3mo ago
    There's a pretty cool article about trying to understand electronics and fix a radio the way we try to understand biology. https://mappingignorance.org/2014/07/02/biologists-cant-understand-biology/ https://mappingignorance.org/2014/07/02/biologists-cant-unde... So now instead of improving the tools of biology so we can actually understand it deeply - we increase complexity of IT so we have to rely on muddy, side-effecty tools of biology to try to infer some of the properties of the systems we made. That's depressing.
    • EvanAnderson 3mo ago
      I feel crushing sadness when I think about how humans made these wonderful deterministic machines that we can understand and manipulate to do our bidding. Within a scant few generations, though, we've managed to effectively turn them into "magic boxes" that, soon, we'll only be able to poke and prod at when they don't do what we want.
  • klabb3 3mo ago
    In engineering you have tolerances to deal with non-determinism. ”Within these bounds” and ”given these assumptions then…” is the foundation of building something _on top of_ those things. LLMs are the same, it relies on heavily exact Turing machines as input but its output is entirely unstable. Even if you can get determinism it will never get anything resembling ”bounds” out of the box. That makes it a poor foundation for building on top of. Ie it’s not a screwdriver, it’s the monkey who’s holding it. I do not understand the need to argue that monkeys are better than screwdrivers at screwing. Just let the monkey be the best version of a monkey.
    • imoverclocked 3mo ago
      You’ve clearly not seen monkeys. /sarcasm I would argue that management is a better discipline to pull from for employing LLMs; It is better equipped to deal with non-determinism and going completely off the rails.
• eptcyka 3mo ago
  If there is something to be said, I have to ask. What specifically is there to be said?
  • zephen 3mo ago
    Non-determinism is not inherently bad. We have had useful non-deterministic tools for (in computer years) several generations. Non-determinism combined with bugginess? That's a terrible combination. It is impossible to gradient-descent your way into a working prompt.
    • wseqyrku 3mo ago
      > Non-determinism is not inherently bad. I would say it is unambiguously defined by the problem you're solving. For example a spellchecker is inherently probabilistic so you shouldn't need hard coded rules here like we did for years. A "bug" then would be considered "weak accuracy" not "crash" or "incorrect behavior", as it might be the case in any other layer. They should add a term for it, maybe "embarrassingly nondeterministic"
      • zephen 3mo ago
        > I would say [whether nondeterminism is bad] is unambiguously defined by the problem you're solving. Sure. You gave an example where it can work. Another example is something like a PCB or chip layout engine. That particular domain (layout) is NP-complete. You'll never have an exhaustive brute force search for the optimal layout, but... You can subsequently easily check whether the produced layout meets all your acceptance criteria or not. Another example that successfully utilizes non-determinism for good outcomes is the application of genetic algorithms to things like antenna design. This works because (a) you have a relatively cheap fitness test; and (b) as with real evolution, the mix of combining working designs and randomly introducing mutations often eventually produces outstanding results. Presumably, if you applied genetic algorithm techniques to, e.g., creating your LLM prompts, you could also get good results, but that would probably quickly get expensive in terms of tokens. So we're left with people just semi-randomly modifying prompts in order to try to tweak results. When it works, it can be amazing. When it doesn't work, it's like a brick wall. I like your "embarrassingly nondeterministic" term, but I somewhat disagree with: > A "bug" then would be considered "weak accuracy" not "crash" or "incorrect behavior", When a lawyer asks an LLM for citations of cases that support his position, he is arguably doing something stupid, because embedding an assertion such as "Show me cases which support X" is just asking for hallucinatory trouble with many current LLMs. Nonetheless, I submit that hallucinations are, by definition, "incorrect behavior" and not merely "weak accuracy." Now, nondeterminism could certainly be useful to the lawyer, in that it could help an LLM make connections that LexisNexis might not have in their database. So asking an LLM for help with legal issues is theoretically not an insane thing to do, but the results need to be checked very carefully.
• tines 3mo ago
  People use the word "determinism" when they really mean something akin to "linearity", i.e. the predictability of a change in input on a change in output. Compilers for example are both deterministic and "more linear" in the sense that I can tell what the output will look like given a change in input (yes yes optimizations violate this to a small degree). LLMs can be made totally deterministic, but a seemingly insignificant change in input can create a drastic change in output, which is the characterstic we don't want.
  • eikenberry 3mo ago
    The generally used term that I'm familiar with for this is "referential transparency", that given the same inputs you'll get the same outputs every time. LLMs can be deterministic (referentially transparent) but almost none are, i.e. when they are given the exact same input they do not return the exact same output.
    • JumpCrisscross 3mo ago
      Wait, what is the light between that definition of referential transparency and determinism?
• archagon 3mo ago
  More than that: it's not an abstraction layer jump, because AI is not an abstraction.