5 ms·

This is a bit of a strawman. The parent comment wasn't suggesting that the inherent defects magically disappear nor was it suggesting that local inference today

by nzeid 2mo ago

This is a bit of a strawman. The parent comment wasn't suggesting that the inherent defects magically disappear nor was it suggesting that local inference today is sufficient for all tasks. At the heart of it, self-hosting liberates your use cases from all the horribly opaque configuration, shadow prompting, etc. And local models are only getting better and more diverse every month.

joshmoody24 2mo ago
> Want it to go away, almost like magic? True, almost like magic and magic are not the same thing, but I'd hesitate to call this a strawman.
john_strinlai 2mo ago
>The parent comment wasn't suggesting that the inherent defects magically disappear that is almost exactly what they said though..? "Want it to go away, almost like magic? Local inference. [...] all of the common LLM defects will go away.
- pessimizer 2mo ago
  "[...]" ----- edit: I suppose I have to spell this out. "[...]" = "When its under your control, and [you're] no longer being forced to hold it wrong," ≈ "liberates your use cases from all the horribly opaque configuration, shadow prompting, etc." and "Almost like magic" is cliché hyperbole. It also means "not magic" in the same way that "almost like a dog" means "not a dog." It was immediately followed by the explicit clarification that was surgically omitted above, in absolute and conscious bad faith. Hence, "[...]"
  - john_strinlai 2mo ago
    i omitted it because it doesnt change my point at all and it makes the quoted text smaller. not because of "absolute and conscious bad faith". people can just read the original from the comment. its not like im quoting from a different website and trying to hide something. look 2 comments above mine and it is right there in full. anyone reading my comment would have seen the original comment already... i have no idea why you took such offense to my comment, but you're reading way too much into it.
  - throw10920 2mo ago
    > "Almost like magic" is cliché hyperbole. It also means "not magic" in the same way that "almost like a dog" means "not a dog." It was immediately followed by the explicit clarification that was surgically omitted above, in absolute and conscious bad faith. Incorrect. The omitted text doesn't change the relevant meaning of the quotation in the slightest. Why are you taking such offense? > Want it to go away, almost like magic? Local inference. When its under your control, and no longer being forced to hold it wrong, all of the common LLM defects will go away. > "Want it to go away, almost like magic? Local inference. [...] all of the common LLM defects will go away. There's no difference between these two statements in the relevant context of the article, which is specifically "long policy documents do not reliably govern agents", or the comments above, which are specifically addressing how that issue is not solved by local agents.
solarkraft 2mo ago
I get your reading. My interpretation is that they’re claiming enshittification and that the local models don’t have that. It’s an understandable view, but I’d be astonished by any local model processing very long context better than any frontier model (and now many racks are we talking).
Aurornis 2mo ago
> This is a bit of a strawman. The parent comment wasn't suggesting that the inherent defects magically disappear The parent comment literally said that the common LLM defects would go away. Direct quote: > all of the common LLM defects will go away
- adrian_b 2mo ago
  Clearly it was meant that they will go away only if you use a custom harness, without the defects that now exist in all vendor-controlled harnesses, because of the non-aligned goals of the vendor and of the user. While it was not spelled explicitly, it should be obvious that nothing can be improved if you run locally an LLM in the same way as the vendor. But you should not do that. It is easy to use better methods for sampling procedures and for prompt and context handling.

by nzeid 2mo ago

joshmoody24 2mo ago

> Want it to go away, almost like magic? True, almost like magic and magic are not the same thing, but I'd hesitate to call this a strawman.

john_strinlai 2mo ago

>The parent comment wasn't suggesting that the inherent defects magically disappear that is almost exactly what they said though..? "Want it to go away, almost like magic? Local inference. [...] all of the common LLM defects will go away.

pessimizer 2mo ago
"[...]" ----- edit: I suppose I have to spell this out. "[...]" = "When its under your control, and [you're] no longer being forced to hold it wrong," ≈ "liberates your use cases from all the horribly opaque configuration, shadow prompting, etc." and "Almost like magic" is cliché hyperbole. It also means "not magic" in the same way that "almost like a dog" means "not a dog." It was immediately followed by the explicit clarification that was surgically omitted above, in absolute and conscious bad faith. Hence, "[...]"
- john_strinlai 2mo ago
  i omitted it because it doesnt change my point at all and it makes the quoted text smaller. not because of "absolute and conscious bad faith". people can just read the original from the comment. its not like im quoting from a different website and trying to hide something. look 2 comments above mine and it is right there in full. anyone reading my comment would have seen the original comment already... i have no idea why you took such offense to my comment, but you're reading way too much into it.
- throw10920 2mo ago
  > "Almost like magic" is cliché hyperbole. It also means "not magic" in the same way that "almost like a dog" means "not a dog." It was immediately followed by the explicit clarification that was surgically omitted above, in absolute and conscious bad faith. Incorrect. The omitted text doesn't change the relevant meaning of the quotation in the slightest. Why are you taking such offense? > Want it to go away, almost like magic? Local inference. When its under your control, and no longer being forced to hold it wrong, all of the common LLM defects will go away. > "Want it to go away, almost like magic? Local inference. [...] all of the common LLM defects will go away. There's no difference between these two statements in the relevant context of the article, which is specifically "long policy documents do not reliably govern agents", or the comments above, which are specifically addressing how that issue is not solved by local agents.

solarkraft 2mo ago

I get your reading. My interpretation is that they’re claiming enshittification and that the local models don’t have that. It’s an understandable view, but I’d be astonished by any local model processing very long context better than any frontier model (and now many racks are we talking).

Aurornis 2mo ago

> This is a bit of a strawman. The parent comment wasn't suggesting that the inherent defects magically disappear The parent comment literally said that the common LLM defects would go away. Direct quote: > all of the common LLM defects will go away

adrian_b 2mo ago
Clearly it was meant that they will go away only if you use a custom harness, without the defects that now exist in all vendor-controlled harnesses, because of the non-aligned goals of the vendor and of the user. While it was not spelled explicitly, it should be obvious that nothing can be improved if you run locally an LLM in the same way as the vendor. But you should not do that. It is easy to use better methods for sampling procedures and for prompt and context handling.