the8thbit

6 ms·

1.
by the8thbit 2y ago
> This is a typical tech echo chamber. There is a significant number of people who make direct purchases through ads. Even if its not, TV ads, newspaper ads, magazine ads, billboards, etc... get exactly 0 clickthrus, and yet, people stil
2.
by the8thbit 2y ago
I don't see why, unless you think they're lying and they filmed their demos, or used some other preexisting model. I didn't ignore the JWST launch just because I haven't been granted to ability to use the telescope.
3.
by the8thbit 2y ago
Her approach might not be the literal peak healthiest, but that doesn't mean its wrong. If she's only having a drink every once in a while and is getting 7+ hours of sleep most nights then the negative effects are probably negligi
4.
by the8thbit 2y ago
Not the person you're responding to, but no, it doesn't really bother me at all. What does bother me is that I don't have confidence in the value of the output, where as if I listen to This American Life, or a podcast or audi
5.
by the8thbit 2y ago
I don't doubt you're right about social media and smartphones rotting our attention spans. But also, peripatetic philosophy is ancient. I spend most of my day sitting. Whether its work, entertainment, or hobbies, most of these thi
6.
by the8thbit 2y ago
While you're right that the Internet contains a lot of wonder and exploration, the vast majority of people (and kids) will not interact much with that part of the Internet, if at all. Additionally, social media platforms have collapsed
7.
by the8thbit 2y ago
> Many of these alternatives failed because they were forced onto users who had already paid over $60 for a game I really doubt that hurt their adoption. Yes, it pissed people off, but that doesn't mean it suppressed adoption. Bei
8.
by the8thbit 2y ago
> 1 - https://www.smh.com.au/national/human-brain-tissue-made-up-o ... holy shit 1/200 molecules in your fucking brain is straight up plastic jesus christ
9.
by the8thbit 2y ago
Extremely pedantic, but is "non-deterministic" really the right language? The same input will always produce the same output, provided you haven't intentionally configured the system to use the model non-deterministically. It
10.
by the8thbit 2y ago
This is absolutely wild.
11.
by the8thbit 2y ago
JPEGs and 3D models are also executable binaries. They, like model weights, contain domain specific instructions that execute in a domain specific and turing incomplete environment. The model weights are the instructions, and those instru
12.
by the8thbit 2y ago
Its not really an analogy. LLMs are quite literally executables in the same way that jpegs are executables. They both specify machine readable (but not human readable) domain specific instructions executed by the image viewer/inference
13.
by the8thbit 2y ago
As I said, "there's a place for that too, of course." I don't think Claude is a good choice if you're trying to prototype a project which uses tools that you don't understand conceptually. However, if you alrea
14.
by the8thbit 2y ago
I've used both Bitbucket and Azure in the corporate world.
15.
by the8thbit 2y ago
I gotta say, Claude is a godsend for building out quick prototypes of ideas, especially when those ideas require domain specific knowledge that you know a little about but aren't specialized in. Which is most interesting programming
16.
by the8thbit 2y ago
Videos and images are software. They are compiled binaries with very domain specific instructions executed in a very non-turing complete context. They are generally not released as open source, and in many cases the source code (the file us
17.
by the8thbit 2y ago
"LLMs are fundamentally different to software and using terms from software just muddies the waters." They're still software, they just don't have source code (yet).
18.
by the8thbit 2y ago
"Be charitable and intellectually curious. What would "open" look like?" To really be intellectually curious we need to be open to the idea that there is not (yet) a solution to this problem. Or in the analogy you laid o
19.
by the8thbit 2y ago
I think you're misunderstanding what I'm saying. I don't think its technically feasible for current models to be open source, because there is no source code to open. Yes, there is a harness that runs the model, but the vast,
20.
by the8thbit 2y ago
> Actually, the fact that the model weights are available means that you can even ignore any limitations that you think are on it, and you'll probably just get away with it. You are also ignoring the fact that the limitations are mi
21.
by the8thbit 2y ago
> No one on the planet understands how the model weights work exactly Which is my point. These models aren't open source because there is no source code to open. Maybe one day we will have strong enough interpretability to generate
22.
by the8thbit 2y ago
"My understanding is that a few years ago, if we knew the degree of interpretability we have today (compared to capability) it would have been devastatingly disappointing." I think this is a situation where both things are true. M
23.
by the8thbit 2y ago
I'm not the person you replied directly to so I can't speak for them, but I did start this thread, and I just wanted to clarify what I meant in my OP, because I see a lot of people misinterpreting what I meant. I did not mean th
24.
by the8thbit 2y ago
"In practice it is intractably hard to make any significant modification to a binary, and even if you could, you would then not be legally allowed to e.g. redistribute." It depends on the binary and the license the binary is relea
25.
by the8thbit 2y ago
"You can modify individual neurons if you are so inclined." You can also modify a binary, but that doesn't mean that binaries are open source. "That's what Anthropic have done with the Claude family of models [1]. .
26.
by the8thbit 2y ago
"You make a good point but those are also just limitations of the technology (or at least our current understanding of it)" Yeah, that is my point. Things that don't have source code can't be open source. "Maybe a
27.
by the8thbit 2y ago
"There's a pretty clear difference between the 'finetuning' offered via API by GPT4 and the ability to do whatever sort of finetuning you want and get the weights at the end that you can do with open weights models.&quot
28.
by the8thbit 2y ago
They don't "[allow] developers to modify its code however they want", which is a critical component of "open source", and one that Meta is clearly trying to leverage in branding around its products. I would like th
29.
by the8thbit 2y ago
Is that what you meant? I don't think releasing the sequence of steps required to produce the model satisfies "open source", which is how I interpreted you, because there is still no source code for the model.
30.
by the8thbit 2y ago
"just the inference code" It's not the "inference code", its the code that specifies the architecture of the model and loads the model. The "inference code" is mostly the model, and the model is not legibl

More ›