alew1

6 ms·

1.
by alew1 11d ago
When you read something like this > Change it to 300 and run it again. You get a different pitch and nothing breaks, because at this level there are no notes yet, just a number. does it make sense? Or perhaps it just seems like something
2.
by alew1 8mo ago
The article presents the fact that we appear to treat non-constituents (eg “in the middle of the”) as “units” to mean that language is more like “snapping legos together” than “building trees.” But linguists have proposed the possibility th
3.
by alew1 1y ago
The algorithms are not deterministic: they output a probability distribution over next tokens, which is then sampled. That’s why clicking “retry” gives you a different answer. An LM could easily (in principle) compute a 50/50 distribut
4.
by alew1 1y ago
Harvard is one of eleven American universities that practice need-blind admissions even for international students, meaning that students are admitted without regard for their financial status (i.e., no explicit preference toward richer stu
5.
by alew1 1y ago
The taxpayer funding is for Harvard's research activities, not for its undergraduate teaching. The undergraduate teaching is funded by tuition (often paid in full by international students) and by returns on the endowment (including so
6.
by alew1 2y ago
Didn't seem to realize that "Still more coherent than the OpenAI lineup" wouldn't make sense out of context. (The actual comment quoted there is responding to someone who says they'd name their models Foo, Bar, Baz.
7.
by alew1 2y ago
Any interpretation (including interpreting the inputs to the neural net as a "prompt") is "slapped on" in some sense—at some level, it's all just numbers being added, multiplied, and so on. But I wouldn't cal
8.
by alew1 2y ago
"Temperature" doesn't make sense unless your model is predicting a distribution. You can't "temperature sample" a calculator, for instance. The output of the LLM is a predictive distribution over the next token
9.
by alew1 2y ago
If I show you a strawberry and ask how many r’s are in the name of this fruit, you can tell me, because one of the things you know about strawberries is how to spell their name. Very large language models also “know” how to spell the word a
10.
by alew1 3y ago
One thought: If you want to be able to remove the static part, you could consider fine tuning without the static part. If you fine tune with, you’re teaching the model that the desired behavior occurs only in the presence of the static part
11.
by alew1 3y ago
But the model ultimately still has to process the comma, the newline, the "job". Is the main time savings that this can be done in parallel (on a GPU), whereas in typical generation it would be sequential?
12.
by alew1 4y ago
> Any question of personhood should be evaluated on the basis that we evaluate ourselves and others: by action and behavior, and not on whether sentience can or cannot arise from this or that configuration of code. But what is action and
13.
by alew1 4y ago
> Unfortunately, that argument applies to you, yourself. Does it? I don’t think it would even apply to a reinforcement learning agent trained to maximize reward in a complex environment. In that setting, perhaps the agent could learn to
14.
by alew1 4y ago
One thing that seems missing from this discussion is that even if LLMs are sentient, there is no reason to believe that we would be able to tell by "communicating" with them. Where Lemoine goes wrong is not in entertaining the pos
15.
by alew1 5y ago
Very cool! The StrangeLoop talk includes an example where you infer that Stove() returns a Stove object. If someone writes something like `f(x).broil()`, do you need to do some kind of type inference to figure out what class f(x) is? What c
16.
by alew1 5y ago
The trick is that your predicate can’t be implemented in Haskell, because the predicate itself requires looking at infinitely many elements.
17.
by alew1 5y ago
Hmm. It seems you posted this after apologizing for being hostile in another thread, where Hera’s capabilities (which are… very different from “Quick Notes”) were explained to you. You’ve made several comments at this point incorrectly summ
18.
by alew1 5y ago
Ah, yep, that’s right. Another way to see it is that we’re interested in the probability that your door has a goat behind it, given that you didn’t need to start over: P(you chose goat | host didn’t choose car) = P(you chose goat, host didn
19.
by alew1 6y ago
I found this recently and was super impressed. A really great (and well-organized) reference!
20.
by alew1 6y ago
I've always been under the IRS Free File income threshold (I've worked as a high school teacher and am now in grad school), but last year after reading this article was the first time I actually filed for free. That was after 5 ye
21.
by alew1 6y ago
(Pretty sure 100% in that sentence means “certainly.”)
22.
by alew1 6y ago
If-then-else is pattern matching on a value of type Bool. I think of general pattern matching on algebraic data types as a generalization.
23.
by alew1 6y ago
The comments here are disheartening, essentially a combination of “It was Google’s right to fire her” and “She seemed ‘difficult.’” For an industry-operated “Ethics in AI” group to have any teeth, it can’t fire people when they do research
24.
by alew1 6y ago
In a sense, probabilistic programming languages are automatic (approximate) integrators. Programs can represent arbitrary measures, and the back-ends provide various methods of estimating integrals with respect to them.
25.
by alew1 6y ago
I also wondered about the prompt design, so I tried feeding GPT-3 the entire text of the linked article up until "[This is one confusion after another." Then I asked it to complete the rest of the explanation about why GPT-3 was w
26.
by alew1 6y ago
Huh. I've been using a Jabra Evolve 75 (in particular, Jabra Evolve 7599-838-199) with my 2018 MacBook Pro and it works just fine with built-in bluetooth. I can even have it simultaneously connected to iPad and laptop and get audio fro
27.
by alew1 7y ago
I’ve seen your Codenames work project — it’s neat! I’ve also played around with word vector games — Robot Mind Meld [1] has you and a robot working together to converge on the same word. [1] http://robotmindmeld.com
28.
by alew1 7y ago
As someone who has done both professionally, I think you're severely underestimating how hard teaching is. I think it's much harder to teach well than to code well. Don't forget, too, that good teachers are experts in the sub
29.
by alew1 7y ago
Good question. Disclaimer: I’m in the lab that made Gen & was on the paper, so not impartial :) Anglican is implemented in Clojure, and can be extended (by writing new Clojure code) to support new general-purpose inference engines. Crea
30.
by alew1 7y ago
This is really cool! And awesome that (anecdotally) it seems to have been an effective tool for teaching high school students about logic and proof. Excited to dig in and learn more at some point about how this relates to formal proof assis

More ›