flebron

10 ms·

1.
by flebron 3mo ago
I think you might be thinking of "AGI" as some sort of point in time, where something happens and everyone all at once has some technology. Not only is the progress towards AGI gradual, it's also very jagged in both capabilit
2.
by flebron 3mo ago
Source for this? The notion of attention dates to a content-addressable lookup during sequence alignment (as well as, concurrently, memory lookups in neural Turing machines). Attention had been used in other models, like GRUs and LSTMs with
3.
by flebron 5mo ago
The standard way of doing MTP is to run the drafter autoregressively for k steps, and then (not concurrently) use the larger model as a verifier for those k tokens at the same time. The larger model can then accept a prefix of those k tok
4.
by flebron 5mo ago
The website asks what they do in Haskell. The answer is property modification and reading, as well as very powerful traversal constructs, use lenses ( https://hackage.haskell.org/package/lens , tutorial at https:/
5.
by flebron 1y ago
I think the point is that a monad is a useful concept _purely_ because of what it _allows_ you to do, and _not_ because of anything syntactical. Those rules that you're obviating there, the commutative squares, are precisely what then
6.
by flebron 1y ago
This is an excellent summary of these techniques :) I like that every single one comes with an example implementation, with shape comments on the tensors. Thanks Stephen!
7.
by flebron 1y ago
Perhaps this chapter can help? https://jax-ml.github.io/scaling-book/tpus/ It's a chip (and associated hardware) that can do linear algebra operations really fast. XLA and TPUs were co-designed, so as long as
8.
by flebron 2y ago
All of them are vectors of embedded representations of tokens. In a transformer, you want to compute the inner product between a query (the token who is doing the attending) and the key (the token who is being attended to). An inductive bia
9.
by flebron 2y ago
No. In the common use of the word fine-tuning, one is in the supervised learning scenario. One has an input prompt, and an output sentence. One teaches the model to say that output in response to that prompt. In the reinforcement learning s
10.
by flebron 3y ago
"Almost nobody except X does Y." and "Z does Y, with Z != X" are consistent. Is your disagreement entirely due to the (possibly nil) distinction between "Almost nobody except X does Y" and "Almost nobody d
11.
by flebron 4y ago
I don't see a link between that Bloomberg article and these kidnappings happening mostly to married men cheating. Nor is it the case that "love motels" are primarily used for cheating. We have plenty of those in Argentina. As
12.
by flebron 4y ago
I think k8sToGo's point is that she never "joined Alphabet", Alphabet did not exist when she joined Larry and Sergey. She joined concurrent with Google's founding as a company, she probably joined when the search engine
13.
by flebron 4y ago
The trifecta of: * There's almost always a simple geometric intuition, and low-dimensional intuition can get you quite far even in high dimensional cases. * You can surprisingly often get by with closing your eyes and saying &quo
14.
by flebron 4y ago
That seems a weird use of the word. By that notion, every convicted criminal is a victim, because the court imposed a sentence that they, presumably, do not like. This removes practically all meaning from the word "victim". In com
15.
by flebron 4y ago
pc = Parsec :) https://en.wikipedia.org/wiki/Parsec I don't know what the G stands for. Possibly luminosity of some kind?
16.
by flebron 4y ago
Well, most interesting properties about computer programs are, in general for all programs, undecidable ( https://en.wikipedia.org/wiki/Rice%27s_theorem ). Undecidability is a closely related notion to unprovability ( ht
17.
by flebron 5y ago
Consider a disconnected domain (say, union of a few open balls in R^n), and f being constant in each connected component, but having different values in each ball. The differential is indeed everywhere 0 in the entire domain.
18.
Reversing an integer hash function(taxicat1.github.io)
87 points by flebron 5y ago|16 comments
19.
by flebron 5y ago
I've always found the definition of monoid objects in a category of endofunctors to be tougher to grasp than the Kleisli-category definition of a monad, at least if one is being formal about "monoid object", and checking that
20.
by flebron 11y ago
Yes.
21.
by flebron 11y ago
"1 bowl of rice" gets interpreted as 1 cup of Sake?
22.
by flebron 11y ago
To a sufficiently advanced alien species, _we_ are neural networks fed with some pictures (and sounds, smells, touches, and tastes) :)
23.
by flebron 11y ago
The report is signed by him, so it's safe to say that report is an accurate recollection of what happened according to him. Having a report on the security of a system be issued months _after_ it is used is completely stupid. This is n
24.
by flebron 11y ago
Unfortunately they did. You can see the request for review here[1], and the actual review here[2]. I know the guy who actually did it (Righetti), he sent an email to the department a few days ago. The quality of that review was absolutely t
25.
by flebron 11y ago
Do you also trust your package maintainer to not mismerge patches and leave you open to security bugs? Your argument is no bueno.
26.
by flebron 11y ago
Honestly, is reversing a binary tree something you need to memorize? I assume that's what he/she means by "invert".
27.
by flebron 12y ago
In C++, "return 0" from main is actually the default (i.e. you don't need to explicitly return), same in C99.
28.
by flebron 12y ago
It's kind of ironic that the author states "In his article, Byorgey hints at what I'm going to say, but I think it deserves to be said again, with slightly different words.", which is essentially what monad tutorials say
29.
by flebron 12y ago
Of course the models I mentioned are not self-contradictory, I've no idea what you mean by that. Your statement was "If the numbers can be arbitrary in size, then you can't compare them in constant time". I was merely po
30.
by flebron 12y ago
My point is indeed that "you can't compare [arbitrary sized numbers] in constant time" is a statement about some model of computation, and not all of them.

More ›