necroforest

5 ms·

1.
by necroforest 2y ago
yeah, and as far as I'm aware there isn't even a definition of "civilization" or "intelligence" that doesn't boil down to "sufficiently like me".
2.
by necroforest 2y ago
I don't entirely disagree with you, but "what products do people want" is overly conservative. Pre-ChatGPT, very few people wanted a (more or less) general purpose chatbot.
3.
by necroforest 2y ago
It's branding (see: TensorFlow); also, pretty much anything (linear) you would do with an arbitrarily ranked tensor can be expressed in terms of vector ops and matmuls
4.
by necroforest 3y ago
cosine similarity is (isomorphic to) "distances to nearby objects". and not all embeddings are word embeddings.
5.
by necroforest 3y ago
If you need to relearn your iPhone/Mac interfaces every year, you might want to get screened for Alzheimers.
6.
by necroforest 3y ago
> Who doesn’t think China has a ten year AI algorithm to takeover Taiwan? anybody who works in either AI or natsec
7.
by necroforest 3y ago
> We arent going to see more progress until we have a way to generalize the compute graph as a learnable parameter That's a bold statement since a ton of progress has been made without learning the compute graph.
8.
by necroforest 3y ago
which laws are broken exactly? it's not remotely settled law that "training an NN = copyright infringement"
9.
by necroforest 3y ago
What's an order of magnitude between friends?
10.
by necroforest 3y ago
Maybe "best word processor for mac" is a better search for you?
11.
by necroforest 3y ago
LLMs are trained in parallel. The model weights and optimizer state are split over a number (possibly thousands) of accelerators. The main bottleneck to doing distributed training like this is the communication between nodes.
12.
by necroforest 3y ago
which jargon here is "just for show"?
13.
by necroforest 3y ago
>it's actually impossible to have a non-Hermitian matrix C such that x†Cx > 0 Nice catch, it's been a few years since I had to think about these details.
14.
by necroforest 3y ago
no, at least not for anything non-contrived.
15.
by necroforest 3y ago
I used Bayesian analysis for an industry problem where we had a lot of data (~1M samples), but the relationship between the observed data and the latent variables of interest was complicated and non-identifiable. In the non-identifiable set
16.
by necroforest 3y ago
JS divergence isn't just the average. it's 0.5(KL(P||M) + KL(Q||M)) where M is the mixture of P and Q: M = 0.5(P + Q)
17.
by necroforest 3y ago
kind of. you can decompose an arbitrary matrix into symmetric and antisymmetric components: R = S + A. Since A = -A^H (anti-symmetric), for any vector x, <x, Ax> = -<x, Ax> => <x, Ax> = 0. So for any matrix where <x,
18.
by necroforest 3y ago
Does anyone use Ada in 2023? Back in college (~2008), I had a prof who was obsessed with it but AFAIK almost nobody used it even back them.
19.
by necroforest 3y ago
Nobody (to good approximation) plays with weights during the peer review process.
20.
by necroforest 3y ago
Having a project page with extra results is very common in computer vision, even in academia.
21.
by necroforest 3y ago
To answer (2): You are token i. In order to see how much of a token j's value v_j you update yourself with, you compare your query q_i with token j's key k_j. This gives you the asymmetry between queries and keys. This is even mor
22.
by necroforest 3y ago
you can't store a function as a tensor. the tensors are the inputs/outputs that flow along the edges of the graph. TF stores things directly as a graph: https://github.com/tensorflow/tensorflow/blob/
23.
by necroforest 3y ago
> AI research has put hardly any effort into building goal-directed agents The entire (enormous) field of reinforcement learning begs to differ.
24.
by necroforest 3y ago
no
25.
by necroforest 3y ago
not in collaboration with openai, one of the authors joined openai before the paper was written and arxiv'd
26.
by necroforest 3y ago
the genius of bfloat16 is that conversion is just truncation / zero padding
27.
by necroforest 4y ago
lol, because whining about your ex is a "basic fact"
28.
by necroforest 4y ago
yes, they are called Program Managers
29.
by necroforest 4y ago
i don't think the NSA has hitmen.
30.
by necroforest 4y ago
...yes. that's exactly what they do.

More ›