_hark

8 ms·

1.
by _hark 5mo ago
Yeah. The safety of the message is underwritten by its state sanction.
2.
by _hark 6mo ago
You literally can do a kind of model PCA, using the Hessian (matrix of second derivatives of the loss function w/r/t the parameters, aka the local curvature of the loss landscape), and diagonalizing. These eigenvectors and eigenva
3.
by _hark 11mo ago
I don't recall Andrej making "next year!" claims, it was always Elon. I found Andrej's talks from that time to be circumspect and precise in describing their ideas and approach, and not engaging in timeline speculation.
4.
by _hark 11mo ago
They really should have just marketed the software "as-is" to whatever extent that is allowed by law. I guess they didn't because deployed automobile software is probably not allowed to be considered experimental. Still, comm
5.
by _hark 1y ago
https://archive.ph/GWBEl
6.
by _hark 1y ago
There aren't merit-based scholarships to any Ivy League schools, they all offer need-based financial aid packages.
7.
by _hark 1y ago
I'm a researcher at Oxford, and I've both taught and studied here and in the US. The undergraduate teaching here is phenomenal. It's incredibly labor intensive for the staff, but the depth and breadth students are exposed to
8.
by _hark 1y ago
I sat the All Souls exam, taking the philosophy specialist papers, though I'm a math/physics/ML guy. It was a lot of fun, I really appreciate that there's somewhere in the world where these kinds of questions are asked i
9.
by _hark 1y ago
Very cool! I've done research on reinforcement/imitation learning in world models. A great intro to these ideas is here: https://worldmodels.github.io/ I'm most excited for when these methods will make a mean
10.
Can Tinygrad Win?(geohot.github.io)
16 points by _hark 1y ago|0 comments
11.
Oxford Ionics acquired by IonQ for 1B(ft.com)
1 points by _hark 1y ago|0 comments
12.
by _hark 1y ago
If FPGAs are competitive on perf/watt, why aren't they more widespread (other than crap software tooling)? Honestly I've asked different hardware researchers this question and they all seem to give different answers.
13.
by _hark 1y ago
Can anyone comment on where efficiency gains come from these days at the arch level? I.e. not process-node improvements. Are there a few big things, many small things...? I'm curious what fruit are left hanging for fast SIMD matrix mul
14.
by _hark 2y ago
Entropy is not absolute! The entropy of some data is well-defined with respect to a model, but the model choice is free. I.e. different models will assign different entropy to the same data. And how do we choose a model...? Well, formally b
15.
by _hark 2y ago
Maybe a correction is needed. Academia has become so gamified. It's supposed to be about ideas, truth, beauty. Too many are in it for the prestige, which has ironically made it less prestigious. Very few true eccentrics left.
16.
The Complexity Dynamics of Grokking(brantondemoss.com)
1 points by _hark 2y ago|0 comments
17.
by _hark 2y ago
Interesting. You need some local structure with global coherence. But you want it to be complex, not too regular. Like a Penrose Tiling.
18.
by _hark 2y ago
Hmm. At least it's still fine to define limits of the complexity for infinite strings. That should be unique, e.g.: lim n->\infty K(X|n)/n Possible solutions that come tom mind: 1) UTMs are actually too powerful, and we should
19.
by _hark 2y ago
Interesting. I guess then we would only be interested in the normalized complexity of infinite strings, e.g. lim n-> \infty K(X|n)/n where X is an infinite set of numbers (e.g. the decimal expansion of some real number), and K(X|n)
20.
by _hark 2y ago
This could also be a hardware startup. If only there were some entrepreneur types around... Presumably there's a market for this in other niches, e.g. weather monitoring, defense/border monitoring, etc... The question is whether t
21.
by _hark 2y ago
The issue with the invariance theorem you point out always bugged me. Let s be an algorithmically random string relative to UTM A. Is it the case that there exists some pathological UTM S, such that K(s|S) (the Kolmogorov complexity of s re
22.
by _hark 2y ago
I agree with you in spirit. I just thought you might be interested in some of the technical details regarding the relationship between compression and generalization! I'll have a paper out next week which makes your point precise, usin
23.
by _hark 2y ago
Great point. This points to the related issue: what do we want to compress? Do we want to compress "the answer", here the arithmetic expression's solution, or do we want to compress the image? You can formalize this with rate
24.
by _hark 2y ago
Say we have some dataset composed of D bytes. Next, say I find some predictive model of the data M, where M is composed of N bytes. Furthermore, let us say that the entropy of the dataset under the model is H bytes. Then, if N + H < D, m
25.
by _hark 2y ago
Claude's API usage absolutely registers (60% the size of OpenAI's), their chat interface just isn't as popular. [1] [1]: https://www.tanayj.com/p/openai-and-anthropic-revenue-breakd...
26.
by _hark 2y ago
I think self-driving technology is really cool, and I hope it reaches high-reliability as soon as possible. But I would rather it is distributed in personally-owned cars than in fleet models. Would be a shame if the only access is through e
27.
by _hark 2y ago
Totally fair points all. Sorry if it came across as condescending! I agree with you that this network probably has not found the source code or something like a minimal description in its weights. Honestly, I'm writing a paper on model
28.
by _hark 2y ago
Sorry I wasn't more clear! I'm referring to the Kolmogorov complexity of the network. The OP said: > With enough computation, your neural net weights would converge to some very compressed latent representation of the source co
29.
by _hark 2y ago
The raw capacity of the network doesn't tell you how complex the weights actually are. The capacity is only an upper bound on the complexity. It's easy to see this by noting that you can often prune networks quite a bit without an
30.
by _hark 2y ago
I'm in the final year of my PhD, and I feel very lucky to be working on the question of what it means for an explanation to be "good". The closest thing to a satisfactory resolution of the question of "reality" in t

More ›