taliesinb

46 ms·

1.
by taliesinb 6mo ago
Interesting. It all seems very brittle, though. And that something has gone very wrong with our ecosystem of tools, languages, and processes when it becomes advisable to massage source until specific passes in a specific version LLVM don&#x
2.
by taliesinb 6mo ago
Amusingly, you need only have circuits of prime depth, though you should probably adjust their widths using something principled, perhaps Euler's totient function.
3.
by taliesinb 6mo ago
And i bet these would be useful in initial and final parts of transformer too. Because syntactic parsing and unparsing of brackets, programming language ASTs, etc is highly recursive; no doubt current models are painfully learning "unr
4.
by taliesinb 6mo ago
There is an obvious implication: since the initial models were trained without loops, it is exceedingly unlikely that a single stack of consecutive N layers represents only a single, repeatable circuit that can be safely looped. It is m
5.
by taliesinb 10mo ago
If you could link to where this is implemented I'd be very grateful!
6.
by taliesinb 10mo ago
Hello! Yes I am curious, how does one deal with cycles in the code hash graph? Mutually recursive functions for example?
7.
by taliesinb 11mo ago
I’ve been super interested in these kinds of cosmic turduckens. See also https://en.wikipedia.org/wiki/Thorne%E2%80%93%C5%BBytkow_obj... and https://en.wikipedia.org/wiki/Quasi-star
8.
by taliesinb 1y ago
Given that whole name binding thing is ultimately a story of how to describe a graph using a tree, I was primed to look for monoidal category-ish things, and sure enough the S and K combinators look very much like copy and delete operators;
9.
by taliesinb 2y ago
Yes, this is a very cool story. But, fascinatingly, integration does in fact have a meaning. First, recall from the OP that d/dX List(X) = List(X) * List(X). You punched a hole in a list and you got two lists: the list to the left of t
10.
by taliesinb 2y ago
Cool! Looks pretty professional. I explored a similar idea once (also implemented in Python, via decorators) to help speed up some neuroscience research that involved a lot of hyperparameter sweeps. It's named after a Borges story abou
11.
by taliesinb 2y ago
The end-game is just dissolving any distinction between compile-time and run-time. Other examples of dichotomies that could be partially dissolved by similar kinds of universal acid: * dynamic typing vs static typing, a continuum that JIT-i
12.
by taliesinb 2y ago
An earnest question: can you elaborate on what Zig got wrong in that respect?
13.
by taliesinb 2y ago
In another life!
14.
by taliesinb 2y ago
Yes, get in touch with me.
15.
by taliesinb 2y ago
Symbolica.ai | London, Australia | REMOTE, INTERNS, VISA We're trying to apply the insights of category theory, dependent type theory, and functional programming to deep learning. How do we best equip neural nets with strong inductive
16.
by taliesinb 2y ago
No worries! Yes, exactly. It's also similar to doing arithmetic with and without units. Sure, you can do arithmetic without units, but when you are actually working with real-world quantities, but you easily get yourself into a muddl
17.
by taliesinb 2y ago
Looks interesting, but your link is broken, can you try again or give us a direct PDF or Arxiv link?
18.
by taliesinb 2y ago
Thanks for highlighting XArray in your other comments. Yup, XArray is great. As are Dex and the various libraries for named axis DL programming within PyTorch and Jax. I never said these things don't exist -- I even mention them in the
19.
by taliesinb 2y ago
While I applaud the OP's exposition, imagine that instead of having axis names live as single-letter variables within einsum, our arrays themselves had these names attached to their axes? It's almost like when we moved from writin
20.
by taliesinb 2y ago
Hey there, a bit late but I've read your other comments and I'd like to get in touch. I happen to be very focused on type derivatives, in the context of applying category theory to AI, having just discovered Conor's original
21.
by taliesinb 3y ago
That is an excellent explanation full of great intuition building! If anyone is interested in a kind of tensor network-y diagrammatic notation for array programs (of which transformers and other deep neural nets are examples), I wrote a pos
22.
by taliesinb 3y ago
Amazing, but that's a lot of channels for a cheap oscilloscope. I'm curious how he did that, I don't see any wiring. Also, he missed an opportunity to bring in the rudiments of transmission line theory -- a twisted pair is a
23.
by taliesinb 3y ago
Wow, I love the interactive wizzing around and the animation, very neat! Way more explanations should work like this. I've recently finished an unorthodox kind of visualization / explanation of transformers. It's sadly not in
24.
by taliesinb 3y ago
QE did not lead to historically high levels of inflation. Most of the money stayed in the financial sector, on banks balance sheets.
25.
by taliesinb 3y ago
I guess we should distinguish money from commodities, though? For money to mean something different to commodities, it must enjoy a privileged legal and social role, by virtue of its ability to satisfy tax obligations to the state and debt
26.
by taliesinb 3y ago
I've been reading up on MMT recently, and I haven't seen adherents (e.g. Mosler) claim that inflation is good. They do provide a heterodox account of how government spending leads to inflation, a sectoral one that focuses on deman
27.
by taliesinb 3y ago
This new stuff aside, there's an exhaustive two part series by Sean Munger that sets out the many interlocking lines of evidence that Oswald acted alone. I am pretty convinced at this point: https://youtu.be/DC8tO16xdrY
28.
by taliesinb 3y ago
Yeah, I thought exactly the same thing when I watching Contact again a few months ago! There are all kinds of fascinating places where you can gain mental leverage by thinking in higher dimensions. For example, the definition of a monoidal
29.
by taliesinb 3y ago
Great to see this kind of visualization gaining prominence! Thinking about matrix algebra in higher dimensions makes everything much more intuitive. I started along this road when I was creating 3D visualizations for the Deep Learning Indab
30.
by taliesinb 3y ago
You might gain some basic intuition from this (partially complete) Numpy tutorial I wrote for the Deep Learning Indaba: https://arrayalgebra.info

More ›