6 ms·

Graphical Linear Algebra

304 points by hyperbrainer 1y ago

lorenzo_medici 1y ago
Appreciate the Claude Makelele praise
- rurban 1y ago
  But nowadays we are calling him the 6. And everybody praises a good 6
Xmd5a 1y ago
Generalized Transformers from Applicative Functors >Transformers are a machine-learning model at the foundation of many state-of-the-art systems in modern AI, originally proposed in [arXiv:1706.03762]. In this post, we are going to build a generalization of Transformer models that can operate on (almost) arbitrary structures such as functions, graphs, probability distributions, not just matrices and vectors. >[...] >This work is part of a series of similar ideas exploring machine learning through abstract diagrammatical means. https://cybercat.institute/2025/02/12/transformers-applicative-functors/ https://cybercat.institute/2025/02/12/transformers-applicati...
MarkusQ 1y ago
I really enjoyed that when it was coming out, and used to follow it with some students. It's a shame it seems to have been abandoned.
- Iwan-Zotow 1y ago
  Who wrote that? Do you know? pawel ... ?
  - mattkrause 1y ago
    Pawel Sobocinski, in collaboration with Filippo Bonchi and Fabio Zanasi https://graphicallinearalgebra.net/about/ https://graphicallinearalgebra.net/about/
theZilber 1y ago
When I read the first meaty chapter about graphs and commutativity I initially thought he just spends too long explaining simple concepts. But then ai realized I would always forget the names for all the mathy c' words - commutativity commutativity, qssociativity... and for the first time I could actually remember commutativity and what it means, just because he tied it into a graphical representation (which actually made me laugh out loud because, initially, I thought it was a joke). So the concept of "x + y = y + x" always made sense to me but never really stuck like the graphical representation, which also made me remember its name for the first time. I am sold.
- gowld 1y ago
  Which chapter is that? It's not in the ToC
  - memoryfault 1y ago
    3!
    - HappMacDonald 1y ago
      Chapter 6, got it
- vanderZwan 1y ago
  It's because the graphs are visual metaphors that encode privileged information[0]. Which is an often overlooked aspect of teaching imo. Your own initial dismissive reaction kind of shows why: people don't really get the point until they realize it works, and even then they're not sure why. [0] https://web.archive.org/web/20140402025221/http://m.nautil.us/issue/6/secret-codes/teaching-me-softly https://web.archive.org/web/20140402025221/http://m.nautil.u...
phforms 1y ago
Years ago when I was reading this (just a couple of chapters, not all of it), it opened my eyes to the power of diagrammatic representation in formal reasoning unlike anything before. I never did anything useful with string diagrams, but it was so fun to see what is possible with this system!
- elric 1y ago
  I had a similar revelation when watching 3Blue1Brown's Calculus series. Had they included those kinds of visual representations in school when I was first learning about Calculus, my understanding (and interest) would have been greatly expanded. Very impressive how some people can create visual representations that enhance understanding.
dtj1123 1y ago
I was never able to get my head around it, but this reminds me somewhat of the zx-calculus: https://en.wikipedia.org/wiki/ZX-calculus https://en.wikipedia.org/wiki/ZX-calculus
dclowd9901 1y ago
> If the internet has taught us anything, it’s that humans + anonymity = unpleasantness. Aka one of my favorite axioms: https://www.penny-arcade.com/comic/2004/03/19/green-blackboards-and-other-anomalies https://www.penny-arcade.com/comic/2004/03/19/green-blackboa...
AntonioL 1y ago
Reminds me of the work from Bob Coecke at the University of Oxford. He came up with a pictorial language for quantum processes.
- gowld 1y ago
  ZX-calculus mentioned in https://news.ycombinator.com/item?id=44532535 https://news.ycombinator.com/item?id=44532535 https://en.wikipedia.org/wiki/ZX-calculus https://en.wikipedia.org/wiki/ZX-calculus
marvinborner 1y ago
It's interesting how some of these diagrams are almost equivalent in the context of encoding computation in interaction nets using symmetric interaction combinators [1]. From the perspective of the lambda calculus for example, the duplication of the addition node in "When Adding met Copying" [2] mirrors exactly the iterative duplication of lambda terms - ie. something like (λx.x x) M! [1]: https://ezb.io/thoughts/interaction_nets/lambda_calculus/2025-04-25_normalization.html https://ezb.io/thoughts/interaction_nets/lambda_calculus/202... [2]: https://graphicallinearalgebra.net/2015/05/12/when-adding-met-copying/ https://graphicallinearalgebra.net/2015/05/12/when-adding-me...
russfink 1y ago
It reads as if Chuck Lorre (The Big Bang Theory) wrote it. Especially chapter two. I love the humor!
webprofusion 1y ago
This is nice, my main criticism would be that it uses the language "easy" and "simple" regularly which is a classic mistake in any instructive text (including docs etc). If the reader was feeling a bit dumb and/or embarrassed that they didn't yet get the concept being explained then this will only make them feel worse and give up. Language like that is often used to make things feel approachable and worry-free, but can have the opposite effect. And never ever, ever write "obvious" in a doc explaining something, because if obviousness was at play they wouldn't be reading your doc.
- RamblingCTO 1y ago
  The good ol' "this proof is trivial so we'll skip it" move.
  - rurban 1y ago
    He should have really used the good ol' QED instead, lol
  - seanhunter 1y ago
    Oh man. The variant I see so infuriatingly often at the moment is “It is clear that these form a Lie algebra/finite abelian group/Hilbert space/bijective map/<whatever other thing that is long-winded or complex to prove> and I encourage the reader to satisfy themselves that this is the case”.
- deleted 1y ago
  [deleted]
- Nevermark 1y ago
  Excellent point. I think about wording like that, like the extraneously explicit meta-content that dumbs down so many story plots. A character explicitly says "That makes me angry". When a better written story would make the anger implicitly obvious. Stories should show not tell. Make a point, make it clear make it concise, and it will be simple for most readers. Don't talk about making a point, or say a point is clear. That is projecting attributes or experiences onto readers. But even a very well written point may not appear simple for some readers. Assume (optimistically!) that there will always be some unusually under-prepared but motivated reader. Hooray if you get them! They can handle a challenge every so often. "Simple" communication is a high priority target, but rarely completely achievable for the total self-selected, beyond intended, audience.
programjames 1y ago
This looks pretty similar to interaction combinators: 1. https://en.wikipedia.org/wiki/Interaction_nets#Interaction_combinators https://en.wikipedia.org/wiki/Interaction_nets#Interaction_c... 2. https://github.com/HigherOrderCO/Bend https://github.com/HigherOrderCO/Bend
vismit2000 1y ago
Immersive Linear Algebra: https://immersivemath.com/ila/index.html https://immersivemath.com/ila/index.html HN: https://news.ycombinator.com/item?id=19264048 https://news.ycombinator.com/item?id=19264048

304 points by hyperbrainer 1y ago

lorenzo_medici 1y ago

Appreciate the Claude Makelele praise

rurban 1y ago
But nowadays we are calling him the 6. And everybody praises a good 6

Xmd5a 1y ago

Generalized Transformers from Applicative Functors >Transformers are a machine-learning model at the foundation of many state-of-the-art systems in modern AI, originally proposed in [arXiv:1706.03762]. In this post, we are going to build a generalization of Transformer models that can operate on (almost) arbitrary structures such as functions, graphs, probability distributions, not just matrices and vectors. >[...] >This work is part of a series of similar ideas exploring machine learning through abstract diagrammatical means. https://cybercat.institute/2025/02/12/transformers-applicative-functors/ https://cybercat.institute/2025/02/12/transformers-applicati...

MarkusQ 1y ago

I really enjoyed that when it was coming out, and used to follow it with some students. It's a shame it seems to have been abandoned.

Iwan-Zotow 1y ago
Who wrote that? Do you know? pawel ... ?
- mattkrause 1y ago
  Pawel Sobocinski, in collaboration with Filippo Bonchi and Fabio Zanasi https://graphicallinearalgebra.net/about/ https://graphicallinearalgebra.net/about/

theZilber 1y ago

When I read the first meaty chapter about graphs and commutativity I initially thought he just spends too long explaining simple concepts. But then ai realized I would always forget the names for all the mathy c' words - commutativity commutativity, qssociativity... and for the first time I could actually remember commutativity and what it means, just because he tied it into a graphical representation (which actually made me laugh out loud because, initially, I thought it was a joke). So the concept of "x + y = y + x" always made sense to me but never really stuck like the graphical representation, which also made me remember its name for the first time. I am sold.

gowld 1y ago
Which chapter is that? It's not in the ToC
- memoryfault 1y ago
  3!
  - HappMacDonald 1y ago
    Chapter 6, got it
vanderZwan 1y ago
It's because the graphs are visual metaphors that encode privileged information[0]. Which is an often overlooked aspect of teaching imo. Your own initial dismissive reaction kind of shows why: people don't really get the point until they realize it works, and even then they're not sure why. [0] https://web.archive.org/web/20140402025221/http://m.nautil.us/issue/6/secret-codes/teaching-me-softly https://web.archive.org/web/20140402025221/http://m.nautil.u...

phforms 1y ago

Years ago when I was reading this (just a couple of chapters, not all of it), it opened my eyes to the power of diagrammatic representation in formal reasoning unlike anything before. I never did anything useful with string diagrams, but it was so fun to see what is possible with this system!

elric 1y ago
I had a similar revelation when watching 3Blue1Brown's Calculus series. Had they included those kinds of visual representations in school when I was first learning about Calculus, my understanding (and interest) would have been greatly expanded. Very impressive how some people can create visual representations that enhance understanding.

dtj1123 1y ago

I was never able to get my head around it, but this reminds me somewhat of the zx-calculus: https://en.wikipedia.org/wiki/ZX-calculus https://en.wikipedia.org/wiki/ZX-calculus

dclowd9901 1y ago

> If the internet has taught us anything, it’s that humans + anonymity = unpleasantness. Aka one of my favorite axioms: https://www.penny-arcade.com/comic/2004/03/19/green-blackboards-and-other-anomalies https://www.penny-arcade.com/comic/2004/03/19/green-blackboa...

AntonioL 1y ago

Reminds me of the work from Bob Coecke at the University of Oxford. He came up with a pictorial language for quantum processes.

gowld 1y ago
ZX-calculus mentioned in https://news.ycombinator.com/item?id=44532535 https://news.ycombinator.com/item?id=44532535 https://en.wikipedia.org/wiki/ZX-calculus https://en.wikipedia.org/wiki/ZX-calculus

marvinborner 1y ago

It's interesting how some of these diagrams are almost equivalent in the context of encoding computation in interaction nets using symmetric interaction combinators [1]. From the perspective of the lambda calculus for example, the duplication of the addition node in "When Adding met Copying" [2] mirrors exactly the iterative duplication of lambda terms - ie. something like (λx.x x) M! [1]: https://ezb.io/thoughts/interaction_nets/lambda_calculus/2025-04-25_normalization.html https://ezb.io/thoughts/interaction_nets/lambda_calculus/202... [2]: https://graphicallinearalgebra.net/2015/05/12/when-adding-met-copying/ https://graphicallinearalgebra.net/2015/05/12/when-adding-me...

russfink 1y ago

It reads as if Chuck Lorre (The Big Bang Theory) wrote it. Especially chapter two. I love the humor!

webprofusion 1y ago

This is nice, my main criticism would be that it uses the language "easy" and "simple" regularly which is a classic mistake in any instructive text (including docs etc). If the reader was feeling a bit dumb and/or embarrassed that they didn't yet get the concept being explained then this will only make them feel worse and give up. Language like that is often used to make things feel approachable and worry-free, but can have the opposite effect. And never ever, ever write "obvious" in a doc explaining something, because if obviousness was at play they wouldn't be reading your doc.

RamblingCTO 1y ago
The good ol' "this proof is trivial so we'll skip it" move.
- rurban 1y ago
  He should have really used the good ol' QED instead, lol
- seanhunter 1y ago
  Oh man. The variant I see so infuriatingly often at the moment is “It is clear that these form a Lie algebra/finite abelian group/Hilbert space/bijective map/<whatever other thing that is long-winded or complex to prove> and I encourage the reader to satisfy themselves that this is the case”.
deleted 1y ago
[deleted]
Nevermark 1y ago
Excellent point. I think about wording like that, like the extraneously explicit meta-content that dumbs down so many story plots. A character explicitly says "That makes me angry". When a better written story would make the anger implicitly obvious. Stories should show not tell. Make a point, make it clear make it concise, and it will be simple for most readers. Don't talk about making a point, or say a point is clear. That is projecting attributes or experiences onto readers. But even a very well written point may not appear simple for some readers. Assume (optimistically!) that there will always be some unusually under-prepared but motivated reader. Hooray if you get them! They can handle a challenge every so often. "Simple" communication is a high priority target, but rarely completely achievable for the total self-selected, beyond intended, audience.

programjames 1y ago

This looks pretty similar to interaction combinators: 1. https://en.wikipedia.org/wiki/Interaction_nets#Interaction_combinators https://en.wikipedia.org/wiki/Interaction_nets#Interaction_c... 2. https://github.com/HigherOrderCO/Bend https://github.com/HigherOrderCO/Bend

vismit2000 1y ago

Immersive Linear Algebra: https://immersivemath.com/ila/index.html https://immersivemath.com/ila/index.html HN: https://news.ycombinator.com/item?id=19264048 https://news.ycombinator.com/item?id=19264048