7 ms·

The Art of Linear Algebra [pdf]

350 points by allending 5y ago

jasode 5y ago
It seems like the "github.com" link would be a better and more canonical url rather than link an opaque url to "githubusercontent.com" https://github.com/kenjihiranabe/The-Art-of-Linear-Algebra https://github.com/kenjihiranabe/The-Art-of-Linear-Algebra
- happy-go-lucky 5y ago
  And, "Graphic Notes on Gilbert Strang's Linear Algebra for Everyone" would be less ambiguous than "The Art of Linear Algebra."
visarga 5y ago
Nice, I was just reviewing SVD and PCA, I grok them but then I forget, this material is useful for remembering the big picture.
ivan_ah 5y ago
Wow this is really well done. It's like a visual-algebraic approach... I've never seen this before. I like how the author sets up a "grammar of matrix multiplications," and then reuses the same patterns in the rest of the document. For people who might not be familiar, these visual notes are inspired by and complement Prof. Strang's new book https://math.mit.edu/~gs/everyone/ https://math.mit.edu/~gs/everyone/ and course https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-linear-algebra-spring-2020/index.htm https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-li... https://www.youtube.com/playlist?list=PLUl4u3cNGP61iQEFiWLE21EJCxwmWvvek https://www.youtube.com/playlist?list=PLUl4u3cNGP61iQEFiWLE2... see also https://news.ycombinator.com/item?id=23157827 https://news.ycombinator.com/item?id=23157827
- hiranabe 5y ago
  Ivan, thanks for sharing the background concepts Prof. Strang has told us. Yes, I am a big fan of him, and I came up with this idea during several email exchanges with him.
- allending 5y ago
  Your books are pretty amazing too Ivan. Have em all.
- deleted 5y ago
  [deleted]
skytreader 5y ago
Instantly fell in love with the presentation! Might give this a read-through just out of pure curiosity. In undergrad, my mnemonic for these operations was visualizing the matrices animated in my head. The more complex ones, it was actually easier for me to remember Scheme functions that represent the algorithm (all expressed via higher-order functions so it was pretty concise); this was unique to my circumstances as an undergrad, not something I can pull off today without reviewing a lot of material. Presenting the operations with color and blocks just gives a more natural "user interface" (lacking a better term) for remembering it!
amelius 5y ago
On the first page they say "if neither a or b are 0", but they haven't defined what it means for a vector to be 0. Also, they say "rank 1 matrix", but they haven't defined the concept of rank yet. Some readers might find this kind of presentation acceptable, but personally I strongly dislike it when concepts are used before they are defined.
- Koshkin 5y ago
  To be fair, it does not look like this text is intended to be a self-contained introduction to linear algebra.
- Grustaf 5y ago
  The concept of an additive zero is built into what it means to be a vector space though.
_wldu 5y ago
This is great for those studying algorithms (DP, DC, FFT, etc.). Very useful.
tishha 5y ago
good notes, very useful!
antegamisou 5y ago
Also check out 3Blue1Brown's Essence of Linear Algebra https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFit...
- deleted 5y ago
  [deleted]
- mathgenius 5y ago
  And here is linear algebra as path counting [1]. This is closely related to path integrals in quantum physics. The rules for combining quantum amplitudes are the rules for combining path counts [2]. This is also how graphical linear algebra works [3]. And since we are all hackers here, if you replace the underlying number system with the min-sum semiring you get things like Dijkstra's algorithm. I really like how all these ideas are related. [1] https://www.youtube.com/watch?v=ei6RfbplYZM https://www.youtube.com/watch?v=ei6RfbplYZM [2] https://www.feynmanlectures.caltech.edu/III_03.html https://www.feynmanlectures.caltech.edu/III_03.html [3] https://graphicallinearalgebra.net/ https://graphicallinearalgebra.net/
  - Grustaf 5y ago
    I watched the path counting one and I want my time refunded please. It had nothing to do with linear algebra. Nothing to do with quantum physics either, except that they used bracket notation for some inexplicable reason. What am I missing?
    - mathgenius 5y ago
      Thanks for the feedback. Do you want your time refunded, or do you want answers to your questions? The two seem to be in contradiction. If it's the former then what is your hourly rate for watching well-intentioned youtube videos made by experts in the field ?
- penguin_booze 5y ago
  Here's part 1 of Pavel Grinfeld's linear algebra series: https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS14F4NxAZudSmv https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS1....
hwers 5y ago
I keep seeing the same material like this (and love it btw) but I keep thinking that it's all only scratching the surface. From what I've seen in abstract algebra this stuff goes way deeper and becomes way more beautiful. I would love the "homemade" explanations and visualizations to start enlightening us about that. E.g. so DIY machine learning folks like the amazing GAN art community that's cropping could get more tooling to pluck easy hanging fruit.
ChicagoBoy11 5y ago
I never understood why we don't do more of this stuff in school, and how calculus instead became the defacto advanced math curriculum in most high-schools. Students grow up working on their basic algebraic operations, solving equations, etc. Liner Algebra introduces them to the universe that lies just beyond those techniques, has very readily applicable uses, lends itself excellently to simulation/connections to computer science (which is super popular to teach now), etc.
- macrolocal 5y ago
  I suspect the answer is partly historical; without a computer, calculations in linear algebra are a pain.
- Grustaf 5y ago
  Both calculus and linear algebra are fundamental for further studies, but I'm not sure linear algebra is more approachable? I personally find geometry and algebra more interesting, but it seems to me that derivatives are more fundamental than matrices. But maybe that's just my bias from being educated like that.
  - dls2016 5y ago
    What if I told you the derivative is a linear operator? (matrix meme style)
    - deleted 5y ago
      [deleted]
    - Grustaf 5y ago
      Of course, but there are plenty of linear operators, that property alone doesn't define it.
      - dls2016 5y ago
        I was responding to derivatives being more fundamental than matrices. My rebuttal is that the derivative is but one in a sea of interesting linear operators.
Matt-Gleich 5y ago
Great visuals
deleted 5y ago
[deleted]
jnurmine 5y ago
Would love to see other things like physics, chemistry, etc. done with this kind of representation. Visualized, concise descriptions.
hiranabe 5y ago
Hello, I'm the author of the article, thanks for the nice comments. This article should have been titled as "Graphic Notes on Linear Algebra for Everyone", and Prof. Strang kindly suggested this big name. I was lucky that this drew this attention. There are some other visuals I'm trying around the area. - Eigenvalues https://anagileway.com/2021/10/01/map-of-eigenvalues/ https://anagileway.com/2021/10/01/map-of-eigenvalues/ -Matrix classification https://anagileway.com/2020/09/29/matrix-world-in-linear-algebra-for-everyone/ https://anagileway.com/2020/09/29/matrix-world-in-linear-alg... When I was an undergraduate, I didn't get this understanding of linear algebra... but after watching all the Prof. Strang's 18.06 classes in MIT OpenCourseWare, now I have much clear view of this area... So I really appreciate his way of teaching. BTW, I even made a T-shirt and sent him ! https://anagileway.com/2020/06/04/prof-gilbert-strang-linear-algebra/ https://anagileway.com/2020/06/04/prof-gilbert-strang-linear...

350 points by allending 5y ago

It seems like the "github.com" link would be a better and more canonical url rather than link an opaque url to "githubusercontent.com" https://github.com/kenjihiranabe/The-Art-of-Linear-Algebra https://github.com/kenjihiranabe/The-Art-of-Linear-Algebra

happy-go-lucky 5y ago
And, "Graphic Notes on Gilbert Strang's Linear Algebra for Everyone" would be less ambiguous than "The Art of Linear Algebra."

visarga 5y ago

Nice, I was just reviewing SVD and PCA, I grok them but then I forget, this material is useful for remembering the big picture.

ivan_ah 5y ago

Wow this is really well done. It's like a visual-algebraic approach... I've never seen this before. I like how the author sets up a "grammar of matrix multiplications," and then reuses the same patterns in the rest of the document. For people who might not be familiar, these visual notes are inspired by and complement Prof. Strang's new book https://math.mit.edu/~gs/everyone/ https://math.mit.edu/~gs/everyone/ and course https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-linear-algebra-spring-2020/index.htm https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-li... https://www.youtube.com/playlist?list=PLUl4u3cNGP61iQEFiWLE21EJCxwmWvvek https://www.youtube.com/playlist?list=PLUl4u3cNGP61iQEFiWLE2... see also https://news.ycombinator.com/item?id=23157827 https://news.ycombinator.com/item?id=23157827

hiranabe 5y ago
Ivan, thanks for sharing the background concepts Prof. Strang has told us. Yes, I am a big fan of him, and I came up with this idea during several email exchanges with him.
allending 5y ago
Your books are pretty amazing too Ivan. Have em all.
deleted 5y ago
[deleted]

skytreader 5y ago

Instantly fell in love with the presentation! Might give this a read-through just out of pure curiosity. In undergrad, my mnemonic for these operations was visualizing the matrices animated in my head. The more complex ones, it was actually easier for me to remember Scheme functions that represent the algorithm (all expressed via higher-order functions so it was pretty concise); this was unique to my circumstances as an undergrad, not something I can pull off today without reviewing a lot of material. Presenting the operations with color and blocks just gives a more natural "user interface" (lacking a better term) for remembering it!

amelius 5y ago

On the first page they say "if neither a or b are 0", but they haven't defined what it means for a vector to be 0. Also, they say "rank 1 matrix", but they haven't defined the concept of rank yet. Some readers might find this kind of presentation acceptable, but personally I strongly dislike it when concepts are used before they are defined.

Koshkin 5y ago
To be fair, it does not look like this text is intended to be a self-contained introduction to linear algebra.
Grustaf 5y ago
The concept of an additive zero is built into what it means to be a vector space though.

_wldu 5y ago

This is great for those studying algorithms (DP, DC, FFT, etc.). Very useful.

tishha 5y ago

good notes, very useful!

antegamisou 5y ago

Also check out 3Blue1Brown's Essence of Linear Algebra https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFit...

deleted 5y ago
[deleted]
mathgenius 5y ago
And here is linear algebra as path counting [1]. This is closely related to path integrals in quantum physics. The rules for combining quantum amplitudes are the rules for combining path counts [2]. This is also how graphical linear algebra works [3]. And since we are all hackers here, if you replace the underlying number system with the min-sum semiring you get things like Dijkstra's algorithm. I really like how all these ideas are related. [1] https://www.youtube.com/watch?v=ei6RfbplYZM https://www.youtube.com/watch?v=ei6RfbplYZM [2] https://www.feynmanlectures.caltech.edu/III_03.html https://www.feynmanlectures.caltech.edu/III_03.html [3] https://graphicallinearalgebra.net/ https://graphicallinearalgebra.net/
- Grustaf 5y ago
  I watched the path counting one and I want my time refunded please. It had nothing to do with linear algebra. Nothing to do with quantum physics either, except that they used bracket notation for some inexplicable reason. What am I missing?
  - mathgenius 5y ago
    Thanks for the feedback. Do you want your time refunded, or do you want answers to your questions? The two seem to be in contradiction. If it's the former then what is your hourly rate for watching well-intentioned youtube videos made by experts in the field ?
penguin_booze 5y ago
Here's part 1 of Pavel Grinfeld's linear algebra series: https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS14F4NxAZudSmv https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS1....

hwers 5y ago

I keep seeing the same material like this (and love it btw) but I keep thinking that it's all only scratching the surface. From what I've seen in abstract algebra this stuff goes way deeper and becomes way more beautiful. I would love the "homemade" explanations and visualizations to start enlightening us about that. E.g. so DIY machine learning folks like the amazing GAN art community that's cropping could get more tooling to pluck easy hanging fruit.

ChicagoBoy11 5y ago

I never understood why we don't do more of this stuff in school, and how calculus instead became the defacto advanced math curriculum in most high-schools. Students grow up working on their basic algebraic operations, solving equations, etc. Liner Algebra introduces them to the universe that lies just beyond those techniques, has very readily applicable uses, lends itself excellently to simulation/connections to computer science (which is super popular to teach now), etc.

macrolocal 5y ago
I suspect the answer is partly historical; without a computer, calculations in linear algebra are a pain.
Grustaf 5y ago
Both calculus and linear algebra are fundamental for further studies, but I'm not sure linear algebra is more approachable? I personally find geometry and algebra more interesting, but it seems to me that derivatives are more fundamental than matrices. But maybe that's just my bias from being educated like that.
- dls2016 5y ago
  What if I told you the derivative is a linear operator? (matrix meme style)
  - deleted 5y ago
    [deleted]
  - Grustaf 5y ago
    Of course, but there are plenty of linear operators, that property alone doesn't define it.
    - dls2016 5y ago
      I was responding to derivatives being more fundamental than matrices. My rebuttal is that the derivative is but one in a sea of interesting linear operators.

Matt-Gleich 5y ago

Great visuals

deleted 5y ago

[deleted]

jnurmine 5y ago

Would love to see other things like physics, chemistry, etc. done with this kind of representation. Visualized, concise descriptions.

hiranabe 5y ago

Hello, I'm the author of the article, thanks for the nice comments. This article should have been titled as "Graphic Notes on Linear Algebra for Everyone", and Prof. Strang kindly suggested this big name. I was lucky that this drew this attention. There are some other visuals I'm trying around the area. - Eigenvalues https://anagileway.com/2021/10/01/map-of-eigenvalues/ https://anagileway.com/2021/10/01/map-of-eigenvalues/ -Matrix classification https://anagileway.com/2020/09/29/matrix-world-in-linear-algebra-for-everyone/ https://anagileway.com/2020/09/29/matrix-world-in-linear-alg... When I was an undergraduate, I didn't get this understanding of linear algebra... but after watching all the Prof. Strang's 18.06 classes in MIT OpenCourseWare, now I have much clear view of this area... So I really appreciate his way of teaching. BTW, I even made a T-shirt and sent him ! https://anagileway.com/2020/06/04/prof-gilbert-strang-linear-algebra/ https://anagileway.com/2020/06/04/prof-gilbert-strang-linear...