10 ms·

A 2020 Vision of Linear Algebra

835 points by organicfigs 6y ago

knzhou 6y ago
Gilbert Strang's linear algebra course blew my mind back in high school, and I still use insights from it every day. Strang has a particular lecturing style where he approaches every topic several times, often beginning many lectures before the main treatment. At first I thought it was a bit confusing, but later I realized it helped build fluency, just like a language class. I'm really thankful to MIT OCW for putting his lectures out for free -- in fact, I think I'll go donate to them now.
- tomjakubowski 6y ago
  +1. Since I took the OCW course, whenever I multiply a matrix and a vector or two matrices by hand, I hear his voice saying, "combinations of columns." Strang must have said those words hundreds of times in it. His lectures stick like nothing else.
- raz32dust 6y ago
  +1, I was very grateful to MIT OCW because when I learned Linear algebra, I could not have afforded it. Later when I got a job, I donated to OCW and I bought his book full price from his own site [1] just as a tribute to the guy. [1] https://math.mit.edu/~gs/linearalgebra/ https://math.mit.edu/~gs/linearalgebra/
  - qorrect 6y ago
    Hey me too! ( All of it )
- willbw 6y ago
  Of course it is up to you what you do with your money but, they have a $17.5B endowment so there may be more needy causes if you were so inclined.
  - arcturus17 6y ago
    OCW opens up top-notch education to anyone and everyone, regardless of social or economic background. I wouldn’t take it for granted even with MIT’s eye-watering endowment, and I doubt donations to it are going to be paying cafeteria lunches for the students. I hope you’re donating and actively contributing to many non-profit projects and that your comment comes from being tired of the world’s injustices rather than from callous impertinence, although I suspect it does not.
    - Judgmentality 6y ago
      I think his point is there are plenty of other noble causes that could use the money a lot more.
    - wegs 6y ago
      Most money which comes into MIT passes through overhead. That means if a foundation donates to MIT, a bit over 1/3 of that money might ends up with whatever they donated to. A bit under 2/3 might go into the general budget (overheads vary by funding source, but the numbers above are from one specific project). On paper, overhead is used for costs of running the place. In practice, it's used for things like upscale faculty clubs, million-dollar executive salaries, $200 million buildings, etc. MIT has among the highest overheads in the academy. Ironically, MIT claims its ocean yacht makes money rather than losing money (which could very well be true). If you're okay with the majority of your money going to graft, donate to MIT. With a project like OCW, which has such a huge cost:benefit ratio, accepting the graft with the donation may be a rational decision, if you subscribe to a system of ethics like utilitarianism. Personally, I almost never donate to a charity where the highest-earner makes more than I do. I think if everyone did that, MIT might lose some of the graft and corruption which has built up there over the years.
      - mattkrause 6y ago
        MIT’s current overhead rate is 50.5%, but that’s pretty standard. Here’s a scatterplot showing lots of research institutions’ rates. When this was published, MIT’s rate was slightly higher (54%). https://www.nature.com/news/indirect-costs-keeping-the-lights-on-1.16376 https://www.nature.com/news/indirect-costs-keeping-the-light... showing actual and calculated rates. This also applies to federal research grants and is meant to cover costs associated with actually hosting the research (rent, utilities, support staff). Foundations can (and often do) negotiate lower rates. I’m not sure how donations are handled, but I don’t think the same F&A rates apply.
- scared2 6y ago
  In High school?
  - gowld 6y ago
    Yes. Linear Algebra is an extension of what is commonly called Algebra 2 or Precalculus in high school. LA and Calculus can be studied independently in any order and then fruitfully combined later.
- bencw 6y ago
  This is very well put. Knowledge has a hierarchical (or perhaps even cyclical!) structure and it's unrealistic to think that a body of knowledge can be taught or learned sequentially.
tomerbd 6y ago
It was a good course I watched it online but I didn't understand much.
potta_coffee 6y ago
Picked up Strang's linear algebra book recently and I'm enjoying it. I've been consistently impressed with the content of MIT books.
jp0d 6y ago
I've been doing the Statistics Micromasters from MIT. It's rigorous and very deep. I look forward to doing this.
frequentnapper 6y ago
Back in uni (2005), we used Dr. Strang's text for linear algebra. When reading the text, I felt like some down-to-earth professor was trying to explain these difficult topics as simply as possible. I remember discovering mit.edu back then and finding precious video lectures that went along with the book after the course. One of the very few times I was so genuinely happy and excited to watch math lectures online :p
- jbd28 6y ago
  We used his book then too, at Drexel in Philadelphia. Our prof at the time invited Dr Strang to guest lecture one time and I remember it being so clear and obvious as he talked that I thought “wow this is why an MIT education is so revered”. I waited after the lecture to personally thank him and have him autograph the textbook; very glad I did in retrospect.
roenxi 6y ago
It is interesting to compare this with 3Blue1Brown's linear algebra introduction on YouTube. He seems to have been the only mathematician who has actually mastered the medium; linear algebra lends itself very well to animations. The mathematicians don't understand how badly they need to animate some of these concepts.
- ekianjo 6y ago
  Agree, world class education in maths there. He understands the importance of examples and visualization and it changes everything.
- gfxgirl 6y ago
  3Blue1Brown's linear algebra animations were fun to watch but they did almost nothing for me except the basic fact that the "linear" part means lines. The rest was effectively preaching to the choir so those that already know linear algebra nodded their heads and idiots like me were still flummoxed
  - domnomnom 6y ago
    Such is life
  - mjburgess 6y ago
    That's interesting. I do think 3B1B's goal is probably to build better intuitions in people who already know it.
    - mesaframe 6y ago
      Yes, those lectures are good enough for intuitions only. For practice one has to read books. I just hope the viewer knows that. Which I have seen is absent among some people.
  - john4532452 6y ago
    Yes. 3Blue1Brown has himself said multiple times no videos is substitute for text books.
tomahunt 6y ago
A 2018 paper by Strang about this approach: https://www.tandfonline.com/doi/abs/10.1080/00029890.2018.1408378 https://www.tandfonline.com/doi/abs/10.1080/00029890.2018.14...
sqlmonkey 6y ago
I started by only reading his book thinking it was enough. I was very wrong, these videos marry themselves beautifully with the content of the book which suddenly became incredibly clear once I started watching the videos. Strang teaching style can also seem odd at first, but don't give up, he is an amazing teacher who makes every concept simple to understand. This course is a true gift.
lcuff 6y ago
As someone who understands nothing of linear algebra, I have to say this "introduction" was gibberish. He may be a fantastic teacher, and perhaps it's a bit much to expect a 4 minute video to teach me anything, but it reminds me of talks from business people where what they're saying is obvious if you already understand it, and completely obscure otherwise.
- wodenokoto 6y ago
  This is not a course, or a primer or an introduction on Linea Algebra. From the course description: > These six brief videos, recorded in 2020, contain ideas and suggestions from Professor Strang about the recommended order of topics in teaching and learning linear algebra.
glram 6y ago
Professor Strang’s lectures helped me greatly during my linear algebra class. I thoroughly appreciated his clear, coherent lecture style. On another note, he is such a nice guy. 10/10.
synaesthesisx 6y ago
Linear algebra was one of those classes I was forced to take in undergrad as an engineering requirement - only to end up appreciating it immensely later on when I realized how many real world problems can be converted to matrix operations.
crdrost 6y ago
So I like this outline. It is very MIT-ish where there is a sense of teaching someone to solve practical engineering problems with matrices. But, I do foresee some difficulties. One thing that I find really difficult, for example, is that I take undergrads who have had linear algebra and ask "what is the determinant?" and seldom get back the "best" conceptual answer, "the determinant is the product of the eigenvalues." Like, this is math, the best answer should not be the only one, but it should be ideally the most popular. We would consider it a failure in my mind if the most popular explanation of the fundamental theorem of calculus was not some variation of "integrals undo derivatives and vice versa". I don't see this approach solving that. Furthermore there is a lot of focus from day one on this CR decomposition which serves to say that a linear transform from R^m to R^n might map to a subspace of R^n with smaller dimension r < min(m,n) and while in some sense this is true it is itself quite "unphysical"—if a matrix contains noisy entries then it will generally only be degenerate in this way with probability zero. (You need perfect noise cancelation to get degeneracies, which amounts to a sort of neglected underlying conserved quantity which is pushing back on you and demanding to be conserved.) In that sense the CR decomposition is kind of pointless and is just working around some "perfect little counterexamples". So it seems weird to see someone say "hold this up as the most important thing!!"
- mansoor_ 6y ago
  Subjective, I find the geometric interpretation of the determinant to be the "best".
- heinrichhartman 6y ago
  > seldom get back the "best" conceptual answer, "the determinant is the product of the eigenvalues." I found that the "best conceptual" answer depends a lot on taste, and what concepts you are familiar with. In this case: - Calculating exact eigenvalues of matrices larger than 4x4 is impractical, since it requires you to solve a polynomial of degree >4. - The EV exist only in algebraically closed fields (complex numbers), while the determinant itself lives in the base field (rationals, reals). How about: - [Geometric Determinant] The determinant is the volume of the polytope (parallel-epiped) spanned by the column vectors of the matrix. - [Coordinate Free Determinant] The determinant is the map induced between the highest exterior powers of the source and target vector spaces (https://en.wikipedia.org/wiki/Exterior_algebra https://en.wikipedia.org/wiki/Exterior_algebra) - I think there is also a representation theoretic version, that characterizes the determinant as invariant under the Symmetric group acting by permutation on the columns/rows of the matrix.
auggierose 6y ago
Have not watched the videos yet, but that seems to me more like an 1820 vision of linear algebra :-) If you look at the order of topics in his book "An Introduction to Linear Algebra", you will find the topic "Linear Transformation" way back in chapter 8! Even after the chapters eigenvalue decomposition and singular value decomposition. But understanding that a matrix is just the representation of a linear transformation in a particular basis is probably the most important and first thing you should learn about matrices ...
- dandanua 6y ago
  I don't think it was this way in 1820, but I agree with your point. Even though I also use Linear Algebra mostly computationally today, the origin of it is in the geometry and I think this connection should come first. Also, "number crunching" is a boring way to learn things. Though, "matrix way" can be good for engineers.
- jacobolus 6y ago
  Gaussian Elimination is indeed from the 1820s. All the rest is more recent than that. The idea of matrix decomposition per se comes from the 1850s. The earliest work on something like the SVD is from the 1870s. You are onto something though. Strang is coming from a direction of numerical computations and algorithms for solving real-world problems. Pure mathematics departments for at least the past maybe 80 years often look down on numerical analysis, statistics, engineering, and natural science, and adopt a position that education of students should be optimized in the direction of helping them prove the maximally general results using the most abstract and technical machinery, with an unfortunate emphasis on symbol twiddling vs. examining concrete examples. By contrast, in the 19th century there was much more of a unified vision and more respect for computations and real-world problems. Gauss himself was employed throughout his career as an astronomer / geodesist, rather than as a mathematician, and arguably his most important work was inventing the method of least squares, which he used for interpreting astronomical observations. With the rise of electronic computers, it is possible that the dominant 2050 vision of linear algebra and the dominant 1900 vision of linear algebra will be closer to each-other than either one is to a 1950 vision from a graduate course in a pure math department.
srean 6y ago
I am familiar with the material of linear algebra but haven't read his books. Could someone who has absorbed linear algebra from different sources and familiar with Strang's books comment on what's good and bad and unique about them. In my time I had picked LA from Ben Noble, Halmos and Axler and the computation side of things from Golub & van Loan.
- mesaframe 6y ago
  I read his book and I would say his book work in complement with his lectures. It is not good enough on it's own.
- T-zex 6y ago
  I'm still a beginner and I think Strang's Linear algebra books are more like a supplement material to his lectures. If you need to build a solid theoretical foundation of linear algebra you'd need to consider other resources too. Having said that, he is explaining many things really well and is helping a lot to build intuition. He is always cautious presenting things that are computationally inefficient and suggests the alternatives. Exercises are too hard for me personally. I'd prefer a more laborious set of exercises helping to cement the material, (as in calculus or usual algebra) and then have one or two problem solving puzzles at the end.
  - srean 6y ago
    So the focus is on different recipes to cook a matrix with ? Different operations one can do on a matrix ? I hope its not just that, that would be very limiting considering what linear algebra is about and capable of.
    - T-zex 6y ago
      No, his books are not recipes. The thing I'm struggling to communicate here is that he's got a more pragmatic style compared to other text books. The material he presents is complete and and he is doing great job making it approachable for non mathematicians. His books usually expand on the subjects he presents at his online lectures. I see them as advanced lecture notes.
praptak 6y ago
I'm currently trying to grok the finite element method. Gilbert Strang's explanation of the transition from the Galerkin method to FEM did more for me in terms of connecting the dots than anything else I could find on the web. And it wasn't even a lecture, just a kind of an interview. I think it's this one: youtube.com/watch?v=WwgrAH-IMOk
- kragen 6y ago
  https://www.youtube.com/watch?v=WwgrAH-IMOk https://www.youtube.com/watch?v=WwgrAH-IMOk I feel like I don't really understand his explanation, because it's kind of vague. But I think that might be because you've seen the equations dozens of times, and I haven't seen them at all, so you were prepared to understand the video.
  - praptak 6y ago
    This makes sense. As said, it was about connecting the dots for me. Also, I don't even claim I fully understand FEM (or even Galerkin), it's just my hobby project.
    - kragen 6y ago
      That sounds interesting! What are you doing with it?
      - praptak 6y ago
        I just want to understand the magic behind static stress analysis. More generally I'm interested in emulating physics behind the rigid body model. Maybe I will create a game prototype based on the mechanics but this is just a vague idea.
penguin_booze 6y ago
I recently came across this rather in-depth series on linear algebra: https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS14F4NxAZudSmv https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS1.... FWIW, I myself have only gone half-way thorugh part 1.
deleted 6y ago
[deleted]
enitihas 6y ago
Another good Linear Algebra book is "Linear Algebra Done Right", which Springer is giving for free right now. Link: https://link.springer.com/book/10.1007/978-3-319-11080-6 https://link.springer.com/book/10.1007/978-3-319-11080-6
- mseri 6y ago
  Came here to say that. It is a wonderful book, and I think it provides a more "modern" approach than the one presented in the videos.
- clarry 6y ago
  There's also a free book "Linear Algebra Done Wrong," which might be worth checking out. https://www.math.brown.edu/~treil/papers/LADW/LADW.html https://www.math.brown.edu/~treil/papers/LADW/LADW.html
- threatofrain 6y ago
  IMO Axler's book should be read either during or after you take an introductory course on Linear Algebra. > You are probably about to begin your second exposure to linear algebra. Unlike your first brush with the subject, which probably emphasized Euclidean spaces and matrices, this encounter will focus on abstract vector spaces and linear maps.
  - vlasev 6y ago
    I whole-heartedly agree. Axler's book is a great stepping stone to more abstract linear algebra.
- bencw 6y ago
  This book is great and very much complementary to Strang's approach in that it leans more towards "abstract" linear algebra.
- AlanYx 6y ago
  Thanks for this -- do you know if there's a consolidated list anywhere of other books Springer is making available for free right now?
brmgb 6y ago
After watching this and having read the comments, I am quite puzzled by the approach American seem to take to linear algebra. Are matrices viewed as the core of the subject in the USA ? My country curriculum introduces linear algebra through group theory and vector spaces. Matrices come later.
- chubot 6y ago
  Yeah I would call this the engineering approach (matrices) vs the mathematical approach (algebra). I took like 3-4 courses in the US involving the engineering approach, starting in high school and continuing through the college as a CS major. That was all that was required. But I also like algebra, so I happened to take a 400-level course that only math majors take my senior of college. And then I got the group theory / vector space view on it. I don't think 95% of CS majors got that. I don't think one is better than the other, but they should have tried to balance it out more. It helps to understand both viewpoints. (If you haven't seen the latter, then picture a 300-page text on linear algebra that doesn't mention matrices at all. It's all linear transformations and spaces.) What country were you taught in? Wild guess: France?
- ojnabieoot 6y ago
  I would not describe his approach as “the US approach” but it is a pretty standard approach to introducing linear algebra to engineers, which is the theme of the course. I was also taught linear algebra this way, by an applied mathematician with a background in chemical engineering: - start by solving Ax=b with row reduction - develop theorems about linear independence and spanning sets of vectors based on these exercises - introduce the determinant from the perspective of linear systems (rather than eg geometry or group theory) - eigenvectors and eigenvalues Later I switched from physics to math and TAed a more “algebraic” approach involving groups/rings/fields. But the matrix-first approach was more helpful for both my physics coursework and later courses in numerical linear algebra.
- tadhgds 6y ago
  I can't say whether or not it is the standard approach but I do know that it is very common in many countries to teach a linear algebra course that is heavy on matrix operations, that you can come away believing that linear algebra is somehow _about_ matrices and their operations. I know many in my university class seemed to believe that. A book I enjoyed is Axler's Linear Algebra Done Right[0], in which, if I remember correctly, doesn't contain a single matrix. [0]https://zhangyk8.github.io/teaching/file_spring2018/linear_algebra_done_right.pdf https://zhangyk8.github.io/teaching/file_spring2018/linear_a...
inshadows 6y ago
Gilbert Strang taught me how to sanely multiply matrices. His Introduction to Linear Algebra is very approachable. It's wildly different experience compared to linear algebra courses I had on university, it actually makes sense and is fun!
elAhmo 6y ago
What would you recommend as a good resource for learning about Linear Algebra in 2020? I am aware of his course on OCW, but wondering is there something more interactive and/or newer than those lectures that has similar quality.
- skywal_l 6y ago
  Frankly, couple with this book, it does hardly get better. You still have 3blue1brown[1] series of video, but it just brush off the surface. [1] https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...
chadcmulligan 6y ago
Wow, Prof Strang is 85 and still teaching! Thats very impressive and inspiring.
pengaru 6y ago
It's ridiculous how much random college-level linear algebra textbook material I stared at before things clicked in the course of just jumping in and exploring 3D graphics and writing my own 3D vector, matrix multiplication and 3D transform headers and using them in making some games in plain C. At some point it's like "Wait, is linear algebra really just about heaps of multiplication and addition? Like every dimension gets multiplied by values for every dimension, and values 0 and 1 are way more interesting than I previously appreciated. That funny identity matrix with the diagonal 1s in a sea of 0s, that's just an orthonormal basis where each corresponding dimension's axis is getting 100% of the multiplication like a noop. This is ridiculously simple yet unlocks an entire new world of understanding, why the hell couldn't my textbooks explain it in these terms on page 1? FML" I'm still a noob when it comes to linear algebra and 3D stuff, but it feels like all the textbooks in the world couldn't have taught me what some hands-on 3D graphics programming impressed upon me rather quickly. Maybe my understanding is all wrong, feel free to correct me, as my understanding on this subject is entirely self-taught.
- Phlogistique 6y ago
  Is it the same issue as the infamous "monads tutorials" problem, where the understanding takes a lot of time to infuse but looks obvious in retrospect when it finally clicks?
  - DagAgren 6y ago
    I am convinced that monads induce a very specific kind of brain damage that makes a person incapable of ever explaining monads.
    - dleslie 6y ago
      Start with a container. M a Then add a way to put things in the container. a -> M a Then add a way to use the thing in the container. M a -> (a -> M b) -> M b
      - 6y ago
vertak 6y ago
Can anyone grow why in the first 5 minutes of part 1 he shows a 3 by 3 matrix multiples by a 1 by 3 vector yet verbally he pulls out of no where this idea that if you have _two_ 1 by 3 vectors that pass through the origin then their linear combinations can be represented by a plane? The jump from the 3D to the 2D has me lost and I gave up
- swiley 6y ago
  I don’t know which video you’re talking about but two non parallel vectors are enough to represent a plane, the normal will be their cross product. Also, if you have 3 dimensional vectors you were always in 3D.
- neutronicus 6y ago
  If you multiply the 1x3 vector by all scalars from -infinity to infinity you get all the points on a line. If you do the same for another 1x3 vector, and it is not parallel to the first, you get all the points on a different line. These two lines define a plane (and the cross product of the two vectors defines its normal vector)
- jcmoyer 6y ago
  The same concept applies in 2D, which might help you build the intuition to understand it in 3D. If you have a vector v=(1,0) that points to the right, you can scale this vector infinitely in that direction by multiplying it by a positive scalar. 5v = (5,0) 62.1v = (62.1,0) Similarly, you can scale that vector infinitely in the opposite direction (i.e. left) by multiplying it by a negative scalar: -987v = (-987,0) If we call this scalar c, the expression cv allows us to represent any point along the X axis simply by varying c, meaning that cv defines a line along that axis. Similarly, we can do the same for a vector w=(0,1) along the Y axis, scaling it by d. Now we have a method for moving to any point on the XY plane simply by varying c and d in the linear combination: cv + dw, meaning that we've defined a plane using two vectors. Two caveats: - this won't work if v and w are parallel; for example, if v = -w (and neither are zero) then we can only move along a line instead of a plane - it also won't work if either of the vectors are zero, because no matter what you multiply by, a zero vector can only represent a single point
katzgrau 6y ago
I had very intelligent linear algebra professor in college but he was, in my opinion, a very poor communicator. I paid attention to lectures and stared at the text, but couldn't really understand the material. For the first part of a linear algebra course, students who don't mind blindly following mechanical processes for solving problems can do very well. Unfortunately I'm one of those people who tends to reject the process until I understand why it works. If it wasn't for Strang's thoughtful and sometimes even entertaining lectures via OCW, I probably would have failed the course. Instead, as the material became considerably more abstract and actually required understanding, I had my strongest exam scores. I didn't even pay attention in class. I finished with an A. Although my first exam was a 70/100, below the class average, the fact that I got an A overall suggests how poorly the rest of the class must have done on the latter material, where I felt my strongest thanks to the videos. So anyway, thank you Gilbert Strang.
- ansible 6y ago
  > I paid attention to lectures and stared at the text, but couldn't really understand the material. For the first part of a linear algebra course, students who don't mind blindly following mechanical processes for solving problems can do very well. I had a similar, though sort of opposite experience. In high school, I breezed through the material, and started teaching myself calculus during the summer to prepare for university. Other than being a lazy student, I had no problems taking the 2nd semester advanced calc 2 and 3 courses my freshman year. I totally get what's being taught. There weren't a ton of practical examples, but I can easily see (for example) what the purpose of integration is, and how and why you'd do it in two or more dimensions. I could work the equations, no problem. Everything is great. Along comes sophomore year, and still thinking I am hot stuff, I take advanced linear algebra and differential equations. More of the same, I thought. Well... we seemed to spend the entire semester just solving different kinds of equations. No explanations given as to what they are for, where they are used, or what the point of any of it was. I struggled, for the very first time. I either got a D or F for the mid-term exam, which was shocking to me. We had one chapter where we were doing something practical. This is where you have a water tank, and a hole in to bottom. Because the pressure lessens as the tank empties, the flow rate is not constant. However, you can solve this via diff equations, and I really grokked it. I finally saw the point for some of what we had been doing. But it was just that one chapter, we skipped any other practical aspects for what we were studying. I did end up pulling out a 'C' with that class, to my relief. Sure, most of the blame for my lousy performance must rest with me, because of my poor study habits. And a little blame can go to the TA, who wasn't a good communicator, so that hour every week was kind of useless. But I also blame the material and how it was presented.
anandrm 6y ago
Just curious .. what really are the usecases where of Linear Algebra is applied ? Any domain of software development ?
- ktta 6y ago
  Any fields that have anything to do with video, image, audio, games, machine learning. Just to have a taste of use cases: compression, filters(image filters for de-noising, HP & LP filters for audio), encoding/decoding, computer vision techniques, cryptography, neural nets, computer graphics (this is where most people learn how to use it in real computer programs)
- justinmeiners 6y ago
  - scale or rotate an image. - root finding algorithm with more than one variable. - graph problems like Google's PageRank - statistical analysis - 3d rendering (projecting a 3d scene onto a 2d image) - solving systems of equation (also see linear programming) Linear algebra is very basic and fundamental to physics and math.
- vlasev 6y ago
  It's applied pretty much everywhere. Most numerical problems have some linear algebra component to them. Physics uses it a lot too. A lot of non-linear problems have a linearization on which you can use linear algebra to obtain approximations. Ideas from linear algebra are used a lot in things like signal processing, quantum mechanics, etc.
irl_zebra 6y ago
I've been wanting to learn linear algebra. I had some exposure in college along with my calc classes, but never really understood it fundamentally. Like it was mentioned, I mostly did matrix transforms but didn't realize fundamentally grasp. I started doing LA on Khan academy, and checked out Linear Algebra Done Right. LADR was a little too much into the deep end for me. KA seemed to be good. One nice thing about KA is that when I didn't quite remember something (i.e. how exactly to multiply a matrix) I could just go to an earlier pre-LA lesson, pick it up, and then go back to LA where I left off. I'm a few lessons in. What do you all recommend for someone like me?
- nafizh 6y ago
  If you want to learn LA by coding, coding the matrix by philip klein is a really good book. He even has his Coursera lecture videos (not available on coursera anymore last I checked) up on his website.
hprotagonist 6y ago
my linear algebra professor began his first lecture with a 5 minute rant informally titled “how you could have gotten an A in differential equations last semester without ever having taken calculus” That certainly got our attention. I’ve always found linear algebra to be kind of ... almost soothing.
- eximius 6y ago
  That's a rant I'd like to hear.
  - hprotagonist 6y ago
    "look guys, at end of class, exam is always long list of silly second order system of differential equations. Well, every time, we can make so-called "guess" that solution looks like e*rt. Why? We know that because professors will only give well-behaved systems on final exams because it's hard to grade the other kind. So we know characteristic polynomials look like so (because of course they do, you can just memorize this) ... so now we lift out the coefficients into nifty thing called _matrix_ and now follow these easy four steps to get roots, plug back in, and incidentally these are "eigenvalues", we'll talk about this later ... Bam. Done. A-, easy. No sweat."
    - eximius 6y ago
      Minus the matrix bits, that's basically how I slogged through DiffEq. Showing up to class was pointless because the prof would make up an equation to exploratively solve and inevitably it would be poorly behaved and the lecture would end with "... And and and for this kind of problem we have to use numerical methods".
balls187 6y ago
Admittedly, I never fully groked linear algebra. Some of the concepts made sense, especially solving for linear systems of equations. Recently, I decided to brush up on my math skills via Youtube videos, and came across this series: https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw It explains Linear Algebra concepts using 2D and 3D vector manipulation, and the animations help me visualize the underlying maths.
longtimegoogler 6y ago
IMO, if one is interested in a computational approach to Linear Algebra, Trefethens book, Numerical Linear Algebra, is the best. That book discusses the actual algorithms used for computation. It is a bit more advanced, but amazingly clear.
ivan_ah 6y ago
For anyone who is already familiar with the Prof. Strang's lectures from previous years, the main new thing in this five-lecture mini-series is he tries to condense the material even further—maximum intuition and power-ideas, instead of the full-length in-class lecture format with derivations. This makes the material difficult to understand for beginners, but makes a great second source in addition to or after a regular LA class. One of the interesting new ways of thinking in these lectures is the A = CR decomposition for any matrix A, where C is a matrix that contains a basis for the column space of A, while R contains the non-zero rows in RREF(A) — in other words a basis for the row space, see https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-linear-algebra-spring-2020/videos/MITRES_18_010S20_LA_Slides.pdf#page=7 https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-li... Example you can play with: https://live.sympy.org/?evaluate=C%20%3D%20Matrix(%5B%5B1%2C4%5D%2C%20%5B3%2C2%5D%2C%20%5B2%2C1%5D%5D)%0A%23--%0AC%0A%23--%0AR%20%3D%20Matrix(%5B%5B1%2C0%2C1%5D%2C%20%5B0%2C1%2C1%5D%5D)%0A%23--%0AR%0A%23--%0AC*R%0A%23--%0A(C*R).rref()%5B0%5D%0A%23--%0A https://live.sympy.org/?evaluate=C%20%3D%20Matrix(%5B%5B1%2C... Thinking of A as CR might be a little intense as first-contact with linear algebra, but I think it contains the "essence" of what is going on, and could potentially set the stage for when these concepts are explained (normally much later in a linear algebra course). Also, I think the "A=CR picture" is a nice justification for where RREF(A) comes about... otherwise students always complain that the first few chapters on Gauss-Jordan elimination is "mind-numbing arithmetic" (which is kind of true...) but maybe if we present the algorithm as "finding the CR-decomposition which will help you understand dozens of other concepts in the remainder of the course" it would motivate more people to learn about RREFs and the G-J algo.
- ivan_ah 6y ago
  Since y'all are code-literate, here the SymPy function for finding the CR-decomposition of any matrix A: def crd(A): """ Computes the CR decomposition of the matrix A. """ rrefA, licols = A.rref() # compute RREF(A) C = A[:, licols] # linearly indep. cols of A r = len(licols) # = rank(A) R = rrefA[0:r, :] # non-zero rows in RREF(A) return C, R Test to check it works: https://live.sympy.org/?evaluate=A%20%3D%20Matrix(%5B%0A%20%20%5B1%2C%204%2C%205%5D%2C%0A%20%20%5B3%2C%202%2C%205%5D%2C%0A%20%20%5B2%2C%201%2C%203%5D%5D)%0A%23--%0AA%0A%23--%0Adef%20crd(A)%3A%0A%20%20%20%20%22%22%22%0A%20%20%20%20Computes%20the%20CR%20decomposition%20of%20the%20matrix%20A.%0A%20%20%20%20%22%22%22%0A%20%20%20%20rrefA%2C%20licols%20%3D%20A.rref()%20%20%23%20compute%20RREF(A)%0A%20%20%20%20C%20%3D%20A%5B%3A%2C%20licols%5D%20%20%20%20%20%20%20%20%20%20%23%20linearly%20indep.%20cols%20of%20A%0A%20%20%20%20r%20%3D%20len(licols)%20%20%20%20%20%20%20%20%20%20%20%23%20%3D%20rank(A)%0A%20%20%20%20R%20%3D%20rrefA%5B0%3Ar%2C%20%3A%5D%20%20%20%20%20%20%20%20%20%23%20non-zero%20rows%20in%20RREF(A)%0A%20%20%20%20return%20C%2C%20R%0A%23--%0AC%2C%20R%20%3D%20crd(A)%0A%23--%0AC%0A%23--%0AR%0A%23--%0AC*R%0A%23--%0A https://live.sympy.org/?evaluate=A%20%3D%20Matrix(%5B%0A%20%...
abecode 6y ago
Two things really made linear algebra click for me: representing camera projections in a computer vision class and spectral graph theory, which basically connects graphs with linear algebra. In both of these, it seems like linear algebra was taken from the electrical engineering domain into computer science, which better fit my perspective.
xchip 6y ago
Every science degree studies algebra in their first course, it should be regarded as something pretty basic. How comes people are still talking about this?
DreamScatter 6y ago
An interesting alternative to linear algebra is geometric algebra. I recommend googling around a bit for geometric algebra and trying out my implementation https://github.com/chakravala/Grassmann.jl https://github.com/chakravala/Grassmann.jl
cashsterling 6y ago
I like all Strang's books... at least the ones I have. I don't have his Learning from Data book, yet... however. I also really like the applied linear algebra book by Boyd Vandenberghe: https://web.stanford.edu/~boyd/vmls/ https://web.stanford.edu/~boyd/vmls/ Free PDF is available on their website. There is Julia and Python code companions for the book and lecture slides from both Profs their websites. Also check out their other books, many of which have free PDF's available. I can also recommend Data-Driven science and engineering by Brunton and Kutz. http://databookuw.com/ http://databookuw.com/ There used to be a free preprint PDF of the book but I can't find it now. Book is totally worth picking up... MATLAB and Python code available. Steve Brunton's lectures on YouTube are pretty damn good and compliment the book well: https://www.youtube.com/channel/UCm5mt-A4w61lknZ9lCsZtBw/featured https://www.youtube.com/channel/UCm5mt-A4w61lknZ9lCsZtBw/fea... Another really cool book is Algorithms for Optimization by Mykel Kochenderfer and Tim Wheeler: https://mitpress.mit.edu/books/algorithms-optimization https://mitpress.mit.edu/books/algorithms-optimization. Julia code used in book.

835 points by organicfigs 6y ago

Gilbert Strang's linear algebra course blew my mind back in high school, and I still use insights from it every day. Strang has a particular lecturing style where he approaches every topic several times, often beginning many lectures before the main treatment. At first I thought it was a bit confusing, but later I realized it helped build fluency, just like a language class. I'm really thankful to MIT OCW for putting his lectures out for free -- in fact, I think I'll go donate to them now.

tomjakubowski 6y ago
+1. Since I took the OCW course, whenever I multiply a matrix and a vector or two matrices by hand, I hear his voice saying, "combinations of columns." Strang must have said those words hundreds of times in it. His lectures stick like nothing else.
raz32dust 6y ago
+1, I was very grateful to MIT OCW because when I learned Linear algebra, I could not have afforded it. Later when I got a job, I donated to OCW and I bought his book full price from his own site [1] just as a tribute to the guy. [1] https://math.mit.edu/~gs/linearalgebra/ https://math.mit.edu/~gs/linearalgebra/
- qorrect 6y ago
  Hey me too! ( All of it )
willbw 6y ago
Of course it is up to you what you do with your money but, they have a $17.5B endowment so there may be more needy causes if you were so inclined.
- arcturus17 6y ago
  OCW opens up top-notch education to anyone and everyone, regardless of social or economic background. I wouldn’t take it for granted even with MIT’s eye-watering endowment, and I doubt donations to it are going to be paying cafeteria lunches for the students. I hope you’re donating and actively contributing to many non-profit projects and that your comment comes from being tired of the world’s injustices rather than from callous impertinence, although I suspect it does not.
  - Judgmentality 6y ago
    I think his point is there are plenty of other noble causes that could use the money a lot more.
  - wegs 6y ago
    Most money which comes into MIT passes through overhead. That means if a foundation donates to MIT, a bit over 1/3 of that money might ends up with whatever they donated to. A bit under 2/3 might go into the general budget (overheads vary by funding source, but the numbers above are from one specific project). On paper, overhead is used for costs of running the place. In practice, it's used for things like upscale faculty clubs, million-dollar executive salaries, $200 million buildings, etc. MIT has among the highest overheads in the academy. Ironically, MIT claims its ocean yacht makes money rather than losing money (which could very well be true). If you're okay with the majority of your money going to graft, donate to MIT. With a project like OCW, which has such a huge cost:benefit ratio, accepting the graft with the donation may be a rational decision, if you subscribe to a system of ethics like utilitarianism. Personally, I almost never donate to a charity where the highest-earner makes more than I do. I think if everyone did that, MIT might lose some of the graft and corruption which has built up there over the years.
    - mattkrause 6y ago
      MIT’s current overhead rate is 50.5%, but that’s pretty standard. Here’s a scatterplot showing lots of research institutions’ rates. When this was published, MIT’s rate was slightly higher (54%). https://www.nature.com/news/indirect-costs-keeping-the-lights-on-1.16376 https://www.nature.com/news/indirect-costs-keeping-the-light... showing actual and calculated rates. This also applies to federal research grants and is meant to cover costs associated with actually hosting the research (rent, utilities, support staff). Foundations can (and often do) negotiate lower rates. I’m not sure how donations are handled, but I don’t think the same F&A rates apply.
scared2 6y ago
In High school?
- gowld 6y ago
  Yes. Linear Algebra is an extension of what is commonly called Algebra 2 or Precalculus in high school. LA and Calculus can be studied independently in any order and then fruitfully combined later.
bencw 6y ago
This is very well put. Knowledge has a hierarchical (or perhaps even cyclical!) structure and it's unrealistic to think that a body of knowledge can be taught or learned sequentially.

tomerbd 6y ago

It was a good course I watched it online but I didn't understand much.

potta_coffee 6y ago

Picked up Strang's linear algebra book recently and I'm enjoying it. I've been consistently impressed with the content of MIT books.

jp0d 6y ago

I've been doing the Statistics Micromasters from MIT. It's rigorous and very deep. I look forward to doing this.

frequentnapper 6y ago

Back in uni (2005), we used Dr. Strang's text for linear algebra. When reading the text, I felt like some down-to-earth professor was trying to explain these difficult topics as simply as possible. I remember discovering mit.edu back then and finding precious video lectures that went along with the book after the course. One of the very few times I was so genuinely happy and excited to watch math lectures online :p

jbd28 6y ago
We used his book then too, at Drexel in Philadelphia. Our prof at the time invited Dr Strang to guest lecture one time and I remember it being so clear and obvious as he talked that I thought “wow this is why an MIT education is so revered”. I waited after the lecture to personally thank him and have him autograph the textbook; very glad I did in retrospect.

roenxi 6y ago

It is interesting to compare this with 3Blue1Brown's linear algebra introduction on YouTube. He seems to have been the only mathematician who has actually mastered the medium; linear algebra lends itself very well to animations. The mathematicians don't understand how badly they need to animate some of these concepts.

ekianjo 6y ago
Agree, world class education in maths there. He understands the importance of examples and visualization and it changes everything.
gfxgirl 6y ago
3Blue1Brown's linear algebra animations were fun to watch but they did almost nothing for me except the basic fact that the "linear" part means lines. The rest was effectively preaching to the choir so those that already know linear algebra nodded their heads and idiots like me were still flummoxed
- domnomnom 6y ago
  Such is life
- mjburgess 6y ago
  That's interesting. I do think 3B1B's goal is probably to build better intuitions in people who already know it.
  - mesaframe 6y ago
    Yes, those lectures are good enough for intuitions only. For practice one has to read books. I just hope the viewer knows that. Which I have seen is absent among some people.
- john4532452 6y ago
  Yes. 3Blue1Brown has himself said multiple times no videos is substitute for text books.

tomahunt 6y ago

A 2018 paper by Strang about this approach: https://www.tandfonline.com/doi/abs/10.1080/00029890.2018.1408378 https://www.tandfonline.com/doi/abs/10.1080/00029890.2018.14...

sqlmonkey 6y ago

I started by only reading his book thinking it was enough. I was very wrong, these videos marry themselves beautifully with the content of the book which suddenly became incredibly clear once I started watching the videos. Strang teaching style can also seem odd at first, but don't give up, he is an amazing teacher who makes every concept simple to understand. This course is a true gift.

lcuff 6y ago

As someone who understands nothing of linear algebra, I have to say this "introduction" was gibberish. He may be a fantastic teacher, and perhaps it's a bit much to expect a 4 minute video to teach me anything, but it reminds me of talks from business people where what they're saying is obvious if you already understand it, and completely obscure otherwise.

wodenokoto 6y ago
This is not a course, or a primer or an introduction on Linea Algebra. From the course description: > These six brief videos, recorded in 2020, contain ideas and suggestions from Professor Strang about the recommended order of topics in teaching and learning linear algebra.

glram 6y ago

Professor Strang’s lectures helped me greatly during my linear algebra class. I thoroughly appreciated his clear, coherent lecture style. On another note, he is such a nice guy. 10/10.

synaesthesisx 6y ago

Linear algebra was one of those classes I was forced to take in undergrad as an engineering requirement - only to end up appreciating it immensely later on when I realized how many real world problems can be converted to matrix operations.

crdrost 6y ago

So I like this outline. It is very MIT-ish where there is a sense of teaching someone to solve practical engineering problems with matrices. But, I do foresee some difficulties. One thing that I find really difficult, for example, is that I take undergrads who have had linear algebra and ask "what is the determinant?" and seldom get back the "best" conceptual answer, "the determinant is the product of the eigenvalues." Like, this is math, the best answer should not be the only one, but it should be ideally the most popular. We would consider it a failure in my mind if the most popular explanation of the fundamental theorem of calculus was not some variation of "integrals undo derivatives and vice versa". I don't see this approach solving that. Furthermore there is a lot of focus from day one on this CR decomposition which serves to say that a linear transform from R^m to R^n might map to a subspace of R^n with smaller dimension r < min(m,n) and while in some sense this is true it is itself quite "unphysical"—if a matrix contains noisy entries then it will generally only be degenerate in this way with probability zero. (You need perfect noise cancelation to get degeneracies, which amounts to a sort of neglected underlying conserved quantity which is pushing back on you and demanding to be conserved.) In that sense the CR decomposition is kind of pointless and is just working around some "perfect little counterexamples". So it seems weird to see someone say "hold this up as the most important thing!!"

mansoor_ 6y ago
Subjective, I find the geometric interpretation of the determinant to be the "best".
heinrichhartman 6y ago
> seldom get back the "best" conceptual answer, "the determinant is the product of the eigenvalues." I found that the "best conceptual" answer depends a lot on taste, and what concepts you are familiar with. In this case: - Calculating exact eigenvalues of matrices larger than 4x4 is impractical, since it requires you to solve a polynomial of degree >4. - The EV exist only in algebraically closed fields (complex numbers), while the determinant itself lives in the base field (rationals, reals). How about: - [Geometric Determinant] The determinant is the volume of the polytope (parallel-epiped) spanned by the column vectors of the matrix. - [Coordinate Free Determinant] The determinant is the map induced between the highest exterior powers of the source and target vector spaces (https://en.wikipedia.org/wiki/Exterior_algebra https://en.wikipedia.org/wiki/Exterior_algebra) - I think there is also a representation theoretic version, that characterizes the determinant as invariant under the Symmetric group acting by permutation on the columns/rows of the matrix.

auggierose 6y ago

Have not watched the videos yet, but that seems to me more like an 1820 vision of linear algebra :-) If you look at the order of topics in his book "An Introduction to Linear Algebra", you will find the topic "Linear Transformation" way back in chapter 8! Even after the chapters eigenvalue decomposition and singular value decomposition. But understanding that a matrix is just the representation of a linear transformation in a particular basis is probably the most important and first thing you should learn about matrices ...

dandanua 6y ago
I don't think it was this way in 1820, but I agree with your point. Even though I also use Linear Algebra mostly computationally today, the origin of it is in the geometry and I think this connection should come first. Also, "number crunching" is a boring way to learn things. Though, "matrix way" can be good for engineers.
jacobolus 6y ago
Gaussian Elimination is indeed from the 1820s. All the rest is more recent than that. The idea of matrix decomposition per se comes from the 1850s. The earliest work on something like the SVD is from the 1870s. You are onto something though. Strang is coming from a direction of numerical computations and algorithms for solving real-world problems. Pure mathematics departments for at least the past maybe 80 years often look down on numerical analysis, statistics, engineering, and natural science, and adopt a position that education of students should be optimized in the direction of helping them prove the maximally general results using the most abstract and technical machinery, with an unfortunate emphasis on symbol twiddling vs. examining concrete examples. By contrast, in the 19th century there was much more of a unified vision and more respect for computations and real-world problems. Gauss himself was employed throughout his career as an astronomer / geodesist, rather than as a mathematician, and arguably his most important work was inventing the method of least squares, which he used for interpreting astronomical observations. With the rise of electronic computers, it is possible that the dominant 2050 vision of linear algebra and the dominant 1900 vision of linear algebra will be closer to each-other than either one is to a 1950 vision from a graduate course in a pure math department.

srean 6y ago

I am familiar with the material of linear algebra but haven't read his books. Could someone who has absorbed linear algebra from different sources and familiar with Strang's books comment on what's good and bad and unique about them. In my time I had picked LA from Ben Noble, Halmos and Axler and the computation side of things from Golub & van Loan.

mesaframe 6y ago
I read his book and I would say his book work in complement with his lectures. It is not good enough on it's own.
T-zex 6y ago
I'm still a beginner and I think Strang's Linear algebra books are more like a supplement material to his lectures. If you need to build a solid theoretical foundation of linear algebra you'd need to consider other resources too. Having said that, he is explaining many things really well and is helping a lot to build intuition. He is always cautious presenting things that are computationally inefficient and suggests the alternatives. Exercises are too hard for me personally. I'd prefer a more laborious set of exercises helping to cement the material, (as in calculus or usual algebra) and then have one or two problem solving puzzles at the end.
- srean 6y ago
  So the focus is on different recipes to cook a matrix with ? Different operations one can do on a matrix ? I hope its not just that, that would be very limiting considering what linear algebra is about and capable of.
  - T-zex 6y ago
    No, his books are not recipes. The thing I'm struggling to communicate here is that he's got a more pragmatic style compared to other text books. The material he presents is complete and and he is doing great job making it approachable for non mathematicians. His books usually expand on the subjects he presents at his online lectures. I see them as advanced lecture notes.

praptak 6y ago

I'm currently trying to grok the finite element method. Gilbert Strang's explanation of the transition from the Galerkin method to FEM did more for me in terms of connecting the dots than anything else I could find on the web. And it wasn't even a lecture, just a kind of an interview. I think it's this one: youtube.com/watch?v=WwgrAH-IMOk

kragen 6y ago
https://www.youtube.com/watch?v=WwgrAH-IMOk https://www.youtube.com/watch?v=WwgrAH-IMOk I feel like I don't really understand his explanation, because it's kind of vague. But I think that might be because you've seen the equations dozens of times, and I haven't seen them at all, so you were prepared to understand the video.
- praptak 6y ago
  This makes sense. As said, it was about connecting the dots for me. Also, I don't even claim I fully understand FEM (or even Galerkin), it's just my hobby project.
  - kragen 6y ago
    That sounds interesting! What are you doing with it?
    - praptak 6y ago
      I just want to understand the magic behind static stress analysis. More generally I'm interested in emulating physics behind the rigid body model. Maybe I will create a game prototype based on the mechanics but this is just a vague idea.

penguin_booze 6y ago

I recently came across this rather in-depth series on linear algebra: https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS14F4NxAZudSmv https://www.youtube.com/playlist?list=PLlXfTHzgMRUKXD88IdzS1.... FWIW, I myself have only gone half-way thorugh part 1.

deleted 6y ago

[deleted]

enitihas 6y ago

Another good Linear Algebra book is "Linear Algebra Done Right", which Springer is giving for free right now. Link: https://link.springer.com/book/10.1007/978-3-319-11080-6 https://link.springer.com/book/10.1007/978-3-319-11080-6

mseri 6y ago
Came here to say that. It is a wonderful book, and I think it provides a more "modern" approach than the one presented in the videos.
clarry 6y ago
There's also a free book "Linear Algebra Done Wrong," which might be worth checking out. https://www.math.brown.edu/~treil/papers/LADW/LADW.html https://www.math.brown.edu/~treil/papers/LADW/LADW.html
threatofrain 6y ago
IMO Axler's book should be read either during or after you take an introductory course on Linear Algebra. > You are probably about to begin your second exposure to linear algebra. Unlike your first brush with the subject, which probably emphasized Euclidean spaces and matrices, this encounter will focus on abstract vector spaces and linear maps.
- vlasev 6y ago
  I whole-heartedly agree. Axler's book is a great stepping stone to more abstract linear algebra.
bencw 6y ago
This book is great and very much complementary to Strang's approach in that it leans more towards "abstract" linear algebra.
AlanYx 6y ago
Thanks for this -- do you know if there's a consolidated list anywhere of other books Springer is making available for free right now?

brmgb 6y ago

After watching this and having read the comments, I am quite puzzled by the approach American seem to take to linear algebra. Are matrices viewed as the core of the subject in the USA ? My country curriculum introduces linear algebra through group theory and vector spaces. Matrices come later.

chubot 6y ago
Yeah I would call this the engineering approach (matrices) vs the mathematical approach (algebra). I took like 3-4 courses in the US involving the engineering approach, starting in high school and continuing through the college as a CS major. That was all that was required. But I also like algebra, so I happened to take a 400-level course that only math majors take my senior of college. And then I got the group theory / vector space view on it. I don't think 95% of CS majors got that. I don't think one is better than the other, but they should have tried to balance it out more. It helps to understand both viewpoints. (If you haven't seen the latter, then picture a 300-page text on linear algebra that doesn't mention matrices at all. It's all linear transformations and spaces.) What country were you taught in? Wild guess: France?
ojnabieoot 6y ago
I would not describe his approach as “the US approach” but it is a pretty standard approach to introducing linear algebra to engineers, which is the theme of the course. I was also taught linear algebra this way, by an applied mathematician with a background in chemical engineering: - start by solving Ax=b with row reduction - develop theorems about linear independence and spanning sets of vectors based on these exercises - introduce the determinant from the perspective of linear systems (rather than eg geometry or group theory) - eigenvectors and eigenvalues Later I switched from physics to math and TAed a more “algebraic” approach involving groups/rings/fields. But the matrix-first approach was more helpful for both my physics coursework and later courses in numerical linear algebra.
tadhgds 6y ago
I can't say whether or not it is the standard approach but I do know that it is very common in many countries to teach a linear algebra course that is heavy on matrix operations, that you can come away believing that linear algebra is somehow _about_ matrices and their operations. I know many in my university class seemed to believe that. A book I enjoyed is Axler's Linear Algebra Done Right[0], in which, if I remember correctly, doesn't contain a single matrix. [0]https://zhangyk8.github.io/teaching/file_spring2018/linear_algebra_done_right.pdf https://zhangyk8.github.io/teaching/file_spring2018/linear_a...

inshadows 6y ago

Gilbert Strang taught me how to sanely multiply matrices. His Introduction to Linear Algebra is very approachable. It's wildly different experience compared to linear algebra courses I had on university, it actually makes sense and is fun!

elAhmo 6y ago

What would you recommend as a good resource for learning about Linear Algebra in 2020? I am aware of his course on OCW, but wondering is there something more interactive and/or newer than those lectures that has similar quality.

skywal_l 6y ago
Frankly, couple with this book, it does hardly get better. You still have 3blue1brown[1] series of video, but it just brush off the surface. [1] https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab https://www.youtube.com/watch?v=fNk_zzaMoSs&list=PLZHQObOWTQ...

chadcmulligan 6y ago

Wow, Prof Strang is 85 and still teaching! Thats very impressive and inspiring.

pengaru 6y ago

It's ridiculous how much random college-level linear algebra textbook material I stared at before things clicked in the course of just jumping in and exploring 3D graphics and writing my own 3D vector, matrix multiplication and 3D transform headers and using them in making some games in plain C. At some point it's like "Wait, is linear algebra really just about heaps of multiplication and addition? Like every dimension gets multiplied by values for every dimension, and values 0 and 1 are way more interesting than I previously appreciated. That funny identity matrix with the diagonal 1s in a sea of 0s, that's just an orthonormal basis where each corresponding dimension's axis is getting 100% of the multiplication like a noop. This is ridiculously simple yet unlocks an entire new world of understanding, why the hell couldn't my textbooks explain it in these terms on page 1? FML" I'm still a noob when it comes to linear algebra and 3D stuff, but it feels like all the textbooks in the world couldn't have taught me what some hands-on 3D graphics programming impressed upon me rather quickly. Maybe my understanding is all wrong, feel free to correct me, as my understanding on this subject is entirely self-taught.

Phlogistique 6y ago
Is it the same issue as the infamous "monads tutorials" problem, where the understanding takes a lot of time to infuse but looks obvious in retrospect when it finally clicks?
- DagAgren 6y ago
  I am convinced that monads induce a very specific kind of brain damage that makes a person incapable of ever explaining monads.
  - dleslie 6y ago
    Start with a container. M a Then add a way to put things in the container. a -> M a Then add a way to use the thing in the container. M a -> (a -> M b) -> M b
    - 6y ago

vertak 6y ago

Can anyone grow why in the first 5 minutes of part 1 he shows a 3 by 3 matrix multiples by a 1 by 3 vector yet verbally he pulls out of no where this idea that if you have _two_ 1 by 3 vectors that pass through the origin then their linear combinations can be represented by a plane? The jump from the 3D to the 2D has me lost and I gave up

swiley 6y ago
I don’t know which video you’re talking about but two non parallel vectors are enough to represent a plane, the normal will be their cross product. Also, if you have 3 dimensional vectors you were always in 3D.
neutronicus 6y ago
If you multiply the 1x3 vector by all scalars from -infinity to infinity you get all the points on a line. If you do the same for another 1x3 vector, and it is not parallel to the first, you get all the points on a different line. These two lines define a plane (and the cross product of the two vectors defines its normal vector)
jcmoyer 6y ago
The same concept applies in 2D, which might help you build the intuition to understand it in 3D. If you have a vector v=(1,0) that points to the right, you can scale this vector infinitely in that direction by multiplying it by a positive scalar. 5v = (5,0) 62.1v = (62.1,0) Similarly, you can scale that vector infinitely in the opposite direction (i.e. left) by multiplying it by a negative scalar: -987v = (-987,0) If we call this scalar c, the expression cv allows us to represent any point along the X axis simply by varying c, meaning that cv defines a line along that axis. Similarly, we can do the same for a vector w=(0,1) along the Y axis, scaling it by d. Now we have a method for moving to any point on the XY plane simply by varying c and d in the linear combination: cv + dw, meaning that we've defined a plane using two vectors. Two caveats: - this won't work if v and w are parallel; for example, if v = -w (and neither are zero) then we can only move along a line instead of a plane - it also won't work if either of the vectors are zero, because no matter what you multiply by, a zero vector can only represent a single point

katzgrau 6y ago

I had very intelligent linear algebra professor in college but he was, in my opinion, a very poor communicator. I paid attention to lectures and stared at the text, but couldn't really understand the material. For the first part of a linear algebra course, students who don't mind blindly following mechanical processes for solving problems can do very well. Unfortunately I'm one of those people who tends to reject the process until I understand why it works. If it wasn't for Strang's thoughtful and sometimes even entertaining lectures via OCW, I probably would have failed the course. Instead, as the material became considerably more abstract and actually required understanding, I had my strongest exam scores. I didn't even pay attention in class. I finished with an A. Although my first exam was a 70/100, below the class average, the fact that I got an A overall suggests how poorly the rest of the class must have done on the latter material, where I felt my strongest thanks to the videos. So anyway, thank you Gilbert Strang.

ansible 6y ago
> I paid attention to lectures and stared at the text, but couldn't really understand the material. For the first part of a linear algebra course, students who don't mind blindly following mechanical processes for solving problems can do very well. I had a similar, though sort of opposite experience. In high school, I breezed through the material, and started teaching myself calculus during the summer to prepare for university. Other than being a lazy student, I had no problems taking the 2nd semester advanced calc 2 and 3 courses my freshman year. I totally get what's being taught. There weren't a ton of practical examples, but I can easily see (for example) what the purpose of integration is, and how and why you'd do it in two or more dimensions. I could work the equations, no problem. Everything is great. Along comes sophomore year, and still thinking I am hot stuff, I take advanced linear algebra and differential equations. More of the same, I thought. Well... we seemed to spend the entire semester just solving different kinds of equations. No explanations given as to what they are for, where they are used, or what the point of any of it was. I struggled, for the very first time. I either got a D or F for the mid-term exam, which was shocking to me. We had one chapter where we were doing something practical. This is where you have a water tank, and a hole in to bottom. Because the pressure lessens as the tank empties, the flow rate is not constant. However, you can solve this via diff equations, and I really grokked it. I finally saw the point for some of what we had been doing. But it was just that one chapter, we skipped any other practical aspects for what we were studying. I did end up pulling out a 'C' with that class, to my relief. Sure, most of the blame for my lousy performance must rest with me, because of my poor study habits. And a little blame can go to the TA, who wasn't a good communicator, so that hour every week was kind of useless. But I also blame the material and how it was presented.

anandrm 6y ago

Just curious .. what really are the usecases where of Linear Algebra is applied ? Any domain of software development ?

ktta 6y ago
Any fields that have anything to do with video, image, audio, games, machine learning. Just to have a taste of use cases: compression, filters(image filters for de-noising, HP & LP filters for audio), encoding/decoding, computer vision techniques, cryptography, neural nets, computer graphics (this is where most people learn how to use it in real computer programs)
justinmeiners 6y ago
- scale or rotate an image. - root finding algorithm with more than one variable. - graph problems like Google's PageRank - statistical analysis - 3d rendering (projecting a 3d scene onto a 2d image) - solving systems of equation (also see linear programming) Linear algebra is very basic and fundamental to physics and math.
vlasev 6y ago
It's applied pretty much everywhere. Most numerical problems have some linear algebra component to them. Physics uses it a lot too. A lot of non-linear problems have a linearization on which you can use linear algebra to obtain approximations. Ideas from linear algebra are used a lot in things like signal processing, quantum mechanics, etc.

irl_zebra 6y ago

I've been wanting to learn linear algebra. I had some exposure in college along with my calc classes, but never really understood it fundamentally. Like it was mentioned, I mostly did matrix transforms but didn't realize fundamentally grasp. I started doing LA on Khan academy, and checked out Linear Algebra Done Right. LADR was a little too much into the deep end for me. KA seemed to be good. One nice thing about KA is that when I didn't quite remember something (i.e. how exactly to multiply a matrix) I could just go to an earlier pre-LA lesson, pick it up, and then go back to LA where I left off. I'm a few lessons in. What do you all recommend for someone like me?

nafizh 6y ago
If you want to learn LA by coding, coding the matrix by philip klein is a really good book. He even has his Coursera lecture videos (not available on coursera anymore last I checked) up on his website.

hprotagonist 6y ago

my linear algebra professor began his first lecture with a 5 minute rant informally titled “how you could have gotten an A in differential equations last semester without ever having taken calculus” That certainly got our attention. I’ve always found linear algebra to be kind of ... almost soothing.

eximius 6y ago
That's a rant I'd like to hear.
- hprotagonist 6y ago
  "look guys, at end of class, exam is always long list of silly second order system of differential equations. Well, every time, we can make so-called "guess" that solution looks like e*rt. Why? We know that because professors will only give well-behaved systems on final exams because it's hard to grade the other kind. So we know characteristic polynomials look like so (because of course they do, you can just memorize this) ... so now we lift out the coefficients into nifty thing called _matrix_ and now follow these easy four steps to get roots, plug back in, and incidentally these are "eigenvalues", we'll talk about this later ... Bam. Done. A-, easy. No sweat."
  - eximius 6y ago
    Minus the matrix bits, that's basically how I slogged through DiffEq. Showing up to class was pointless because the prof would make up an equation to exploratively solve and inevitably it would be poorly behaved and the lecture would end with "... And and and for this kind of problem we have to use numerical methods".

balls187 6y ago

Admittedly, I never fully groked linear algebra. Some of the concepts made sense, especially solving for linear systems of equations. Recently, I decided to brush up on my math skills via Youtube videos, and came across this series: https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw It explains Linear Algebra concepts using 2D and 3D vector manipulation, and the animations help me visualize the underlying maths.

longtimegoogler 6y ago

IMO, if one is interested in a computational approach to Linear Algebra, Trefethens book, Numerical Linear Algebra, is the best. That book discusses the actual algorithms used for computation. It is a bit more advanced, but amazingly clear.

ivan_ah 6y ago

For anyone who is already familiar with the Prof. Strang's lectures from previous years, the main new thing in this five-lecture mini-series is he tries to condense the material even further—maximum intuition and power-ideas, instead of the full-length in-class lecture format with derivations. This makes the material difficult to understand for beginners, but makes a great second source in addition to or after a regular LA class. One of the interesting new ways of thinking in these lectures is the A = CR decomposition for any matrix A, where C is a matrix that contains a basis for the column space of A, while R contains the non-zero rows in RREF(A) — in other words a basis for the row space, see https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-linear-algebra-spring-2020/videos/MITRES_18_010S20_LA_Slides.pdf#page=7 https://ocw.mit.edu/resources/res-18-010-a-2020-vision-of-li... Example you can play with: https://live.sympy.org/?evaluate=C%20%3D%20Matrix(%5B%5B1%2C4%5D%2C%20%5B3%2C2%5D%2C%20%5B2%2C1%5D%5D)%0A%23--%0AC%0A%23--%0AR%20%3D%20Matrix(%5B%5B1%2C0%2C1%5D%2C%20%5B0%2C1%2C1%5D%5D)%0A%23--%0AR%0A%23--%0AC*R%0A%23--%0A(C*R).rref()%5B0%5D%0A%23--%0A https://live.sympy.org/?evaluate=C%20%3D%20Matrix(%5B%5B1%2C... Thinking of A as CR might be a little intense as first-contact with linear algebra, but I think it contains the "essence" of what is going on, and could potentially set the stage for when these concepts are explained (normally much later in a linear algebra course). Also, I think the "A=CR picture" is a nice justification for where RREF(A) comes about... otherwise students always complain that the first few chapters on Gauss-Jordan elimination is "mind-numbing arithmetic" (which is kind of true...) but maybe if we present the algorithm as "finding the CR-decomposition which will help you understand dozens of other concepts in the remainder of the course" it would motivate more people to learn about RREFs and the G-J algo.

ivan_ah 6y ago
Since y'all are code-literate, here the SymPy function for finding the CR-decomposition of any matrix A: def crd(A): """ Computes the CR decomposition of the matrix A. """ rrefA, licols = A.rref() # compute RREF(A) C = A[:, licols] # linearly indep. cols of A r = len(licols) # = rank(A) R = rrefA[0:r, :] # non-zero rows in RREF(A) return C, R Test to check it works: https://live.sympy.org/?evaluate=A%20%3D%20Matrix(%5B%0A%20%20%5B1%2C%204%2C%205%5D%2C%0A%20%20%5B3%2C%202%2C%205%5D%2C%0A%20%20%5B2%2C%201%2C%203%5D%5D)%0A%23--%0AA%0A%23--%0Adef%20crd(A)%3A%0A%20%20%20%20%22%22%22%0A%20%20%20%20Computes%20the%20CR%20decomposition%20of%20the%20matrix%20A.%0A%20%20%20%20%22%22%22%0A%20%20%20%20rrefA%2C%20licols%20%3D%20A.rref()%20%20%23%20compute%20RREF(A)%0A%20%20%20%20C%20%3D%20A%5B%3A%2C%20licols%5D%20%20%20%20%20%20%20%20%20%20%23%20linearly%20indep.%20cols%20of%20A%0A%20%20%20%20r%20%3D%20len(licols)%20%20%20%20%20%20%20%20%20%20%20%23%20%3D%20rank(A)%0A%20%20%20%20R%20%3D%20rrefA%5B0%3Ar%2C%20%3A%5D%20%20%20%20%20%20%20%20%20%23%20non-zero%20rows%20in%20RREF(A)%0A%20%20%20%20return%20C%2C%20R%0A%23--%0AC%2C%20R%20%3D%20crd(A)%0A%23--%0AC%0A%23--%0AR%0A%23--%0AC*R%0A%23--%0A https://live.sympy.org/?evaluate=A%20%3D%20Matrix(%5B%0A%20%...

abecode 6y ago

Two things really made linear algebra click for me: representing camera projections in a computer vision class and spectral graph theory, which basically connects graphs with linear algebra. In both of these, it seems like linear algebra was taken from the electrical engineering domain into computer science, which better fit my perspective.

xchip 6y ago

Every science degree studies algebra in their first course, it should be regarded as something pretty basic. How comes people are still talking about this?

DreamScatter 6y ago

An interesting alternative to linear algebra is geometric algebra. I recommend googling around a bit for geometric algebra and trying out my implementation https://github.com/chakravala/Grassmann.jl https://github.com/chakravala/Grassmann.jl

cashsterling 6y ago

I like all Strang's books... at least the ones I have. I don't have his Learning from Data book, yet... however. I also really like the applied linear algebra book by Boyd Vandenberghe: https://web.stanford.edu/~boyd/vmls/ https://web.stanford.edu/~boyd/vmls/ Free PDF is available on their website. There is Julia and Python code companions for the book and lecture slides from both Profs their websites. Also check out their other books, many of which have free PDF's available. I can also recommend Data-Driven science and engineering by Brunton and Kutz. http://databookuw.com/ http://databookuw.com/ There used to be a free preprint PDF of the book but I can't find it now. Book is totally worth picking up... MATLAB and Python code available. Steve Brunton's lectures on YouTube are pretty damn good and compliment the book well: https://www.youtube.com/channel/UCm5mt-A4w61lknZ9lCsZtBw/featured https://www.youtube.com/channel/UCm5mt-A4w61lknZ9lCsZtBw/fea... Another really cool book is Algorithms for Optimization by Mykel Kochenderfer and Tim Wheeler: https://mitpress.mit.edu/books/algorithms-optimization https://mitpress.mit.edu/books/algorithms-optimization. Julia code used in book.