8 ms·

I always see linear algebra on HN and many people comment on how they never understood the subject. This makes me ask: what exactly is it that people don't get

I always see linear algebra on HN and many people comment on how they never understood the subject. This makes me ask: what exactly is it that people don't get about linear algebra? What makes it appear to be a difficult subject? As someone who has used linear algebra almost every day in some form over the last decade, it's hard to get a perspective of what aspects are challenging to the beginner. And since I TA courses that involve linear algebra, it is good to know where the problems are.

wodenokoto 11y ago
What it is, I think was my major problem. Matrix multiplication, ranks etc. are some of the most random seeming operations you can tell a student to memories.
- dalke 11y ago
  My view is that the people who get to matrix algebra didn't have problem understanding the random seeming operation of previous math courses. Consider the existence of the "FOIL" mnemonic for multiplying two binomials (https://en.wikipedia.org/wiki/FOIL_method https://en.wikipedia.org/wiki/FOIL_method ), or SOH-CAH-TOA, where it was easier for me to just remember the definitions than the acronym. I think those are hints that some people consider those to be 'random seeming'. When I took algebra, I remember being tripped by all the new vocabulary. Why do mathematicians say something is "Abelian" instead of 'commutative'? Why is it important to have terms like "group", "ring", and "vector space"? Sure, I learned the definitions, but at that level they were 'random seeming'.
ashishb4u 11y ago
I think it's mostly because mathematics is taught as is with little or no attention to practical applications of the theory. For instance, till date, I could not see how eigen values/vectors are useful.
- cousin_it 11y ago
  One nice application is that the eigenvectors of a stochastic matrix correspond to stationary states of the Markov chain. That leads naturally to a conversation about PageRank, which is super useful because it made billions of dollars for some people. Then you could talk about the use of eigenvectors for principal component analysis in data science, which is making a lot of money for other people right now. If these examples are not useful, then I don't know what is.
  - ashishb4u 11y ago
    Thanks, the application to PageRank looks interesting. I shall resurrect my knowledge on eigenvectors and eigenvalues and make some sense out of it.
- cabinpark 11y ago
  Interesting. Eigenvalues and eigenvectors are one of the most fundamental applications of linear algebra. In physics, quantum mechanics is a giant eigenvalue-eigenvector problem. One application I really like is in machine learning: the eigenface algorithm.
  - neuronic 11y ago
    That might all be the case. I would simply like a thoughtful introduction to the topic about where this abstraction comes from. What is the idea behind it and what was the motivation for its invention? After I understand the plain English concept, then give me the math notation and proof and applications. The way it works today is often to simply leave out the first part and I believe this is why many people find it hard to develop intuition and a real understanding of the concept. For man it's just there, you memorize it, you apply it, you take the result and simply have faith that it is what you need because some book/ prof said to solve problem X use eigenvalues.
    - zornthewise 11y ago
      The simple idea is that while linear maps(matrices) are generally a combination of rotations, scaling and other stuff in higher dimensions, sometimes they will act on a vector simply by scaling it. Since scaling is a really easy operation to understand, the space generated by these vectors will be really easy to understand. Note that different vectors can be scaled by different amounts. Now it often turns out that the space generated like this is actually the whole of the space under consideration and this really simplifies the linear map we started with. Hope that helps.
dagw 11y ago
I found that the level of abstraction that linear algebra is taught at is really weird. On the one hand it was never really taught in connection with algebra in general, or put into a wider mathematical context. On the other hand we were never really shown any real world used of linear algebra or how the concepts maps onto practical applications. It wasn't before years later when A) I'd taken more abstract algebra courses that introduced concepts like tensors and fields and B) I'd taken more practical computational courses where I had to do things like use linear algebra in 3D graphics and use eigenvectors to do dimensionality reduction and PCA, that I really understood the subject and its place in the world. Just saying here's a random object (that we'll call a "matrix"), here's some random steps (that we'll call "taking a determinant"), now memorize how to apply the steps to the object and see if the result is zero, didn't lead to much deeper understanding.
- neuronic 11y ago
  I think what courses often fail at is to specifically mention that all of these things are abstractions invented by people to deal with a specific issue and then WHY IT HELPS solving this issue. Whenever I studied Linear Algebra it was simply "here's a matrix, here is some algorithm, use it to get some number seemingly coming out of nowhere and just believe us that this number is what you really need." Note that I am not immediately interested in mathematical proof of why this method is correct. I want a plain English explanation of what is going on but instead you usually just get a bunch of notation to digest.
dalke 11y ago
I see that chapter 10, on eigenvalues/eigenvectors, is "Coming soon!". Eigenvectors never made intuitive sense to me. As with the various decompositions (Cholesky, LU, etc), I could apply the math as algorithms to follow, but never got to the point where felt I could apply them to new problems. Then again, in practice, I've only needed eigenvectors once since college, and it was more a rote implementation described in a paper. (In other words, don't feel like you should educate me on them here.) Since you are a TA, don't you get some idea of where your students struggle with linear algebra?
- cabinpark 11y ago
  Interesting. I used linear algebra with quantum mechanics where eigenvectors represent quantum state of the Hamiltonian which is how I initially understood them. And as for TAing, have you ever TA'd? You definitely get a feeling but I can count numerous times when I've stood in front of the tutorial class and asked if there are any questions only to get no response back. I think it's a symptom of first years. I've TA'd calculus as well and I get similar responses. It's very frustrating sometimes.
  - dalke 11y ago
    I have TA'ed. Asking for questions in public doesn't help much. Students rarely like to admit they don't know something. I was thinking more of when you look at their assignments. There are often multiple ways to approach a problem, and the route chosen can reveal something about one's level of comfort. It doesn't help that university TAs get almost no training in how to be a TA. At least, I didn't.
    - cabinpark 11y ago
      Yeah I know. It's hard to squeeze information out of them. I actively tell them to please, please come to me if they don't understand something and I have office hours for a reason. Yet no one takes advantage. At a certain point, you can only do so much since I'm a grad student and not a lecturer and my time is finite. The problem I've found with assignments though is that people copy and cheat. Many times someone will do very well on assignments and then do absolutely terrible on midterms and finals. It's very frustrating. I remember one course where everyone did nearly perfect on the assignments and yet the final and midterm followed the standard bell curve.
yardie 11y ago
I took LA in university and I really wish they had offered it in high school or even middle school. It was a requirement for my major but not a requirement for any of my other math courses so I treated it like an elective. It was only after I took the course and started applying it that I realized how applicable it was to almost every math problem. Instead of calculus for business majors or art majors they really should be using LA instead. At the moment I'm trying to teach my son LA but I can't find any books for 8-10yo
gopowerranger 11y ago
In almost every case, confusion arises when the "how" is taught with no explanation of the "why". Why does that formula work? Why did you use that formula? Whole worlds were opened up to me when I learned that the fundamentals of trigonometry, like the simple computation of sin of an angle, are all related at its base to a square but no math teacher ever made that connection for me.
dagw 11y ago
The other aspect that I found with LA compared to almost any other math subject I took was that it was almost entirely mechanical with little to no connection between my understanding of the subject and my grade. The exam questions where mostly along the lines of "find the eigenvector" and "find the determinant" rather than "what's an eigenvector" and "what's a determinant", so I got good grades despite not having any idea how to answer the latter questions, or really having any idea what I was doing. Understanding was neither necessary nor encouraged.
- Ocerge 11y ago
  This, exactly. I could find an orthonormal basis or an eigenvector, but I never quite wrapped my head around their importance, especially visually.
noobie 11y ago
What's challenging to me is my inability to develop intuition about LA simply because most of the time I have no idea what (the purpose) I am doing. How does one build intuition about LA?
cokernel 11y ago
At my (American, liberal arts, flyover country) university, linear algebra was my first real mathematics course. (The prior classes were calculus and a "proof" class, AKA intro to intro to number theory.) I'd been introduced to the determinant, Gauss--Jordan elimination, and so on, in high school, but all in a very mechanical way. I think this transition between mechanically generating solutions to problems using pattern matching and some basic algorithms on the one hand, and the more mathematically mature approach of exploring problem spaces using pattern matching, some basic algorithms, and intuition can be difficult for many people. It's not necessarily the material -- you can get used to almost anything, and even convince yourself it's easy or obvious with enough familiarity -- but the lack of intuition. When you're first learning linear algebra, its fundamental unity is not obvious, especially if the instructor does not take pains to point it out. (And even if the instructor does take pains to point it out -- well, it's hard to understand why the instructor is saying we could do this computation this way or that way.) So in the absence of existing intuition or any perception of unity, linear algebra becomes another target for pattern matching and basic algorithms. As it happens, I've never, ever felt like I didn't "get" linear algebra. However, I almost always feel like I "get" it now and all my prior conceptions of it were a confused muddle.
ThrustVectoring 11y ago
It's a global vs sequential learner thing. Linear Algebra is usually taught as a giant pile of tricks, techniques, and terminology. This is tractable for sequential learners like myself - I really enjoy picking up new tools in a narrow context just for it's own sake. Highly global learners basically can't learn anything this way. So, in short, my advice is "add more context."

wodenokoto 11y ago

What it is, I think was my major problem. Matrix multiplication, ranks etc. are some of the most random seeming operations you can tell a student to memories.

dalke 11y ago
My view is that the people who get to matrix algebra didn't have problem understanding the random seeming operation of previous math courses. Consider the existence of the "FOIL" mnemonic for multiplying two binomials (https://en.wikipedia.org/wiki/FOIL_method https://en.wikipedia.org/wiki/FOIL_method ), or SOH-CAH-TOA, where it was easier for me to just remember the definitions than the acronym. I think those are hints that some people consider those to be 'random seeming'. When I took algebra, I remember being tripped by all the new vocabulary. Why do mathematicians say something is "Abelian" instead of 'commutative'? Why is it important to have terms like "group", "ring", and "vector space"? Sure, I learned the definitions, but at that level they were 'random seeming'.

ashishb4u 11y ago

I think it's mostly because mathematics is taught as is with little or no attention to practical applications of the theory. For instance, till date, I could not see how eigen values/vectors are useful.

cousin_it 11y ago
One nice application is that the eigenvectors of a stochastic matrix correspond to stationary states of the Markov chain. That leads naturally to a conversation about PageRank, which is super useful because it made billions of dollars for some people. Then you could talk about the use of eigenvectors for principal component analysis in data science, which is making a lot of money for other people right now. If these examples are not useful, then I don't know what is.
- ashishb4u 11y ago
  Thanks, the application to PageRank looks interesting. I shall resurrect my knowledge on eigenvectors and eigenvalues and make some sense out of it.
cabinpark 11y ago
Interesting. Eigenvalues and eigenvectors are one of the most fundamental applications of linear algebra. In physics, quantum mechanics is a giant eigenvalue-eigenvector problem. One application I really like is in machine learning: the eigenface algorithm.
- neuronic 11y ago
  That might all be the case. I would simply like a thoughtful introduction to the topic about where this abstraction comes from. What is the idea behind it and what was the motivation for its invention? After I understand the plain English concept, then give me the math notation and proof and applications. The way it works today is often to simply leave out the first part and I believe this is why many people find it hard to develop intuition and a real understanding of the concept. For man it's just there, you memorize it, you apply it, you take the result and simply have faith that it is what you need because some book/ prof said to solve problem X use eigenvalues.
  - zornthewise 11y ago
    The simple idea is that while linear maps(matrices) are generally a combination of rotations, scaling and other stuff in higher dimensions, sometimes they will act on a vector simply by scaling it. Since scaling is a really easy operation to understand, the space generated by these vectors will be really easy to understand. Note that different vectors can be scaled by different amounts. Now it often turns out that the space generated like this is actually the whole of the space under consideration and this really simplifies the linear map we started with. Hope that helps.

dagw 11y ago

I found that the level of abstraction that linear algebra is taught at is really weird. On the one hand it was never really taught in connection with algebra in general, or put into a wider mathematical context. On the other hand we were never really shown any real world used of linear algebra or how the concepts maps onto practical applications. It wasn't before years later when A) I'd taken more abstract algebra courses that introduced concepts like tensors and fields and B) I'd taken more practical computational courses where I had to do things like use linear algebra in 3D graphics and use eigenvectors to do dimensionality reduction and PCA, that I really understood the subject and its place in the world. Just saying here's a random object (that we'll call a "matrix"), here's some random steps (that we'll call "taking a determinant"), now memorize how to apply the steps to the object and see if the result is zero, didn't lead to much deeper understanding.

neuronic 11y ago
I think what courses often fail at is to specifically mention that all of these things are abstractions invented by people to deal with a specific issue and then WHY IT HELPS solving this issue. Whenever I studied Linear Algebra it was simply "here's a matrix, here is some algorithm, use it to get some number seemingly coming out of nowhere and just believe us that this number is what you really need." Note that I am not immediately interested in mathematical proof of why this method is correct. I want a plain English explanation of what is going on but instead you usually just get a bunch of notation to digest.

dalke 11y ago

I see that chapter 10, on eigenvalues/eigenvectors, is "Coming soon!". Eigenvectors never made intuitive sense to me. As with the various decompositions (Cholesky, LU, etc), I could apply the math as algorithms to follow, but never got to the point where felt I could apply them to new problems. Then again, in practice, I've only needed eigenvectors once since college, and it was more a rote implementation described in a paper. (In other words, don't feel like you should educate me on them here.) Since you are a TA, don't you get some idea of where your students struggle with linear algebra?

cabinpark 11y ago
Interesting. I used linear algebra with quantum mechanics where eigenvectors represent quantum state of the Hamiltonian which is how I initially understood them. And as for TAing, have you ever TA'd? You definitely get a feeling but I can count numerous times when I've stood in front of the tutorial class and asked if there are any questions only to get no response back. I think it's a symptom of first years. I've TA'd calculus as well and I get similar responses. It's very frustrating sometimes.
- dalke 11y ago
  I have TA'ed. Asking for questions in public doesn't help much. Students rarely like to admit they don't know something. I was thinking more of when you look at their assignments. There are often multiple ways to approach a problem, and the route chosen can reveal something about one's level of comfort. It doesn't help that university TAs get almost no training in how to be a TA. At least, I didn't.
  - cabinpark 11y ago
    Yeah I know. It's hard to squeeze information out of them. I actively tell them to please, please come to me if they don't understand something and I have office hours for a reason. Yet no one takes advantage. At a certain point, you can only do so much since I'm a grad student and not a lecturer and my time is finite. The problem I've found with assignments though is that people copy and cheat. Many times someone will do very well on assignments and then do absolutely terrible on midterms and finals. It's very frustrating. I remember one course where everyone did nearly perfect on the assignments and yet the final and midterm followed the standard bell curve.

yardie 11y ago

I took LA in university and I really wish they had offered it in high school or even middle school. It was a requirement for my major but not a requirement for any of my other math courses so I treated it like an elective. It was only after I took the course and started applying it that I realized how applicable it was to almost every math problem. Instead of calculus for business majors or art majors they really should be using LA instead. At the moment I'm trying to teach my son LA but I can't find any books for 8-10yo

gopowerranger 11y ago

In almost every case, confusion arises when the "how" is taught with no explanation of the "why". Why does that formula work? Why did you use that formula? Whole worlds were opened up to me when I learned that the fundamentals of trigonometry, like the simple computation of sin of an angle, are all related at its base to a square but no math teacher ever made that connection for me.

dagw 11y ago

The other aspect that I found with LA compared to almost any other math subject I took was that it was almost entirely mechanical with little to no connection between my understanding of the subject and my grade. The exam questions where mostly along the lines of "find the eigenvector" and "find the determinant" rather than "what's an eigenvector" and "what's a determinant", so I got good grades despite not having any idea how to answer the latter questions, or really having any idea what I was doing. Understanding was neither necessary nor encouraged.

Ocerge 11y ago
This, exactly. I could find an orthonormal basis or an eigenvector, but I never quite wrapped my head around their importance, especially visually.

noobie 11y ago

What's challenging to me is my inability to develop intuition about LA simply because most of the time I have no idea what (the purpose) I am doing. How does one build intuition about LA?

cokernel 11y ago

At my (American, liberal arts, flyover country) university, linear algebra was my first real mathematics course. (The prior classes were calculus and a "proof" class, AKA intro to intro to number theory.) I'd been introduced to the determinant, Gauss--Jordan elimination, and so on, in high school, but all in a very mechanical way. I think this transition between mechanically generating solutions to problems using pattern matching and some basic algorithms on the one hand, and the more mathematically mature approach of exploring problem spaces using pattern matching, some basic algorithms, and intuition can be difficult for many people. It's not necessarily the material -- you can get used to almost anything, and even convince yourself it's easy or obvious with enough familiarity -- but the lack of intuition. When you're first learning linear algebra, its fundamental unity is not obvious, especially if the instructor does not take pains to point it out. (And even if the instructor does take pains to point it out -- well, it's hard to understand why the instructor is saying we could do this computation this way or that way.) So in the absence of existing intuition or any perception of unity, linear algebra becomes another target for pattern matching and basic algorithms. As it happens, I've never, ever felt like I didn't "get" linear algebra. However, I almost always feel like I "get" it now and all my prior conceptions of it were a confused muddle.

ThrustVectoring 11y ago

It's a global vs sequential learner thing. Linear Algebra is usually taught as a giant pile of tricks, techniques, and terminology. This is tractable for sequential learners like myself - I really enjoy picking up new tools in a narrow context just for it's own sake. Highly global learners basically can't learn anything this way. So, in short, my advice is "add more context."