EDIT: thank you all for the great responses!
[0]: https://www.math.brown.edu/streil/papers/LADW/LADW_2017-09-0...
[1]: https://www.amazon.com/Linear-Algebra-Its-Applications-5th/d... — PDFs exist.
[2]: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x...
Like others here I recommend 3B1B, which may be what you are looking for visually, but whatever you end up with it is absolutely crucial that you do exercises. Do many of them. It is the only way to get better.
His course at MIT is legendary, completely available online https://ocw.mit.edu/courses/18-06-linear-algebra-spring-2010...
And there’s so much good linear algebra stuff on YouTube from 3brown1blue.
If you can do one thing now, watch this Veritasium video to disprove the myth that you’re a visual learner: https://youtu.be/rhgwIhB58PA.
I don't think I've been able to find any particularly good visual LinAlg books - most of what you're trying to achieve is actually quite abstract and I found the classic books a little confusing.
As an addendum - if you live stateside, classes at community colleges may be quite inexpensive and fairly approachable.
https://www.youtube.com/watch?v=rhgwIhB58PA&feature=youtu.be
It's worth calling out that the video itself doesn't support some of the blunt arguments being made here. The point of the video is that it's likely that everyone does better with a multimodal approach. It thus remains reasonable to seek out books that do a good job with visual representations! No visual components, or, worse, bad diagrams are, according to the video, an impediment to everyone, not just people who have used a disfavored term in their Ask HN question. :)
I like 3Blue1Brown as much as everyone else, it's an achievement and kind of a joy to watch, but my experience was that, after many go-rounds over the years, the thing that made any of this actually stick was doing exercises. I tend to bang the less abstract ones out in Sage: https://www.sagemath.org (you have to do a little bit of extra work to make sure Sage isn't doing too much of the work for you.
I'm a fan of Strang's approach. But I'm bad at linear algebra, so, grain of salt.
I think it's great for an intermediate student, or someone who's also watching Strang's lectures.
This is highly important. Linear algebra is applicable to so many fields, but learning linear algebra for say... Graphics Programmers, is a completely different feel from learning linear algebra for an Electrical Engineer Signals-and-systems engineer.
Graphics programmers largely need to learn "how to use" matricies. Emphasis on associative properties. Emphasis on non-communitive operations.
In contrast, Electrical Engineers / Signals-and-systems want to learn linear-algebra as a stepping stone to differential calculus. In this case, you're going to be focusing more on eigen-values, spring-mass systems / resonant frequencies, applicability to calculus and other tidbits (how linear algebra relates to the Fourier Transform).
----------
The graphics programmer (probably) doesn't need to learn eigenvalues. So any textbook written as "linear algebra for graphics programmers" can safely skip over that.
The electrical engineer however needs all of this other stuff as "part" of the linear algebra class.
I'm sure other fields (statistics, error-correction codes/galois fields, abstract algebra, etc. etc.) have "their own ways" of teaching linear algebra that is most applicable to them.
Yes, "linear algebra" is broadly applicable. But instead of trying to "learn all of it", you should instead focus on the "bits of linear algebra that is most applicable to the problems you face". That shrinks down the field, increases the "pragmatism" of your studies.
Later, when you're more familiar with "some bits" of linear algebra, you can then take the next step of generalizing off of your "seed knowledge".
--------
I personally never was able to learn linear algebra from a linear algebra book.
Instead, I relearned linear algebra 4 or 5 times as the "basis" of other maths I've learned. I learned it for differential calculus. I relearned linear algebra for signals. I relearned linear algebra for Galois fields/CRC-codes/Reed Solomon. I relearned linear algebra for graphics.
Yes, it seems inefficient, but I think my "focus" isn't strong enough to just study it in the abstract. I needed to see the "applicable" practice to encourage myself to learn. Besides, each time you "relearn" linear algebra, its a lot faster than the last time.
For example, if some particular method is computationally efficient relative to others, the text makes a note of it, and has lots of computational examples. Most of the examples could be set up fairly straightforwardly with something like a Python notebook and Numpy for matrices. It also covers things like computational errors wrt floating-point operations when doing vector and matrix calculations, efficient algorithms for approximating eigenvalues of a matrix, etc.
And!, the full text is available on archive.org with a free account:
“Linear Algebra for Everyone”
Perhaps a game development book is even more visual? I haven't read it (yet), but this book is getting recommendations: https://gamemath.com/book/
I don't know if the H&S book is beginner friendly, but what I found good about it was studying linear algebra and differential equations at the same time, i.e. treating them as closely related topics rather than separate ones. So you could use your physical intuition about (say) a harmonic oscillator (mass on a spring, the archetypal second order ODE), then see how the 2nd order equation can be separated into a system of first order ODE's, and solved by finding matrix eigenvalues.
That worked better for me than the abstract linear algebra approach that was purely about vector spaces with nothing going on in them. It showed real sensible motivations of linear algebra.
However, there's no magic bullet that will let you learn linear algebra in a couple of hours. At some point you have to sit down and work to figure it out. The field has university departments researching it, so there's a lot more to it than just multiplying m×p by p×n matrices.
I don't know what you mean exactly by beginner, but assuming you have some level of mathematical maturity, UT Austin has an edx course you can audit for free, "Linear Algebra Foundations to Frontiers", and FastAI also have a pretty good free video series/course on it too.
It's definitely not the norm compared to many of the other listings in this thread but it definitely gave me a better understanding of many algebraic properties and helped build an intuition around spaces, vectors, products, etc.
It doesn't have a ton of graphics, to which you might snub your nose at it (you mentioned visual learning), but the graphics it does have are incredibly useful for building a geometric understanding of what linear algebra concepts map to. The subsection on quaternions and pseudoscalars is one of the best descriptions of such in my experience.
I don't remember how I found this guy but watching him feels more like learning from a friend who's extremely knowledgeable about linear algebra rather than sitting in a university course.
that book, i think, is fantastic. i was a TA for a graduate (and undergraduate) level course using it in Urbana-Champaign around 25 years ago. It's just a great book.
Basically, go through the exercises in the book, but use numpy, Julia, MATLAB, or similar. Learn the "Matrix API" back to front.
I've successfully taught a number of devs without a strong mathematics background with this method. The fast feedback loop this approach affords allows a good intuition to be built up very quickly.
I realize the answers will differ for different people/situations, but I'd be curious to know what has or hasn't worked for others.
Hania Uscka-Wehlou: https://www.udemy.com/user/hania-uscka-wehlou/
Great for visual learners: full of colours, highlights relevant parts of the equations, visualisations, etc.
I recommend enrolling in a class, explain the instructor about your goal and s/he can find topics and walk you through them.
Also I'm sure it's been mentioned here already, but the MIT Linear Algebra course by Gilbert Strang was absolutely phenomenal. Really made it click for me.
http://www.faculty.luther.edu/~macdonal/laga/
It's a great and very geometric presentation of linear algebra that takes it through to geometric algebra. A very nice presentation of a coherent set of geometric techniques.
Practical Linear Algebra - Gerald Farin & Dianne Hansford.
IIRC, the first edition was based on a work designed to educate design workers at a car firm who either didn't have the background or needed shoring up. So from the beginning it was aimed at the common person. It has since been developed over the years to it's current form.
All I know is this book has great illustrations / great analogies and was written in a way that worked _for me_ where other books did not and was a springboard to more advanced texts which as you can tell by the comments there are many. I highly reccomend.
Almost every topic is illustrated with an application or illustrative example, and the applications are very interesting.
https://ocw.mit.edu/courses/res-18-008-calculus-revisited-co...
Higher math tends to be abstract; you can't visualize higher-dimensional linear algebra concepts directly. The standard resources (Strang, Axler, etc) are worth the effort.
Applied learning is the best way to learn linear algebra.
For the graphical part, start with, say, (3,7). Regard that as the coordinates in the standard X-Y coordinate system of a point on the plane. So, the X coordinate is the 3 and the Y coordinate is the 7. You could get out some graph paper and plot the thing. So, more generally, given two numbers x and y, (x,y) is the coordinates of a point in the plane. We call (x,y) a vector and imagine that it is an arrow from the origin to an arrow head at point (x,y). Then we can imagine that we can slide the vector around on the plane, keeping its length and direction the same.
What we did for the plane and X-Y we could do for space with X-Y-Z. So, there a vector would have three coordinates. Ah, call them components.
Now in linear algebra, for a positive integer n, we could have a vector with n components. For geometric intuition, what we saw in X-Y or X-Y-Z is usually enough.
We can let R denote the set of real numbers. Then R^n denotes the set of all of the vectors with n components that are real numbers. Our R^n is the leading example of a vector space. Sometimes it is good to permit the components to be complex numbers, but that is a little advanced. And the components could be elements of some goofy finite field from abstract algebra, but that also is a bit advanced.
Here we just stay with the real numbers, elements of R.
Okay, suppose someone tells us
ax + by = s
cx + dy = t
where the a, b, c, d, s, t are real numbers.
We want to know what the x and y are. Right, we can think of the vector (x,y). Without too much work we can show that, depending on the coefficients a, ..., t, the set of all (x,y), that fits the two equations has none, one, or infinitely many (points, vectors) solutions.
So, that example is two equations in two unknowns, x and y. Well, for positive integers m and n, we could have m equations in n unknowns. Still, there are none, one, or infinitely many solutions.
C. F. Gauss gave us Gauss elimination that lets us know if none, one, or infinitely many, find the one, or generate as many as we wish of the infinite.
We can multiply one of the equations by a number and add the resulting equation to one of the other equations. We just did an elementary row operation, and you can convince yourself that the set of all solutions remains the same. So, Gauss elimination is to pick elementary row operations that make the pattern of coefficients have a lot of zeros so that we can by inspection read off the none, one, or infinitely many. Gauss elimination is not difficult or tricky and programs easily in C, Fortran, ....
Quite generally in math, if we have a function f, some numbers a and b, and some things, of high generality, e.g., our vectors, and it is true that for any a, b and things x and y
f(ax + by) = af(x) + bf(y)
then we say that function f is linear. Now you know why our subject is called linear algebra.
A case of a linear function is Schroedinger's equation in quantum mechanics, and linear algebra can be a good first step into some of the math of quantum mechanics.
Let's see why those equations were linear: Let
f(x,y) = ax + by
Then
f[ c(x,y) + d(u,v)]
= f[ (cx, cy) + (du,dv) ]
= f(cx + du, cy + dv)
= a(cx + du) + b(cy + dv)
= c(ax) + d(au) + c(by) + d(bv)
= c(ax + by) + d(au + bv)
= cf(x,y) + df(u,v)
Done!
This linearity is mostly what makes linear algebra get its mathematical theorems and its utility in applications.
We commonly regard the plane with coordinates X-Y as 2 dimensional and space with coordinates X-Y-Z as 3 dimensional. If we study dimension carefully, then the 2 and 3 are correct. Similarly R^n is n dimensional.
We can write
ax + by = s
cx + dy = t
as x(a,c) + y(b,d) = (s,t)
So, (a,c), (b,d), and (s,t) are vectors, and x and y are coefficients that let us write vector (s,t) as a linear combination of the two vectors (a,c) and (b,d).
Apparently the superposition in quantum mechanics is closely related to this linear combination.
Well suppose these two vectors (a,c) and (b,d) can be used in such a linear combination to get any vector (s,t). Then, omitting some details, (a,c) and (b,d) span all of R^2, are linearly independent, and form a basis for the vector space R^2.
Sure, the usual basis for R^2 is just
(1,0)
(0,1)
And that our basis has two vectors is because R^2 is 2 dimensional. Works the same in R^n -- n dimensional and a basis has n vectors that are linearly independent.
Now for some geometric intuition, given vectors in R^3 (x,y,z) and (u,v,w), then for coefficients a and b, the set of all
a(x,y,z) + b(u,v,w)
forms, depending on the two vectors, a point, a line, or a plane through (0,0,0) -- usually a plane and, thus, a vector subspace of dimension 0, 1, 2, usually 2.
And this works in R^n: We can have vector subspaces of dimension 0, 1, ..., n. For a subspace V of dimension m, 1 <= m <= n, there will be a basis of m linearly independent vectors in subspace V.
Let's explain matrix notation: Back to
ax + by = s
cx + dy = t
On the left side, let's rip out the x and y and write the rest as
/a b\
| |
\c d/
The (x,y) we can now call a 1 x 2 matrix. But we really want its transpose, 2 x 1 as
/x\
| |
\y/
We want to define the matrix product
Av
ax + by = s
cx + dy = t
That is, Av is the transpose of (s,t).
If we have a vector u and coefficients a and b and define matrix addition in the obvious way, we can have
A(au + bv) = aAu + bAv
= a(Au) + b(Av)
= (aA)u + (bA)v
And matrix multiplication is associative, and the usual proof is just an application of the interchange of summation signs for finitely many terms.
We can define the length of a vector and the angle between two vectors. then multiplying two vectors by an orthogonal matrix U does not change the length or angle of two vectors.
Then for any orthogonal matrix U, all it does is reflect and/or make a rigid rotation.
We can also have a symmetric, positive definite matrix S. What S does is stretch a sphere into an ellipsoid (the 3 dimensional case does provide good intuition). Then A can be written as SU. That is, all A can do is rotate and reflect and then move a sphere into an ellipsoid. That is the polar decomposition and is the key to much of the most advanced work in linear algebra. Turns out, once we know more about orthogonal and symmetric matrices, the proof is short.
That's enough for a fast introduction!
Linear and Geometric Algebra