Linear algebra

How to study linear algebra when the computation makes sense but nothing else does

Linear algebra is the course where students get high marks on the computation and cannot say what any of it means. You can row-reduce, invert, and find eigenvalues without ever forming a picture of what a matrix is — and then the second half of the course, which is all about that picture, becomes an unrelated set of theorems to memorise. The subject is much easier if you build the geometry first: a matrix is a transformation of space, and almost every definition in the course is a statement about what that transformation does.

10 min readSubjects

One idea the whole course hangs from

A matrix is a function that takes vectors to vectors, and does it linearly — grid lines stay parallel and evenly spaced, the origin doesn't move. Every other concept in a first course is a question about that function.

ConceptThe computationThe question it's really asking
DeterminantA formula involving cofactorsBy what factor does this transformation scale area or volume — and does it flip orientation?
RankCount the pivotsHow many dimensions does the output actually occupy?
Null spaceSolve Ax = 0What gets crushed to zero — what information does this transformation destroy?
InvertibilityDeterminant nonzeroCan you undo it? You can't, if anything was crushed.
EigenvectorSolve (A − λI)x = 0Which directions does this transformation not rotate, only stretch?
BasisIndependent spanning setWhat's the minimum set of directions needed to describe everything here?

Study the right-hand column and the left-hand column becomes bookkeeping. Study only the left and you'll pass the first midterm and struggle from there.

Build the picture deliberately

  1. 1

    Work everything in 2D first, by hand, with a drawing

    Take a unit square. Apply a 2×2 matrix to its corners. Draw the result. Do this for a rotation, a shear, a scaling, a reflection and a singular matrix. Half an hour, once, and it will carry the entire course.

  2. 2

    For each 2×2 example, compute the determinant and check it against the drawing

    The determinant is the area of the transformed square, signed. Seeing the number match the picture converts a formula into a fact about the world.

  3. 3

    Find the eigenvectors of your examples visually before computing them

    For a shear, which arrows don't change direction? Guess, then compute, then check. Guessing first is what makes the computation mean something.

  4. 4

    Do the singular case deliberately

    Apply a rank-1 matrix and watch the square collapse onto a line. Everything about null spaces, invertibility and zero determinants is visible in that one picture, and it's the picture most students never draw.

  5. 5

    Only then generalise to n dimensions

    Almost nothing in the theory changes; you just can't draw it. But you can now reason by analogy from a picture you own, which is what people mean when they say they have intuition for the subject.

The definitions have to be exact

Linear algebra is many students' first properly abstract course, and it punishes approximate definitions in a way that calculus doesn't. "Linearly independent means they're different directions" is roughly right and will lose you the proof question. The exact statement — no nontrivial linear combination equals zero — is what you actually manipulate.

  • Write each definition out from memory weekly. Word for word. These are arbitrary in the way vocabulary is arbitrary, and belong in spaced review.
  • Attach a picture and a counterexample to each. The counterexample is what prevents the approximate version from creeping back in.
  • Know which definitions are equivalent. Half the theorems in the course are statements that two definitions coincide — invertible, nonzero determinant, full rank, trivial null space are all the same thing, and knowing that turns a dozen theorems into one.
  • Build the equivalence list explicitly. Most courses have one big theorem gathering these; if yours doesn't state it, assemble it yourself. It's the single most useful page you'll write.

Proofs, for people who haven't done proofs

If this is your first proof-based course, the difficulty is usually not the mathematics — it's that you've never been shown what a proof is supposed to look like. Three habits fix most of it.

  • Start by writing what you're allowed to assume and what you must reach. Most stuck proofs are stuck because the goal was never written down.
  • Say the definition out loud. A large fraction of first-course proofs are literally the definition unpacked and repacked, and students look for cleverness that isn't required.
  • Study the proof structure separately from the content. Direct, contrapositive, contradiction, induction. There are perhaps five, and knowing which one a proof uses before reading it makes it comprehensible.

Then use the Feynman technique on proofs specifically: cover the proof and try to reproduce the argument, not the algebra. The step where you stall is the step that was doing the work.

Where the exam marks actually are

In most first courses the marks split roughly: 40% computation you can grind, 30% definitions and short theory, 30% proof or application. Students who study only by doing problem sets are optimising for the first 40% and hoping. Check your own past papers rather than trusting that split — but check them early enough to change what you're doing.

The computational marks also reward speed and accuracy, which is a separate skill from understanding. Row reduction under time pressure with no arithmetic slips is worth practising as a drill, the same way calculus execution is.

If you're taking it for machine learning or engineering

The applied route tempts you to skip the theory and learn the operations. It works until it doesn't: eigendecomposition, rank and projection are not exotic corners of the theory, they're the things that appear the moment you touch dimensionality reduction, least squares or anything with a covariance matrix in it. The geometric picture is what makes those applications comprehensible rather than magical, and it's the cheapest part of the course to acquire.

If you're studying alongside programming work, implementing the operations yourself once — a 2×2 solver, a projection, a power iteration — is unusually good revision, because code cannot be approximately right. How to learn to code and how to study computer science cover that side.

Common questions

Why is linear algebra so hard to understand?

Because it's usually taught computation-first, so you learn to row-reduce and invert without ever forming a picture of what a matrix does. Once the geometric idea — a matrix is a linear transformation of space — is in place, most of the later definitions become questions about that transformation rather than isolated theorems.

What's the best way to learn linear algebra intuitively?

Work in two dimensions by hand with drawings first. Apply a rotation, a shear, a scaling and a singular matrix to a unit square and draw each result. Half an hour of that gives you a picture to reason from for the rest of the course, including in dimensions you can't draw.

How do I understand eigenvalues and eigenvectors?

Eigenvectors are the directions a transformation doesn't rotate — it only stretches or shrinks them — and the eigenvalue is that stretch factor. Guess them visually for simple 2D matrices before computing, then check. The computation makes sense once you know what it's looking for.

Do I need to memorise linear algebra definitions?

Exactly, word for word. Linear algebra punishes approximate definitions in a way calculus doesn't, because the proofs manipulate the precise statement. Keep them in a spaced review queue alongside a picture and a counterexample for each.

How do I get better at linear algebra proofs?

Write down what you may assume and what you must reach before starting — most stuck proofs never stated the goal. Then unpack the definition, since a large share of first-course proofs are precisely that. Study the handful of proof structures separately from the content.

Can I skip the theory if I only need linear algebra for machine learning?

Not usefully. Rank, projection and eigendecomposition are exactly what appears in dimensionality reduction, least squares and covariance work — they're not obscure corners of the theory. The geometric picture is the cheapest part of the course to get and the part that makes the applications comprehensible.

Try it on your own material

Upload your notes, slides or lecture recordings and get a tutor that answers only from them — and says so when they don't cover it.

$12/month, or $8/month billed yearly · cancel in two clicks