Posts tagged with matrix

For what is the theory of determinants? It is an algebra upon algebra; a calculus which enables us to combine and foretell the results of algebraical operations, in the same way as algebra itself enables us to dispense with the performance of the special operations of arithmetic. All analysis must ultimately clothe itself under this form.

I have in previous papers defined a ‘Matrix' as a rectangular array of terms, out of which different systems of determinants may be engendered, as from the womb of a common parent; these cognate determinants being by no means isolated in their relations to one another , but subject to certain simple laws of mutual dependence and simultaneous deperition.

James Joseph Sylvester, 1851


(a relevant fact is Cramer’s rule: knowing only the determinants of submatrices you can find the eigenvectors, which are the stable fixed-points under the matrix operation)

from the same source, quoting Sylvester’s Apotheosis of Algebraical Quantity (1884):

A matrix … regarded apart from the determinant … becomes an empty schema of operation, … only for a moment looses the attribute of quantity to emerge again as quantity, … of a higher and unthought-of kind, … in a glorified shape-as an organism composed of discrete parts, but having an essential and undivisible unity as a whole of its own .… The conception of multiple quantity thus rises on the field of vision.

The romantic view expressed there doesn’t sound very different, to me, of Deleuze on calculus


A farmer’s Markov transition matrix.

  • At every step of the life cycle, the farmer (well, the plant, really…) contends with new natural enemies: wind, birds, mice, disease, varmints, frost, deer, ….
  • This is a not-really-Markov transition matrix.
  • Some states (such as “into a deer’s mouth”) that a plant could come to occupy aren’t shown.
  • In the final row I made up some number for volume change.
  • I guess the bottom right is the difference between a perennial and an annual, right? Like if that 1 were a 0 the plant would be an annual.

pics from Wikipedia or Purdue



In many showers there are two taps: Hot and Cold. You use them to control both the pressure and the temperature of the water, but you do so indirectly: the pressure is controlled by the sum of the (position of the) two taps, while the temperature is controlled by their difference. Thus, the basis you are given:

Hot = (1,0)
Cold = (0,1)

Isn’t the basis you want:

Pressure = (1,1)
Temperature = (1,−1)

Alon Amit

(image would be the basis isomorphism)


other bases:


Here’s a physically intuitive reason that rotations ↺

Matrix Transform

(which seem circular) are in fact linear maps.

If you have two independent wheels that can only roll straight forward and straight back, it is possible to turn the luggage. By doing both linear maps at once (which is what a matrix
\begin{pmatrix} a \rightsquigarrow a  & | &  a \rightsquigarrow b  & | &  a \rightsquigarrow c \\ \hline b \rightsquigarrow a  & | &  b \rightsquigarrow b  & | &  b \rightsquigarrow c \\ \hline c \rightsquigarrow a  & | &  b \rightsquigarrow c  & | &  c \rightsquigarrow c   \end{pmatrix}
or Lie action does) and opposite each other, two straights ↓↑ make a twist ↺.

Or if you could get a car | luggage | segway with split (= independent = disconnected) axles

to roll the right wheel(s) independently and opposite to the left wheel(s)

, then you would spin around in place.

Once you’re comfortable with 2-arrays and 2-matrices, you can move up a dimension or two, to 4-arrays or 4-tensors.

You can move up to a 3-array / 3-tensor just by imagining a matrix which “extends back into the blackboard”. Like a 5 × 5 matrix. With another 5 × 5 matrix behind it. And another 5 × 5 matrix behind that with 25 more entries. Etc.

The other way is to imagine “Tables of tables of tables of tables … of tables of tables of tables.” This imagination technique is infinitely extensible.

\begin{bmatrix}  \begin{bmatrix} \begin{bmatrix} a & b \\ c & d \end{bmatrix} & \begin{bmatrix} e & f \\ g & h \end{bmatrix} \\ \\ \begin{bmatrix} j & k \\ l & m \end{bmatrix} & \begin{bmatrix} n & o \\ p & q \end{bmatrix} \end{bmatrix} & \begin{bmatrix} \begin{bmatrix} r & s \\ t & u \end{bmatrix} & \begin{bmatrix} v & w \\ x & y \end{bmatrix} \\ \\ \begin{bmatrix} z & a' \\ b' & c' \end{bmatrix} & \begin{bmatrix} d' & e' \\ f' & g' \end{bmatrix} \end{bmatrix} \\ \\ \begin{bmatrix} \begin{bmatrix} h' & j' \\ k' & l' \end{bmatrix} & \begin{bmatrix} m' & n' \\ o' & p' \end{bmatrix} \\ \\ \begin{bmatrix} q' & r' \\ s' & t' \end{bmatrix} & \begin{bmatrix} u' & v' \\ w' & x' \end{bmatrix} \end{bmatrix} & \begin{bmatrix} \begin{bmatrix} y' & z' \\ a'' & b'' \end{bmatrix} & \begin{bmatrix} c'' & d'' \\ e'' & f'' \end{bmatrix} \\ \\ \begin{bmatrix} g'' & h'' \\ j'' & k'' \end{bmatrix} & \begin{bmatrix} l'' & m'' \\ n'' & o'' \end{bmatrix} \end{bmatrix} \end{bmatrix}

If that looks complicated, it’s just because simple recursion can produce convoluted outputs. Reading the LaTeX (alt text) is definitely harder than writing it was. (I just cut & paste \begin{bmatrix} stuff \end{bmatrix} inside other \begin{bmatrix} … \end{bmatrix}.)

(The technical difference between an array and a tensor: an array is a block which holds data. A tensor is a block of numbers which (linearly) transform matrices / vectors / tensors. Array = noun. Tensor = verb.)

As the last picture — the most important one — demonstrates, a 4-array can be filled with completely plain, ordinary, pedestrian information like age, weight, height.

Inside each of the yellow or blue boxes in the earlier pictures, is a datum. What calls for the high-dimensional array is the structure and inter-relationships of the infos. Age, height, sex, and weight each belongs_to a particular person, in an object-oriented sense. And one can marginalise, in a statistical sense, over any of those variables — consider all the ages of the people surveyed, for example.

One last takeaway:

  • Normal, pedestrian, run-of-the-mill, everyday descriptions of things = high-dimensional arrays of varying data types.

Normal people speak about and conceive of information which fits high-D arrays all the time. “Attached” (in the fibre sense) to any person you know is a huge database of facts. Not to mention data-intensive visual information like parameterisations of the surface of their face, which we naturally process in an Augenblick.


This is trippy, and profound.

The determinant — which tells you the change in size after a matrix transformation 𝓜 — is just an Instance of the Alternating Multilinear Map.


(Alternating meaning it goes + − + − + − + − ……. Multilinear meaning linear in every term, ceteris paribus:

\begin{matrix} a \; f(\cdots  \blacksquare  \cdots) + b \; f( \cdots \blacksquare \cdots) \\ = \shortparallel | \ | \\ f( \cdots a \ \blacksquare + b \ \blacksquare \cdots) \end{matrix}    \\ \\ \qquad \footnotesize{\bullet f \text{ is the multilinear mapping}} \\ \qquad \bullet a, b \in \text{the underlying number corpus } \mathbb{K} \\ \qquad \bullet \text{above holds for any term } \blacksquare \text{ (if done one-at-a-time)})


Now we tripThe inner product — which tells you the “angle” between 2 things, in a super abstract sense — is also an instantiation of the Alternating Multilinear Map.

In conclusion, mathematics proves that Size is the same kind of thing as Angle

Say whaaaaaat? I’m going to go get high now and watch Koyaanaasqatsi.



In the world of linear approximations of multiple parameters and multiple outputs, the Jacobian is a matrix that tells you: if I twist this knob, how does that part of the output change?


(The Jacobian is defined at a point. If the space not flat, but instead only approximated by flat things that are joined together, then you would stitch together different Jacobians as you stitch together different flats.)

Pretend that a through z are parameters, or knobs you can twist. Let’s not say whether you have control over them (endogenous variables) or whether the environment / your customers / your competitors / nature / external factors have control over them (exogenous parameters).

And pretend that through F are the separate kinds of output. You can think in terms of a real number or something else, but as far as I know the outputs cannot be linked in a lattice or anything other than a matrix rectangle.

In other words this matrix is just an organised list of “how parameter c affects output F”. 

Notan bene — the Jacobian is just a linear approximation. It doesn’t carry any of the info about mutual influence, connections between variables, curvature, wiggle, womp, kurtosis, cyclicity, or even interaction effects.

A Jacobian tensor would tell you how twisting knob a knocks on through parameters h, l, and p. Still linear but you could work out the outcome better in a difficult system — or figure out what happens if you twist two knobs at once.


In maths jargon: the Jacobian is a matrix filled with partial derivatives.

A matrix ℳ represents a sequence of + and × operations. At the end you’ve linearly transformed a space (sheared it, expanded it, rotated it — but kept the origin where it is.)

sheared Mona Lisa

Did the amount of stuff in the picture change when you did that? If you kept everything in proportion then det |ℳ| = 1. If not, then det |ℳ| ≠ 1.


If the amount of stuff increased by 10% then det |ℳ|=1.1. If you effectively shrank the picture in half, then det |ℳ|=.5. And so on.

The determinant |ℳ| is the change in volume after the linear transformation.

This metaphor extends to 3-D and beyond.

  • If water is flowing linearly in a stream, then |ℳ| needs to be 1, or else water (matter) would be being created.
  • If money is flowing linearly in a billion-dimensional economic system, then |ℳ| is hopefully just a little bit above 1, if value is being created. (Central banks need to print |ℳ| times more money to prevent deflation.) 
  • And a hundred-dimensional linear dynamical system's phase space grows by |ℳ| at every step.