Posts tagged with vectors

Double integrals ∫∫ƒ(x)dA are introduced as a “little teacher’s lie” in calculus. The “real story” requires “geometric algebra”, or “the logic of length-shape-volume relationships”. Keywords

• multilinear algebra
• Grassmann algebra / Grassmanian
• exterior calculus
• Élie Cartán’s differential-forms approach to tensors

These equivalence-classes of blobs explain how

• volumes (ahem—oriented volumes!)
• areas (ahem—oriented areas!)
• arrows (vectors)
• numbers (scalars)

"should" interface with each other. That is, Clifford algebra or Grassman algebra or "exterior algebra" or "geometrical algebra" encodes how physical quantities with these dimensionalities do interface with each other.

(First the volumes are abstracted from their original context—then they can be “attached” to something else.)



EDIT:user mrfractal points out that Clifford algebras can only have dimensions of 2,4,8,16,… https://en.wikipedia.org/wiki/Clifford_algebra#Basis_and_dimension Yes, that’s right. This post is not totally correct. I let it fly out of the queue without editing it and it may contain other inaccuracies. I was trying to throw out a bunch of relevant keywords that go along with these motivating pictures, and relate it to equivalence-classing, one of my favourite themes within this blog. The text here is disjointed, unedited, and perhaps wrong in other ways. Mostly just wanted to share the pictures; I’ll try to fix up the text some other time. Grazie.

(Source: arxiv.org)

my illustration of the first isomorphism theorem, which says you can replace an arrow ƒ:X→Y by a sequence of arrows surjection ∘ bijection ∘ injection.

(Source: tjsullivan.org.uk)

## philosophy & quantum mechanics

The isomorphismes links (last three tweets) might be worth sharing with everyone. (I’ve been accused that this site is hard to browse—sorry!)

@isomorphisms haha, yep.

— Clark (@jtc_19) April 15, 2013

By the way! OCW has course notes on the standard weird phenomena and also the standard interpretation of the physics. http://ocw.mit.edu/courses/linguistics-and-philosophy/24-111-philosophy-of-quantum-mechanics-spring-2005/lecture-notes/

Consider ℂ, the field of complex numbers, as a 1-dimensional vector space. The balanced sets are ℂ itself, the empty set and the open and closed discs centered at 0 (visualizing complex numbers as points in the plane). Contrariwise, in the two dimensional Euclidean space there are many more balanced sets: any line segment with midpoint at (0,0) will do.

As a result, ℂ and ℝ² are entirely different as far as their vector space structure is concerned.

(Source: Wikipedia)

a smooth field of 1-vectors in 3-D

(Source: thievess)

hi-res

## Tangent Space & getting Punched in the Face

I used to think vectors were the same as points. They’re lists of numbers, right? Tuples with each item ∈ the same domain (or same data type). But … they’re not.

The way vector mathematics describes the forces of the world is to instantiate one linear-algebra at each point on the surface where a force is imparted.

When I strike a ball with my foot, look at the exact contact point (and pretend the ball doesn’t squish across my foot—even though it does, because my pegas are so poderosas). There’s a tangent space at that contact point and the exact particulars of my posture, shape of my foot|shoe, and so on determine the force vector that I use to bend the ball like Beckham.

Newton’s f=ma corrected Aristotle’s theory of motion

Vacuum isn’t possible: Vacuum doesn’t occur, but hypothetically, terrestrial motion in a vacuum would be indefinitely fast.

So we know we only need that one strike and then inertia minus drag convolved with lay-of-the-land will determine a full path for the ball: till my teammate’s head contacts it—another tangent space, another vector—and we score.

That’s football for simple tangent spaces—the Aristotle-v-Newton scenario. Now onto the most poetic sport, where two holonomic dynamical systems dance within each other’s gyre, moving in and out of each other’s movements and—occasionally—osculating a tangent that links one part of me with one part of you.

All of the logic about connecting vectors head-to-tail with parallelograms is only to reason about one single strike. The whole linear algebra on force vectors is a complete examination of the possibilities of the variations and the ways to strike at that exact same point.

To talk about striking different points requires a connection (parallel transport). [And remember if you hook my cheek versus jab my forehead, those arenotparallel so even more maths is involved. Also hard tissue (bone) versus soft tissue (cheek).]

Striking the ball on the side (to spin it) or under (to chip it) would be a connection on the S² manifold of the ball — moving the point of tangency — which is a different algebra’s worth of logic. Landing a punch on a different part of me is also a connection (that would be parallel transport of the strike vector on the surface manifold (with boundary) of my face/torso/armpit/whatever).

Keeping my torso over the ball is even yet another calculus—one that I suspect is much, much more complicated. Even though Julian James Faraway and others work on these questions of whole-body mechanics, I don’t think ∃ a complete mathematical theory of how all of the fixed holonomic parts of bodies that have very similar morphogenetic shapes (similar ratios of forearm to humerus length, etc) interact—how soft tissue and hard tissue in the usual places interact and specific angles can make a judoist with excellent technique and little strength able to whirl the body of a weight-lifter around her fulcrum.

Or how this guy can deliver a lot of force with proper dynamical posture (“shape”) when he’s clearly weak and fat. I can start to imagine the beginnings of something like that but it doesn’t obviously fit into the tangent space points & vectors story, except in a very complicated way of vectors connected to vectors connected to vectors, with each connection (not the same as the parallel-transport connection sense of the word I used above! Back to the base namespace) holonomically constrained or even "soft-constrained" in the case of soft tissue. Same with a blow landing parallel-transported different places on the surface of my body. The transport takes care of the initial strike vector but not how those forces travel and twist through my skeleton, soft tissues, down to the floor (either through my feet or through my *rse if the blow knocks me down). And sure, you could FEM-approximate my interior with a 3-D mesh and a deep geometric logic—and even make the ribs more rigid than the liver—but that’s just going to make you want higher maths even more as you grapple with the army of ε’s you’ll convoke with all the approximations, dx size, parameters, and so on.

Or why it’s better to swing the bat this way, or teach your body to do its swimming strokes in that motion, or various other dynamical or static particulars of human body shape and internal anatomical facts.



BTW, an amateur ultimate fighter once told me that the order of importance for winning a fight is:

1. technique (brasilian jiu jitsu technique)
2. balance
3. agility
4. strength

He didn’t mention tolerance for getting punched in the head but he seemed to put up with it remarkably. PS: judo guys have sexy bodies.

### Linear Transformations will take you on a Trip Comparable to that of Magical Mushroom Sauce, And Perhaps cause More Lasting Damage

Long after I was supposed to “get it”, I finally came to understand matrices by looking at the above pictures. Staring and contemplating. I would come back to them week after week. This one is a stretch; this one is a shear; this one is a rotation. What’s the big F?

The thing is that mathematicians think about transforming an entire space at once. Any particular instance or experience must be of a point, but in order to conceive and prove statements about all varieties and possibilities, mathematicians think about “mappings of the entire possible space of objects”. (This is true in group theory as much as in linear algebra.)

So the change felt by individual ink-spots going from the original-F to the F-image would be the experience of an actual orbit in a dynamical system, of an actual feather blown by a bit of wind, an actual bullet passing through an actual heart, an actual droplet in the Mbezi River pulsing forward with the flow of time. But mathematicians consider the totality of possibilities all at once. That’s what “transforming the space” means.

$\large \dpi{300} \bg_white \begin{pmatrix} a \rightsquigarrow a & | & a \rightsquigarrow b & | & a \rightsquigarrow c \\ \hline b \rightsquigarrow a & | & b \rightsquigarrow b & | & b \rightsquigarrow c \\ \hline c \rightsquigarrow a & | & c \rightsquigarrow b & | & c \rightsquigarrow c \end{pmatrix}$

What do the slots in the matrix mean? Combing from left to right across the rows of numbers often means “from”. Going from top to bottom along the columns often means “to”. This is true in Markov transition matrices for example, and those combing motions correspond with basic matrix multiplication.

So there’s a hint of causation to this matrix business. Rows are the “causes” and columns are the “effects”. Second row, fifth column is the causal contribution of input B to the resulting output E and so on. But that’s not 100% correct, it’s just a whiff of a hint of a suggestion of a truth.

The “domain and image” viewpoint in the pictures above (which come from Flanigan & Kazdan about halfway through) is a truer expression of the matrix concept.

• [ [1, 0], [0, 1] ] maps the Mona Lisa to itself,
• [ [.799, −.602], [.602, .799] ] has a determinant of 1 — does not change the amount of paint — and rotates the Mona Lisa by 37° counterclockwise,
• [ [1, 0], [0, 2] ] stretches the image northward;
• and so on.

MATRICES IN WORDS

Matrices aren’t* just 2-D blocks of numbers — that’s a 2-array. Matrices are linear transformations. Because “matrix” comes with rules about how the numbers combine (inner product, outer product), a matrix is a verb whereas a 2-array, which can hold any kind of data with any or no rules attached to it, is a noun.

* (NB: Computer languages like R, Java, and SAGE/Python have their own definitions. They usually treat vector == list && matrix == 2-array.)

Linear transformations in 1-D are incredibly restricted. They’re just proportional relationships, like “Buy 1 more carton of eggs and it will cost an extra $2.17. Buy 2 more cartons of eggs and it will cost an extra$4.34. Buy 3 more cartons of eggs and it will cost an extra \$6.51….”  Bo-ring.

In scary mathematical runes one writes:

$\large \dpi{200} \bg_white \begin{matrix} y \propto x \\ \textit{---or---} \\ y = \mathrm{const} \cdot x \end{matrix}$

And the property of linearity itself is written:

$\large \dpi{200} \bg_white \begin{matrix} a \cdot f(\cdots, \; \blacksquare , \; \cdots) + b \cdot f( \cdots, \; \blacksquare,\; \cdots) \\ = \\ f( \cdots, \; a \cdot \blacksquare + b \cdot \blacksquare, \; \cdots) \end{matrix} \\ \\ \qquad \footnotesize{\bullet f \text{ is the linear mapping}} \\ \qquad \bullet a, b \in \text{the underlying number corpus } \mathbb{K} \\ \qquad \bullet \text{above holds for any term } \blacksquare}$

Or say: rescaling or adding first, it doesn’t matter which order.



The matrix revolution does so much generalisation of this simple concept it’s hard to imagine you’re still talking about the same thing. First of all, the insight that mathematically abstract vectors, including vectors of generalised numbers, can represent just about anything. Anything that can be “added” together.

And I put the word “added” in quotes because, as long as you define an operation that obeys commutativity, associativity, and distributes over multiplication-by-a-scalar, you get to call it “addition”! See the mathematical definition of ring.

• The blues scale has a different notion of “addition” than the diatonic scale.
• Something different happens when you add a spiteful remark to a pleased emotional state than when you add it to an angry emotional state.
• Modular and noncommutative things can be “added”. Clock time, food recipes, chemicals in a reaction, and all kinds of freaky mathematical fauna fall under these categories.
• Polynomials, knots, braids, semigroup elements, lattices, dynamical systems, networks, can be “added”. Or was that “multiplied”? Like, whatever.
• Quantum states (in physics) can be “added”.
• So “adding” is perhaps too specific a word—all we mean is “a two-place input, one-place output satisfying X, Y, Z”, where X,Y,Z are the properties from your elementary school textbook like identity, associativity, commutativity.

So your imagination is usually the limiting reagent in defining “addition”.

But that’s just vectors. Matrices also add dimensionality. Linear transformations can be from and to any number of dimensions:

• 1→7
• 4→3
• 1671 → 5
• 18 → 188
• and X→1 is a special case, the functional. Functionals comprise performance metrics, size measurements, your final grade in a class, statistical moments (kurtosis, skew, variance, mean) and other statistical metrics (Value-at-Risk, median), divergence (not gradient nor curl), risk metrics, the temperature at any point in the room, EBITDA, not function(x) { c( count(x), mean(x), median(x) ) }, and … I’ll do another article on functionals.

In contemplating these maps from dimensionality to dimensionality, it’s a blessing that the underlying equation is so simple as linear (proportional). When thinking about information leakage, multi-parameter cause & effect, sources & sinks in a many-equation dynamical system, images and preimages and dual spaces; when the objects being linearly transformed are systems of partial differential equations, — being able to reduce the issue to mere multi-proportionalities is what makes the problems tractable at all.

So that’s why so much painstaking care is taken in abstract linear algebra to be absolutely precise — so that the applications which rely on compositions or repetitions or atlases or inversions of linear mappings will definitely go through.



Why would anyone care to learn matrices?

Understanding of matrices is the key difference between those who “get” higher maths and those who don’t. I’ve seen many grad students and professors reading up on linear algebra because they need it to understand some deep papers in their field.

• Linear transformations can be stitched together to create manifolds.
• If you add Fourier | harmonic | spectral techniques + linear algebra, you get really trippy — yet informative — views on things. Like spectral mesh compressions of ponies.
• The “linear basis” and “linear combination” metaphors extend far. For example, to eigenfaces or When Doves Cry Inside a Convex Hull.
• You can’t understand slack vectors or optimisation without matrices.
• JPEG, discrete wavelet transform, and video compression rely on linear algebra.
• A 2-matrix characterises graphs or flows on graphs. So that’s Facebook friends, water networks, internet traffic, ecosystems, Ising magnetism, Wassily Leontief’s vision of the economy, herd behaviour, network-effects in sales (“going viral”), and much, much more that you can understand — after you get over the matrix bar.
• The expectation operator of statistics (“average”) is linear.
• Dropping a variable from your statistical analysis is linear. Mathematicians call it “projection onto a lower-dimensional space” (second-to-last example at top).
• Taking-the-derivative is linear. (The differential, a linear approximation of a could-be-nonlinear function, is the noun that results from doing the take-the-derivative verb.)
• The composition of two linear functions is linear. The sum of two linear functions is linear. From these it follows that long differential equations—consisting of chains of “zoom-in-to-infinity" (via "take-the-derivative") and "do-a-proportional-transformation-there" then "zoom-back-out" … long, long chains of this, can amount in total to no more than a linear transformation.

• If you line up several linear transformations with the proper homes and targets, you can make hard problems easy and impossible problems tractable. The more “advanced-mathematics” the space you’re considering, the more things become linear transformations.
• That’s why linear operators are used in both quantum mechanical theory and practical things like building helicopters.
• You can understand dynamical systems, attractors, and thereby understand love better through matrices.

## How to multiply matrices

This is for my homies in maths class.

Mathematical matrices are blocks of numbers, arrayed in 2-D. (Higher-dimensional array-verbs are called tensors.)

1. Left “times” right equals target. Each entry in the target is the result of a series of +'s and ×'s along the red and blue. A long sum of pairwise products.

2. Your left hand goes across and your right hand goes up/down.
where
.
3. There need to be as many abcdefg's as there are 1234567's or else the operation can't be done.
4. Also you can tell how big the output matrix will be. There can be three blue rows so the output has three rows. There can be four red columns so the output has four columns.
5. This is the “inner product” because multiplying vector-shaped blocks (tall blocks) like Aᵀ•B results in an equal or smaller sized output.

(There is also an “outer product" which is a different way of combining the info from the two matrices. That gives you an equal or larger shaped result when you multiply vector/list-shaped tall blocks A∧B.)
6. Try playing around with this one or that one.

Matrix multiplication is the simplest example of a linear operator, the broad class of which explains quantum mechanics and ODE’s. You can also apply different matrices at different points as in a vector field — on a flat surface or a curvy, holey surface.

## Vector fields

Vector fields pervade. I think about them every time I throw a frisbee in wind.

In a social context, I think about vectors of intent attached to people talking at a party — vectors of flirtation, vectors of eye movement and attention, and more abstract vectors representing jokes, topics of discussion, dance moves, or songs that are playing.

Also when I’m thinking about international trade or just the local flows of money in my community, it’s natural to use the vector-field metaphor to “see” the flows.

I also think of history (at different scales) using vector fields. Wars are like nation-states or soldiers aiming weapon vectors at each other. Commerce has many more dimensions since goods and money are both multi-dimensional. Ideas and culture also transmit in a vector-field-like way. Epidemics — well, there’s a reason mosquitoes are referred to as disease vectors.

Information flows, thoughts, internet bits — anything that can be characterised as a vector, you can expand that thought into a more complicated vector-field thought. Turbulent versus laminar flows of ideas and culture? Maybe it wouldn’t deserve a research grant but it’s fun to think about.

There are pretty obvious physical examples of vector fields — rivers, wind, geological eroding forces, magnetism, gravity, flying machines, bridge engineering, parachute design, weather patterns, your entire body as it does martial arts or dances. Being measurable, these are the source of most of the neat vector-field pictures you can find online.

(Or you find programmatically simple theoretical vector fields like the above: a vector facing [−y,x] is attached to every point (x,y). So for instance the point (3,4) has a pointer going out −4 south and 3 east, which equals a total force of 5.)

The same metaphors and visualisations, though, are open to interpretation as social or economic variables too. For example a profitable business is more of a “sink” or attractor for 1-D money flows, while a benefactor is a “source”. Likewise a blog that receives lots of links and traffic is a 2-D attractor on the graph of the web — and Google recognises that as PageRank.

I know of at least one paper that tries to best economists’ utility theory models by imagining a person on a 1-D vector field, trying to avoid minus signs and find a path to plus signs in the space.

There is also a game theory connection. Basins of attraction can draw you into a locally optimal place that is not globally optimal. You can imagine examples in the evolution of animals, in company policies or business practices, or in whole economic systems.

On the one hand it may seem frivolous or crackpottical to generalise these concrete physical concepts to the social or psychological. On the other hand — that’s the power of the generality of mathematics!

Vector fields are surfaces or spaces with a vector at each point. That’s the mathematical definition.