Posts tagged with complexity

Both direct sum and tensor product are standard ways of putting together little Hilbert spaces to form big ones. They are used for different purposes. Suppose we have two physical systems…. Roughly speaking, if … a physical system’s … states are either of A OR of B, its Hilbert space will be [a] direct sum…. If we have a system whose states are states of A AND states of B, its Hilbert space will be [a] tensor product….
MEASURE SPACE   disjoint union  Cartesian product
HILBERT SPACE   direct sum      tensor product

John Baez


Michael Conover: Information Visualization for Large-Scale Data Workflows

  • data geometry
  • memes
  • visual analysis of program structure
  • visual analysis of propaganda
  • image
  • compare last week’s analysis and share with colleagues
  • geom_bin2d rather than geom_point(alpha=...) in ggplot2
  • ggpairs
  • automated grading: in addition to unit testing, 1) parse syntax trees of submissions, 2) define edit distance between them, 3) induces a network structure, 4) identify clusters, 5) give feedback to a representative member of the cluster and cc: everyone else

The illusion that Nature … adopts the same complexity scale used by people … arises each time we face … configuration [space]…. The second law of Thermodynamics, for example, has been interpreted in many textbooks as though Nature exhibits an incurable tendency to disrupt order.

Nature, of course, could not prefer one state of affairs to any other simply because we found an elegant description to the former, not more than the sequence HHHHHH is preferred to any other sequence in a coin-flipping experiment. The second law implies only that a … system tends to “escape” from any narrow region of phase space toward regions of larger volume.

The illusion of an irreversible trend toward disorder originates with the fact that the volume occupied by states to which people can find concise descriptions … is extremely small compared to the entire space of possibilities. The escape from the [describable] to the [formless] is merely a perceptual distortion of the underlying transition from the narrow to the wider, as people fail to record the much more frequent transitions from the complex to the complex.

Judea Pearl, On the connection between the complexity and credibility of inferred models

via Artemy Kolchinsky

In the beginning, at the birth of computing, there were no programming languages. Programs looked something like this:
00110001 00000000 00000000
00110001 00000001 00000001
00110011 00000001 00000010
01010001 00001011 00000010
00100010 00000010 00001000
01000011 00000001 00000000
01000001 00000001 00000001
00010000 00000010 00000000
01100010 00000000 00000000

That is a program to add the numbers from one to ten together, and print out the result (1 + 2 + ... + 10 = 55). It could run on a very simple kind of computer.
Marijn Haverbeke


Consider the problem of transmitting information so that the receiver can decode the message with high probability.

We wish to find codewords (sequences of input symbols to a channel) that are mutually far apart in the sense that their noisy versions (available at the output of the channel) are distinguishable.

This is equivalent to sphere packing in high dimensional space.

Thomas A. Cover & Joy A. Thomas, Elements of Information Theory

What happens if, instead of doing a linear regression with sums of monomial terms, you do the complete opposite? Instead of regressing the phenomenon against image , you regressed the phenomenon against an explanation like image ?

I first thought of this question several years ago whilst living with my sister. She’s a complex person. If I asked her how her day went, and wanted to predict her answer with an equation, I definitely couldn’t use linearly separable terms. That would mean that, if one aspect of her day went well and the other aspect went poorly, the two would even out. Not the case for her. One or two things could totally swing her day all-the-way-to-good or all-the-way-to-bad.

The pattern of her moods and emotional affect has nothing to do with irrationality or moodiness. She’s just an intricate person with a complex utility function.

If you don’t know my sister, you can pick up the point from this well-known stereotype about the difference between men and women:

a well-known stereotype: women are complex, men are simple

"Men are simple, women are complex.” Think about a stereotypical teenage girl describing what made her upset. "It’s not any one thing, it’s everything.”

I.e., nonseparable interaction terms.

I wonder if there’s a mapping that sensibly inverts strongly-interdependent polynomials with monomials — interchanging interdependent equations with separable ones. If so, that could invert our notions of a parsimonious model.

Who says that a model that’s short to write in one particular space or parameterisation is the best one? or the simplest? Some things are better understood when you consider everything at once.

Complex systems are ones with a large effective number of strongly-interdependent variables.

This excludes both low-dimensional systems, and high-dimensional ones where the variables are either independent, or so strongly coupled that only a few variables effectively determine all the rest.
Cosma Rohilla Shalizi


Moscow by Lee Jang Sub

Moscow by Lee Jang Sub