Posts tagged with probability

risk, however measured, is not positively related to (rational) expected returns. It goes up a bit as you go from Treasuries, or overnight loans, to the slightly less safe BBB bonds, or 3 year maturities. But that’s it, that’s all you get for merely taking the psychic pain of risk.

Just as septic tank cleaners do not make more than average, or teachers of unruly students do not make more than average, merely investing in something highly volatile does not generate automatic compensation. Getting rich has never been merely an ability to withstand some obvious discomfort.

Fun coursera on virology.

  • Viruses are so numerous (10³⁰) and filling up everywhere. It gives this Boltzmann flavour of ‘enough stuff” to really do statistics on.

  • Viruses are just a bundle of {proteins, lipids, nucleic acids} with a shell. It’s totally value-free, no social Darwinism or “survival of the fittest” being imbued with a moral colour. Just a thing that happened that can replicate.
  • Maybe this is just because I was reading about nuclear spaces (⊂ topological vector spaceand white-noise processes that I think of this. Viruses have a qualitatively different error structure than Gaussian. Instead of white-noise it’s about if they can get past certain barriers, like:
    • survive out in the air/water/cyanide
    • bind to a DNA
    • spread across a population
    • adapt to the host’s defences
  • … it seems like a mathematician or probabilist could use the viral world of errors to set out different assumptions for a mathematical object that would capture the broad features of this world that’s full of really tiny things but very different to gas particles.
  • Did I mention that I love how viral evolution is totally value-neutral and logic-based?
  • Did I mention how I love that these things are everywhere all the time, filling up the great microspace my knowledge had left empty between man > animals > plants > > bacteria > > minerals?

i.e. think about the number of rearrangements of AAAAABBBBBBB and then revalue A as + and B as − — or whatever.
Requires knowing that combinatorics can be thought of in terms of counting injections, bijections, etc. rather than real-life examples like cards or coin flips.

i.e. think about the number of rearrangements of AAAAABBBBBBB and then revalue A as + and B as − — or whatever.

Requires knowing that combinatorics can be thought of in terms of counting injections, bijections, etc. rather than real-life examples like cards or coin flips.


Wheel Of Fortune

Sors immanis et inanis, rota tu volubilis, status malus, vana salus semper dissolubilis. #oFortunaRoll #newRickRoll
— JD Long (@CMastication) September 22, 2012


The Cauchy distribution (?dcauchy in R) nails a flashlight over the number line

and swings it at a constant speed from 9 o’clock down to 6 o’clock over to 3 o’clock. (Or the other direction, from 3→6→9.) Then counts how much light shone on each number.


In other words we want to map evenly from the circle (minus the top point) onto the line. Two of the most basic, yet topologically distinct shapes related together.


You’ve probably heard of a mapping that does something close enough to this: it’s called tan.
Since tan is so familiar it’s implemented in Excel, which means you can simulate draws from a Cauchy distribution in a spreadsheet. Make a column of =RAND()'s (say column A) and then pipe them through tan. For example B1=TAN(A1). You could even do =TAN(RAND()) as your only column. That’s not quite it; you need to stretch and shift the [0,1] domain of =RAND() so it matches [−π,+π] like the circle. So really the long formula (if you didn’t break it into separate columns) would be =TAN( PI() * (RAND()−.5) ). A stretch and a shift and you’ve matched the domains up. There’s your Cauchy draw.

In R one could draw three Cauchy’s with rcauchy(3) or with tan(2*(runif(3).5)).


What’s happening at tan(−3π/2) and tan(π/2)? The tan function is putting out to ±∞.

I saw this in school and didn’t know what to make of it—I don’t think I had any further interest than finishing my problem set.

File:Hyperbola one over x.svg

I saw as well the ±∞ in the output of flip[x]= 1/x.

  • 1/−.0000...001 → −∞ whereas 1/.0000...0001 → +∞.

It’s not immediately clear in the flip[x] example but in tan[x/2] what’s definitely going on is that the angle is circling around the top of the circle (the hole in the top) and the flashlight of the Cauchy distribution could be pointing to the right or to the left at a parallel above the line.

Why not just call this ±∞ the same thing? “Projective infinity”, or, the hole in the top of the circle.

At a purely formal level, one could call probability theory the study of measure spaces Ω with total measure one ∑Ω=1, but that would be like calling number theory the study of strings of digits which terminate. At a practical level, the opposite is true…

it is the events and their probabilities that are viewed as being fundamental, with the sample space Ω being [forgotten] as much as possible, and with the random variables and expectations being viewed as derived concepts. …

However, it is possible to … abstract… one step further, and view the algebra of random variables and their expectations as being … foundational …, and ignoring both the presence of the original sample space, the algebra of events, or the probability measure.

Terry Tao 0, 5

Hat tip to Qiaochu Yuan, who echoes:

The traditional mathematical axiomatization of probability, due to Kolmogorov, begins with a probability space P and constructs random variables as certain functions P→ℝ. But start doing any probability and it becomes clear that the space P is de-emphasized as much as possible; the real focus of probability theory is on the algebra of random variables. It would be nice to have an approach to probability theory that reflects this.

Qiaochu is interested in extending to the noncommutative case to

As I’ve remarked elsewhere, noncommutative is normal. In life as well as in QM. In your typical Euclidean xy plane the directions don’t matter—and from this same-etry we get

Practically speaking, in life, though, the direction you’re goingdoesmatter, either because the evolutionary field only has manna in the Canaanic direction, or because the playing field itself is tilted, by gravity (general relativity) or entropy or financial regulations or competition or something else.

  • It’s easier to lie down than jump.
  • It’s easier to fail than succeed.
  • Portfolio variation in the upward direction has a notably different effect on LPs’ attitudes during phone calls than does portfolio variation in the downward direction.
  • You can’t go back in time and say the words you should have said.
  • Speaking of words, word order matters.

Noncommutative is normal. Looking forward to stealing these mathematicians’ great ideas on free probability!

In the Public Encyclopedia’s (present) discussion of the hypothetical existence of a magnetic monopole

in nature, among the possible fundamental particles, exemplifies both (and maybe >2) “sides” in the debate over what probability means:

Magnetism in bar magnets and electromagnets does not arise from magnetic monopoles, and in fact there is no conclusive experimental evidence that magnetic monopoles exist at all in the universe.

Since Dirac’s 1931 paper[8] , several systematic monopole searches have been performed. Experiments in 1975[10] and 1982[11] produced candidate events that were initially interpreted as monopoles, but are now regarded as inconclusive.[12]Therefore, it remains an open question whether or not monopoles exist.

Further advances in theoretical particle physics, particularly developments in grand unified theories and quantum gravity, have led to more compelling arguments[which?] that monopoles do exist. Joseph Polchinski, a string-theorist, described the existence of monopoles as "one of the safest bets that one can make about physics not yet seen”.[13]These theories are not necessarily inconsistent with the experimental evidence. In some theoretical models, magnetic monopoles are unlikely to be observed, because they are too massive[why?] to be created in particle accelerators, and also too rare in the Universe to enter a particle detector with much probability.[13] (According to these models, there may be as few as one monopole in the entire visible universe.[14])

Here are a few potential explanations of how one is to arrive at a probability number:

  • opinion — it’s just Joseph Polchinski’s opinion
  • frequentism — Europeans never observed a black swan before exploring the New World, therefore black swans have 0% chance of existing.
  • frequentism + how hard you’ve searched — the probability comes attached with a confidence number. If you’ve stayed within the city limits of Minneapolis your entire life, you should attach a low confidence to your search for tarantulas the size of your head. But we’ve tried very hard to find monopoles, and haven’t. So a “more confident” zero on that one.
  • Dutch Books — could we arbitrage Joseph Polchinski’s “sure thing” bet?
  • authority, credibility, expertise — who exactly is this Joseph Polchinski character, anyway? And who says he’s such an expert? Is he an interested party? I don’t believe what vested interests and biased sources say, even if it happens to be true.
  • propensity — good gravy, I don’t even get to invoke the famous “coin has an innate propensity to tend to certain heads/tails ratio” because it would get us nowhere in terms of “Do monopoles have a propensity to exist or not?”. Anyway propensity merely passes the buck even in the cases where it does make sense.
  • reason & facts — there is no conclusive evidence that monopoles exist, yet they haven’t been proven impossible. I will withhold my opinion and it would be unreasonable to assign a probability mass to either alternative. We’re simply somewhere ∈ [0%, 100%] at this time.
  • model strength — some of these models sound suspect. It’s constructed “just so” that there’s only one monopole in the universe? Very convenient for you, when you want to say monopoles exist and we just haven’t seen them yet. Pull the other one!

All of the stochastic maths is done with the Kolmogorov axioms, i.e. it’s done with measure spaces with a fixed | finite | constant measure (= 100% of the probability mass) without connecting that to “how likely” a one-off event is. (Much like some maths you could pass off as financial modelling “is just" the theory of martingales = fair repeated bets.) But it needn’t have be called “likelihood”, it could have been “fuzzy truthiness” or “believability” or “motions of a fixed-volume-but-infinitely-divisible liquid”. As Cosma Shalizi puts it here:

Probabilities are numbers that tell us how often things happen.

Mathematicians are anxious to get on with talking about ergodicity, Markov transition matrices, and large-deviations theory. What you’re seeing in this block quote is the handoff between mathematicians and philosophers—essentially the mathmos say “You take it from here to the firm foundation” and philosophers, so far, haven’t been able to.


Is there a problem in practice due to not having a sound foundation on our concept of probability? Yes. It’s not secure to move forward with the rear flank uncovered. The lax attitude toward probability and “We’ll do the best with what we can” lets us make up numbers for the {pessimistic, neutral, optimistic} scenarios of our forecasting spreadsheets.

Think about when some consequential decision by a powerful group depends on the value of one parameter. It could be

  • the likelihood of Floridian home prices decreasing by more than 5% in a year,
  • the likelihood of [foreign country X] attacking "us" in response to Y,
  • the likelihood of RHIC creating a strangelet and swallowing the world in a minisecond,
  • the likelihood of construction on the new power plant going over budget,
  • the likelihood of borrowing rates staying this low for another 5 years,
  • the likelihood of real GDP rising at least 2%/year during the next 10 years,
  • the likelihood of our borrowing rate quadrupling
  • the likelihood that your college degree will “be worth it” to you
  • the likelihood of this whole startup thing actually working.

and I get to either rely on

  • historical data (“home prices have always gone up before”, “we haven’t seen any problems with financial derivatives yet”, “correlation with a Gaussian copula has always worked so far”),
  • reason and facts (and multiply an endless debate among the experts),
  • or gut (throw in some numbers that sound pessimistic, optimistic, and neutral, and we’ll see how the forecast behaves).

We got nuthin’.

One of my old jobs was at a private equity firm. One rule of thumb I learned there may be useful to would-be entrepreneurs. To myself, I call it the rule of "Just add water." Like one of those bath toys that grows to a larger size of the exact same thing when you add water, the perfect investment is a business that grows to a larger size of the exact same thing when you just add money.

This is not meant to deter anyone who’s already on a different path or to be some master theory of finance. I just think it’s an easy-to-remember model that a will-be entrepreneur can use to check ideas against when planning a new “growth business”. (i.e., not the vineyard you’re going to operate in retirement; not the splogs you run passively on the side to augment your regular income; not the community-enhancing business you’re doing less for money than to make the world a more interesting place. Just businesses that are hoping to get acquired by a large corp or else attract investment to grow to a medium-to-large size)

So. What does an ideal investment look like to an investor? It looks like “I put in money and get out more money later”. It doesn’t involve

  • taking chances,
  • having to run the business (unless they are actually great in that business area),
  • and especially not giving someone a chance because everyone deserves a chance.

Here is my fantasy model of the perfect business to invest in. Let’s say the Six Flags corporation has built its first rollercoaster park in Ohio and it is doing very well. It cost $130 million to build and it nets $10 million in profits per year. If you do some annuity maths (from the geometric series) you’ll see that that’s a decent business. Let’s ignore all complications and say that that profit stream is worth a net $7.6mm today. (WolframAlpha’s number if I use 6% interest rate and just assume the theme park depreciates to zero after 30 years of constant profits) Which is a big number for one person but small when it’s divided 100 ways.

Nevertheless the value that’s been proved by the Six Flags team is not just a $7,600,000 net addition of wealth but $7,600,000 in that region of Ohio. In other words that number can be multiplied. All you have to do is: just add money.

Well me and my people, we have connections to people who already made it and now want their money to work for them. “Having the money work for them” means paying us a management fee to look for businesses like this Six Flags and then bet on sure things. If we have a sure thing like this to bet on, then we can subscribe as much funds as we need to.

So the initial cost of a Six Flags was $130mm and let’s say there are 19 other locations with the exact same stats (number of people with a certain income in a certain radius, competition, etc) where the management team has convinced us they can duplicate the exact same business with the exact same cash flows. It would take them >15 years to save up enough money to build a new Six Flags in just one of those locations, but here is an opportunity for capital to come in and speed up the business’ growth. Now we multiply $7.6mm of NPV by twenty = $152mm of present value.

Then we have to figure out how to actually structure this deal, that’s another complicated question. When do investors get their money and how? How much is the investors’ capital worth as a percentage of the growth? Does the management team get stretched too thin or can they hire and train enough people. (This is called “operations” = actually doing things like running a business, not just elocution and planning as the financiers do).

In reality there are going to be more factors like repair costs and risks, risk of lawsuits, interest rates, other opportunities, appropriate size of the investment, and much more. Anything that makes this business not just an annuity complicates things. That’s why I say this is a fantasy model.

But I think the basic story behind the duplication of Six Flagses is basically what investors love to see. Isn’t it what you would love to see if you were an investor; had made your fortune running paper mills; and just wanted to sit back, relax, and live off your massive dosh now?
Zhang Yin
Here is something that already works perfectly, all the kinks have been straightened out, it’s just a formula that’s been proven to work. All these Six Flags management people need is money, which I happen to have, and nothing else from me (I don’t know how to run a Six Flags), and then the investors can multiply out the Six Flags formula to all of our benefit.


The present zeitgeist notwithstanding, the driving force of capitalism is not solving social problems. Asking those questions can be a good way to look for ideas, but it is not sufficient for extracting dinero from customers/clients, which is the actual driving force. Social problem + investment = solution is a naïve way some beginning entrepreneurs think, and it essentially puts all of the risk and all of the work onto the investor—which is not a value proposition for them. (I.e., you are relying on your investment partner being a fool—so then how will you really feel after you’ve bilked him/her/them and living on ill-gotten wealth?)

So that’s the foolish way to think “Just add money”: I am going to make the next Groupon, all I need is some programmers and a million dollars and then we can get started doing this thing! Why is this going to work? Because people are stupid. They’ll buy anything. What, do the work without getting paid? You crazy! What I just described is not “Just add money”, it’s “Just add everything”. Some people deceive themselves, though, thinking they could be rich if only somebody gave them the million-dollar OK to pursue their “idea”. (Well, they would be rich, but it wouldn’t be from the idea succeeding. And business costs could eat a million in short order anyway.) One of your “jobs” as “the boss”—what you’re contributing to the situation, and what you’re getting paid for—is a plan that, with good execution and getting people on board with you and relationships and everything else, is going to add to the world a “machine” that causes people to hand over money, either in large amounts or many times, over decades-plus time period. That is, you’re creating new revenue streams that your investors (if you’re taking investors) are buying into. (There are some markets where it takes a lot of money to start up or where a huge advertising budget can make/break the business. I don’t pretend to understand those, though, so I can’t offer any useful advice there.)

I’ll grant there are other ways to win investors over—like, they are half in it for personal interest in a subject area, or they half just want to change the world like you do, or Instagram just sold for a $billion and they are gunning from the hip for the next big score, etc. To me, as an entrepreneur, you can hope for that kind of luck, but you can’t control luck. You do have it within your control to solve all of the problems down to the point where more money = multiplication and the multiplication will bring in enough returns for everyone to share and both parties walk away satisfied. If you offer people an obviously good deal you will get bites and you can use that goal to sieve your ideas at the outset.

distances in the tree to the path connecting the corners in a uniform spanning tree of a 200×200 grid
by Russ Lyons

distances in the tree to the path connecting the corners in a uniform spanning tree of a 200×200 grid

by Russ Lyons


If you’ve read Stats 101 at your local institution of schooling and refinement, you know the difference between false positives and false negatives.

  • False positive. Oncologist, to patient: Oh my God. This is terrible. Just terrible. Patient: What? Is it bad, Doc? Oncologist: Oh, not you. My son’s handwriting. It’s terrible! Practically illegible.”
  • False negative: Pregnancy test:  Eight months later: Waaaah!

False positive is when the canary has spent several years building up an immunity to iocaine mine gas; you stroll in and die. False negative is when the canary dies of canary-pneumonia in a gas-free mine; you scurry away and miss out on $500bn worth of shale coal.

twitter bird


For algorithmic traders, a “signal” is the switch that tells your software “Buy! Buy!” or “Sell! Sell!”

  • Computer: Just give me the signal, boss, and I’ll buy 10,000 shares of the company that makes IcyHot.
  • Trader: Let’s see … gold is up 50% on the year … the underlying is retracing between the third and fourth Fibonacci levels  … the volatility of the DOW is below 30 … it’s raining in Moscow … and my Alabama state government newsfeed just flashed the word “indubitably” … throw down the iron condor! Hard!!!

If I think about looking for a signal, I think about: when should I do this trade?


The idea behind this exercise is to have a computer search through data streams for you and tell you what’s a good time to trade.

If you take the perspective that the only thing you can control is your bet size (and not what the market will do), then it becomes clear that the choice is not only about {yes, no} but also about [£0, £100bn].

Accordingly, something that tells you when not to trade can be just as valuable as something that tells you when to trade.

The most obvious non-trading scenario is the Federal Open Market Committee. Say you normally trade forex intraday, close out all your positions when you leave the screen, and that’s your game. Right after the FOMC announcement, market movements may be drastic and will have little to do with what you normally bet on — unless you trade FOMC announcements specifically. But the point is it’s a separate modelling problem.

In theory at least, any strategy should be improvable if you can accurately identify conditions when the strategy fails. Removing losers will add to your PnL just as surely as adding winners.

I’ll make up a fake example with fake data (aka, lies). Say your strategy is to trade in the direction of momentum of S&P 500 E-mini’s iff the directionality has been sustained for at least 70% of the last five minutes, and to pull out the trade iff the fraction of price movements in your direction falls below 70%.

Looking at each hour of the past year, the least profitable hour for this trade, statistically, has been 12:00-1:00 New York time. So if you had followed the exact same instructions but closed out all positions and never traded during lunchtime, your PnL during 2011 would have been higher than the strategy as originally stated. (Of course, this example works only because one hour had to be the least profitable. But the same difficulty—distinguishing real patterns from numerical mirages—inheres in signal identification as well as anti-signal identification. If you identify a real cause & effect then the anti-signal should work.)

Any Statistical Model

Say you are trying to calculate when, where, and wieviel an advertiser should bid in a DMP for internet ad space. You take as inputs known or presumed data about site visitors, indexed by cookie, and produce as (eventual) output a list of which ads to buy, when, and how much to bid for them.

Here the same anti-signal concept could apply.

Instead of thinking, What are some damned good characteristics in this space? or Should I try another algorithm? This other paper says RandomForests aren’t as good as Breiman says. , think What data is the AI really failing on? You can remove those data from the training set and decline to make recommendations about cookies within that hull.

Say you are scanning a number of text resumes on a site like Indeed <aff link> and trying to figure out whose application you should invite for a geomodelling job. Just as much as searching for positive keywords like “Petrel”, you might want to filter out negative keywords like “definately”.

Say you are training your machine to learn when tweets will be effective and when not. Instead of shoving every tweet through the lingpipe, first filter out the non-English-language tweets.

OK, that last example is really obvious. I am not claiming that anti-signals are novel. It’s just a word I made up for something that’s common sense. But coining the word reminds me when I look at a modelling problem, to turn the problem upside-down and ask if there’s any low-hanging fruit on the other side.