Man, I should really start taking notes as I read, like at the end of each chapter. It feels like such a pain to go back and take notes on every chapter after I’ve finished the whole book. Alas, I never took notes at all in school, and apparently I still haven’t come very far.
Thoughts on the subject of being fooled by randomness, in July 2020: the ongoing dialogue around the current pandemic is like one giant demonstration of humans being utterly misled when thinking about randomness and statistics, from government leaders and scientists on down to individuals. ALL the biases and mistakes outlined below are present in the current dialogue.
Humans are bad at telling the difference between noise and meaning, and underestimate the share of randomness in everything. Especially “specialists” like scientists and economists. Some thinkers believe humans are basically rational, and some believe humans are basically irrational.
This book is composed of three parts:
Part I is about “visible and invisible histories and the elusive property of rare events (black swans).”
Part II is about cognitive biases in probability.
Part III is about practical and philosophical aids in the face of our irrationality.
Solon was a Greek legislator who believed that you can’t really evaluate how someone’s life went until you know it all the way to the end. Things that came via luck can easily be taken away by luck; things that came without need of luck are more robust.
Chapter 1: If you’re so rich, why aren’t you so smart?
Nero is a successful-enough trader. His MO is to trade conservatively and never put himself at risk of blowing up (losing so badly that your career is basically over), which also means he’s not as rich as others, but he’s set up his life so that he has everything he needs and wants.
By contrast, John is a much more “successful” trader who makes a lot more money, but by investing in assets that leave one vulnerable to blowing up, such as high-yield junk bonds. John is also unaware of his risk because he is just following what other people do, and not actually understanding what he’s doing.
Intellectual contempt does not control personal envy.
I highlighted this and the below quote, because I sometimes doubt myself and envy people who dove into some field and found success. Whereas I feel like I’ve invested more time into being generally wiser and “thinking” and writing, which can feel like wasted time.
Maybe if he had pushed himself harder or had sought the right opportunity–instead of “thinking,” writing articles and reading complicated papers.
John finally blows up in 1998, so Nero is vindicated.
There appears to be a correlation between positive performance (whether thanks to luck or skill) and increased serotonin, which makes one carry oneself more confidently and leader-like, which can change even day to day.
Behaviorial scientists believe that one of the main reasons why people become leaders is not from what skills they seem to possess, but rather from what extremely superficial impression they make on others through hardly perceptible physical signals–what we call today “charisma,” for example. The biology of the phenomenon is now well studied under the subject heading “social emotions.” Meanwhile some historian will “explain” the success in terms of, perhaps, tactical skills, the right education, or some other theoretical reason seen in hindsight.
In addition, there seems to be curious evidence of a link between leadership and a form of psychopathology (the sociopath) that encourages the non-blinking, self-confident, insensitive person to rally followers.
This one is interesting because I noticed it myself in the documentary King of Kong, and I was really wondering why everybody in the movie seems so enamored with Billy Mitchell, who comes off (to me) like a total psychopath. And I was so disappointed that the movie doesn’t mention that at all. Which made the experience of watching it kind of funny-horrifying.
Recall that Nero can be considered prosperous but not “very rich” by his day’s standards. However, according to some strange accounting measure we will see in the next chapter, he is extremely rich on the average of lives he could have led…
You can more accurately evaluate your life/profession if you take into account all possible outcomes, not only the ones that actually happen. That way you account for professions or decisions that come with huge risks that can wipe a lot of people out, as opposed to only looking at the success cases.
Chapter 2: A bizarre accounting method
More on the “possible worlds” concept from Chapter 1. Some stuff in real life is like playing Russian roulette but with thousands of chambers and just one bullet: it’s easy to forget (or be unaware) that you are actually still playing, and there still exists a risk of losing everything.
Particularly thoughtful are those who had to abandon scientific studies because of their inability to keep focused on a narrowly defined problem.
I just relate to this–intellectual curiosity but without any desire to specialize super narrowly.
As a derivatives trader I noticed that people do not like to insure against something abstract; the risk that merits their attention is always something vivid.
Referencing the famous Kahneman/Tversky experiment.
In addition to such problems with the perception of risk, it is also a scientific fact, and a shocking one, that both risk detection and risk avoidance are not mediated in the “thinking” part of the brain but largely in the emotional one (the “risk as feelings” theory). The consequences are not trivial: It means that rational thinking has little, very little, to do with risk avoidance. Much of what rational thinking seems to do is rationalize one’s actions by fitting some logic to them.
This is one of the many reasons that journalism may be the greatest plague we face today–as the world becomes more and more complicated and our minds are trained for more and more simplification.
Borrowed wisdom can be vicious. I need to make a huge effort not to be swayed by well-sounding remarks. I remind myself of Einstein’s remark that common sense is nothing but a collection of misconceptions acquired by age eighteen.
Correctness vs. intelligibility.
From the standpoint of an institution, the existence of a risk manager has less to do with actual risk reduction than it has to do with the impression of risk reduction.
You see this in the software security world as well.
Chapter 3: A mathematical meditation on history
Mathematics is principally a tool to meditate, rather than to compute.
Referring to Monte Carlo math as a way of thinking.
Path vs. outcome: a path is the series of events, not just the end result.
Stochastic processes refer to the dynamics of events unfolding with the course of time. Stochastic is a fancy Greek name for random. This branch of probability concerns itself with the study of the evolution of successive random events–one could call it the mathematics of history. The key about a process is that it has time in it.
Monte Carlo mathematics vs. “true” or theoretical mathematics based on formulas
As for Keynes, to the literate person he is not the political economist that tweed-clad leftists love to quote, but the author of the magisterial, introspective, and potent Treatise on Probability.
Just a note to check out the Keynes book.
People are bad at learning from history, both other people’s and their own.
A mistake is not something to be determined after the fact, but in the light of the information until that point.
It’s wrong to judge the quality of a decision by its outcome.
Bad trades catch up with you, it is frequently said in the markets. Mathematicians of probability give that a fancy name: ergodicity. It means, roughly, that (under certain conditions) very long sample paths would end up resembling each other.
Each one would revert to his long-term properties.
For a journalist, silence rarely surpasses any word.
Most daily news is noise. Yet journalists get paid to say something, even when there is no new information to add. Unnecessary information (noise) is not just of zero value but of negative value.
Robert Shiller in 1981 posited that markets aren’t perfectly efficient (contrary to what financial theory asserted at the time), because prices often “overreact” and swing more than the funadmentals they are supposed to reflect.
You should favor older traders–those who have been exposed to markets the longest without blowing up. Because that means they have probably survived rare/random events, which is harder than e.g. riding a bull market on pure luck and then blowing up at the first rare event. Those who have survived, have most likely done so because they know how to protect themselves from the downside.
Noise vs. meaning: the more often you check your portfolio, the more useless the data AND the more stressful (because of the ups and downs).
Chapter 4: Randomness, nonsense, and the scientific intellectual
You can computer-generate very literary- or philosophical-sounding texts. It’s okay to enjoy irrational things and randomness in personal or aesthetic life.
Chapter 5: Survival of the least fit–can evolution be fooled by randomness?
The example of “Carlos” and emerging-markets bonds in 1998 and the danger of not having a stop-loss (a point at which you will sell, decided in advance). Especially combined with an information bubble, where all your friends have the same opinions as you.
And, at any point in time, the richest traders are often the worst traders. This, I will call the cross-sectional problem: At a given time in the market, the most successful traders are likely to be those that are best fit to the latest cycle.
The example of “John” and high-yield bonds, also in 1998. Leveraging yourself (putting in your own money to be able to borrow much more money to trade with) is probably a bad idea.
Both examples are characterized by overestimation of one’s own beliefs/abilities and denial when faced with reality.
Chapter 6: Skewness and asymmetry
Wherever you have asymmetry in outcomes, median can be very different from average or expected. Actual expectation is probability x the magnitude of the outcome.
The problem with the terms bullish and bearish is that people use them without accounting for the magnitude of the outcome: by how much they expect the market to go up or down.
You can profit from events that happen very rarely but come with a large payoff.
When looking at data, people are in the habit of blindly removing outliers. Instead, you need to think about whether you’re in a situation where it actually makes sense to remove outliers, or one where you need to pay special attention to outliers.
Shallow history / naive empiricism: “this has never happened before.” Actual history: “things that never happened before do happen.”
The peso problem: things that show no volatility tend to also be vulnerable to rare events. One of them is the Mexican peso.
Chapter 7: The problem of induction
No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion.
You can’t prove that something is always true, you can only disprove it. That’s the problem of induction, and the danger of improperly relying on empiricism.
I suddenly felt financially insecure and feared becoming an employee of some firm that would turn me into a corporate slave with “work ethics” (whenever I hear work ethics I interpret inefficient mediocrity).
I needed the backing of my bank account so I could buy time to think and enjoy life.
There was all along lurking in my mind the idea that these researchers had missed a point, but I did not quite know what it was. It was not what they knew, it was how they knew it, that was the subject of my annoyance.
Karl Popper proposed that there are only two kinds of theories:
- Theories that are known to be wrong, as they were tested and adequately rejected (he calls them falsified).
- Theories that have not yet been known to be wrong, not falsified yet, but are exposed to be proved wrong.
So a theory can never be right or verified. Also, a theory that cannot be falsified under any conditions is not a scientific theory.
More practically to me, Popper had many problems with statistics and statisticians. He refused to blindly accept the notion that knowledge can always increase with incremental information–which is the foundation of statistical inference…Sir Karl feared that some type of knowledge did not increase with information–but which type we could not ascertain.
I speculate in all of my activities on theories that represent some vision of the world, but with the following stipulation: No rare event should harm me. In fact, I would like all conceivable rare events to help me.
Popper believed that any idea of Utopia is necessarily closed owing to the fact that it chokes its own refutations. The simple notion of a good model for society that cannot be left open for falsification is totalitarian. I learned from Popper, in addition to the difference between an open and a closed society, that between an open and a closed mind.
Causality is easier to commit to memory…The effect of such compression is the reduction in the degree of detected randomness.
I want to take the best of what the past can give me without its dangers. Accordingly, I will use statistics and inductive methods to make aggressive bets, but I will not use them to manage my risks and exposure.
This is called a stop loss, a predetermined exit point, a protection from the black swan.
The monkeys-on-typewriters thought experiment, and survivorship biases.
Chapter 8: Too many millionaires next door
The virtue of capitalism is that society can take advantage of people’s greed rather than their benevolence, but there is no need to, in addition, extol such greed as a moral (or intellectual) accomplishment.
Books like The Millionaire Next Door suffer from a double survivorship bias. First, the authors focused only on the winners, and not everyone who tried the same things and didn’t win. Secondly, it focuses on what happened during a decades-long bull market; the same strategies tried at any other time in history, or in another country, might not have worked so well.
Chapter 9: It is easier to buy and sell than fry an egg
Trading is a profession that doesn’t strictly require any skill (in the way that the arts, cooking, dentistry etc do), and in any such profession, it is possible to succeed by pure luck, and not as uncommon as one would think. A Monte Carlo simulation shows that if you start with a big enough pool, you can have many people who make money for, say, 5 years in a row, out of pure luck.
In real life, we then retroactively attribute the success of such people to their competence.
The birthday problem (when there are 23 people in a room, the probability of there being 2 people with the same birthday is about 50%) demonstrates how easy it is for human intuition to be bad at estimating probabilities.
Backtesting or data snooping is fitting the rule to the past data. When doing so, you can always find some rule that appears perfectly correlated with the data, e.g. finding a correlation between some stock’s price and the temperature somewhere in the world.
A random series will always present some detectable pattern.
Even the fathers of statistical science forgot that a random series of runs need not exhibit a pattern to look random; as a matter of fact, data that is perfectly patternless would be extremely suspicious and appear to be man-made. A single random run is bound to exhibit some pattern–if one looks hard enough…real randomness does not look random!
Chapter 10: Loser takes all–on the nonlinearities of life
Life is unfair in a nonlinear way. Chance events, combined with a positive feedback loop, can allow a small initial bit of randomness to spiral into a huge advantage.
Path-dependent outcome: the probability of success can increase with each subsequent success. This breaks our familiar math of probability.
This summarizes why there are routes to success that are nonrandom, but few, very few, people have the mental stamina to follow them…Most people give up before the rewards.
Chapter 11: Randomness and our mind: we are probability blind
Kahneman and Tversky, etc.
If your mind operates by series of different disconnected rules, these may not be necessarily consistent with each other, and if they may still do the job locally, they will not necessarily do so globally.
Some heuristics. System I and System II.
Failures of probability thinking in the OJ Simpson trial.
I am glad to be a trader taking advantage of people’s biases but I am scared of living in such a society.
The false-positives quiz. How to think about options trading.
Absence of evidence vs. evidence of absence. (Saying you found no evidence of some effect != saying you found evidence of NO effect.)
The maximum of an average != the average maximum (and the former is less volatile).
Chapter 12: Gambler’s ticks and pigeons in a box
Chapter 13: Carneades comes to Rome: on probability and skepticism
The right to contradict oneself–to revise your opinion and avoid being “married to your position.” (And the danger of being unwilling to contradict yourself or your past views.)
Chapter 14: Bacchus abandons Antony
The inverse skills problem: the higher up the corporate ladder, the lower the evidence of an individual’s contribution.
More appropriately, what they have is skill in getting promoted within a company rather than pure skills in making optimal decisions–we call that “corporate political skill.”
On some additional benefits of randomness:
Subway riders are freer of their schedule, and not just because of the higher frequency of trains. Uncertainty protects them from themselves.
A slightly random schedule prevents us from optimizing and being exceedingly efficient, particularly in the wrong things. This little bit of uncertainty might make the diner relax and forget the time pressures. He would be forced to act as a satisficer instead of a maximizer–research on happiness shows that those who live under a self-imposed pressure to be optimal in their enjoyment of things suffer a measure of distress.
I am convinced that we are not made for clear-cut, well-delineated schedules. We are made to live like firemen, with downtime for lounging and meditating between calls, under the protection of protective uncertainty.
Standing on one leg
We favor the visible, the embedded, the personal, the narrated, and the tangible; we scorn the abstract.