Part 1 - Black Swans
For those who are not familiar with the idea behind 'Black Swan' I will explain. It is a simple idea all about induction. Imagine someone were to ask the question "what colour are swans?" If you have ever only experienced white swans then you would say all swans were white. You would base this on your experience. The key here is that no matter how many white swans you see it will not confirm the fact that all swans are white. Even if you studied swans all your life and never see anything other than a white swan you cannot confirm the fact. It only takes 1 black swan to destroy the entire proposition that all swans are white. If you were to believe this fact (all swans are white) then then seeing a black swan would be earth shattering (not really but you get the point). Your whole perspective would be overturned.
Induction is a tricky thing. Yet we rely upon it everyday. In the book Taleb takes this basic issue of the 'Black Swan' and outlines mankind's inability to deal with uncertainty. It is human nature to think inductively, to base their future on what has occurred in the past. Because of this we only predict things (even then inaccurately) based upon a set of experiences we have had. Most of these experiences do not include 'Black Swan' type events. We have never seen a black swan therefore all swans are and will be white.
It is this blindness to uncertainty (the fact that we cannot know the future based upon the past) that permeates every aspects of our lives. Black swans affect us in the form of stock market crashes, natural disasters and in many more subtle ways (terrorism).
Taleb talks at length about our predictions. He really has it in for statisticians and economists who regularly provide predictive models all of which affect policy in government, banking, medicine and in fact just about everything.
The key things I took away from this book when it comes to prediction was
1) The people who predict rarely look back at past predictions and test their accuracy. They simply update their predictions ignoring the the fact that the methods that produced the previous ones are the same as they use for the modern predictions and see no difficulty in this.
2) Understand the error rate. If you are going to predict then understand the error margins. The further out the bigger the effect of the error rendering the far out predictions to be useless. If you do not understand this then don't predict.
3) Silent evidence. Predictions and past performance favours only the winners.
What do I mean by 3. He shows that winners are not necessarily winners because of what they have done or proved. There is a famous approach to marketing stocks. Send 1000 people a marketing letter predicting which way the stock market will go, up or down. The following week do the same thing but for each 50% send 50% up and 50% down. After a number of weeks one or two of the recipients will be amazed that your company will have predicted every rise and fall of the market over a long period (too long to be deemed to be lucky).
In fact many of the winners we see today fall foul of the silent evidence. You only get to see the miraculous winner and not the other 999 losers. The losers are never seen.
If you can grasp these simple points then next time the trending graph goes up in a supplier presentation think about what they are saying.
I had one such presentation the other day. They predicted the rise costs of IT. The fact that all their previous predictions have been wildly out did not stop them. Or even stir them to say 'watch out may be inaccurate'. They followed up with an interesting set of stats.
We have 800 users of this IT system. Here are 5 of them that made a good (but not great saving). The question should be "but what about the other 795?" Yes it is cynical to make such a statement (they could not fit them on) but the fact is that if 800 people made that saving you would want to make such a claim.
So next time the graph goes up,
1) ask the relevant questions
2) ensure that you ask about the silent evidence
Look at these things through more sceptical eyes and it is almost funny quite how bad some suppliers can be.
Induction is a tricky thing. Yet we rely upon it everyday. In the book Taleb takes this basic issue of the 'Black Swan' and outlines mankind's inability to deal with uncertainty. It is human nature to think inductively, to base their future on what has occurred in the past. Because of this we only predict things (even then inaccurately) based upon a set of experiences we have had. Most of these experiences do not include 'Black Swan' type events. We have never seen a black swan therefore all swans are and will be white.
It is this blindness to uncertainty (the fact that we cannot know the future based upon the past) that permeates every aspects of our lives. Black swans affect us in the form of stock market crashes, natural disasters and in many more subtle ways (terrorism).
Taleb talks at length about our predictions. He really has it in for statisticians and economists who regularly provide predictive models all of which affect policy in government, banking, medicine and in fact just about everything.
The key things I took away from this book when it comes to prediction was
1) The people who predict rarely look back at past predictions and test their accuracy. They simply update their predictions ignoring the the fact that the methods that produced the previous ones are the same as they use for the modern predictions and see no difficulty in this.
2) Understand the error rate. If you are going to predict then understand the error margins. The further out the bigger the effect of the error rendering the far out predictions to be useless. If you do not understand this then don't predict.
3) Silent evidence. Predictions and past performance favours only the winners.
What do I mean by 3. He shows that winners are not necessarily winners because of what they have done or proved. There is a famous approach to marketing stocks. Send 1000 people a marketing letter predicting which way the stock market will go, up or down. The following week do the same thing but for each 50% send 50% up and 50% down. After a number of weeks one or two of the recipients will be amazed that your company will have predicted every rise and fall of the market over a long period (too long to be deemed to be lucky).
In fact many of the winners we see today fall foul of the silent evidence. You only get to see the miraculous winner and not the other 999 losers. The losers are never seen.
If you can grasp these simple points then next time the trending graph goes up in a supplier presentation think about what they are saying.
I had one such presentation the other day. They predicted the rise costs of IT. The fact that all their previous predictions have been wildly out did not stop them. Or even stir them to say 'watch out may be inaccurate'. They followed up with an interesting set of stats.
We have 800 users of this IT system. Here are 5 of them that made a good (but not great saving). The question should be "but what about the other 795?" Yes it is cynical to make such a statement (they could not fit them on) but the fact is that if 800 people made that saving you would want to make such a claim.
So next time the graph goes up,
1) ask the relevant questions
2) ensure that you ask about the silent evidence
Look at these things through more sceptical eyes and it is almost funny quite how bad some suppliers can be.
posted by So long and thanks for all the fish at
4:41 pm
0 Comments:
Post a Comment
Subscribe to Post Comments [Atom]
<< Home