What do you really mean when you say you are 90% sure about something? Are you really sure about what it means to be 90% sure? Probably not, according to several studies which show people are consistently overconfident about their beliefs, pretty much like the cat in the picture below:
One trick that makes it easier to visualize what a confidence level of 90% really means requires converting that confidence into a bet involving money. Here’s how.
Let’s say you are 90% sure that the stock market will close higher in 2010 than it did in 2009. If that is really the case i.e. if you are really 90% sure about your prediction, then you should be willing to win $1 in a bet but accept a loss of $9 if you lose. Why?
We already know that either you are right or you are wrong about your prediction. In fact, you are going to be right 90% of the time according to you. Thats what being 90% sure means. If that turns out to be the case, then the expected value of your winning is 90% of $1. That comes to +$0.90
You are going to be wrong 10% of the time according to you. That’s what being 90% sure means – that you will have an error rate of only 10%. If that turns out to be the case, then the expected value of your loss is 10% of $9. That comes to -$0.90.
Therefore, if you are 90% confident that the market will close higher in 2010 than in 2009, then this is a fair bet because it involves an equal exchange.
Put in those words, you may balk at the prospect of losing $9 versus winning only $1. But that is exactly what 90% confidence level means.
When your beliefs are framed in the form of money bets instead of confidence levels, you have a better chance of understanding the odds implied by your beliefs. Keep that in mind when you express confidence in your predictive abilities.
Watch your language!
Fundoo Professor 
John Allen Paulos makes more or less the same argument in the book ‘Innumeracy’.