The perils of vague language in predictions

[back to all posts]

Feb 2023

Have you ever used terms like "likely" or "a serious possibility" to indicate the likelyhood or probably of some future event? If so, you may be unintentionally miscommunicating.

"Superforecasting: The Art and Science of Prediction" has an interesting anecdote on how even the very best fall prey to this:

A 1951 National Intelligence Estimate report concluded "an attack on Yugoslavia in 1951 should be considered a serious possibility". A few days later, after a confusing discussion with a senior official, Sherman Kent went back to his team asked each person what they thought "a serious possibility" meant. Alarmingly, answers ranged from odds of 80:20 to 20:80! The phrase was so vague that it could create dangerous misunderstandings. It was worse than useless!

Sherman Kent suggested a solution — the terms used by analysis should have standardized numerical meanings. So e.g. something is termed "probable" if there is a 63% to 87% chance it would happen.

Kent's scheme was simple and greatly reduced the room for confusion. But unfortunately it was never adopted.