with No Comments

Post No.: 0707survivorship


Furrywisepuppy says:


The ‘survivorship bias’ or ‘survival bias’ is a type of selection bias. It’s the error of focusing on the people/things that made it past some selection process while ignoring those who/that didn’t; usually because they’re unseen or voiceless.


A school that produced 6 of the 10 highest-grade students in town can lead us to believe that this school necessarily offers an outstanding education – when this school could’ve simply been a relatively large school in this town? We can answer this question by enquiring about the grades of all the other students from that school too; not just those who made the top 10 in town.


So the survivorship bias can lead to the false belief that those who succeeded have some special property rather than because of coincidence or chance.


A mutual fund manager could set up several different portfolios then choose to highlight to prospective clients the one that performed best according to its Sharpe ratio, while it keeps hidden the many more that performed badly. (The past doesn’t always predict the future anyway. Statistics is a part of science but the past reveals a probability for – not what’s going to definitely happen in – the future. We must also question which statistics matter most e.g. the past 5 years or the past 120 minutes of a goalkeeper’s performance?)


So the survivorship bias can lead to overly optimistic beliefs because the failures are ignored.


Pseudoscience is frequently peddled this way – a business selling some unproven alternative medicine that supposedly cures a disease will report on every positive, but not negative, testimonial. Moreover, those who died after using this product cannot personally tell us about their potential regrets for using it! Something risky can therefore be made to sound safe because we only get to hear the views of those who survived. “Just take the jump” the survivors exhort(!) We might know that some people died but their voices won’t be included in the conversations, hence a survivorship bias. The full picture is missing.


In any context, we’re just really poor at looking at the total picture when that’s where the best truths are. The overall picture normally shows us that most issues are more complex and nuanced than at first thought.


Generally, there’s a bias when journalists interview people to highlight their stories and viewpoints – those who volunteer to be interviewed might just be the ‘good/acceptable faces’ of their industries? After listening to them, we might therefore end up with the impression that they’re all good. But of course the ‘bad/unacceptable faces’ don’t want the publicity and they constantly refuse to give any interviews unless they can control every aspect of these events.


So we must always ask who decided not to, or couldn’t, give us an interview? We must also wonder about whom we’re personally not interested in enough to hear about?


Always consider whom we’re not hearing from, like the unscrupulous corporations hiding under ‘confidentiality’ and who don’t want their unethical practices reported. Who didn’t want us to know exactly what they do? Who doesn’t want to tell us exactly how they got or made their money? Do those who volunteered to be interviewed truly represent the rest of their group? Someone might stand out because they’re rare, even amongst their own group? Deceased people can’t defend themselves, even with a right to reply. So consider if the missing voices are dead? Too disabled? Have they emigrated? Did they go out of business? Lose their investments? Are they too embarrassed to publicly speak about their stories? If someone died, would they be televising the episode?


Alternatively, journalists might go undercover to report on a particularly egregious business when it doesn’t represent how its industry behaves as a whole.


Are the reporters just capturing the voices of those who are saying, “Thank you for the liberation” and ignoring all those who are barking, “Get lost for the invasion”? Reporters might be able to always find at least one citizen who’ll support a particular angle. (This shows us that when we use the word ‘they’ to refer to others – as in the example ‘they, in France, love cheese’ – this seldom refers to ‘everyone’ in that country or group even though the word is used for succinctness and can be okay if understood as a generality; as long as this generality is fair. So ‘they’ seldom means ‘all of them’.)


We may hear from false ‘experts’ who got their long-range forecasts correct because they’re the ones coming forwards or accepting interviews. Meanwhile, those who guessed wrong are avoiding the discussion of their misses or aren’t being asked for interviews – which is like asking the lottery winners how they chose their numbers while ignoring all those who lost their money(!)


Will abusive or neglectful parents be the ones who’ll volunteer to have their family life scrutinised on screen if they’re wondering why their children are disruptive in school?


An interviewed winner may claim that his/her superstitious rituals gave him/her the edge. But how many losers also believed in superstitions too? Alas, no one cares to interview them deeply to learn what they all did. And interviewing spiritual advisors won’t elicit an impartial answer – they’re obviously going to say that paying for such expertise is worth it!


We’ve therefore got to question the data points we’re not seeing, like someone who’s incredibly studious is a winner, and because he/she is a winner, he/she gets interviewed on TV about his/her habits and routines – but we’re not asking and hearing about the potentially thousands of other, in some cases more, studious contestants who didn’t win.


What about the single-minded, determined, dedicated, brave, ambitious, hard-working, self-believing and morally good people who didn’t catch a break? I do understand the issue of what else are we going to say to kids – don’t bother believing in yourself, don’t work hard and don’t be good?(!) Yet we’d be misleading them if we suggested that this was the full story of the road to success.


As media consumers, we’re intrigued to hear about the life stories of successful athletes who experienced ‘tough love’ when young (while the stories about them also receiving enough gentle supportive love are downplayed by journalists even when they’re also autobiographically disclosed). Meanwhile, we don’t get or want to hear about the stories of those who experienced ‘tough love’ (especially if gentle supportive love wasn’t involved in their childhoods too) and didn’t turn into top sports stars – perhaps because they gave up on sport as soon as they could due to the premature pressure placed on them by their own parents, or simply because they didn’t have the opportunities of receiving decent tuition and training. We don’t hear about the thousands of hours of hard work that the last-placed athlete put in too, so we might end up believing that it was only the podium-placed finishers who trained hard and the rest were just lazy because the world is simple – everybody gets what they deserve(!)


It’s also important to understand that what gets published is only a fraction of all the scientific research that actually gets conducted, because academic journals are biased towards publishing only novel research that finds some link or effect rather than none.


And if the general media only reports on ‘interesting novel results’ and ignores all ‘boring obvious results’ then we might spot that one study that claims to show that ‘exercise is bad for you’ but not notice the hundreds of studies that show that ‘exercise is good for you’. This means that we can end up believing in the wrong overall conclusion.


We need the ‘God’s eye view’, outside view or ‘statistical thinking’ to see the fuller picture. This means that we need to also be aware of the voiceless, ignored, dismissed or ‘mere statistic’ – whom may actually make up the vast majority of the individuals; like in the case of one death being a tragedy while a million is a statistic. We might have to speculate these figures but sometimes we can find out e.g. we can calculate the probability of people losing after playing a particular casino game, research the percentage of businesses that fold before 1 or 5 years of starting, or work out for ourselves how many competitors don’t win gold medals after making the starting line after putting in years of hard training and eating right. Perhaps the losers received less funding to afford the latest, top-of-the-range equipment or the best coaching? Alas, we’re not interested in their stories, even though they might reveal that they fundamentally did nothing different to what the winner did regarding what was reasonably in their control.


We’ll find on the bookshelves the (auto)biographies of those who claim that dreaming big, visualising success, personal skill, industriousness, an unshakeable self-belief and ignoring one’s detractors were key to their success. Theses books are typically present-biased (with an overly-coherent narrative of history), one-sided (from the perspective of the experience of their own lives – like for all of us regarding our own lives) and are self-serving narratives of what actually happened (where they’re typically framed as the maverick ‘rule/mould-breaking’ hero). They sift through their journey and typically cherry-pick evidence of how they merited all the wealth and respect they received, without external advantages. We as readers might then take this as the recipe for success. Well it kind of is – in the sphere of selling the books of entrepreneurs! And by buying their books, looking for the ‘secret mindset to getting rich’, they’ll be made even richer. Yet most readers who follow their advice won’t.


And we’d have known why if only we’d heard the stories of those who failed despite their competence, diligence, sacrifices, etc.. (Auto)biographies are typically biased to seldom mention luck as an additional factor for success – especially those books that are trying to preach a ‘formula for success’. And there could be far more of these stories untold because stubborn single-mindedness and ignoring critics who advise you that your goal is unrealistic is more typically a recipe for disaster! Many have bankrupted themselves for taking on excessive risks in their single-minded pursuit for wealth. Plenty with this mindset have ended up in financial ruin – but we don’t, or don’t want to, hear about their stories so much or at all. Numerous professional miscarriages have been committed by those who broke rules or were convinced they knew best when they didn’t.


It’s thus a survivorship-biased selection of (auto)biographies we see in bookstores. It’s like hearing only the anecdotes of those who felt cured after taking an alternative medicine or who won the lottery. We don’t want to hear about the stories of those who failed despite doing all the same things as the successful did that they reasonably could. It’s not to say that being ambitious, industrious, dogged, etc. is pointless – it’s to say that if we learn more from mistakes then we’re missing much of the most useful lessons. Strategies or gambles that are ‘against the odds’ ought to be self-explanatory as requiring luck being on your side. Something, like a rags-to-riches story, may stick out precisely because ‘you rationally shouldn’t have put your money on it happening but it did’.


We need to always combat confirmation bias and seek the full picture to know the best truth – and this means enquiring about disconfirming evidence too, even when it doesn’t fit a neat, coherent, simple tale.


There are tons more examples of the survivorship bias. We need holistic knowledge to generate fairer holistic views. Instead of a bias towards listening to what we’re interested in hearing about – we need to listen to the voices we’re not interested in hearing about too. What about all those who stayed silent or are being ignored? Who did the journalists not (want to) interview? What would the dead say if they were able to sell their stories instead of being clumped together in a condensed statistic?




Comment on this post by replying to this tweet:


Share this post