Post No.: 0742
Furrywisepuppy says:
‘Regression to the mean’ isn’t about how your significant other’s behaviour reverts back to cruelty once your birthday is over. It’s about how, if you’ve just had an extreme result, the next result is quite likely going to be less extreme, and moreover closer to the expected long-term mean, or average. This applies to random or independent samples/trials i.e. where one sample/trial doesn’t affect the next, like one dice roll doesn’t affect the next. It applies to a proportionally lesser degree the less independent the samples/trials are.
Regression towards the mean is important to understand because when, say, a mid-table football team wins 9-0 in a match, we can end up assuming that this is the start of a meteoric rise for them… just to be disappointed if they lose or draw the next match. A gambler who wins well one day will most likely have a much poorer following day.
Also, when we’re at an extremely low point, anything we try will often seem effective because frankly things couldn’t get any worse. So hiring a new manager will likely appear to improve matters after a string of several unexpectedly poor results from the outgoing manager. Or if you have a cold and then gradually feel better over the following days – it most probably had nothing to do with that alternative medicine you took but simply a natural function of healing and the regression back to your long-term average non-ill state.
Because we tend to praise people when they do well and be harsh on them when they do badly yet, because of regression, people tend to perform relatively more poorly after they’ve just performed exceedingly above-par, and vice-versa – this explains why it often appears like we get punished for rewarding others, and rewarded for punishing others. So one’s personal experience may seem to prove that criticism is more effective than praise, but in reality it’s far more likely simply due to regression to the mean. Hence many of us overestimate the effectiveness of punishment and underestimate the effectiveness of rewards, and bosses are often wrongly rewarded for punishing others and wrongly punished for rewarding them.
Rewards for improved performances work better than punishments for mistakes, but regression to the mean can make it seem like the opposite is true. When someone does something perfectly and gets rewarded for it, they’ll likely do worse the next time. Or when someone does something terribly and gets berated for it, they’ll likely do better the next time. But when you do something perfectly, you’ll likely not repeat this feat the next time. Or when you do something terribly, you’ll likely not do quite as bad the next time. In other words, your performance regresses back towards your average performance. Your results will naturally fluctuate around your average so if it was high last time, it’ll likely be low next time, and vice-versa; and rewards and punishments may have little or nothing to do with it i.e. the feedback isn’t always a causal factor. So a child being good one day will tend to have a relatively bad day the next, and vice-versa; and it may have nothing to do with their parents punishing them on the days they misbehave or rewarding them on the days they behave.
The regression towards the mean is most apparent when there’s lots of randomness or noise in a measurement (e.g. referee decisions over a season) compared with something that’s more path dependent (e.g. skill development).
If one trial to the next is completely independent from the other and each permutation has a fixed probability, like fair coin tosses, then the average will always stay fixed. If one trial to the next isn’t completely independent from the other, like one rugby match to the next isn’t totally independent because the teammates will be learning more about each other, then the long-term average can potentially shift – but it’ll typically do so only very gradually over time.
Extreme outliers tend to regress to the mean more greatly. Regression to the mean highlights the value of carrying out re-trials or repeat clinical trials, and the need to compare to a control group (preferably a placebo tested on a group drawn from the same population as the treatment group) when researching if an intervention really works, by seeing if any improvements occur more often than pure random chance. The more average events happen more commonly, thus they’ll dilute the effects of any extreme events the more events that occur.
When you’re at the top and you’ve won the lot, the only possible change is down, and vice-versa – it’s simple logic. Yet most people don’t expect regression in many contexts where it occurs and, when they come across an occurrence, they often invent spurious causal explanations for it. The universe on a certain level does operate according to cause-and-effect, but we frequently overestimate our own ability to cause the effects we see. Natural regression is incompatible with the belief that an outcome is maximally representative of the inputs that we perceive of as key (e.g. the perception that someone’s success is directly correlated with, and therefore caused by, the level of their own efforts and nothing else of importance).
In a talent show, someone who gives the most spellbinding first performance is likely to do relatively worse, while someone who scrapes through is likely to do relatively better, in the next round. But when they do, we’re too eager to read into, overweight and over-extrapolate their relative trajectories or momentums and assume they’ll always continue on their trend instead of more likely soon reverse and regress back to their personal long-term average performance. And so the second person may get voted to go through to the following round because they’re the one who seems to be on an upward path, even though the first person might still have done better overall on countback. The ‘most improved player’ is going to be more likely someone who was rubbish at the start! And the higher we are, the further we can fall, and vice-versa. Post No.: 0670 examined over-extrapolated conclusions.
We cannot expect certain statistics to keep rising or falling year-on-year or generation-on-generation (e.g. how much sex people are having or how much alcohol people are consuming) – it’s the longer multi-year or multi-generational trend that’s more important to note.
Because success is a function of both talent and luck (and a big success is a function of a little bit more talent and a hell of a lot more luck) – lucky people can be rewarded too early, like a trader after merely one exceptional year. Luck doesn’t always average out for each individual in reality but our best guess is that it should if there are repeated trials – thus the trader will likely perform worse the next year and it might be more the case that their first year was abnormally outstanding rather than their second year was unusually abysmal. If it’s a footballer and they transfer to a loftier team, they mightn’t play as well now because their performances will naturally regress. (It could also be because it was the combination with their former teammates and style of play that allowed them to shine that prior season.) And the more extreme the luck then the more regression we should expect. So don’t be surprised if the previously best often fail to meet future expectations.
Regression to the mean is incredibly common – it happens whenever the correlation between any two measures is imperfect i.e. fluctuates. Imperfect correlation and regression go furry paw-in-paw.
We do need to be wary that some seemingly independent trials aren’t completely independent from each other though. For example, although each season or each match is separate from the next, it could be the case that there’s a carried-over confidence of a goal-scoring striker from one separate match to another thus they’ll continue taking their chances and score more in subsequent games; or the way that players who’ve been playing well in recent past matches will get marked more tightly by opponents in subsequent matches thus they’ll subsequently score less. Winning or losing the last game can affect a team’s dressing room morale and thus performance in the next game. Even referee decisions appear to be influenced by previous matches – if a referee was criticised for being too soft one week, they’ll appear to overcompensate and be too harsh the next, and vice-versa, regarding similar incidents. The sentiments or forecasts of politicians, economic analysts and even ultra-rich individuals (like Elon Musk) can themselves influence the outcomes of markets, hence they’re not independent to the predictions they make. So some events that appear independent are in fact dependent on each other. However, when it comes to superstitions – a supporter who’s wearing his/her ‘lucky pants’ while watching the match at home won’t affect the outcome(!)
One shouldn’t expect everybody in a group to improve by the same amount. You should want to factor in regression – which means expecting more from the low performers and less (or even a dip) from the high performers. Causal factors such as choking after trying to defend a lead or letting go of pressure for having nothing more to lose after an atrocious first half could be true too – yet they don’t need to be true for the regression of luck to statistically occur.
Rich women tend to marry men who are less wealthy than them. There’s nothing to read into this though because it’s logical that rich women will statistically find it hard to find men who are richer than them. The correlation between the incomes of spouses is imperfect. No casual explanation is required. But our minds are strongly biased towards seeking causal explanations and don’t cope well with ‘mere statistics’. This is a problem of both ‘system one’ (which looks for a causal interpretation) and ‘system two’ (which can struggle to understand maths and statistics, and is incessantly prodded by system one to demand a causal interpretation). Correlation doesn’t necessarily mean causation anyway.
Extreme predictions and a willingness to predict rare events from scant evidence are both manifestations of system one. Because confidence arises from the coherency of the best story you can tell from the limited evidence available, which may be an extreme result – your intuitions will tend to deliver predictions that are too extreme (in order to coherently match this exceptionally extreme evidence) and you’ll also be inclined to put too much faith in them. And because your mind already has what it perceives is a reasonable way to intuitively come up with confident answers, you won’t naturally identify regression from personal experience i.e. we won’t naturally question our intuitions to look for a better method of thinking because our intuitions are so confident in themselves! Even when regression is identified, our intuitions will still urge us to give everything a causal story that’ll almost always be over-simple, over-neat, or maybe plain wrong. Therefore regression must be learnt about by system two in effortful statistics lessons.
We tend to read too much into isolated evidence in general, like a poor performance from a normally exceptional team – out of 63 games per season, you should expect a few mediocre performances from even the best in the world. Other kinds of statistics-related errors we frequently make include rudimentary ones like believing that if we hear a figure like ‘1 in 5 people are x’, then if there are 5 people in a room, 1 of them must be x. That’d be like hearing a statistic that 60% of the world’s human population is Asian then assuming every 6 in 10 people in the room we’re standing in must be Asian, even if we’re in Chile or Malaysia! That statistic applies to the global population and not necessarily to any more local population, never mind any particular individual in it – as if we all must be 60% Asian(!)
Woof!
Comment on this post by replying to this tweet:
