with No Comments

Post No.: 0044control group


Furrywisepuppy says:


Scientific experiments involving new products should really compare to the best existing alternative treatments if one exists (represented fairly e.g. with appropriate dosages) as a control group – rather than to nothing, a weak alternative, a placebo, or to a non-representative control. This is because we aren’t really interested in knowing if something is better than nothing or merely better than the worst existing alternative if effective treatments already exist. (In many, although not all, cases, doing or changing anything when in need of a solution is likely better than doing nothing or carrying on as before e.g. when thirsty, consuming snow that’s just been picked up from the ground will hydrate you compared to drinking nothing, but this is not very useful information if you have more effective alternatives available with fewer side-effect risks – in this case the risks of hypothermia and ingesting harmful bacteria. Certainly never eat yellow snow because urine is not sterile plus it won’t hydrate you despite the persistent myths!)


Correlated with some kind of intervention may be an improvement of a person’s health over time – but pertinent questions to ask include whether this person would’ve improved over time without the intervention anyway (so whether the intervention actually did anything at all, or time and one’s own natural bodily mechanisms simply healed them), whether they improved in spite of this intervention (so whether it actually made things worse than things could’ve been even though there was a net improvement), or whether they could’ve improved even more or faster with another kind of intervention (so whether some alternative intervention would’ve been even better). We cannot know these answers without an appropriate control group or groups to compare with. Woof!


Therefore you must pay great attention to exactly what the control group is. Other possible questions to ask are – should a control group receive the best alternative treatment that is available in the whole world or that is available in the country the product is concerned? There can also be a problem when outsourcing clinical trials to poorer countries too, where a control group will receive such a relatively low quality of ‘standard health care’ that almost any intervention will be better by comparison.


Failing to compare to the best existing alternative treatment is the main reason why any calorie-restricting crash or fad diet will help you to lose weight, at least in the short-term – but important questions include whether one will lose fat as opposed to just water or muscle mass, whether they come with any harmful or unpleasant side-effects, whether they are sustainable for the long-term, and whether other possible diets will result in even more fat loss over the same duration, have fewer unpleasant side-effects and/or are more physically and psychologically sustainable.


Experiments must also seek out and fully report the cons or side-effects too, not just the pros or positives – because the benefits become more limited or inconsequential if e.g. a drug can reduce blood pressure but risks causing stroke. Of course, it’s routine practice in the field of advertising to only stress the positives of one’s products whilst trying to hide their negatives, but this doesn’t give a potential consumer the full picture to make the best-informed decisions. This is one area where advertising regulations can benefit consumers.


So almost any treatment is better than doing nothing but this isn’t always useful information – the best practice is to compare to the best reasonable alternative as a control group (e.g. something close or better in availability and cost). The control group, as well as the cons or side-effects, are things we must actively look out for when assessing any science news or claim made in a product commercial.




Comment on this post by replying to this tweet:


Share this post