Links to learn more: Meta-analyses, Risk of Bias Tool, GRADE criteria, Cohen’s d
This was a great paper, probably the best I’ve read all year. This was a systematic review and meta-analysis that just came out in JAMA Internal Medicine on the question of whether meditation programs improve psychological stress and well-being. What made this stand apart from many other similar systematic reviews is the quality of the methodology and the quality of the studies that they would allow into the meta-analysis.
Let’s talk a bit about background information. Meditation is usually divided into three groups. You have your mindfulness meditation, your concentration meditation, and your transcendence meditation. For example, MBSR or mindfulness based stress reduction is a common form of meditation studied often in the literature. There are actually a lot of studies out there on meditation. The problem is that most of them are observational studies which leave us open to self-selection bias among other issues. Also, many trials which have controls use non-active controls like a wait list control. This doesn’t really control for the placebo effect, while it might control for other aspects such as fluctuations in the normal progression of the disease. These authors wanted to ask, is meditation’s effect more than just placebo? For this they needed to look at active controls.
This was a systematic review and meta-analysis. They only looked at actively controlled RCTs. They also divided their results into studies with non-specific active controls and specific active controls. The former would control for many of the placebo effect elements so if meditation was more effective than control we would know it was more than just placebo effect. These are efficacy studies. The latter compare meditation head to head with some other intervention we know works like progressive muscle relaxation for example. These are comparative effectiveness studies.
Even with these strict criteria they found 47 studies that met their criteria which included over 3500 participants. They used Cohen’s d for effect size. This is appropriate. For more information watch the Cohen’s d video. Most of the trials included were short interventions but some were longer and ranged up to 5 years in length. They used an excellent risk of bias assessment (for more on that watch the Risk of Bias Tool videos). And they used the GRADE criteria for assessing the overall quality of the estimate of effect. This is an excellent evaluation tool I think you will see more and more of. To learn more watch the GRADE video.
Now let’s get into the results. What we see here is moderate GRADE level evidence (this is an excellent level of evidence by the way – very little makes it to high) for mindfulness meditation having a statistically significant effect on anxiety, depression and pain over placebo. We see this effect for anxiety and depression both in the short term and the long term. We see that the effect size reported is 0.38 for anxiety, 0.30 for depression and 0.33 for pain. What do these numbers mean? These are Cohen’s d effect sizes. For more on understanding Cohen’s d watch this video. We tend to use cutoffs for Cohen’s d. We say anything above 0.2 is a small effect size and above 0.5 is a moderate effect size. So all of these fall into a “small effect size” category. But this could still be clinically significant as we will see in a little bit. We have to ask how they measure up to effect sizes of commonly used interventions such as anti-depressants for example.
We also see low level evidence that meditation programs improve stress and distress as well as mental health related quality of life measures. And it also found that there was no effect on positive affect outcomes like a sense of well-being. But again this is low level evidence so this is very likely to be changed based on new research.
What are some issues with this study? Well most trials were short term. This is true but again we see an effect even with these short term interventions. There was also very strict inclusion criteria which means we may be missing important studies. This is also true but remember the whole point was to get the highest quality evidence they could at the expense of inclusiveness. Of course we know the placebo effect attenuates the ‘verum’ effect but the authors wanted to know if meditation had a ‘true’ effect over that of placebo to help guide clinicians in recommending it.
So what’s the take home? Well, we have good quality evidence that meditation is effective for anxiety, depression and pain. And while the effects are “small” by the Cohen’s d cutoffs they stack up well with current interventions. For example they are on par with the effects of anti-depressants. These effects are not placebo effects. They are statistically and clinically significant. And finally, interestingly, there was no effect on positive affect like a sense of well-being. This was very interesting. The authors make an interesting point here which is that traditionally meditation was a skill that was taught over years and took a lifetime to master. All of these interventions were short term. So it is possible that long term training would positively affect a sense of well-being. However, what is so interesting is that even in a short term training program they have significant effects on anxiety, depression and pain.