Lack of rigorous data: may cause irritability

7/21/2014

This past weekend, Dr. Alice Roberts published an article on the combined oral contraceptive pill in The Guardian. I was excited to see a media article on contraception by a scientist and a Professor of Public Engagement, assuming it'd be evidence-based and teach me more on writing about science for lay audiences.

The article implies a link between premenstrual dysphoric disorder (PMDD) and oral contraceptive pills, based in part on the opinion of Peter Greenhouse, a consultant in sexual health who sat next to Roberts at a conference dinner. Roberts writes:

"[T]here was something else that Greenhouse said that immediately caught my attention: for some women with more extreme cyclical mood changes, going on the pill could be disastrous."

A strong assertion! Intrigued, I kept reading, eager for a discussion of the epidemiological evidence behind this statement (which contradicts information in resources like the evidence-based, frequently-updated book, Contraceptive Technology).

To my surprise, Roberts' article did not contain any epidemiological evidence to back up this assertion. She discussed her distressing post-partum experience on the pill, and concluded that all women should be "warned about these potentially incredibly damaging side-effects of such a routinely dished-out contraceptive pill." She also claimed that mention of this possible effect is "distinctly lacking in the warning notes that come with the pills“ - which is inaccurate (see Ovrette package insert, but note that package inserts often include a laundry list of conditions that may or may not have to do with the medication).

Shouldn't we expect more critical thinking on health-related issues from scientists, especially those who engage frequently with the public?

How should we weigh anecdotal evidence?

While anecdotal evidence has a role in medicine, it "should play only a very minor and clearly defined role". Yet, anecdotal stories can feel "so powerful that they cause people to ignore contrary evidence", and confirmation bias (favoring information that confirm's one's beliefs, and selectively ignoring other information) compounds the problem. As the saying goes; the plural of anecdote is not data (which is more systematically collected and documented, and which can better consider potential confounding factors).

For someone (especially a scientist! with a large online platform!) to posit anecdotal evidence as a basis for guiding clinical practice is irresponsible. That does NOT mean that this potential association should not be considered in well-designed scientific studies.

On Twitter, someone noted that the anecdotal story was interesting, but asked for scientific studies to back up the claims. Roberts replied:

.@Shingle_beach @hollygriggspall On PMS & PMDD: http://t.co/v2Yq1TlZVI On the pill & mood changes: http://t.co/oEKU6CrEeb
— Prof Alice Roberts (@DrAliceRoberts) July 20, 2014

While I was glad she provided at least something , when I looked at the study shared, the poor quality of the study (for making any causal inference, at least) leapt off the page. This is a cross-sectional study (which can never be used for causal inference), with no control group (meaning, no ability to know if outcomes were related to the pill, or something else), which excluded women who did not report side effects (thereby inflating the reported proportion of those who did report side effects). It was also unclear whether all women reporting mood changes had reported negative mood changes (other studies have found improvements in mood as a result of hormonal contraception, so this is important).

Random, not-so-useful citation: 1, evidence-based medicine: 0.

Back in Twitterverse, (conspiracy?) theories were already flying - such as (I'm paraphrasing):
- research isn't done on this issue because scientists shrug off its importance and ignore women's voices, or
- studies do suggest links, but those studies are suppressed by Big Pharma, or
- (By Dr. Roberts herself): general practitioners won't speak out on this issue because of:

@Shingle_beach @hollygriggspall @bengoldacre [whispers] Big Pharma.
— Prof Alice Roberts (@DrAliceRoberts) July 20, 2014

Despite claims (from folks who had clearly not searched the literature) that very few pertinent studies existed because of cold-hearted scientists and Big Pharma suppression, it took about 30 minutes to compile a list of about 30 relevant studies.

OK Chelsea...so what do YOU think the data say on this issue?

Personally, I don't form hard opinions on these important issues until I've read the data (or a well-done systematic review) myself. On quick read most studies I saw did not appear to support a link between oral contraceptive pills and mood disorders/depression/etc, which aligns with information summarized in Contraceptive Technology. But the point of this blog is not to try to reach a conclusion on that issue; I'd only do that after a substantial investment of time and effort with the data (and I certainly wouldn't make suggestions for clinical practice without putting in that effort!).

My philosophy is that we should always remain open to new scientific information, and never, ever cherry-pick evidence - which is why I am such a big fan of systematic reviews. I hope to look at these data in-depth soon, but for the time being, I've started a list of the studies at the end of this post. Please feel free to add to this list* (hopefully without regard for the study conclusions; as that part comes later).

Sooooo, then, what's your point?!

That WOMEN DESERVE BETTER when it comes to having access to accurate, scientifically-based information about important health issues - especially from people who are trusted by the public. We deserve a reliance on evidence (and an openness to valid, methodologically-based critique of evidence!) from journalists, bloggers, advocates, providers, scientists...from everybody. Why? Because relying on rigorous scientific evidence is the only way that we can make true progress towards improved health for all.

Anecdotal stories are highly meaningful to each individual, should not be ignored by clinicians, and are sometimes important signals for larger issues. But scientists and others must be careful not to rely on anecdotes instead of data when looking at issues on a population level. This article (and the many others like it) risks scaring women away from a method that might otherwise be a good fit for them, which is no laughing matter when over 50% of pregnancies in the United States are unintended (and nearly 50% in England and Wales), and when unintended pregnancy may also be associated with mood disorders or complicate treatment for women with pre-existing mood disorders. Women need options; what works well for you may not work well for me, and vice versa.

Roberts concluded her article by noting that the whole thing "makes [her] just a little bit angry. Not irrationally angry. Not irritable. Just proportionately, understandably angry."

You and me both, sister. You and me both.

References:
Mood swings? It might be more than PMS… By Alice Roberts. The Guardian/The Observer. Published July 19, 2014.

Hatcher RA, Trussell J, Nelson A, Cates W, Kowal D. Contraceptive Technology, 20th Revised Edition. Atlanta, GA: Ardent Media, Inc., 2011.

*Some studies identified on this issue, presented in no particular order, and with no claim that this search was comprehensive or systematic.
http://www.ncbi.nlm.nih.gov/pubmed/24043440
http://www.ncbi.nlm.nih.gov/pubmed/23944249
http://www.ncbi.nlm.nih.gov/pubmed/22467147
http://www.ncbi.nlm.nih.gov/pubmed/17629629
http://www.ncbi.nlm.nih.gov/pubmed/17161120
http://www.ncbi.nlm.nih.gov/pubmed/11479103
http://www.ncbi.nlm.nih.gov/pubmed/3978343
http://www.ncbi.nlm.nih.gov/pubmed/22465115
http://www.ncbi.nlm.nih.gov/pubmed/23864301
http://www.ncbi.nlm.nih.gov/pubmed/24015872
http://www.ncbi.nlm.nih.gov/pubmed/23219471
http://www.ncbi.nlm.nih.gov/pubmed/23121822
http://www.ncbi.nlm.nih.gov/pubmed/23131613
http://www.ncbi.nlm.nih.gov/pubmed/22673038
http://www.ncbi.nlm.nih.gov/pubmed/23152475
http://www.ncbi.nlm.nih.gov/pubmed/22537684
http://www.ncbi.nlm.nih.gov/pubmed/22136510
http://www.ncbi.nlm.nih.gov/pubmed/23833619
http://www.ncbi.nlm.nih.gov/pubmed/22166277
http://www.ncbi.nlm.nih.gov/pubmed/21840911
http://www.ncbi.nlm.nih.gov/pubmed/18288601
http://www.ncbi.nlm.nih.gov/pubmed/18599013
http://www.ncbi.nlm.nih.gov/pubmed/18760521
http://www.ncbi.nlm.nih.gov/pubmed/19041441
http://www.ncbi.nlm.nih.gov/pubmed/17314012
http://www.ncbi.nlm.nih.gov/pubmed/17713839
http://www.ncbi.nlm.nih.gov/pubmed/16580672
http://www.ncbi.nlm.nih.gov/pubmed/16206030
http://www.ncbi.nlm.nih.gov/pubmed/14710055
http://www.ncbi.nlm.nih.gov/pubmed/12128235
http://www.ncbi.nlm.nih.gov/pubmed/22473394
http://www.ncbi.nlm.nih.gov/pubmed/19067135
http://www.ncbi.nlm.nih.gov/pubmed/17688380
http://www.ncbi.nlm.nih.gov/pubmed/17362710
http://www.ncbi.nlm.nih.gov/pubmed/16299641
http://www.ncbi.nlm.nih.gov/pubmed/15236788

5 Comments

Ruth Livingstone link

7/22/2014 12:08:23 am

Ah, the famous trial of one. How I agree with you on this. Anecdotes are wonderfully powerful things but should never be used as a substitute for proper scientific evidence.

And now for a counter-anecdote. Having been on the OCP for over 20 years, the only time I had crashing PMT was when I was off the pill, during my brief trying-to-get-pregnant interludes. Murder? Yes, I could have. Thank goodness for the sanity of controlled hormones :)

Millie Kieve link

8/5/2014 06:03:19 am

Anecdotal evidence that repeats and repeats the same story becomes evidence...many drugs later withdrawn were discovered to cause harm by the people who took them, not by those who prescribed them.

This is why we now have patient Yellow Card reporting. This has been proven in a recent study to produce signals missed by the medical profession.

We have received around 150 reports of depression, self harming and suicidal thoughts and actions linked to Dianette - a contraceptive only licensed as treatment for acne, due to higher risk of blood clots. The MHRA reviewed 100 of the reports and the manufacturer added a warning to patient information. The warnings of depression on Dianette leaflets were in terms that avoided saying the drug can CAUSE depression. Most of the women who contacted APRIL (charity) said 'the cloud lifted' when they stopped the drug. Sadly many were prescribed antidepressants and not told by their health professional prescriber the drug may be the cause and a 'drug holiday' could be the better choice for someone with symptoms that could be an adverse drug reaction (ADR).

Pharmacogenetics plays a part in the metabolising of drugs as well as the hormonal cycle and this is rarely taken into account.

Chelsea Polis

8/5/2014 07:28:09 am

Hi Millie,

Thank you for your comment. However, saying "anecdotal evidence that repeats and repeats the same story becomes evidence" indicates a lack of understanding of how scientific evidence is generated.

Anecdotes allow us to generate hypotheses, but once an issue is examined in a well-designed and conducted scientific study, the *scientific* value of those anecdotes diminish greatly (although that experience is still very important to that individual person). This is because (well done) studies minimize the effects of a number of uncontrolled confounding variables and other biases, and help us better understand whether an effect is actually causal.

The issue of oral contraceptive pills and mental health has been, as I point out in my post, assessed in a number of scientific studies. In my opinion, the next step is to do a well-done systematic review of all of the scientific literature on this issue (one which pays careful attention to the quality of the studies), to clarify what we know and what we don't yet know, based on existing data. That will tell us if additional studies are indeed needed, and if so, what they should look like. I have offered to collaborate with Alice Roberts on such a systematic review of this issue, but to date she has not contacted me to pursue this offer.

Adverse event reporting systems, like Yellow Card reporting, are incredibly useful and important in post-marketing research to detect possible signals about previously unknown issues. However, it can be difficult to draw casual inference from this information.

If the public is concerned about an issue, they should advocate for a clear understanding of the existing evidence base, and then, if appropriate, for support of additional well-conducted scientific studies. But we must rely on the findings of well-conducted studies once they are available, and it is critical not to cherry-pick evidence - which is counterproductive in achieving improved public health.

Millie Kieve link

8/5/2014 11:36:39 pm

Dear Chelsea
I advocate on behalf of patients and bereaved families, they have little energy to do this alone. I feel I am banging my head against many brick walls - the pharmaceutical giants not only will not spend money to discover drugs for orphan diseases such as Ebola, they wont spend money on studies to discredit financially lucrative drugs they are producing including many 'me too' drugs (drugs that do the same as some already available so called 'blockbuster' drugs.)

The comment about repeated complaints from the public being of no scientific relevance, discounts the fact that much of the so called scientific evidence fails in not looking for harms but only looking for the benefits of drugs. Many studies are biased in favour of the drug. Many studies with negative findings are not published. You must know about AbbVie and other large pharmaceutical manufacturers who took out an injunction to stop the European Medicines Agency from having access to the raw data from clinical trials.

I fully support your effort to do a 'scientific study' and will help you if I can. Where will the money come from for such a study? How will you obtain the data?

Chelsea Polis

8/6/2014 03:18:30 am

Hi Millie,

In response to your reply above, I am glad there are people who advocate for improved health; there are many different pathways of doing that.

I feel badly that you feel you are banging your head against a brick wall regarding pharma companies, but I do not think those points relate to the topic of my post, and am not sure why you are including it here.

I agree with you that some studies with negative findings can be harder to publish. However there are methods to attempt to statistically assess for publication bias, and some systematic reviews include funnel plots to assess for this possibility. While I am sympathetic to your concern that some studies may be biased due to financial interests, a good epidemiologist is often able to look at the methodology of a study to determine whether or not it is scientifically sound. Science is ultimately a self-correcting mechanism, though sometimes that process can be slow.

So, while your concerns about pharmaceutical interests are not unfounded, I do not think they justify blanket sentiments that science cannot be trusted, or that anecdotes are more important than well-conducted, peer-reviewed studies. That is simply not the case.

Finally, I think you misunderstand what I said; I am not proposing to conduct a scientific study including original data collection on this issue at the moment. I am proposing to collaborate with Dr. Alice Roberts - a professor of public engagement in science with a large online presence who has spoken publicly about this topic by relying on anecdote rather than discussing what current evidence does and does not tell us - on a systematic review of existing literature. A well-conducted systematic review of the evidence would be the best foundation to move forward in better understanding if any causal relationship exists between oral contraceptive pills and mental health issues.

Your comment will be posted after it is approved.