How an unethical company (Daysy) responded to retraction of their study

UPDATE 9/9/19: The company that manufactures Daysy is now the subject of a legal investigation. More information is available here: http://www.khuranapc.com/daysy-investigation. If you purchased a Daysy, please be sure to read about the investigation and determine if you would like to speak with these lawyers to share your experience. (NOTE: as of 9/20/19 that link is no longer functional. I will share additional updates as they become available.)


In May 2019, 404 days after I submitted a commentary detailing egregious flaws in data collection and analysis of a study purporting to estimate contraceptive effectiveness of the Daysy thermometer -- that study was retracted from the journal Reproductive Health. The retraction note says: “Independent post-publication peer review has confirmed that there are fundamental flaws in the methodology which mean that the conclusions are unreliable due to selection bias and the retrospective self-reporting of whether pregnancies were intentional.”
 
Science reporter Stephanie Lee covered the retraction story in Buzzfeed. Prior writings had investigated Daysy and detailed unethical behaviors by Valley Electronics (Daysy’s manufacturer) – including that the company kicked people out of online forums for asking questions, and used manipulative language in their marketing materials. In a comment on PubPeer, I also detailed unethical behaviors by Valley Electronics.
 
I’m glad to have helped to remove junk science from the literature. Hopefully, fewer people will be made vulnerable to unintended pregnancy via misleading, unsupported claims. While the retraction was very important, it does not fully solve the problem. Elsewhere, I’ve described the concerning asymmetry between the rapid spread of misinformation on social media and the long, slow process of addressing misinformation in the scientific literature, and how this mismatch can ultimately impact public health.
 
So, I have more to say.

This blog discusses how Daysy responded after the retraction of their study. Existing/potential customers should be aware of how unethical this company is. This situation also serves as a case study for the public health community.

I'll likely also post a follow up blog when possible, to discuss thoughts around the experience of the retraction process, in hopes of stimulating discussion around whether retraction guidance can be improved to better protect public health and the integrity of the scientific literature.

---

HOW DID DAYSY RESPOND TO THE RETRACTION OF THEIR FLAWED STUDY?

• Daysy recklessly rejected the Editor-in-Chief’s decision to retract
  • The retraction notice stated: “The authors do not agree with this retraction. In particular, the authors are of the opinion that the retrospective design of their underlying study is described and that the respective disadvantages are discussed transparently in this article." 
  • However, the problem is not simply that the study was retrospective. As my commentary clearly noted, there were fatal flaws in both the way the data were collected, as well as the way the data were analyzed. The authors have never addressed the numerous flaws described in my commentary.

• Daysy misrepresented their evidence base to their customers and downplayed the significance of the retraction
  • Customers on social media were rightfully concerned when they learned of the retraction. On Facebook, the company responded: “The reliability of Daysy is well studied. The retraction of the study says nothing about the reliability of Daysy’s prediction of the fertility status. Various clinical trials and observational studies display that Daysy can predict the fertility status with highest reliability. An overview of our studies can be found here: https://usa.daysy.me/reliability/”
  • However, Daysy has never been tested in a clinical trial. On the website (screenshot below), they list two studies (neither are clinical trials) – Freundl 1998, and the retracted Koch 2018 study (it is astounding that they continue to list the study even after retraction). Read below for more on the Freundl 1998 study.

Screenshot from Daysy website after the retraction.

• Daysy pointed worried customers to a prior study that they know is flawed
  • Daysy appears to simply be replacing their misleading 99.4% estimate (from Koch 2018) with a misleading 99.3% estimate (from an earlier study by Freundl in 1998) – rather than acknowledging that at present, no reliable estimate exists at all.
  • The Freundl 1998 study was not on the Daysy device, but on devices that preceded it. Like Koch 2018, it had substantial scientific flaws. For example, my commentary noted:
    • “in addition to multiple concerns pertaining to the methods of pregnancy ascertainment in Freundl 1998 (collected via a mailed, retrospective questionnaire), the calculation which generated a 0.7 perfect use Pearl Index used an incorrect denominator which included all cycles, rather than only cycles of perfect use, which would inflate perfect use effectiveness rates.”
  • Even Dr. Freundl himself told Daysy (according to a reporter I spoke with) not to use his study to support marketing their device for contraception. He publicly disavowed using poorly done retrospective surveys for such purposes in this interview: 

Quote by Dr. Freundl in a July 2018 interview with reporter Dalmeet Singh Chawla.

• Daysy perpetuated linguistic confusion to avoid getting in additional trouble with the FDA
  • Valley Electronics (Daysy's manufacturer) has a long history of ignoring FDA rules. They received two FDA warning letters in 2010 (here and here) for marketing their products as contraception without approval. They received another FDA warning letter in 2013 for the multiple concerns (those were addressed in 2014). 
  • Following publication of my commentary last year, Daysy started replacing the phrase 99% “effective” with 99% “accurate”. This does nothing to address the actual problem: no matter what word is used, their estimates (from either Koch or Freundl) are inherently meaningless and misleading, since the data and analysis these estimates were based on were flawed.
  • As demonstrated when Nicole Jardim interviewed Daysy Ambassador Holly Grigg-Spall, Daysy admits that this linguistic slight-of-hand is necessary to avoid additional enforcement action by the FDA. It is illegal to market Daysy as for contraceptive purposes in the US – and Daysy has brazenly done so anyway.

Excerpt of interview by Nicole Jardim with Daysy brand ambassador Holly Grigg-Spall

• Daysy made many other outrageous claims…far too many to list
  • One ridiculous example includes telling their customers that Koch 2018 was retracted due to concerns of an “anonymous reviewer”. I've been anything but anonymous; the retraction note itself, which cites me, makes this rather obvious.

 
I continue to be appalled by the blatant ethical violations by Daysy. Perhaps the single comment that best summarizes Daysy’s lack of integrity came from their Director of Medical Affairs, Dr. Niels van de Roemer. In 2017, I contacted him to privately and directly express concerns about their misleading marketing practices. I asked if Daysy had ever been tested in a prospective study before being marketed for contraceptive purposes. He responded that they’d be interested in such a study but that the “costs and benefits are out of all proportion”.
 
Let that sink in. A company marketing a $330 device as a contraceptive method in over 30 countries did not see it fit to spend money to conduct the appropriate studies, which would have enabled them to market the device honestly, and avoid misleading people into unintended pregnancies. I would not be surprised if somebody who experienced an unintended pregnancy while using Daysy files a class action lawsuit against Valley Electronics for misleading marketing. This company has no shame or integrity, and I am pleased to have played a major role in shining light on their deceptive and dishonest practices.

UPDATE: shortly after posting this blog, I learned that the company has posted a statement on their website about the retraction. It is available here, and repeats many of the problems described above. It's good that they note they will do a study on Daysy effectiveness; I certainly look forward to reading it closely when it is done to see if it provides adequate evidence for the device. But in the meantime, it does not appear that the company plans to stop marketing the device until it has adequate evidence in hand. Perhaps the "costs and benefits" are all out of proportion...