4. Scientific Hoaxes and the Predatory Paradox
Past, Present, and Future

In 2015, Johannes Bohannon, along with three coauthors, published an article titled ‘Chocolate with High Cocoa Content as a Weight Loss Accelerator’ in the International Archives of Medicine (Bohannon and others 2015).1 The article reported results from a study that divided participants into three groups, with a different diet assigned to each group, and concluded that ‘Subjects of the chocolate intervention group experienced the easiest and most successful weight loss’ (p. 1). When the study was published, Bohannon and his team also produced a press release that was widely circulated and well received. Soon after the article and accompanying press release were published, the study’s findings were picked up and reported by several high-profile media outlets with dramatic headlines such as ‘Slim by Chocolate’ and ‘Why You Must Eat Chocolate Daily’ (Bohannon 2015).

‘Johannes Bohannon’ is actually a science journalist whose real name is John Bohannon. In a personal account published later, Bohannon (2015) described the article as an intentional hoax that he and his coauthors had carried out in response to a request from a German film crew who was making a documentary on the ‘junk-science diet industry’. To implement the hoax, Bohannon and his coauthors created an ‘Institute of Diet and Health’ that existed only as a website, and he assumed the name ‘Johannes Bohannon’ as lead author of the study. Bohannon (2015) narrates the rest of the events as follows:

Other than those fibs, the study was 100 percent authentic. My colleagues and I recruited actual human subjects in Germany. We ran an actual clinical trial, with subjects randomly assigned to different diet regimes. And the statistically significant benefits of chocolate that we reported are based on the actual data. It was, in fact, a fairly typical study for the field of diet research. Which is to say: It was terrible science. The results are meaningless, and the health claims that the media blasted out to millions of people around the world are utterly unfounded. (para. 3)

As Bohannon’s account makes clear, his team’s research was conducted and accurately reported in the article, so there is nothing false about the publication in that sense. However, the study enrolled only fifteen participants and, thus, the dramatic conclusions that the article reported were not statistically sound, and nowhere in the article was the sample size mentioned. Another problem was that the authors had not received Institutional Review Board approval to recruit participants and conduct their research (Schwitzer 2015). Such approval would usually be indicated in the text of the manuscript that is submitted to a journal.

These omissions reflect basic problems in the quality of the science, and they would have been detected if the article had undergone a legitimate peer-review process. The article had not undergone peer review, but rather, according to Bohannon (2015), the journal’s editors accepted the version that was submitted, without requesting any revisions, and they published it as soon as they received the publication fee of 600 Euros from the authors. As Bohannon explains, he had submitted the piece to International Archives of Medicine, as well as nineteen other journals, all of which he believed to be, in his words, ‘fake’. His belief was proven correct when the ‘paper was accepted for publication by multiple journals within 24 hours’ (Bohannon 2015: para. 2). By Bohannon’s own account, the ‘Chocolate with High Cocoa Content’ hoax study was intended to expose weaknesses in our system of reporting scientific findings to expert audiences as well as the wider public:

If a study doesn’t even list how many people took part in it, or makes a bold diet claim that’s ‘statistically significant’ but doesn’t say how big the effect size is, you should wonder why. But for the most part, we don’t. Which is a pity, because journalists are becoming the de facto peer review system. And when we fail, the world is awash in junk science. (Bohannon 2015: para. 5)

In short, Bohannon and others’ (2015) hoax tested the system, and the system failed. The journal editors who accepted and published the article, as well as the science journalists who reported uncritically on the study, looked bad as a result.

Scientific hoaxes such as Bohannon’s have been studied by communication scholars for their rhetorical characteristics and effects. For example, Finneman and Thomas (2018) identify ‘central characteristics of hoaxing’ as ‘untruths framed as truths’, carried out by ‘someone in a position of power over a public and influence via entertainment’ (p. 353), and Reilly (2020) defines hoaxes as ‘ambiguous forms of communication that channel deception, humour, and mischief in the targeting of victims and the approbation of audiences’ (p. 266–67). Along similar lines, scientific hoaxes have been characterized as opportunistic rhetorical acts: ‘as reactions to and perpetuations of a particular kairos — an opportunity to speak up’ (Walsh 2006: 3). Secor and Walsh (2004) offer a more comprehensive definition, suggesting that a hoax is ‘characterized by a constellation of rhetorical features: it involves the production of discourse, a perpetrator with intentions, an audience that is first engaged and then duped’ (p. 71).

In this chapter, we take a broader view of scientific hoaxes such as Bohannon’s. Specifically, through a close examination of Bohannon’s ‘Chocolate with High Cocoa Content’ hoax in the context of other notable scientific hoaxes, we explore the various types of weakness that such hoaxes can expose in the larger information ecosystem of scholarly publishing. As we argue, Bohannon and his colleagues’ ‘Chocolate with High Cocoa Content’ hoax is interesting not only for the rhetorical effect of exposing one specific journal as predatory, or for drawing attention to an isolated instance of poor media reporting practice, but because of the flaws that it exposes in our entire system of scholarly communication. Thus, scientific hoaxes further complicate any neat distinction between journals that are predatory and those that are not. Hoaxes have, in some cases, exposed specific journals as predatory. But in other cases, they have had effects beyond those that the author anticipated, exposing major weaknesses or fraudulent practices not only at journals or publishers suspected to be predatory but also at the most prestigious and well-respected journals. More importantly, publishing hoaxes have unintentionally exposed weaknesses in the mechanisms that we have long relied on to ensure research quality. For example, hoaxes have exposed flaws in even the best journals’ peer-review systems, and when hoax articles continue to be cited in subsequent literature — sometimes even after retraction — they lead us to question our habit of relying on citation counts as a measure of research quality.

A Closer Look at the ‘Chocolate with High Cocoa Content’ Hoax

Bohannon and his colleagues’ ‘Chocolate with High Cocoa Content’ article was retracted shortly after it was published, and the editors of International Archives of Medicine published a retraction notice dated 10 June 2015 (Editorial Office 2015). The editors’ decision to retract this article ostensibly served to correct the scientific record and prevent the erroneous data reported in the published study from being circulated in subsequent literature. This manner of correcting the scientific record is an important purpose of retractions, as defined by the Committee on Publication Ethics (COPE):

Retraction is a mechanism for correcting the literature and alerting readers to publications that contain such seriously flawed or erroneous data that their findings and conclusions cannot be relied upon. Unreliable data may result from honest error or from research misconduct. (Wager and others 2009: 202)

Although the editors’ retraction of Bohannon’s hoax article seems to correct the scientific record, it does not entirely align with these COPE guidelines. For example, the editors’ language in the retraction notice is confusing. The retraction notice published on the journal’s website states that the manuscript had been published ‘accidentally’ and that it ‘was finally rejected and never published as such.’ Furthermore, the journal editors did not fully follow the COPE guidelines, which state that ‘Notices of retraction should… be linked to the retracted article wherever possible (i.e., in all electronic versions)’ (Wager and others 2009: 201). It is not clear why, but in this case, the retracted article is no longer available at the journal’s website. In fact, making the situation even more confusing, the Digital Object Identifier (DOI) for the original article now links to a study called ‘The Comparison of Resilience and Spirituality in Addicted and Non-Addicted Women’ (Ramezani and others 2015). This is extremely problematic, given that the sole purpose of a DOI is to provide a unique and persistent identifier that links to a published article, and it is the publisher’s responsibility to maintain the integrity of that link (‘How the ‘Digital Object Identifier’ Works’ 2001). In short, this retraction seems to be intended as a ‘mechanism for correcting the literature’, as the COPE guidelines advise, but the error that needed to be corrected was not an error made by researchers; rather, it was, according to this retraction notice, an error made by the publisher.

The series of events preceding the article’s retraction is also subject to dispute. According to a 28 May 2015 Retraction Watch post, the hoax article was available on the journal’s website on the morning of 28 May but appears to have been retracted later that same day (Schwitzer 2015). A 28 May 2015 Facebook post (Perez 2015) from someone who appears to be one of the journal’s editors also indicates the article was retracted on that same date. By contrast, the retraction notice that appears on the journal’s website (Editorial Office 2015) indicates 10 June 2015 as the publication date of the notice; it is not clear why the retraction notice would be published almost two weeks after the article was retracted on 28 May. In addition to this confusion about the date of retraction, these different sources also offer varying accounts of the series of events that preceded retraction. Perez’s Facebook post suggests that the article was never accepted by the journal — that it was published ‘by mistake’, based on a misunderstanding of a managing editor who had been copied on Bohannon’s initial email submission. The Facebook post even claims that sometime after the article was accidentally published due to this miscommunication, ‘the manuscript was rejected by the editorial board.’ However, Bohannon offers a different narrative of events. He quotes an email acceptance notice that is also quoted by Retraction Watch. In this personal email correspondence that Bohannon had apparently shared with Retraction Watch, the journal editors praised Bohannon and his colleagues’ article for its high quality and potential contributions:

I’m contacting to let you know your manuscript ‘Chocolate with High Cocoa Content as a Weight-Loss Accelerator’ has been pointed by our editors as an outstanding manuscript and could be accepted directly in our premier journal *International Archives of Medicine.* (McCook 2015)2

Adding even more confusion, Perez’s Facebook post was edited on 10 June 2015, almost two weeks after its initial 28 May publication, to add the second paragraph, which seems to be an attempt to account for the journal’s problematic set of actions. This 10 June edit also contributes to a discrepancy about the length of time that the fake article was available on the journal’s website. For instance, Perez’s (2015) Facebook post suggests it was online for only a few days (from Thursday one week to Monday of the next week, with no specific dates given). However, Retraction Watch indicates that the article was accepted soon after Bohannon submitted it in early March and was not retracted until 28 May 2015.

It may be easy to look at this situation in retrospect and conclude that so many problems are apparent in this article that no one could ever take it seriously, and because it was retracted and is no longer available, its damage was minimal. In fact, any science journalist or academic researcher who took time to assess the International Archives of Medicine’s online presence would have had good reason to question the credibility of this journal. For instance, a search in PubMed Central’s journal list reveals that the journal was indexed from 2008 until 2014 but is ‘no longer participating’ (‘PMC Journal List’ [n.d.]). Articles published in the journal are full of grammatical errors, indicating a lack of attention to copy editing, at the very least, and probably indicating an absence of peer review as well. The journal publishes a lot of articles on a wide range of topics. For example, Volume 14, published in 2014, includes fifty articles. These are indexed in PubMed Central as the entire volume, without any indication of issue numbers. Categories include Case Reports, Hypothesis, Original Research, Reviews, and Short Reports. The journal website linked from its current publisher, iMedicalPublisher.com, describes the journal as ‘The new megajournal on all areas of medicine’, and says the journal is ‘Really international’ (‘International Archives of Medicine’ [n.d.]). These obvious cues should be enough to raise questions about the journal’s legitimacy, even aside from the glaring scientific flaws already noted in the hoax article itself. However, none of these factors, and even the fact that the article was retracted, were obvious enough to stop the article from being cited in subsequent literature. In fact, when this chapter was written, the article had been cited twenty-eight times, according to Google Scholar.3 Although some of these citations refer to the article as an example of a publication in a predatory journal, other citations are in legitimate journal articles that genuinely cite the scientific findings reported in the article. Specifically, as shown in Table 4.1, almost one-third of these citations (eight out of twenty-eight) cite the article to support a scientific claim.

Table 4.1 Citations of Bohannon Hoax Article. © STEPP Research Team.

None of the eight publications that cite the Bohannon hoax article to support scientific claims acknowledges that the publication was retracted, even though all these publications appeared well after the 10 June 2015 publication date of the International Archives of Medicine’s retraction notice. Instead, these eight publications cite the Bohannon hoax article as if it were any other legitimate peer-reviewed scientific text. For example, a 2017 review article (Ramos and others 2017) cites the Bohannon study, along with several other sources, to support the claim that appears in the following sentence: ‘In addition, despite the fact that cocoa products commercially available are frequently high-caloric foodstuffs, they have been reported to have a similar [anti-obesity] effect in humans’ (p. 5). The note that cites the Bohannon and others article includes a CrossRef link that links to the DOI, which now links to a different article, as noted above. This 2017 review article, interestingly, is published in the journal Antioxidants, which is published by MDPI. The Bohannon hoax article is also cited in another article (Rodríguez-Lagunas and others 2019) published in the MDPI journal Molecules. This article cites the Bohannon hoax article to support the claim that ‘anti-obesity actions of cocoa have been reported’ (p. 7). Oddly, the Bohannon hoax article is also cited at the end of the following sentence, which does not seem to have anything to do with the findings reported by Bohannon and his coauthors: ‘Regarding the health questionnaire, the university students reported, logically, a good health status, far away from suffering chronic diseases involving neoplasm and cardiovascular diseases, the main causes of death in the Spanish population’ (p. 8). These citations of the Bohannon hoax article, well after it was retracted and without any acknowledgement that it was a hoax, raise some serious concerns about the quality of content published in these two MDPI journals. It appears, at least in these two cases, that authors are citing literature without paying much attention to its quality — or maybe without even reading the texts they are citing — and peer reviewers are not catching these sloppy citation practices.

Another citation of Bohannon and others appears in a 2019 article in the Journal of Agricultural and Food Chemistry (Camps-Bossacoma and others 2019). This article, ‘Role of Theobromine in Cocoa’s Metabolic Properties in Healthy Rats’, links to the Bohannon hoax article through Google Scholar instead of CrossRef, but like the other citations, this one cites the study without any acknowledgement that it is fraudulent or that it has been retracted. Specifically, the Bohannon hoax article is cited in the following sentence: ‘Cocoa effects on body weight increase have already been reported both in animal models and humans, […] and it has been postulated that cocoa is a weight loss accelerator’ (p. 3611). Unlike the other two journals, published by MDPI, this journal is published by the American Chemical Society, described on its website as follows: ‘As a non-profit scientific organization with more than 140 years’ experience, we are a champion for chemistry, its practitioners and our global community of members’ (‘About ACS’ [n.d.]). This organizational affiliation may seem to grant legitimacy to the journal. Again, though, it is troubling that the Bohannon hoax study is cited uncritically, as a source within the authoritative scientific record, to support a statement about cocoa’s weight-loss properties. This must lead us to question the overall legitimacy of the article and the soundness of the science that it reports.

Persistent citation of retracted articles is not uncommon. In fact, Retraction Watch keeps a list of retracted articles that have received the highest number of citations (‘Top 10 Most Highly Cited Retracted Papers’ [n.d.]). The top article on the list was published in 2013 in New England Journal of Medicine and retracted in 2018 (Estruch and others 2013). The Retraction Watch site reports that the article has 2735 citations. Next on their list is a Lancet article published in 1998 and retracted in 2018; Retraction Watch indicates 1509 citations of this article. A recent study exposes the serious nature of this problem by examining a high-profile retraction case that involved the work of Scott S. Reuben (Bornemann-Cimenti and others 2016). Reuben was a well-established medical researcher who studied pain medicine. In 2009, it was discovered that he had fabricated data in many of his published studies, which led journals to retract twenty-five of his published articles. Bornemann-Cimenti and others (2016) track the extent to which Reuben’s articles continued to accrue citations for many years after they were retracted. Although some of the citations note that the article has been retracted, many do not (Peng and others 2022).

According to Hyland (1999), ‘the attribution of propositional content to another source’ (p. 341) is a fundamental rhetorical feature of academic writing. Problematic citation practices include the citation of retracted articles, before or after the article is retracted (Bolland and others 2022); citation, knowingly or unknowingly, of articles published in predatory journals (Akça and Akbulut 2021); or citing an article without having read it (Akça and Akbulut 2021; Wetterer 2006). Offering an especially concerning example, Wetterer tells the story of an outright falsehood that was perpetuated for many years among ant scientists, about the extinction of a particular ant species that allegedly occurred on the Atlantic Islands of Madeira due to the invasion of an exotic species. According to Wetterer’s analysis, this myth was perpetuated and became accepted as scientific fact reported in peer-reviewed journal articles because of practices that he labels ‘quotation error and citation copying’ (p. 352). In short, scientists were repeating the myth that this species had been made extinct, based on a misinterpretation of an 1898 article (Stoll 1898) that ended up being widely cited as the source of this information. Wetterer speculates that Stoll’s report was misinterpreted because his article was published in German, and the person who initially cited it mistranslated his findings; thus, the erroneous belief that Stoll’s report was based on first-hand evidence became widely cited in subsequent literature, even though, according to Wetterer, Stoll never claimed to have access to first-hand evidence.

Thus, like many of the other phenomena explored in this book, the phenomenon of sloppy citation practices is nothing new; it is well documented in science, and the phenomenon of poor citation practices has been documented and studied well before the relatively recent development and proliferation of predatory journals. In the case of the Bohannon hoax, articles that cited it later are articles that were presumably subject to actual peer review, and they link directly to the Bohannon and others’ study, which was retracted shortly after it was published, without acknowledging that it was retracted. An especially puzzling aspect of this case is that it is not clear how the authors of these articles would have located a correct version of Bohannon and others’ study, given that its DOI — which is supposed to provide a unique and persistent link to a digital publication — does not even link correctly to the retracted article.

In short, the Bohannon ‘Chocolate with High Cocoa Content’ hoax may have been designed to expose substandard publishing practices at predatory journals, but it ultimately exposed a much larger, more complex set of problems. Everything was broken in this situation, including the various checks and balances that are expected to detect bad science and prevent it from being shared with the public. The science reported in this article was conducted, but it was conducted poorly by anyone’s standards. A thorough peer review would have detected the obvious flaws in the research and would have prevented the paper from being published. A proper retraction, meaning that the retraction notice would appear along with the original article at the original DOI, and a more careful vetting of the literature by scientists after the article was published, may have prevented the retracted article from subsequent citations. However, in addition to all these scientific problems, the science journalists who reported on the findings also should have done due diligence by asking questions to relevant experts about the science behind the article.

Retractions of published articles are not uncommon, even at journals that are not predatory. In fact, the frequency of article retractions is on the rise, and even the most prestigious journals in their respective disciplines are not immune (Bornemann-Cimenti and others 2016). The extent to which the citation of a retracted article will alter the course of science depends on the nature of the evidence that is cited. If the evidence is in line with most other evidence, then the effect may not be too great, but if it is the only study that is cited to support a claim that departs from other available evidence, then this is obviously problematic. For all these reasons, retractions of published articles, whether they are published in a ‘predatory’ journal or not, are something that researchers and other stakeholders in scholarly publishing need to be aware of.

Bohannon’s Other Hoax: The Science Sting

Bohannon’s chocolate diet hoax followed closely after a larger sting operation he had carried out for the journal Science in 2013 (Vergano 2013b). In this sting operation, Bohannon acted on behalf of Science, producing a fake research article on lichen as a supposed cure for cancer. Unlike the cocoa diet article, which reported results of research that the authors had conducted, the lichen article’s contents were entirely fabricated. Bohannon used false author names, submitting the paper to several journals and varying the author’s name from one journal to the next, and he reported conclusions that were completely unfounded. He even suggested that the new drug was immediately available to patients, without conducting any clinical trials to ensure its safety. Another important distinction between the two hoax attempts is that Bohannon never moved forward with publishing the lichen cancer article. Rather, in the Science sting operation, whenever Bohannon received an acceptance notice from a journal, he withdrew the article and informed the editors he had discovered flaws in the research (Lowe 2013). Thus, Bohannon’s fake cancer article never became part of scientific record, unlike the cocoa diet study.

This 2013 sting was a much larger operation than the 2015 cocoa diet hoax. Also, whereas the cocoa diet hoax article specifically targeted journals that Bohannon had reason to believe were predatory, the Science sting operation targeted open access journals much more broadly. In fact, it is clear in some of the discourse surrounding this sting operation that the operation, sponsored by Science, was explicitly intended to expose weaknesses in open access publishing. For example, in the article that Bohannon (2013) published after the hoax, he says the following:

Over the past 10 months, I have submitted 304 versions of the wonder drug paper to open-access journals. More than half of the journals accepted the paper, failing to notice its fatal flaws. Beyond that headline result, the data from this sting operation reveal the contours of an emerging Wild West in academic publishing. (p. 60)

This language is noteworthy for the extent to which it vilifies open access publishing, identifying open access publishing, broadly defined, as the target for the sting operation, failing to make any distinction between those journals that are legitimate and those that are not, and failing to direct any attention to the question of the quality of a journal’s peer-review procedures, aside from whether the journal adopts open access publishing practices. Bohannon then proceeds to make some inaccurate claims about open access publishing, such as the following statements:

Most of the players are murky. The identity and location of the journals’ editors, as well as the financial workings of their publishers, are often purposefully obscured. (p. 60)

As the sting operation unfolded, Bohannon’s generalizations about open access publishing did not hold up. In fact, the sting operation resulted in a wide range of responses from the target journals, working against the prospect of making any widespread generalizations about the quality of open access journals. Specifically, Bohannon submitted the lichen article to 304 open access journals, making only slight changes to the versions that he submitted to each journal. Of these 304 submissions, 255 journals agreed to review the article, and 157 accepted it for publication (Lowe 2013). Among these journals, it turned out that some of the high-profile publishers that publish exclusively open access content proved themselves to be most resistant to the sting attempt. For example, two Hindawi journals rejected the spoof article, whereas two Elsevier journals accepted it (Bohannon 2013). Hindawi is a well-known open access publisher whose mission is clearly stated on their website: ‘Maximizing the impact of research through openness. Because science works best when research is open’ (‘Open Access Publishing for the Scientific Community’ [n.d.]: para. 1). Several of Hindawi’s journals were at one time included on Beall’s list (Berger and Cirasella 2015). However, Hindawi journals were later removed from Beall’s list, with the acknowledgment that they had improved their practices since the time they were initially added to the list. Beall is quoted in an interview that occurred later as saying ‘I reanalyzed [Hindawi] and determined that it did not belong on the list […] It was always a borderline case’ (Butler 2013: 434).

Only one journal that was targeted in this sting operation — PLOS ONE — rejected the paper outright because of ethical concerns. However, even though many open access journals came out looking better than expected, Bohannon (2013) ultimately concluded that Beall’s list is a better predictor of flawed procedures than the Directory of Open Access Journals (DOAJ):

The results show that Beall is good at spotting publishers with poor quality control: For the publishers on his list that completed the review process, 82% accepted the paper. Of course that also means that almost one in five on his list did the right thing — at least with my submission. A bigger surprise is that for DOAJ publishers that completed the review process, 45% accepted the bogus paper. ‘I find it hard to believe,’ says Bjørnshauge, the DOAJ founder. ‘We have been working with the community to draft new tighter criteria for inclusion.’ Beall, meanwhile, notes that in the year since this sting began, ‘the number of predatory publishers and predatory journals has continued to escalate at a rapid pace.’ (p. 64)

This disparity between Beall’s list and DOAJ is significant in context of discussions about the ‘watchlist’ versus ‘safelist’ approach (Koerber and others 2020), in so far as it suggests that Beall’s watchlist was a more accurate predictor of suspect journals than DOAJ’s safelist. But it also works against the possibility of making broad generalizations about any category of journal in that neither method of categorizing journals proved to be airtight in this situation.

This series of events may explain why at the end of his commentary, in the ‘Coda’ section, Bohannon (2013) adopts a more generous stance toward open access:

From the start of this sting, I have conferred with a small group of scientists who care deeply about open access. Some say that the open-access model itself is not to blame for the poor quality control revealed by Science’s investigation. If I had targeted traditional, subscription-based journals, Roos told me, ‘I strongly suspect you would get the same result.’ But open access has multiplied that underclass of journals, and the number of papers they publish. (p. 65)

This outcome of Bohannon’s Science sting operation, once again, exposes the problem of making any simple distinction between predatory and non-predatory publishers.

Aftermath of the Bohannon–Science Sting Operation: Reform, Fallout, and Backlash

It could be argued that Bohannon’s Science sting operation had some positive impact. For instance, Bohannon notes in a published interview that both the DOAJ and the Open Access Scholarly Publishers Association (OASPA) made changes to their databases after the Bohannon sting operation (Davis 2013; see also Teixeira da Silva and others 2018). However, the sting operation also led to significant backlash against Science, leading to questions about the journal’s editorial practices. For example, in a blog post published on 3 October 2013, Michael Eisen responded to Bohannon’s sting operation by drawing attention to a 2010 Science article that had previously sparked controversy and criticism of traditional peer review and publishing processes (Eisen 2013). In this situation, which has come to be known as the ‘#arseniclife’ case, it was not so much an intentional hoax as a situation that raises questions about the changing nature of the relationship between science journalism and traditional peer-reviewed journals.

The ‘#arseniclife’ case began with a 29 November 2010 press release from the National Aeronautics and Space Administration (NASA) that captured international attention with its announcement of a news conference scheduled for 2 December ‘to discuss an astrobiology finding that will impact the search for evidence of extraterrestrial life’ (Brown and Weselby 2010: 1). The ‘astrobiology finding’ to which the press release referred was from a research team that had purportedly discovered the existence of an organism that could use arsenic instead of phosphorus as a foundation for its DNA. If this had been true, it would have been the only organism of its kind and could have suggested that the organism came from a different planet other than Earth (Rosen 2012). On 2 December 2010, the day of the NASA news conference, a news story published on the NASA website (Phillips 2010) made even bolder claims about the study:

NASA-supported researchers have discovered the first known microorganism on Earth able to thrive and reproduce using the toxic chemical arsenic. The microorganism, which lives in California’s Mono Lake, substitutes arsenic for phosphorus in the backbone of its DNA and other cellular components. (para. 1)

This claim later turned out to be false and based on poor-quality science; the microorganism did require phosphorus, but the science was conducted in a way that did not make that apparent. However, NASA’s press release was written and taken up in a way that suggested their scientists had discovered an organism that would cause us to expand the scientific definition of life in a way that would make the existence of extraterrestrial life much more likely. The Science publication (Wolfe-Simon and others 2011) was later refuted, as it was found in replication studies that the organism did need tiny amounts of phosphorus and that the arsenic was not actually incorporated into its genetic structure as the initial research team had reported (Vergano 2012).

In short, this case showed how even the most high-profile journals can publish science of questionable quality — ironically, in this case, it was the initial excitement and desire to be the first to break the news that led Science to publish these NASA findings too quickly, without good-quality peer review. In fact, a USA Today investigation later revealed that Science editors were so eager to be the ones to publish this breaking news that they purposely sent the article to peer reviewers whom they knew would not be too critical (Vergano 2013a). In other words, the desire for profit and the need to keep feeding the high-impact machine led to the same outcomes — publishing of substandard science — that critics love to point out in open access or so-called predatory journals.

One of the major problems in this situation, as critics observed, was that by 2 December 2010, when Science published online the actual peer-reviewed papers that had been the subject of all this hype, Internet chatter had largely debunked the paper’s findings, so that when the paper was published in its final version after the initial embargo period, it was old news. In Rosen’s (2012) opinion, this case should cause us to question the value of tried-and-true practices of withholding scientific evidence under journal embargoes and so on. Her stance is that allowing debate to play out freely as soon as evidence is available, in the manner that is allowed by online media, can result in better-quality science than the slower mechanism of traditional peer review and withholding findings until they make their way into print. As she says,

Perhaps this is clear, but the reason to have ‘proper’ methods of engagement is because they ostensibly will produce better science and better science journalism. But in this one case study, we can see how the opposite is true: The ‘proper’ paths of engagement produced uninformed hype, poor science, and kept the sources – both human and paper – away from a conversation that was simmering with genuine enthusiasm and curiosity. The best science – and the best science writing – could come when we allow those natural levels of interest to have a field day with the research and researchers that are out there. (para. 21)

In short, the ‘#arseniclife’ case, as it came to be known, exposed weaknesses in the kind of journal whose integrity is not typically called into question, proving that a high-impact subscription journal such as Science can be just as guilty of publishing substandard science as open access journals. As some have suggested, the important takeaway from the #arseniclife case was that, rather than focus the spotlight on open access publishing, or predatory journals, we should be taking a critical look at peer review, which is not always acting as the gatekeeper that we expect it to be, even at the most prestigious journals. As Eisen (2013) states in a commentary that addresses both the Bohannon sting operation and the #arseniclife scandal,

First, and foremost, we need to get past the antiquated idea that the singular act of publication — or publication in a particular journal — should signal for all eternity that a paper is valid, let alone important. Even when people take peer review seriously, it is still just represents the views of 2 or 3 people at a fixed point in time. To invest the judgment of these people with so much meaning is nuts. And its far worse when the process is distorted — as it so often is — by the desire to publish sexy papers, or to publish more papers, or because the wrong reviewers were selected, or because they were just too busy to do a good job. If we had, instead, a system where the review process was transparent and persisted for the useful life of a work (as I’ve written about previously), none of the flaws exposed in Bohannon’s piece would matter. (para. 14)

As Eisen’s commentary suggests, it is worth noting that the ‘legitimate’ scientific article in the #arseniclife scandal, which was published at a top-tier journal, ultimately led to some of the same conclusions about the broken nature of our scientific publishing system as the intentional hoax and sting operations discussed above.

Concluding Thoughts and Some Practical Advice

Scientific hoaxes have a long history, originating long before the relatively recent concern about predatory journals. For example, well-known examples in the nineteenth century include Edgar Allan Poe’s 1844 ‘balloon hoax’, in which he published a hoax article in The Sun newspaper about someone making a three-day balloon trip across the Atlantic in a gas balloon (Poe 1844), and Richard Adam Locke’s 1835 ‘Moon Hoax’, a six-day series in The Sun which made fabricated scientific claims about the discovery of life on the moon (Vida 2012). In more recent times, the 1996 Sokal hoax and a 2018 spin-off that has come to be known as the ‘Sokal Squared’ hoax were both published by high-profile journals that would not be considered predatory (Mounk 2018). These hoaxes were targeting a different aspect of academic publishing than Bohannon: their target was the tendency toward over-reliance on jargon, even in legitimate peer-reviewed journals. The original Sokal hoax, in particular, occurred well before the term ‘predatory’ was coined to refer to suspicious journals and publishing practices.

Even while acknowledging this long history, we might consider a different perspective on scientific hoaxes in the present moment given that, as explained in a recent Aspen Institute report, our information ecosystem in the twenty-first-century United States perpetuates a state of ‘information disorder’ that is especially harmful to those who live in ‘marginalized communities’ (‘Final Report’ 2021: 12). This harm is attributed to several factors, including the ‘news deserts’ in which 65 million Americans are reported to live (Simpson 2019) as well as the often-overlooked reality that many marginalized communities have valid reasons to mistrust expert information, based on the long history of scientific research that has abused them (Jaiswal and others 2020). An especially egregious example is a recent Senate report showing that black people were targeted more than other groups by Russians interfering with the 2016 election (Mak 2019).

Along these lines, some critics have questioned the ethics of scientific hoaxes and stings such as those enacted by Bohannon and Sokal. The criticism of Bohannon foregrounds multiple concerns: the fact that Bohannon’s hoaxes wasted the time and resources of multiple journals, without issuing an apology, and the fact that he violated a basic principle of scholarly publishing ethics by submitting the same piece to multiple journals at that same time (Al-Khatib and Teixeira da Silva 2016; Teixeira da Silva and Al-Khatib 2016). By contrast, Sokal’s hoax is criticized more generally for its deception and its potential to create mistrust of the academic enterprise (Fish 2000).

Even while acknowledging this long history of scientific hoaxes, we must also acknowledge that information and misinformation are able to spread more quickly now than ever before. In this context, if an idea resonates with something that audiences want to hear, it is going to spread even more quickly, regardless of the quality of the information. These trends are also related to other phenomena in scholarly communication such as the bias against publishing negative findings. The stakes have never been higher, and as academic researchers, or other professionals in the scholarly enterprise, we have a crucial role to play in safeguarding the knowledge produced by this enterprise.

Misinformation and disinformation are commonly depicted as phenomena that should concern us regarding public communication of scientific information (e.g., ‘Final Report’ 2021). Thus, when we hear about ‘fake news’, for example, we are usually talking about public audiences who seek out information that aligns with their beliefs, regardless of accuracy, or who do not possess adequate critical thinking skills to know the difference between accurate and inaccurate information. Far less attention has been paid to the potential for misinformation, disinformation, or ‘fake news’ in the context of expert discourse. As this chapter has demonstrated, this potential is significant, and it is not only something that we see in predatory journals but is occurring with some frequency in high-profile, seemingly credible, peer-reviewed journals produced by the major publishing companies. In all these ways, publishing hoaxes raise our awareness of the extent to which predatory journals should be understood as part of a complex ecology; these journals take advantage of the very same weaknesses that are often exposed by scientific hoaxes. Thus, publishing hoaxes have made an important contribution to the ‘predatory paradox’ that is this book’s focus.

Practical Applications

Several mechanisms have been developed to guard against the various forms of scholarly misinformation addressed in this chapter. These mechanisms include the DOI to provide a unique identifier for published research texts and the ‘Open Researcher and Contributor ID’ (ORCID) to provide a unique identifier for authors and contributors to published research. Becoming familiar with these mechanisms is one strategy that researchers and other stakeholders can use to fortify themselves against scholarly misinformation and fraud (see Table 4.2). Although none of these systems is foolproof, understanding how they operate, what they mean, and their limitations, will make authors and other stakeholders in scientific research better informed and equipped to detect fake publications, whether they are published in journals considered predatory or not. The suggested activities below are designed to introduce these systems and provide some practice in using them.

Key Takeaways

• Scientific hoaxes such as Bohannon’s ‘Chocolate’ hoax have exposed weaknesses in the scholarly publishing ecosystem.
• Scientific hoaxes have served to expose fraudulent practices in both predatory and legitimate journals.
• Article retractions are supposed to correct the scientific record, but often do not, for a variety of reasons.
• Retracted articles often continue to be cited well after their retractions.
• Peer review, which should be the gatekeeping mechanism in scholarly publishing, often fails to detect hoaxes, or is not conducted at all by journals claiming to use peer review.
• Some critics question the ethics behind scientific hoaxes.
• Mechanisms such as DOIs and ORCID identifiers are intended to provide unique identifiers for authors and published research to help prevent unethical practices.

Discussion Questions

1. Look back at some of the news articles that cited Bohannon’s ‘Chocolate’ study. At a casual glance, is there anything in the news articles that might make you suspicious about the science that was presented? Discuss why or why not.
2. What are some benefits that have come from Bohannon’s ‘Chocolate’ hoax?
3. What are some of the ethical concerns related to scientific hoaxes?
4. Have you ever come across a retracted article while conducting research? What did you do? Would you still use it?

Activities

1. Johannes Bohannon was listed as the first author of the ‘Chocolate with High Cocoa Content’ hoax article. Search for ‘Johannes Bohannon’ in ORCID (https://orcid.org/). (Note: you can click on ‘advanced search’ to narrow your results by adding an institutional affiliation).
   1. Try searching for some of the other coauthors as well. What results do you find? How could a search like this have been helpful to each of the stakeholder groups who fell victim to the ‘Chocolate with High Cocoa Content’ hoax?
   2. Stakeholders who were addressed in this chapter include academics who cited the article and science journalists who reported on it. Can you think of other stakeholders who were potentially impacted by this hoax?
2. Try searching in ORCID (https://orcid.org/) for someone who is a well-known author in your discipline.
   1. Is this author indexed in ORCID? If not, try searching for names of other well-known authors in your discipline until you find someone who is indexed in ORCID.
   2. What types of information are you able to obtain from these authors’ ORCID profiles?
   3. How might this type of information be helpful in assessing the legitimacy of a published text?
3. The DOI assigned to Bohannon and others’ ‘Chocolate’ hoax article when it was initially published is 10.3823/1654.
   1. Try searching for this DOI in the DOI search tool (doi.org). What do you find?
   2. How could this tool have helped the various stakeholders who fell victim to the ‘Chocolate with High Cocoa Content’ hoax?
4. Table 4.1 lists articles that had cited Bohannon and others’ hoax article according to a Google Scholar search carried out on 11 May 2022. Take a close look at each of these citations.
   1. Do these citations include a DOI for Bohannon and others’ article, and if so, is it the same DOI listed in Question 3 (10.3823/1654)?
   2. Do you see evidence that the authors citing this article could have used any of the tools referenced here to screen out this bad citation and avoid citing it?
5. The Open Ophthamology Journal uses Crossmark for all its recent articles.
   1. Click on the Crossmark icon for a recent article in the journal. Here is one example of a recent article that uses Crossmark, but you will find several others to choose from: https://openophthalmologyjournal.com/VOLUME/14/PAGE/82/FULLTEXT/
   2. What information does this link provide about the article?
   3. How does Crossmark assist you in assessing the overall quality of this journal? Do you see any other cues on this journal’s website that give you an indication of this journal’s quality?

References

‘About ACS’. [n.d.]. American Chemical Society, https://www.acs.org/content/acs/en/about.html?sc=180808_GlobalFooter_od

Akça, Sümeyye and Akbulut, Müge. 2021. ‘Are Predatory Journals Contaminating Science? An Analysis on the Cabells’ Predatory Report’, Journal of Academic Librarianship, 47.4: 1–11, https://doi.org/10.1016/J.ACALIB.2021.102366

Alkalaj, Miso. 2017. ‘Crisis of Confidence in Science, 2: Lies, Damned Lies, and Statistics’, Objektiv, https://www.dnevnik.si/1042762387

Al-Khatib, Aceil and Teixeira da Silva. Jaime A. 2016. ‘Stings, Hoaxes and Irony Breach the Trust Inherent in Scientific Publishing’, Publishing Research Quarterly, 32.3: 208–19, https://doi.org/10.1007/S12109-016-9473-4

Arias-Castro, Ery. 2019. Principles of Statistical Analysis: Statistical Inference Based on Randomization (Cambridge, UK: Cambridge University Press)

Beall, Jeffrey. 2018. ‘Scientific Soundness and the Problem of Predatory Journals’, in Pseudoscience: The Conspiracy against Science, ed. by Allison Kaufman and James Kaufman (Cambridge, MA: The MIT Press), 283–99, https://doi.org/10.7551/mitpress/10747.001.0001

Berger, Monica, and Cirasella, Jill. 2015. ‘Beyond Beall’s List: Better Understanding Predatory Publishers’, College & Research Libraries News, 76.3: 132–35, https://doi.org/10.5860/crln.76.3.9277

Bohannon, Johannes., Koch, Diana., Home, Peter and Driehaus, Alexander. 2015. ‘Chocolate with High Cocoa Content as a Weight-Loss Accelerator’, International Archives of Medicine, 8.55: 1–8, https://en.wikipedia.org/wiki/File:Chocolate_with_high_Cocoa_content_as_a_weight-loss_accelerator.pdf⁴

Bohannon, John. 2013. ‘Who’s Afraid of Peer Review?’, Science, 342.6154: 60–65, https://doi.org/10.1126/SCIENCE.342.6154.60

——. 2015. ‘I Fooled Millions Into Thinking Chocolate Helps Weight Loss. Here’s How.’, Gizmodo, https://gizmodo.com/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800

Bolland, Mark J., Grey, Andrew and Avenell, Alison. 2022. ‘Citation of Retracted Publications: A Challenging Problem’, Accountability in Research, 29.1: 18–25, https://doi.org/10.1080/08989621.2021.1886933

Bornemann-Cimenti, Helmar., Szilagyi, Istvan S. and Sandner-Kiesling, Andreas. 2016. ‘Perpetuation of Retracted Publications Using the Example of the Scott S. Reuben Case: Incidences, Reasons and Possible Improvements’, Science and Engineering Ethics, 22.4: 1063–72, https://doi.org/10.1007/S11948-015-9680-Y

Brown, Dwayne, and Weselby, Cathy. 2010. ‘NASA Sets News Conference on Astrobiology Discovery; Science Journal Has Embargoed Details Until 2 p.m. EST On Dec. 2’, National Aeronautics and Space Administration (NASA), https://www.nasa.gov/home/hqnews/2010/nov/HQ_M10-167_Astrobiology.html#:~:text=WASHINGTON%20%2D%2D%20NASA%20will%20hold,of%20life%20in%20the%20universe.

Butler, Declan. 2013. ‘Investigating Journals: The Dark Side of Publishing’, Nature, 495: 433–35, https://doi.org/10.1038/495433A

Camps-Bossacoma, Mariona., Garcia-Aloy, Mar., Saldana-Ruiz, Sandra., Cambras, Trinitat., González-Domínguez, Raúl and others. 2019. ‘Role of Theobromine in Cocoa’s Metabolic Properties in Healthy Rats’, Journal of Agricultural and Food Chemistry, 67.13: 3605–14, https://doi.org/10.1021/ACS.JAFC.8B07248

da Costa, Gabriel Gonçalves. 2021. ‘Lowering the P-Value Threshold to Alleviate Information Crisis in Nutritional Epidemiology – a Possible Temporary Solution?’, OSF Preprints, https://doi.org/10.31219/osf.io/ezmyj

‘Crossref Announces CrossCheck Plagiarism Detection Service’. 2008. Crossref, https://www.crossref.org/news/2008-04-15-crossref-announces-crosscheck-plagiarism-detection-service/

Davis, Phil. 2013. ‘Post Open Access Sting: An Interview with John Bohannon’, The Scholarly Kitchen, https://scholarlykitchen.sspnet.org/2013/11/12/post-open-access-sting-an-interview-with-john-bohannon/

‘Distinguish Yourself in Three Easy Steps’. [n.d.]. ORCID, https://orcid.org/

Editorial Office. 2015. ‘Retraction Notice on ‘Chocolate with High Cocoa Content as a Weight-Loss Accelerator’, International Archives of Medicine, 8, http://imed.pub/ojs/index.php/iam/article/view/1087

Eisen, Michael. 2013. ‘I Confess, I Wrote the Arsenic DNA Paper to Expose Flaws in Peer-Review at Subscription Based Journals’, It Is Not Junk, https://www.michaeleisen.org/blog/?p=1439

Elkhateeb, Yomna Ali Moustafa Marzok, and Mohammed AL Harbi, Razan. 2018. ‘Comparative Study on Health Knowledge about Biochemical Effect of Chocolate on Human Health among Females and Males of Hail University’, International Journal of Gastroenterology, 2.1: 1–6, http://dx.doi.org/10.11648/j.ijg.20180201.11

Estruch, Ramón., Ros, Emilio., Salas-Salvadó, Jordi., Covas, Maria-Isabel., Corella, Dolores and others. 2013. ‘Primary Prevention of Cardiovascular Disease with a Mediterranean Diet’, The New England Journal of Medicine, 368.14: 1279–90, https://doi.org/10.1056/NEJMOA1200303

‘Final Report: Commission on Information Disorder’. 2021. The Aspen Institute, https://www.aspeninstitute.org/publications/commission-on-information-disorder-final-report/

Finneman, Teri, and Thomas, Ryan J. 2018. ‘A Family of Falsehoods: Deception, Media Hoaxes and Fake News’, Newspaper Research Journal, 39.3: 350–61, https://doi.org/10.1177/0739532918796228

Fish, S. 2000. ‘Professor Sokal’s Bad Joke’, in The Sokal Hoax: The Sham That Shook the Academy, 1st edn, ed. by Lingua Franca (Lincoln, NE: University of Nebraska Press), 81–84

‘Funder Registry’. 2020. Crossref, https://www.crossref.org/services/funder-registry/

Gauthier, Benoît. 2016. ‘Impact = Context × Influence: Evaluation, Evidence and Policy in Canadian Government Contexts’, in Evidence and Policy Symposium, 1–14, https://digitalcollections.cdu.edu.au/nodes/view/4668

Giraldo, M., Toro, J.M., Arango, C.M., Posada, L.G. and Garcia, H.I.. 2017. ‘Ensayo Clínico Aleatorizado y Controlado Del Efecto Del Consumo de Cacao En Pacientes Con Resistencia a La Insulina’, Acta Médica Colombiana, 42.2: 90–96, https://doi.org/10.36104/amc.2017.988

Goldschmidt, Vivian. [n.d.]. ‘Lose Weight, Get In Shape, And Build Your Bones With These 7 Scrumptious Foods’, Save Our Bones, https://saveourbones.com/lose-weight-get-in-shape-and-build-your-bones-with-these-7-scrumptious-foods/

Grass, Robert N., and Stark, Wendelin J. 2015. ‘The Dissipation Rate of News in Online Mass Media Evaluated by Chemical Engineering and Process Control Tools’, AIChE Journal, 62.4: 1104–11, https://doi.org/10.1002/aic.15103

‘How the “Digital Object Identifier” Works’. 2001. Bloomberg, https://www.bloomberg.com/news/articles/2001-07-22/online-extra-how-the-digital-object-identifier-works#xj4y7vzkg

Hyland, Ken. 1999. ‘Academic Attribution: Citation and the Construction of Disciplinary Knowledge’, Applied Linguistics, 20.3: 341–67, https://doi.org/10.1093/APPLIN/20.3.341

‘International Archives of Medicine’. [n.d.]. International Medical Publisher, https://imedpub.jimdo.com/journals/list-of-journals/international-archives-of-medicine/

Jaiswal, J., LoSchiavo, C. and Perlman, D. C. 2020. ‘Disinformation, Misinformation and Inequality-Driven Mistrust in the Time of COVID-19: Lessons Unlearned from AIDS Denialism’, AIDS and Behavior, 24.10: 2780, https://doi.org/10.1007/S10461-020-02925-Y

Kawakami, Mark T. 2020. ‘How (Not) to Search for the Truth and (Perhaps) Improve the Human Condition’, Toyohogaku, 63.3: 291–319, http://doi.org/10.34428/00011523

Koerber, Amy., Starkey, Jesse C., Ardon-Dryer, Karin., Cummins, R. Glenn., Eko, Lyombe and others. 2020. ‘A Qualitative Content Analysis of Watchlists vs Safelists: How Do They Address the Issue of Predatory Publishing?’, Journal of Academic Librarianship, 46.6, https://doi.org/10.1016/j.acalib.2020.102236

Kyas, M., Springer, J. D., Pedersen, J. T. and Chkoniya, V. 2021. ‘Data Analysis in the Shipping Industry: EShip Case Study–Problem Statement’, in Handbook of Research on Applied Data Science and Artificial Intelligence in Business and Industry (Hershey, PA: IGI Global), 381–400

Lammey, Rachael. 2014. ‘The Basics of CrossRef Extensible Markup Language’, Science Editing, 1.2: 76–83, https://doi.org/10.6087/KCSE.2014.1.76

Lee, Younghyun, Hee Yang, Gihyun Hur, Jiwoo Yu, Sumin Park, and others. 2019. ‘5-(3′,4′-Dihydroxyphenyl)-γ-Valerolactone, a Metabolite of Procyanidins in Cacao, Suppresses MDI-Induced Adipogenesis by Regulating Cell Cycle Progression through Direct Inhibition of CDK2/Cyclin O’, Food & Function, 10.5: 2958–69, https://doi.org/10.1039/C9FO00334G

Lowe, Derek. 2013. ‘An Open Access Trash Heap’, Science, https://www.science.org/content/blog-post/open-access-trash-heap

Mak, Tim. 2019. ‘Senate Report Finds Russians Used Social Media Mostly to Target Race in 2016’, National Public Radio, https://www.npr.org/2019/10/08/768319934/senate-report-russians-used-used-social-media-mostly-to-target-race-in-2016#:~:text=Senate%20Report%20Finds%20Russians%20Used,Target%20Race%20In%202016%20%3A%20NPR&text=Tiny%20Desk-,Senate%20Report%20Finds%20Russians%20Used%20Social%20Media%20To%20Target%20Race,to%20a%20new%20bipartisan%20report.

Malinowski, Grzegorz M. 2019. ‘Uncertainty of Science and Decision-Making – Problems with Evidence-Based Policy’, Education of Economists and Managers, 54.4: 9–29, https://econjournals.sgh.waw.pl/EEiM/article/view/1828/1988

——. 2020. ‘Niepewność nauki a działanie, czyli – problemy z evidence–based policy’, Metody Ilościowe w Badaniach Ekonomicznych, 21.1: 49–69, https://doi.org/10.22630/MIBE.2020.21.1.6

McCook, Alison. 2015. ‘Chocolate-Diet Study Publisher Claims Paper Was Actually Rejected, Only Live ‘for Some Hours.’ Email, However, Says…’, Retraction Watch, https://retractionwatch.com/2015/05/28/chocolate-diet-study-publisher-claims-paper-was-actually-rejected-only-live-for-some-hours-email-however-says/

Moore, D.A. 2016. ‘Preregister If You Want To’, American Psychologist, 71.3: 238–39, https://doi.org/10.1037/a0040195

Mounk, Yascha. 2018. ‘What an Audacious Hoax Reveals About Academia’, The Atlantic, https://www.theatlantic.com/ideas/archive/2018/10/new-sokal-hoax/572212/

Mukerji, N. 2017. ‘The Third Commandment: Make Credible Assumptions’, in The 10 Commandments of Common Sense (New York: Springer)

Müller, Phillip. 2021. ‘Concerning Levels of Significance- Analysing Methods to Detect and Estimate P-Hacking’ (Unpublished Bachelor’s Thesis, Karlsruhe Institute of Technology), https://www.researchgate.net/profile/Philip_Mueller9/publication/352906629_Concerning_Levels_of_Significance_-_Analysing_Methods_to_Detect_and_Estimate_P-Hacking/links/60df2a3792851ca9449fd67a/Concerning-Levels-of-Significance-Analysing-Methods-to-Detect-and-Estimate-P-Hacking.pdf

‘Open Access Publishing for the Scientific Community’. [n.d.]. Hindawi, https://www.hindawi.com/

Peck, Lori Nicole. 2021. ‘Risk Portrayal and Actionability of Human Papillomavirus Coverage in Popular Magazines’ (Unpublished Master’s Thesis, Idaho State University)., https://www.proquest.com/docview/2621020323?pq-origsite=gscholar&fromopenview=true

Peng, Hao., Romero, Daniel M. and Horvát, Emoke Ágnes. 2022. ‘Dynamics of Cross-Platform Attention to Retracted Papers’, PNAS, 119.25: 16119–8, https://doi.org/10.1073/PNAS.2119086119

Perez, Carlos V. 2015. ‘Public Disclaimer [Facebook Status Update]’, Facebook [Group: International Archives of Medicine], https://www.facebook.com/groups/iamedicine/posts/10153415208752082/

Phillips, Tony. 2010. ‘Discovery of “Arsenic-Bug” Expands Definition of Life’, NASA Science, https://science.nasa.gov/science-news/science-at-nasa/2010/02dec_monolake/

‘PMC Journal List’. [n.d.]. NIH National Library of Medicine, https://www.ncbi.nlm.nih.gov/pmc/journals/

Poe, Edgar Allan. 1844. ‘The Balloon Hoax [Archived Full Text]’, New York Sun (New York City), https://www.eapoe.org/works/tales/ballhxa.htm

Ponce, Ana Rita de Heaton Ayres. 2018. O Papel Dos Comunicados de Imprensa No Sensacionalismo Em Notícias de Ciência (Unpublished Master’s Thesis: Universidada Nova De Lisboa), https://run.unl.pt/handle/10362/54180?mode=simple

Rahman, S. N., and Citrakesumasari, N. A. T. 2018. ‘Effect of Cocoa on Body Weight, Waist Circumference and Visceral Fat Patient with Central Obesity’, International Journal of Science and Healthcare Research, 3.1: 45–47, https://inrein.com/ijshr.com/IJSHR_Vol.3_Issue.1_Jan2018/IJSHR_Abstract.007.html

Ramezani, Tahereh., Behzadifard, Samaneh., Parvaresh, Nooshin., Jahani, Younes., Bakhtyari, Fahimeh and others. 2015. ‘The Comparison of Resilience and Spirituality in Addicted and Non-Addicted Women’, International Archives of Medicine, 8.55: 1–8, https://doi.org/10.3823/1654

Ramos, Sonia., Martín, María Angeles and Goya, Luis. 2017. ‘Effects of Cocoa Antioxidants in Type 2 Diabetes Mellitus’, Antioxidants, 6.4: 84, https://doi.org/10.3390/ANTIOX6040084

Reilly, Ian. 2020. ‘Public Deception as Ideological and Institutional Critique: On the Limits and Possibilities of Academic Hoaxing’, Canadian Journal of Communication, 45.2: 265–85, https://doi.org/10.22230/CJC.2020V45N2A3667

Rodríguez-Lagunas, Maria J., Vicente, Filipa., Pereira, Paula., Castell, Margarida and Pérez-Cano, Francisco J. 2019. ‘Relationship between Cocoa Intake and Healthy Status: A Pilot Study in University Students’, Molecules, 24.4: 812, https://doi.org/10.3390/MOLECULES24040812

Rosen, Rebecca J. 2012. ‘The Case (Study) of Arsenic Life: How the Internet Can Make Science Better’, The Atlantic, https://www.theatlantic.com/technology/archive/2012/07/the-case-study-of-arsenic-life-how-the-internet-can-make-science-better/259581/

Sauerwein, Tessa. 2019. ‘Framework Information Literacy – Aspekte Aus Theorie, Forschung Und Praxis’, Bibliothek Forschung Und Praxis, 43.1: 126–38, https://doi.org/10.1515/bfp-2019-2027

Schwitzer, Gary. 2015. ‘Should the Chocolate-Diet Sting Study Be Retracted? And Why the Coverage Doesn’t Surprise a News Watchdog’, Retraction Watch, https://retractionwatch.com/2015/05/28/should-the-chocolate-diet-sting-study-be-retracted-and-why-the-coverage-doesnt-surprise-a-news-watchdog/

Secor, Marie, and Walsh, Lynda. 2004. ‘A Rhetorical Perspective on the Sokal Hoax’, Written Communication, 21.1: 69–91, https://doi.org/10.1177/0741088303261037

Simpson, April. 2019. ‘As Local News Outlets Shutter, Rural America Suffers Most’, Stateline, https://www.pewtrusts.org/en/research-and-analysis/blogs/stateline/2019/10/21/as-local-news-outlets-shutter-rural-america-suffers-most

Steel, E.A., Liermann, M. and Guttorp, P. 2019. ‘Beyond Calculations: A Course in Statistical Thinking’, The American Statistician, 73.sup1: 392–401, https://doi.org/10.1080/00031305.2018.1505657

Stoll, O. 1898. ‘Zur Kenntniss Der Geographischen Verbreitung Der Ameisen’, Mitteilungen Der Schweizerischen Entomologischen Gesellschaft, 10: 120–26

Stylianou, M. 2022. ‘The Role of Dark Chocolate in the Diet and Consumer Sentiments’ (Unpublished thesis, University of Thessaly), https://ir.lib.uth.gr/xmlui/bitstream/handle/11615/51872/19107.pdf?sequence=1

Teixeira da Silva, Jaime A., and Al-Khatib, Aceil. 2016. ‘Questioning the Ethics of John Bohannon’s Hoaxes and Stings in the Context of Science Publishing’, KOME, 4.1: 84–88, https://doi.org/10.17646/KOME.2016.16

Teixeira da Silva, Jaime A., Dobránski, Judit., Alkhatib, Aceil and Pangiotis, Tsigaris. 2018. ‘Challenges Facing the DOAJ (Directory of Open Access Journals) as a Reliable Source of Open Access Publishing Venues’, Journal of Educational Media and Library Science, 55.3: 349–58, https://doi.org/10.6120/JoEMLS.201811_55(3).e003.BC.BE

‘Top 10 Most Highly Cited Retracted Papers’. [n.d.]. Retraction Watch, https://retractionwatch.com/the-retraction-watch-leaderboard/top-10-most-highly-cited-retracted-papers/

Vergano, Dan. 2012. ‘Discovery of an Arsenic-Friendly Microbe Refuted’, ABC News, https://abcnews.go.com/Technology/discovery-arsenic-friendly-microbe-refuted/story?id=16737658

——. 2013. ‘Glowing Reviews on “arseniclife” Spurred NASA’s Embrace’, USA Today, https://www.usatoday.com/story/tech/columnist/vergano/2013/02/01/arseniclife-peer-reviews-nasa/1883327/

——. 2013. ‘Fake Cancer Study Spotlights Bogus Science Journals’, National Geographic, https://www.nationalgeographic.com/pages/article/131003-bohannon-science-spoof-open-access-peer-review-cancer

Vida, István Kornél. 2012. ‘The “Great Moon Hoax” of 1835’, Hungarian Journal of English and American Studies, 18.1/2: 431–41

Wager, Elizabeth., Barbour, Virginia., Yentis, Steven and Kleinert, Sabine. 2009. ‘Retractions: Guidance from the Committee on Publication Ethics (COPE)’, Maturitas, 64.4: 201–3, https://doi.org/10.1016/J.MATURITAS.2009.09.018

Walsh, Lynda. 2006. Sins against Science (New York: State University of New York Press)

Wetterer, James K. 2006. ‘Quotation Error, Citation Copying, and Ant Extinctions in Madeira’, Scientometrics, 67.3: 351–72, https://doi.org/10.1556/SCIENT.67.2006.3.2

Wolfe-Simon, Felisa., Blum, Jodi Switzer., Kulp, Thomas R., Gordon, Gwyneth W., Hoeft, Shelley E. and others. 2010. ‘A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus’, Science, 332.6034: 1163–66, https://doi.org/10.1126/SCIENCE.1197258

Wooven, L., and Snider, T. 2018. Chocolate: Superfood of the Gods (Detroit, MI: Lotus Press)