Reply to Author's Response Philip Morris International used the e-cigarette, or vaping, product use associated lung injury (EVALI) outbreak to market IQOS heated tobacco
Reply to Author's Response Philip Morris International used the e-cigarette, or vaping, product use associated lung injury (EVALI) outbreak to market IQOS heated tobacco | Tobacco Control (bmj.com)
Authors’ Response Reveals Several New and Serious Issues
Replication attempts are one of the self-correcting mechanisms of science, and we thank the Authors for their response to our concerns and their attempt to replicate aspects of their study[1]. Regrettably, they have failed to adequately address the central point raised in our letter of 23rd April 2021, namely that the title and conclusions of their original Article are patently invalid and have no basis in fact or evidence[2]. Instead of strengthening their argument in support of the Article’s findings and conclusions, the Authors’ response considerably weakens them. Strikingly, the Authors reveal several new and serious issues and yet maintain that their “principle finding is unchanged”.
Methodological Problems
The Authors acknowledge that they were unable to replicate an important aspect of their original analysis, namely that a Philip Morris International (PMI) News Article[3] published on its website (falsely described as a “press release”) was “republished […] in 14 additional news outlets”. In their response, they note that “Our original assertion that there were 14 duplicate articles is not supported by our replication analysis”. This failure to replicate a key finding—in their own proprietary database, which several of them co-developed—is concerning. The Authors provide no explanation for the irregularity.
Notably, on 20th April 2021, we were able to source these 14 articles in Tobacco Watcher since they were clearly marked as “additional coverage” of the PMI News Article. Of the total, 5 of the “duplicate articles” were no longer accessible online and none of the remaining 9 mentioned IQOS at all. All of them were reporting on the EVALI outbreak and flavor ban proposals, a fact that we raised in our April letter to the Authors.
The Authors note that Tobacco Watcher is “a dynamic resource with continuous data collection and processing”. We could not find any information about the computational environment of the database, nor does there appear to be any public information about factors that can impact reproducibility and replicability. Specifically, we could not find any published information about how it collects, processes, and analyzes data, how it is version controlled and what data management plans are in place. However, a poster presentation by some of the Authors indicates that media articles are “automatically coded” using “natural language processing, trained on human-coded data”[4]. It is therefore difficult to understand why results relating to media articles from 2019—generated from an automatic coding process—could abruptly change over the course of two months in 2021. Of course, it is entirely plausible that the 14 articles had been falsely coded, and that this error was manually addressed at some point between April 20th and June 10th, 2021. The Authors should explain if this was the case, or whether changes to the underlying Tobacco Watcher algorithms caused the change—and if so, how. They should also provide an explanation of why they remain confident in the rest of their trend analyses, given the apparent instability of Tobacco Watcher over time.
Because of the Authors’ failure to replicate this important aspect of their study, there is no valid evidence to show any direct link between the PMI News Article and the increase in media coverage of IQOS and EVALI. The Authors’ invalid assumption that the PMI News Article was “republished” was presumably relied upon to support their title and conclusions and yet the evidence for this has now been shown to be nonexistent—not just by our analysis, but also by the Authors’ subsequent failure to replicate their initial finding.
Confounding Factors Ignored
Given that the Authors have now shown that they do not have any evidence to support a direct, causal relationship between the PMI News Article and subsequent news coverage including both the terms IQOS and EVALI, potential confounders become central to the question of whether their title and conclusions are supported or not. Whilst they now concede that their evidence merely demonstrates that a causal relationship is “plausible”, they continue to ignore or dismiss other more likely explanations. Many confounding factors were laid out in our April letter—with sources beyond those emanating solely from PMI—and they provide important information about the news media environment that could credibly have caused a rise in articles mentioning IQOS and EVALI (including in particular the official press releases[5] by PMI and Altria on 25th September 2019—announcing that their proposed merger talks were over and that the companies would focus on launching IQOS in the U.S.— as well as a press release from the U.S. Centers for Disease Control related to EVALI). Generally, in epidemiology, sociology and other fields, there are at least three main criteria that should be met in order to establish a causal relationship: the cause occurs before the effect; the cause and the effect covary; and a lack of plausible alternative explanations[6]. By dismissing the most plausible explanations for the increase in media coverage, and failing to provide any credible alternative evidence to explain their hypothesis, the overall analysis and conclusion presented by the Authors fails to meet these criteria for causality.
Attempt to Shift the Burden of Proof
The Authors make logical errors in defense of their Article. They state that “PMI has not provided any verifiable disclosure about how they circulated these materials [the PMI News Article] beyond their publication and availability on internet search engines and social media”. Any perceived lack of disclosure on the part of PMI does not render the Authors’ claims any more valid. To the contrary, the burden of proof is on them to provide evidence for their assertions. In the three months since we first wrote to the Authors to express our concerns, it would have been perfectly possible for them to: (a) list which news articles actually refer to the PMI News Article and/or use the phrases ‘IQOS’ AND ‘EVALI’ (rather than ‘vaping’ and ‘illness’, which are not identical search terms); and (b) rapidly and independently verify whether any of the journalists writing about both IQOS and EVALI had seen the PMI News Article by simply asking them.
Unsubstantiated, Yet Definitive Language
The title “Philip Morris International used the e-cigarette, or vaping, product use associated lung injury (EVALI) outbreak to market IQOS heated tobacco” is a statement purporting to be factual. However, the Authors now state that their original findings and conclusions are merely “plausible” and admit “we cannot know PMI’s intent”. Plausibility is not a statement of fact or truth. Instead, it implies that an argument is not fully formed, appears specious, superficially fair or reasonable, but in fact could be either right or wrong. As such, the title and other conclusory allegations in the Article are—by the Author’s own admission—not supported. Having ignored or discounted evidence that makes the Author’s conclusions implausible, the final paragraph of their response makes clear that in the absence of evidence, their conclusion rests solely on the Author’s distrust of the industry.
Summary
In summary, the conclusion that “Philip Morris International used the e-cigarette, or vaping, product use associated lung injury (EVALI) outbreak to market IQOS heated tobacco” can be disproven using the authors’ own tool, an analysis method that actually takes into account concomitant news events, the Authors’ acknowledgment that they failed to replicate an important aspect of their findings, and their concession that their title is merely “plausible”. The entire premise of the EVALI/IQOS Article is patently invalid and substantial revision or retraction of this paper remains urgently warranted.
[2] https://www.qeios.com/read/NLZDBR (see supplementary data file)
[3] https://www.pmi.com/media-center/news/lung-illnesses-associated-with-use-of-vaping-products-in-the-us-the-facts (Note: this News Article was published on 24th September 2019. The Authors response inaccurately states a publication date of 24th September 2021)
[5] Unlike official press releases circulated to news agencies via press services, which reach more than a hundred thousand news outlets worldwide instantly and directly (see https://services.businesswire.com/public-relations-services/press-release-distribution), the PMI News Article was only published on PMI.com and via our social media accounts, and as such would only have received organic traffic. [6] e.g., Chambliss, Daniel & Schutt, Russell. (2018). Making Sense of the Social World: Methods of Investigation.