Peer-Reviewed Publications

      Synergy, suppression and immorality: forward differences of the entropy function

      Whittaker, J.; Martin, F.; Xiang, Y.
      Published
      Jan 18, 2015
      Link to publication
      Summary

      Conditional mutual information is important in the selection and interpretation of graphical models. Its empirical version is well known as a generalised likelihood ratio test and that it may be represented as a difference in entropy. We consider the forward difference expansion of the entropy function defined on all subsets of the variables under study. The elements of this expansion are invariant to permutation of their suffices and relate higher order mutual informations to lower order ones. The third order difference is expressible as an, apparently assymmetric, difference between a marginal and a conditional mutual information. Its role in the decomposition for explained information provides a technical definition for synergy between three random variables. Positive values occur when two variables provide alternative explanations for a third; negative values, termed synergies, occur when the sum of explained information is greater than the sum of its parts. Synergies tend to be infrequent; they connect the seemingly unrelated concepts of suppressor variables in regression, on the one hand, and unshielded colliders in Bayes networks (immoralities), on the other. We give novel characterizations of these phenomena that generalise to categorical variables and to higher dimensions. We propose an algorithm for systematically computing low order differences from a given graph. Examples from small scale real-life studies indicate the potential of these techniques for empirical statistical analysis.

      PMIScience.com is operated by Philip Morris International for the purpose of publishing and disseminating scientific information about Philip Morris International’s efforts in support of its smoke-free product portfolio. This site is a global site for use by scientists, the public health and regulatory communities, and other stakeholders with an interest in tobacco policy. The purpose of this site is not advertising or marketing, nor is it directed at any specific market. It is not intended for use by consumers. New tobacco products sold in the United States are subject to FDA regulation; therefore the content of this site is not intended to make, and nor should it be construed as making, any product related claims in the United States without proper FDA authorization. ​
      ​
      Reduced Risk Products ("RRPs”) is the term we use to refer to products that present, are likely to present, or have the potential to present less risk of harm to smokers who switch to these products versus continuing smoking. PMI has a range of RRPs in various stages of development, scientific assessment and commercialization. All of our RRPs are smoke-free products that deliver nicotine with far lower quantities of harmful and potentially harmful constituents than found in cigarette smoke.