Peer-Reviewed Publications

      Strengths and limitations of microarray-based phenotype prediction: lessons learned from the IMPROVER Diagnostic Signature Challenge

      Tarca, A. L.; Lauria, M.; Unger, M.; Bilal, E.; Boue, S.; Kumar Dey, K.; Hoeng, J.; Koeppl, H.; Martin, F.; Meyer, P.; Nandy, P.; Norel, R.; Peitsch, M. C.; Rice, J. J.; Romero, R.; Stolovitzky, G.; Talikka, M.; Xiang, Y.; Zechner, C.
      Published
      Nov 15, 2013
      DOI
      10.1093/bioinformatics/btt492
      PMID
      23966112
      Link to publication
      Topic
      Summary

      Motivation: After more than a decade since microarrays were used to predict phenotype of biological samples, real-life applications for disease screening and identification of patients who would best benefit from treatment are still emerging. The interest of the scientific community in identifying best approaches to develop such prediction models was reaffirmed in a competition style international collaboration called IMPROVER Diagnostic Signature Challenge whose results we describe herein. Results: Fifty-four teams used public data to develop prediction models in four disease areas including multiple sclerosis, lung cancer, psoriasis and chronic obstructive pulmonary disease, and made predictions on blinded new data that we generated. Teams were scored using three metrics that captured various aspects of the quality of predictions, and best performers were awarded. This article presents the challenge results and introduces to the community the approaches of the best overall three performers, as well as an R package that implements the approach of the best overall team. The analyses of model performance data submitted in the challenge as well as additional simulations that we have performed revealed that (i) the quality of predictions depends more on the disease endpoint than on the particular approaches used in the challenge; (ii) the most important modeling factor (e.g. data preprocessing, feature selection and classifier type) is problem dependent; and (iii) for optimal results datasets and methods have to be carefully matched. Biomedical factors such as the disease severity and confidence in diagnostic were found to be associated with the misclassification rates across the different teams. Availability: The lung cancer dataset is available from Gene Expression Omnibus (accession, GSE43580). The maPredictDSC R package implementing the approach of the best overall team is available at www.bioconductor.org or http://bioinformaticsprb.med.wayne.edu/.

      PMIScience.com is operated by Philip Morris International for the purpose of publishing and disseminating scientific information about Philip Morris International’s efforts in support of its smoke-free product portfolio. This site is a global site for use by scientists, the public health and regulatory communities, and other stakeholders with an interest in tobacco policy. The purpose of this site is not advertising or marketing, nor is it directed at any specific market. It is not intended for use by consumers. New tobacco products sold in the United States are subject to FDA regulation; therefore the content of this site is not intended to make, and nor should it be construed as making, any product related claims in the United States without proper FDA authorization. ​
      ​
      Reduced Risk Products ("RRPs”) is the term we use to refer to products that present, are likely to present, or have the potential to present less risk of harm to smokers who switch to these products versus continuing smoking. PMI has a range of RRPs in various stages of development, scientific assessment and commercialization. All of our RRPs are smoke-free products that deliver nicotine with far lower quantities of harmful and potentially harmful constituents than found in cigarette smoke.