Where are we at regarding species translation? A review of the sbv IMPROVER challenge

Animal models serve an important purpose in fundamental research aimed at understanding the molecular networks underpinning biological and cellular mechanisms. In drug development and toxicological risk assessment, compound testing usually starts by in vitro experiments, followed by in vivo testing using rodents as a mammalian model. This practice assumes that such animal models respond to active substances through similar mechanisms. Yet in spite of many effective uses of animal models, numerous findings have also been unsuccessfully translated to humans, as evidenced by the failure of many clinical trials. These failures clearly stem from our lack of detailed understanding of inter-species differences that precludes naïve translation of knowledge derived from animal models directly to human, but also further distinction among available rodent models, including xenograft, genetic or syngeneic models. Such limitations are driving the development of increasingly more complex in vitro systems, ranging from 3D-organoid tissue cultures derived from primary cells to so-called human Organs-on-chips (Huh et al., 2013) and Human-on-a-chip (Marx et al., 2012). However, it is still early days, and replacing complex model organisms with a battery of in vitro systems that adequately recapitulates human biology is a journey that has just begun. In this context, a pertinent question to ask is ‘to what extent perturbations induced in rodent cells are conserved and hence predictable, in human cells?’ This important question was addressed in the form of a challenge titled ‘Species Translation Challenge’ in the framework of sbv IMPROVER (systems biology verification, Industrial Methodology for PROcess VErification in Research) (Meyer et al., 2012), a collaborative initiative by IBM Research and Philip Morris International (PMI) R&D that uses crowd-sourcing approaches to address fundamental questions in systems biology. For 4 months, starting in May 2013, the Species Translation Challenge was articulated around four sub-challenges focusing on different aspects of translatability within and between species. The results of this challenge provide insights on (i) the accuracy of predictions in the context of the diversity and dimensionality of data types generated at various levels of the biological system; (ii) the importance of temporal information (more time points) to gain resolution in sequential molecular events, especially if the activation time window is shifted between species; (iii) heterogeneity of results from similar computational methods, which is probably due to different methodological choices made at distinct steps of the process (e.g. data preprocessing, feature selection, classification algorithm) that are problem dependent; (iv) a consensus network that describes the conservation and divergence of biological pathways and interactions in response to a subset of stimuli. A total of 28 teams, comprising 51 scientists from 14 countries across Europe, North America, Asia and Australia submitted predictions. The submissions of each team were evaluated by an independent panel of scientists and the results of the Species Translation Challenge were presented, and best performers were rewarded at the sbv IMPROVER Symposium last October 2013 in Athens, Greece. Three best-performing teams were recognized in the first sub-Challenge. Team AMG involved scientists from University of California Santa Barbara (USA), University of Groningen (Netherlands) and Rutgers University (USA). The other best performers in this sub-Challenge were Clemson University (USA) and Wayne State University (USA). Team AMG was also best performing in sub-Challenges 2 and 3. Five teams were recognized as joint best performers in sub-Challenge 4: one from Max Planck Institute for Dynamics of Complex Technical Systems (Germany), a Swiss team from University of Lausanne and Institute of Bioinformatics, one from Pacific Northwest National Laboratory (USA) and two separate teams from University of Pittsburgh (USA). Details of each team are published on the sbv IMPROVER website (http://www.sbvimprover.com).