Offering safer alternatives to cigarettes, such as e-cigarettes and heated tobacco products, to smokers who are not willing to quit could reduce the harm caused by smoking. Extensive and rigorous scientific studies are conducted to assess the relative risk of such potentially modified risk tobacco products compared with that of smoking cigarettes. In addition to the peer review of publications reporting individual studies, we aimed to gauge the plausibility of the evidence to the scientific community and appreciate likely necessary additions prior to regulatory submission. Therefore, we sponsored a two-tier peer review organized by an independent third party who identified, recruited, and managed 7 panels of 5–12 experts whose identity remains unknown to us. The reviewers had access to all publications and raw data from preclinical and clinical studies via a web portal. The reviewers were asked questions regarding study design, methods, quality of data, and interpretation of results to judge the validity of the conclusions regarding the relative effects of the Tobacco Heating System 2.2 compared with cigarettes. Once their conclusions were submitted, the experts had the opportunity to participate in an anonymized online debate with their fellow panel members. We present here the results obtained from this innovative peer review effort which revealed supportive or very supportive of the study methods and results, and support the robustness of the studies and validity of the conclusions.