HARMONY2022 - FROG analysis - standard for reproducibility and curation of constraint-based models

Описание: Constraint-based models are used to investigate metabolism in diverse conditions; particularly genome-scale metabolic models (GEMs) with thousands of reactions provide opportunities to analyze organism-specific metabolism. Community standards for consistent model reconstruction, curation, and sharing are crucial to ensure reproducibility, reliability, and FAIR sharing of the models. MEMOTE, a community tool was developed for standardized quality assessment of the models. Currently, it is not possible to assess whether constraint-based models including Genome-Scale metabolic models (GEM) are reproducible because these models often have multiple solutions and the numerical values are not always enumerated in the published manuscripts. Here we initiate a community effort for standardized assessment of model reproducibility, which is currently lacking. Reproducibility of results is the cornerstone of science and its assessment is an essential part of the curation of a model in the repository such as BioModels. Following discussions at dedicated breakout sessions at HARMONY2020, COMBINE2020, HARMONY2021 and COMBINE2021 we have now developed FROG analysis, an ensemble of analysis of constraint-based models to generate standardized numerically reproducible reference dataset (aka FROG report). We have also developed a collection of tools that generates FROG reports in a standardized schema. We propose that the modelers share FROG reports along with their model which can be used by the modeler or curator to independently assess the reproducibility of a model. FROG analysis is currently used in BioModels’ workflow for the curation of constraint-based models along with the MEMOTE test suite. To allow retrospective curation of previously published models, we propose that the model authors submit a miniFROG report in addition to autogenerated FROG report. The miniFROG report is a manually created data table in a standardized schema that lists at least a couple of results described in the manuscript and compares them against the model results from the FROG report. Sharing FROG and miniFROG reports with model (SBML-FBC) files will enable reproducibility assessment and curation and enhance reuse, extension, integration of constraint-based models for new knowledge generation.