Open Problems for Single-Cell Analysis

Benchmarking formalized challenges in single-cell analysis

OpenProblems is an open-source platform for benchmarking and formalizing computational tasks in single-cell analysis. The platform aims to facilitate the development of new methods for single-cell omics by defining mathematical interpretations of tasks, providing publicly available gold-standard datasets in standardized formats, defining quantitative metrics to evaluate success, and ranking state-of-the-art methods in continuously updated leaderboards. OpenProblems is hosted on GitHub and evaluated using AWS with support from the Chan Zuckerberg Initiative.

Viash is being used in OpenProblems to streamline the development, execution, and sharing of methods, metrics and dataset loaders. It allows users to define their components as reusable modules that can be combined to form pipelines, and it manages dependencies, versioning, and execution on different computing environments. Viash also provides a user-friendly interface for configuring pipelines and visualizing results. By using Viash, OpenProblems can provide a flexible and scalable platform for benchmarking single-cell analysis methods.