Alessandro Cosentino

Title: 

Metriq-data: an open, cross-platform dataset of quantum benchmark results

Abstract: 

We will present metriq-data, an open collection of schema-validated benchmark records across quantum devices, providers, physical implementations, and benchmark protocols. Our goal is for metriq-data to become the most comprehensive public dataset of quantum benchmark results, supporting continuous benchmarking and making it possible to track performance trends over time as quantum computing stacks evolve.

The current dataset includes results from thirteen QPUs, spanning superconducting and trapped-ion processors, evaluated on system-level and application-inspired benchmarks. This shared dataset enables analyses that are difficult to perform from isolated studies alone: identifying which benchmarks move together, which act as proxies for others, and where residual structure points to distinct hardware or workflow limitations. We will also discuss reproducibility and governance challenges exposed by the dataset, including compilation choices, metadata gaps, timing information, cost, and benchmark inclusion.  

The goal of the talk is also to invite submissions: we welcome new results, new devices, better metadata, and community feedback on what the next Metriq suite should measure.