Benchmarks

A benchmark runs a variety of experiments that pose different challenges to a Congestion Control Algorithm (CCA) under test. The goal of benchmark is to evaluate a CCA holistically with a broad test coverage instead of cherry-picking favorable results. Thereby, strengths, weaknesses, and design tradeoffs of CCAs may be revealed.

A CCA may perform well in one scenario and may fail in another. Therefore, the experiments set up many different network and traffic scenarios. A variety of metrics quantify the performance.

A benchmark runs each experiment type at least once. Furthermore, experiments are often repeated with different experiment parameters.