Experiments

Experiments are run to evaluate the performance of congestion control algorithms in specific scenarios. The network, traffic, and monitoring are the three components that specify an experiment:

  • The network sets up the network elements, namely nodes and links. Nodes are connected to each other via links. Together, nodes and links form a network topology that can be visualized in graph.

  • The flows specify when applications running on nodes generate data. Data is generated according to traffic models that determine the exact timings of transmissions by an application.

  • Monitoring observes the network elements and data transmissions throughout the duration of an experiment. Based on these observations, metrics are captured and saved for later evaluations.

Components of an experiment.

Experiments can have parameters that modify the evaluation conditions. In ccperf, experiments are often repeated with different parameters to vary the side conditions congestion control algorithms face.

An experiment traverses three stages throughout its life.

  • Setup: In the setup step the experiment is prepared for execution. Network elements, traffic, and monitoring are set up according to the experiment type and experiment parameters.

  • Execution: The execution step runs the experiment. While the experiment runs, metrics are saved to a database.

  • Reporting: The metrics are post-processed for the performance evaluation.

Experiment Types

ccperf uses a variety of experiments to evaluate Congestion Control Algorithms (CCAs) holistically. Each experiment type is designed to test one or more performance criteria of a CCA. The experiment types have unique identifiers and are organized in groups that bundle related types together. The following table gives an overview about the different experiments types including links to the documentation.

Group Name Identifier
Steady-state Single Flow steady_state_single_flow
Synchronous Fairness steady_state_synchronous_fairness
Start-time Fairness steady_state_start_time_fairness
RTT Fairness steady_state_rtt_fairness
Transient-state Step Function Response responsiveness_step
Pulse Wave Response responsiveness_pulse_wave
Network Variation Scalability scalability
Loss Resilience loss_resilience
Deep Buffers deep_buffers
Shallow Buffers shallow_buffers
AQM aqm
ECN ecn
ACK Compression ack_compression
Traffic Policing traffic_policing
Dynamic Networks Replay Rate Trace replay_rate_trace
Random Walk random_walk
Competition Reno Fairness reno_fairness
Cubic Fairness cubic_fairness
Vegas Fairness vegas_fairness
TCP On/Off Traffic tcp_on_off
UDP Background Traffic udp_background