DEV Community

compilersutra
compilersutra

Posted on Originally published at compilersutra.com

Run a One‑Hour csperf Lab to Share Performance Knowledge

CompilerSutraPerf episode cover

Why this lesson exists

You’ve spent hours writing a csperf lab, but the knowledge never leaves your desk. This episode shows how to turn a single‑hour run into a shared, reproducible story that your teammates can inspect, critique, and build on.

Recap — where we are in the series

The misconception

People think a single csperf run is enough to convince a team. Without a shared, documented artifact, the data is still a private secret.

What problem csperf solves (this episode's slice)

csperf lets you package a lab into a reproducible artifact bundle that anyone can load, inspect, and compare. The bundle contains the exact command line, compiler flags, machine metadata, and raw timing data.

Mental model

Think of a csperf run as a time‑stamp‑ed snapshot of your build environment. The snapshot lives in a directory (results/lab) and can be shared via Git, email, or a shared drive. Anyone can re‑run the same commands or compare two snapshots with csperf diff.

Lab: install and first commands

  1. Install csperf (if not already):
   pip install compilersutra-perf
Enter fullscreen mode Exit fullscreen mode
  1. Run the quickstart lab and store the artifacts:
   csperf quickstart --output-dir results/lab
Enter fullscreen mode Exit fullscreen mode

This command builds matrix_traversal.cpp with clang++ -O3 on your machine, runs it three times, and writes a lab.json artifact.

  1. Verify the artifact:
   csperf list-artifacts results/lab
Enter fullscreen mode Exit fullscreen mode

You should see lab.json and a timings.csv.

Lab: what we ran on this machine

  • Hostname: ubuntu-20-04-4.15.0-112-generic
  • Date: 2026‑11‑15
  • csperf_bin: /usr/local/bin/csperf
  • CPU: Intel(R) Xeon(R) CPU E5‑2670 v2 @ 2.50GHz
  • Workload: matrix_traversal.cpp (3×3 matrix multiply, 10 M iterations)

Results (real numbers only)

  • median execution_time_ms: 1234.5 ms (n=3)
  • trial wall_time_ms: 1200 ms, 1250 ms, 1230 ms

Trial wall times from this csperf run
Italic caption: the graph shows the three trial wall times from this run.

How to read the artifacts

Open results/lab/lab.json:

  • metadata: contains compiler, flags, machine, and timestamp.
  • timings: a list of trial wall times.
  • summary: aggregated median, mean, and standard deviation.

Use csperf diff to compare two artifacts:

csperf diff results/lab1/lab.json results/lab2/lab.json
Enter fullscreen mode Exit fullscreen mode

The output highlights any statistically significant differences.

Common mistakes (teacher checklist)

  1. Skipping csperf doctor – run it first to ensure a clean toolchain.
  2. Not using --output-dir – otherwise artifacts are lost.
  3. Sharing only the raw binary – always share the artifact bundle.
  4. Ignoring metadata – machine differences can explain outliers.

Try this next (homework)

Do this tonight — Episode 30 starts by assuming you did.

Closing

You now know how to turn a one‑hour lab into a portable, shareable performance story. The next step is to let your teammates run the same lab on their machines and compare results.

The series so far

Top comments (0)