📎header

📖 How to Read the Results

Status Indicators

Status Meaning Threshold
✅ pass Performance within expected bounds Below warning threshold
⚠️ warn Performance elevated; investigate on regression Small repos: >⚙️small_repo_warning_ms ms  |  Large repos: >⚙️large_repo_warning_ms ms  |  openclaw: >⚙️openclaw_warning_ms ms
❌ critical Performance regression requiring immediate attention Small repos: >⚙️small_repo_critical_ms ms  |  Large repos: >⚙️large_repo_critical_ms ms  |  openclaw: >⚙️openclaw_critical_ms ms

Small repo: repos where mean < large-repo warning threshold. Large repo: repos where mean ≥ large-repo warning threshold. openclaw has its own thresholds (~14k files, significantly larger than the rest).

🔬 Measurement Methodology

Timing

Outlier Removal

Corpus

The real-world corpus contains 14 open-source repositories of varying sizes: small agent frameworks (~50 files) through large codebases like openclaw (~14,000 files). This range tests both startup-dominated and I/O-dominated performance profiles.

🔥 Reading Flamegraphs

Flamegraphs are generated using cargo-flamegraph with Linux perf and inferno. The profiling binary uses release-with-debug profile for accurate symbol resolution.

📈 CV% (Coefficient of Variation)

CV% = (stdev / mean) × 100. A low CV% indicates stable, consistent measurements. High CV% (>10%) may indicate:

Results with high CV% should be treated with caution; re-running benchmarks on a quieter system is recommended.

📎footer