Performance Benchmarks

What we measure

Rank	Command / artifact	Answers
1 (CI)	examples/cachegrind_harness.rs via scripts/run_cachegrind_bench.sh	Instruction-count deltas on fixed CON parse / skip / write / float paths
2 (PR)	benchmarks/ + asv.conf.json (ASV) → asv-spyglass compare	Python surface wall-time on PR vs base
3	benches/compare_readers.py	Same CON text vs ASE ase.io.eon and eOn-style C sscanf
4	benches/multiformat_traj.py	Equal-geometry wall times: ASE XYZ/extXYZ/CON vs readcon CON
5	benches/ase_traj_vs_con.py	ASE .traj / NetCDF / XYZ vs readcon.read_chemfiles vs native CON
6	benches/h5md_vs_con.py	MDAnalysis H5MD / h5py positions vs CON / chemfiles XYZ
local	cargo bench (Criterion)	Rust microbench latency; optional PR job saves baselines

Regression gates in CI:

• Cachegrind I-refs — path-filtered + weekly job (ci_cachegrind.yml); commits docs/source/_generated/cachegrind_results.* on main when counts change.

• Python ASV + spyglass — every PR (ci_benchmark.yml); comment via asv-perch.

Peer scripts under benches/ are for local re-runs. JSON under benches/results/ is what you produce when you run them — this page does not embed host wall-clock tables.

CI Cachegrind (instruction-count gate)

Workflow ci_cachegrind.yml runs Valgrind Cachegrind on examples/cachegrind_harness.rs when CON-path sources (or this doc) change on main, on a weekly schedule, or on manual dispatch. With CACHEGRIND_FEATURES=chemfiles it also records conversion scenarios.

Current committed include (refreshed by that job):

Cachegrind instruction counts (CI-refreshable)

Scenario	I refs	Notes
parse_multi_2x4	6,695,543	2-frame multi CON (50×)
forward_multi_2x4	4,928,105	forward() skip (50×)
convel_multi	7,565,875	coords + velocities (50×)
parse_100_frames	30,777,846	100× tiny_cuh2 (10×)
forward_100_frames	6,967,063	skip 100 frames (10×)
parse_cuh2_218	12,795,215	218-atom frame (20×)
float_fast_float2	20,003,368	5-col fast-float2 (10k)
float_std_parse	32,893,863	5-col str::parse (10k)
write_100_frames	17,301,283	buffer writer (10×)
chemfiles_xyz_path	9,677,042	XYZ path → ConFrame (50×)
chemfiles_xyz_memory	9,526,006	XYZ memory → ConFrame (50×)
chemfiles_select_name_O	6,080,433	selection name O (50×)

Generated 2026-09-30T22:08:35Z from commit 0577f6b (Cargo features: chemfiles-from-sources). Metric: Valgrind Cachegrind I refs. Lower is better for the same scenario. Comparable across commits on the same CI image/Valgrind.

Wall-clock Criterion tables elsewhere on this page are illustrative unless re-run for a release. PR workflow Benchmark PR still uses Criterion + critcmp for latency deltas.

Reproduce (needs Valgrind):

scripts/run_cachegrind_bench.sh
# writes docs/source/_generated/cachegrind_results.{json,rst}

I-refs for a fixed binary are stable across noisy shared runners; wall-clock medians on GHA are not. That is why Cachegrind is the commit-stable gate and ASV is the PR Python surface compare.

Python ASV (PR surface)

Workflow Benchmark PR (ci_benchmark.yml):

1. Matrix builds base and PR with maturin develop --features python,chemfiles --release.

2. Runs ASV on benchmarks/ with -E existing:$(which python) (no re-install mid-bench).

3. asv-spyglass compare writes results/comparison.txt (job fails if empty).

4. ci_bench_commenter.yml posts that file via asv-perch.

A separate criterion-benchmark job may save Criterion baselines; it uses continue-on-error: true so a Rust microbench miss does not fail the workflow conclusion that asv-perch requires.

Suite (calls shipped readcon only):

Class	API timed
TimeReadConTiny	read_con_string / read_con / iter_con on multi-frame tiny CON
TimeReadConCuh2	read_con on multi-frame cuh2.con (218 atoms/frame)
TimeChemfilesXyz	read_chemfiles multi-frame XYZ → ConFrame (skipped if lean)
TimeSelectAtoms	frame.select_atoms("name Cu") (skipped if lean)

Local:

maturin develop --features python,chemfiles --release
# uv pip install asv numpy ase
asv machine --yes
asv run -E "existing:$(which python)" --set-commit-hash "$(git rev-parse HEAD)" \
  --record-samples --quick
uvx --from "git+https://github.com/airspeed-velocity/asv_spyglass.git" \
  asv-spyglass compare .asv/results/*/<base>*.json .asv/results/*/<pr>*.json \
  .asv/results/benchmarks.json \
  --label-before main --label-after pr

Config: asv.conf.json. Results: .asv/ (gitignored).

Peer scripts (named host)

Cite only JSON that records host, UTC date, commit, and Cargo features. benches/results/*_terra.json and the host / date_utc fields in ase_traj_vs_con.json and h5md_vs_con.json are the paper-facing artifacts. A number without those fields is a local plot, not a table.

Chemfiles ingress vs native CON is an equal-geometry wall comparison: same atoms and frame count, XYZ through readcon.read_chemfiles versus readcon.read_con on the CON text. Cachegrind already has chemfiles_xyz_path / chemfiles_xyz_memory I-refs. The wall ratios live in ase_traj_vs_con.json (ratio_chemfiles_to_con_over_con) and h5md_vs_con.json (ratio_chemfiles_xyz_over_con). That answers the ingress cost. It is not a “conversion atoms/s” vanity metric: XYZ has no constraints, atom_id, or JSON, so the chemfiles path is a different payload.

H5MD framing for the same geometry: h5py position/value is a dense coordinate array and is faster than a full CON parse. MDAnalysis H5MD cost is the Python Universe API, not the format. CON carries cell, constraints, atom_id, and JSON. Report all three; do not drop H5MD.

# Same CON text vs ASE ase.io.eon and eOn-style C sscanf
uv run --with matplotlib --with numpy --with ase python benches/compare_readers.py \
  --out benches/results/compare_readers.json

# Equal-geometry multi-frame: ASE XYZ/extXYZ/CON vs readcon CON
python benches/multiformat_traj.py --fixtures cuh2 --ladder 100 --repeats 5 \
  --out benches/results/multiformat_traj.json

# ASE .traj / NetCDF / multi-frame XYZ vs chemfiles→CON vs native CON
maturin develop --features python,chemfiles --release
python benches/ase_traj_vs_con.py --frames 100 --repeats 5

# H5MD via MDAnalysis / h5py vs CON (uncompressed + gzip + optional zstd size/load)
uv pip install MDAnalysis h5py ase numpy
python benches/h5md_vs_con.py --frames 100 --repeats 5
# writes h5md_size_bytes, con_size_bytes, con_gz_size_bytes, con_zst_size_bytes,
# readcon_con_ms, readcon_con_gz_ms, readcon_con_zst_ms (null if zstd feature off)

Implementation details that show up in Cachegrind and peer scripts (facts about the code, not a promise about every host):

• Atom floats: fast-float2 (float_fast_float2 vs float_std_parse Cachegrind scenarios)

• Line views: zero-copy over the input buffer (MemchrLines)

• Atom vectors: sized from the CON header before filling

• File load: read_to_string below 64 KiB, mmap at/above (MMAP_THRESHOLD in src/compression.rs)

• Multi-frame parallel parse: Rayon when buffer ≥ 48 KiB and parallel is on (PARALLEL_BYTES_THRESHOLD in src/iterators.rs). Sequential vs pinned workers: examples/wall_scale_harness.rs writes benches/results/wall_scale.json (host, date, commit). That JSON is Table io in paper/cpc/main.tex. Do not mix it with the Python readcon.read_con wall table.

• Skip without atoms: forward / forward_fast (Cachegrind forward_*)

h5py position/value alone loads a coordinate array (not cell / constraints / atom_id / JSON). benches/h5md_vs_con.py times MDAnalysis H5MD and h5py as peer load paths on equal geometry, and reports **CON.gz / optional CON.zst** sizes and load times next to uncompressed CON. Gzip of multi-frame CON is often the same order as positions-only H5MD for near-repeated rare-event frames; that is a size fact, not advice to store campaigns as H5MD instead of CON + readcon-db. Compressed CON always fully decompresses before parse (no mmap); uncompressed ≥ 64 KiB uses memmap2.

Plots under docs/orgmode/img/ are produced by benches/make_plots.py from JSON you generate; regenerate after peer runs if you need them current.

Parsing throughput (from peer JSON via make_plots.py)

Feature coverage vs parse speed (from peer JSON via make_plots.py)

Public API model and hot path

**Load (full frames)**

• read_all_frames / ConFrameIterator / Python iter_con / read_first_frame

• Skip payload: count_frames / forward_fast when atoms are not needed

• Coordinates on a loaded frame: SoA / Python coords_array()

• No separate public “coords-only” trajectory load

**Python**

Multi-frame parse paths release the GIL via py.detach (src/python.rs) around the Rust work; iter_con yields full frames.

Criterion (local Rust latency)

cargo bench runs benches/iterator_bench.rs. Prefer Cachegrind for I-ref regressions and ASV for the Python PR surface. The PR Criterion job is optional artifact collection only; the posted comment is ASV/spyglass.

Memory

Peak RSS depends on host and whether all frames are materialised. Streaming:

let iter = ConFrameIterator::new(&contents);
for result in iter {
    let frame = result?;
    // process, then drop
}

Per frame the library keeps type symbols as shared Arc<str> (per type, not per atom), pre-sized atom storage from headers, and parses floats from borrowed line slices (no intermediate atom-line String).

Peak RSS plot (from historical peer JSON; re-run make_plots.py to refresh)

Feature matrix plot

CON v2 feature matrix vs common formats (make_plots.py)

CON v2 on the wire can carry positions, velocities, forces, unit cell, per-direction constraints, atom_id, structured JSON, compression, multi-frame concatenation, and streaming iteration — see The CON File Format Specification.

Statistical analysis (optional)

bayescomp can fit Gamma-family models to Criterion JSON and compare_readers.py timings when you want credible intervals; it is not part of the PR gate.

Reproduce (one place)

# Python ASV (PR surface)
maturin develop --features python,chemfiles --release
asv machine --yes
asv run -E "existing:$(which python)" --set-commit-hash "$(git rev-parse HEAD)" \
  --record-samples --quick

# CON peers / multi-format / traj / H5MD — see Peer scripts above

# Plots from your JSON
uv run --with matplotlib --with numpy python benches/make_plots.py

# Cachegrind I-refs (needs Valgrind)
scripts/run_cachegrind_bench.sh

# Rust Criterion
cargo bench