Developer workflow and release guidelines ¶

1 Development setup ¶

1.1 Prerequisites ¶

Rust >= 1.88 (edition 2024)
pixi for environment management
cocogitto (cog) for conventional commits and changelog

1.2 Getting started ¶

git clone https://github.com/lode-org/readcon-core.git
cd readcon-core
pixi install

# Run tests
pixi r test

# Run with all features (needs capnproto)
pixi r test-all

# Build release
pixi r build

1.3 Environment-specific workflows ¶

# Python bindings
pixi r -e python python-build
pixi r -e python python-test

# Julia bindings
pixi r -e julia julia-test

# Documentation
pixi r -e docs docbld
# Docs output in docs/build/

2 Commit conventions ¶

The project uses conventional commits enforced by cocogitto. The CI lint check rejects non-conforming commit messages.

Recognized commit types (from cog.toml):

Type	Changelog section	Use for
feat	Features	New user-facing functionality
enh	Enhancements	Improvements to existing features
fix	Bugfixes	Bug corrections
bug	Bugfixes	Alias for fix
doc	Documentation	Documentation-only changes
tst	Tests	Test additions or modifications
bld	Buildsystem	Build system, CI, dependency changes
maint	Maintenance	Refactoring, cleanup, internal changes
bench	Benchmarks	Benchmark additions or modifications
gen	Generated	Auto-generated file updates
dat	Data	Test data or resource changes

Examples:

feat: add convel format support with optional velocity fields
enh: integrate fast-float2 for f64 parsing hot path
fix: handle empty velocity section in multi-frame files
doc: update tutorials with Julia convel example
bld: add CMakeLists.txt for cmake subproject use
tst: add roundtrip test for convel writer

3 Branching and workflow ¶

main is the release branch. All PRs target main.
Feature branches: feat/<description>
Bugfix branches: fix/<description>
Keep commits logical and atomic. Soft-reset and rebase as needed (never on main).
The CI runs on every PR: tests, lint, coverage, and benchmark regression checks (via Criterion + critcmp + asv-perch).

4 Testing ¶

# Core Rust tests
cargo test

# All features (parallel, rpc, python)
cargo test --all-features

# Meson build with valgrind leak checking
meson setup bbdir -Dwith_tests=True -Dwith_examples=True
meson test -C bbdir

# Benchmarks
cargo bench
# or: pixi r bench

Test data lives in resources/test/. Use the test_case! macro in integration tests to locate test files.

5 Continuous integration ¶

5.1 Workflows ¶

Workflow	File	Trigger	Purpose
Python wheels	`python_wheels.yml`	`v*` tag, PR	Build wheels for 5 platforms, publish to PyPI
Benchmark PR	`ci_benchmark.yml`	PR to main	Run Criterion benchmarks on base and PR
Comment benchmark results	`ci_bench_commenter.yml`	benchmark complete	Post comparison table as PR comment
Coverage	`coverage.yml`	push, PR	Code coverage via tarpaulin
Lint	`lint.yml`	push, PR	Conventional commit check + large file audit

5.2 Benchmark regression detection ¶

The project uses a two-workflow pattern for safe benchmark PR comments:

ci_benchmark.yml runs cargo bench on both the base commit and the PR head in parallel (matrix strategy). It saves Criterion baselines as artifacts.
ci_bench_commenter.yml triggers on workflow_run completion. It downloads the Criterion results, compares them with critcmp, and posts the comparison table as a PR comment using asv-perch.

The workflow_run split is required for fork PRs to have write access for posting comments (GitHub security model).

Regressions above 10x trigger automatic conversion to draft PR.

6 Release process ¶

6.1 Version files ¶

Versions are tracked in five places that must stay synchronized:

Cargo.toml (version = "X.Y.Z")
meson.build (version: 'X.Y.Z')
pyproject.toml (version = "X.Y.Z")
pixi.toml (version = "X.Y.Z")
docs/source/conf.py (release = "X.Y.Z")

6.2 Complete release checklist ¶

Ensure all tests pass and the branch is clean:
```
cargo test
cargo test --all-features
```
Bump the version in all five files listed above.

Generate the changelog:

cog changelog --at vX.Y.Z > /tmp/cl.md
# Review, then prepend to CHANGELOG.md

Regenerate the README from readme_src.org:

# Export org to markdown (emacs batch or manual)
# Verify README.md reflects current features

Commit the version bump:

git add Cargo.toml Cargo.lock meson.build pyproject.toml \
        pixi.toml docs/source/conf.py CHANGELOG.md README.md
git commit -m "chore(version): vX.Y.Z"

Tag the release (annotated, signed if GPG is configured):
```
git tag -a vX.Y.Z -m "vX.Y.Z"
```

Push to both remotes:

git push origin main --tags
git push upstream main --tags

Publish to crates.io:
```
cargo publish
```
The v* tag push triggers the python_wheels.yml workflow which builds wheels for Linux (x86₆₄, aarch64), macOS (Intel, ARM), and Windows (x86₆₄), then publishes to PyPI via OIDC trusted publisher.

Create the GitHub release with artifacts:

cargo build --release

# Package shared/static library + headers for C/C++/Julia
mkdir -p /tmp/pkg/{lib,include}
cp target/release/libreadcon_core.so /tmp/pkg/lib/
cp target/release/libreadcon_core.a /tmp/pkg/lib/
cp include/readcon-core.h /tmp/pkg/include/
cp include/readcon-core.hpp /tmp/pkg/include/
tar czf readcon-core-vX.Y.Z-linux-x86_64.tar.gz -C /tmp pkg/

# Package Julia bindings
tar czf ReadCon.jl-vX.Y.Z.tar.gz -C julia ReadCon/

# Create release (attach capnp schema too)
gh release create vX.Y.Z \
  readcon-core-vX.Y.Z-linux-x86_64.tar.gz \
  ReadCon.jl-vX.Y.Z.tar.gz \
  schema/ReadCon.capnp \
  --repo lode-org/readcon-core \
  --title "vX.Y.Z" \
  --notes-file /tmp/cl.md

Verify all distribution channels:
- crates.io shows the new version
- PyPI shows the new version (wait for CI)
- GitHub Releases has the tarball, Julia package, and capnp schema

6.3 Initial PyPI setup (first release only)¶

For the very first release, PyPI trusted publisher is not yet configured. Publish manually:

# Build wheels
maturin build --release --features python

# Upload (first time, creates the project on PyPI)
uvx twine upload target/wheels/*

Then configure trusted publisher on PyPI:

Go to https://pypi.org/manage/project/readcon/settings/publishing/
Add a new pending publisher:
- Owner: lode-org
- Repository: readcon-core
- Workflow: python_wheels.yml
- Environment: pypi

After this, all subsequent tagged releases auto-publish via OIDC.

7 Adding new features ¶

7.1 Feature gates ¶

Optional functionality is gated behind Cargo features:

Feature	Dependencies	Purpose
parallel	rayon	Multi-frame parallel parse
rpc	capnp, capnp-rpc, tokio, etc.	Cap’n Proto RPC serving
python	pyo3	Python bindings

Add new optional features in Cargo.toml under [features].

7.2 Adding a new binding ¶

Add the binding source under src/ (e.g., src/python.rs) or as a separate package (e.g., julia/ReadCon/).
Gate behind a feature flag if it adds dependencies.
Add tests under tests/ or the binding’s own test directory.
Document in docs/orgmode/bindings.org and docs/orgmode/tutorials.org.
Add a pixi environment and tasks if applicable.

7.3 Updating the C header ¶

The C header include/readcon-core.h is generated by cbindgen. After modifying src/ffi.rs:

cbindgen --config cbindgen.toml --crate readcon --output include/readcon-core.h -- src/lib.rs

Or let the meson/cmake build regenerate it.

The C++ header include/readcon-core.hpp is hand-maintained and must be updated manually to match any changes to CAtom, CFrame, or the FFI function signatures.