Library architecture

Overview

The library is an hourglass: a Rust core, a C ABI in ffi.rs, and thin wrappers (PyO3, Julia ccall, C++ RAII). Every language talks to the same rkr_* surface; none reimplements the parser.

Python (PyO3)    Julia (ccall)    C++ (RAII header)
     \                |               /
      \               |              /
       +------- C FFI (ffi.rs) -----+
                      |
            Rust core library
       types | parser | writer | iterators

Core types (types.rs)

FrameHeader

9-line header metadata (cell, angles, atom counts, masses, spec_version, metadata). The spec_version field (u32) records the CON spec version from the JSON metadata line. The metadata field (BTreeMap<String, serde_json::Value>) stores additional key-value pairs from the JSON, preserving unrecognized keys through round-trips.

AtomDatum

Single atom data (symbol, coordinates, fixed flag, atom_id (original atom index before type-based reordering), optional velocity / force / charge / spin / magmom / displacement / spread fields).

ConFrame

Complete frame (header + atom data / SoA columns for declared sections).

ConFrameBuilder

Builder pattern for constructing frames programmatically. Accepts an optional metadata map; the builder always sets spec_version to CON_SPEC_VERSION. Exposes typed setters for common keys like energy, frame_index, time, etc.

Symbol strings use Arc<str> to avoid per-atom string clones within a type block and ensure thread-safety for parallel parsing.

Parser (parser.rs)

parse_line_of_n<T>

Generic whitespace-separated value parser for header lines.

parse_line_of_n_f64

fast-float2 specialized parser for the coordinate/velocity hot path.

parse_line_of_range_f64

Flexible parser accepting between min and max columns, padding from defaults. Used for atom lines where column 5 (atom_id) is optional.

parse_frame_header

Consumes 9 header lines. Detects JSON metadata on line 2 (starts with {) for spec v2+; non-JSON line 2 triggers legacy mode (spec_version = 1). Malformed JSON (starts with { but invalid) produces an error. Reserved metadata keys are checked against the spec schema.

parse_single_frame

Header + coordinate blocks. Accepts 4-column atom lines (column 5 defaults to sequential index). In validate mode, coordinate labels, symbols, fixed flags, atom ids, cell geometry, atom counts, and masses are checked before the frame is accepted.

• Section parsers (parse_velocity_section, parse_force_section, and the scalar / 3-vector paths for energies / charges / spins / magmoms / displacements / spreads) :: Optional blocks after coordinates. Spec-v2+ files declare sections in JSON metadata; files without a sections key can still auto-detect velocities by blank separator. Declared sections must be present at their declared position. If JSON metadata sets validate to true, section parser paths verify component symbols, exact labels, fixed masks, and atom ids against the coordinate blocks before attaching section data.

Writer (writer.rs)

ConFrameWriter<W: Write>

Generic buffered writer.

• Line 2 is always serialized as a JSON object containing con_spec_version plus any entries from frame.header.metadata. The original prebox_header[1] value is overwritten. Managed keys (con_spec_version and sections) are regenerated by the writer; when validate=true, the writer emits an empty sections array for coordinate-only frames.

• Writes header, coordinate blocks, then any declared optional sections present on the frame (has_velocities, has_forces, has_energies, has_charges, has_spins, has_magmoms, has_displacements, has_spreads) in sections order.

Iterators (iterators.rs)

ConFrameIterator

Lazy frame-by-frame parser with next() and forward() (skip without parsing atom data).

read_all_frames()

Convenience function using memmap2 for large trajectory files. With the parallel feature, Rayon parse turns on automatically when the buffer is at least 48 KiB.

read_all_frames_with_threads(path, n)

None or 0 is that automatic policy, 1 is sequential, n > 2= pins a Rayon pool (sequential fallback when parallel is off).

• parse_frames_parallel() / parse_frames_parallel_with_threads() :: Rayon-based parallel parsing behind the parallel feature gate.

FFI layer (ffi.rs)

Opaque handle pattern:

RKRConFrame

Opaque Rust frame handle.

CFrame / CAtom

Transparent C structs for direct data access.

• Iterator lifecycle: read_con_file_iterator -> con_frame_iterator_next -> rkr_frame_to_c_frame -> free_c_frame -> free_rkr_frame -> free_con_frame_iterator.

Version and spec constants:

#define RKR_CON_SPEC_VERSION 2

Compile-time spec version for #if guards.

rkr_con_spec_version()

Runtime library spec version query.

rkr_library_version()

Returns library version string (e.g. "0.5.2"). The pointer is static; do not free it.

Per-frame metadata accessors:

rkr_frame_spec_version(handle)

Returns the spec version stored in a parsed frame’s header (the version the file was written with, not the library’s version).

rkr_frame_metadata_json(handle)

Returns the full JSON metadata line as a heap-allocated C string. The caller must free with rkr_free_string().

rkr_frame_energy(handle), rkr_frame_time(handle), etc.

Typed accessors for common metadata keys.

rkr_read_all_frames

Same automatic policy as read_all_frames for the features the library was built with. The published C prefix enables parallel.

rkr_read_all_frames_n_threads(filename, num_frames, n_threads)

0 auto, 1 sequential, n_threads > 2= a Rayon pool.

rkr_has_parallel_support()

1 when Rayon is linked.

RKRStatus

Prefixed C-compatible status codes such as RKR_STATUS_SUCCESS, RKR_STATUS_NULL_POINTER, and RKR_STATUS_BUFFER_TOO_SMALL.

rkr_status_message(status)

Returns a static human-readable message for every status value.

• Builder FFI functions preserve the full atom payload. Existing boolean fixed helpers remain available, while \*_fixed_mask variants accept per-axis fixed flags. Force and velocity+force variants call the same Rust ConFrameBuilder paths as native Rust.

C++ wrapper (readcon-core.hpp)

Header-only RAII wrappers with lazy caching:

ConFrameIterator

Range-based for loop support.

ConFrame

Cached accessors for cell, angles, atoms, headers, velocities, forces, and metadata helpers.

ConFrameWriter

RAII file writer.

ConFrameBuilder

Overloads for boolean fixed flags and std::array<bool, 3> masks, with velocity, force, and velocity+force atom constructors.

Python wrapper (python.rs)

The PyO3 layer stores ConFrame.atoms as a live Python list and ConFrame.metadata as a live Python dict. Writers validate metadata as JSON-compatible values and convert the live atom list into a Rust ConFrame at write time. read_first_frame(path) uses the Rust first-frame reader; iter_con(path) exposes a Python iterator for loop-oriented frame processing.

ASE conversion maps all-fixed atoms to FixAtoms and partial masks to FixCartesian, preserving atom_id through a named ASE array.

Julia wrapper (wrapper.jl)

The Julia ccall layer exposes typed frame metadata fields populated from C FFI accessors. write_con(path, frames; precision=6) reconstructs opaque frame handles through the C builder API and writes them through the Rust writer, preserving velocities, forces, per-axis fixed masks, masses, atom ids, and JSON metadata.

C ABI install layout (cargo-c)

The [package.metadata.capi.*] keys in Cargo.toml drive cargo-c. Running cargo cinstall --prefix=$PREFIX --libdir=$PREFIX/lib --library-type cdylib produces:

Asset	Source	Install path
C header	include/readcon-core.h	$PREFIX/include/readcon-core.h
C++ header	include/readcon-core.hpp	$PREFIX/include/readcon-core.hpp
pkg-config file	cargo-c / CMake / Meson (same name)	$PREFIX/lib/pkgconfig/readcon-core.pc
cmake package	cmake/readcon-core-config.in.cmake	$PREFIX/lib/cmake/readcon-core/
Linux shared object	target/.../libreadcon_core.so	$PREFIX/lib/libreadcon_core.so{,.0.X,.0.X.Y}
macOS dylib	target/.../libreadcon_core.dylib	$PREFIX/lib/libreadcon_core.dylib
Windows DLL + import lib	target/.../readcon_core.dll{,.lib}	$PREFIX/{bin,lib}/readcon_core.dll{,.lib}

The header is shipped pre-generated at include/readcon-core.h. CMake (FetchContent / find_package), Meson (wrap / pkg.generate filebase: readcon-core), and cargo-c (generation = false) all install that file. None of them run cbindgen. The cxx source tarball (scripts/package-cxx.sh → readcon-core-cxx-$VERSION.tar.gz on the GitHub Release) is the FetchContent / wrapdb URL. The prebuilt C ABI tarball (scripts/package-clib.sh → readcon-core-clib-$VERSION-$target.tar.gz) is the Julia / Fortran / pkg-config consumer path; attach it to an existing tag with .github/workflows/c_lib_tarball.yml (workflow_dispatch tag). Windows + chemfiles is not a clib asset. Maintainers only: scripts/regen-capi-headers.sh.

The generation = false flag in [package.metadata.capi.header] tells cargo-c to copy it from the install assets list rather than running cbindgen at install time. Optional metatensor C exports are gated (READCON_CORE_HAS_METATENSOR); prefer include/readcon-metatensor.h (metatensor.h first). Fat builds write target/<profile>/readcon-metatensor.env for include/lib. Ownership: tensor_block_into_raw_mts then mts_block_free only (bindings page). Maintainers may draft a header with scripts/regen-capi-headers.sh. That tool is not a consumer or CI dependency. CI compiles C/C++ against the committed header and checks that the library exports the declared rkr_* symbols.

The [package.metadata.capi.pkg_config] filename = "readcon-core" override is load-bearing: without it cargo-c defaults the .pc filename to the [lib] name (readcon_core), which downstream packagers check for at the hyphenated name.

C ABI stability contract (rkr_*)

The published C surface is the rkr_* API in the shipped include/readcon-core.h. Consumers compile against that header; they do not run cbindgen. The C++ wrapper (include/readcon-core.hpp) is a header-only RAII layer over the same symbols.

What is frozen in 0.14.x **

• Symbol prefix rkr_* and the opaque handle types (RKRConFrame, RKRConFrameWriter, RKRConFrameBuilder, CConFrameIterator).

• Ownership rules in the header comment: caller-owned handles, free_rkr_frame_array vs free_rkr_frame, rkr_free_string for heap char*, process-static rkr_library_version.

• Sentinel policy: NaN for absent floats, UINT64_MAX for absent unsigned integers, NULL for absent strings.

• Error policy: functions that allocate or validate return NULL or a negative RKRStatus; they do not abort the process. Builder rkr_frame_builder_build returns NULL on MassMismatch and other ParseError cases.

• Feature-gated symbols (RKR_STATUS_FEATURE_DISABLED) stay in the header; a lean build returns that status instead of hiding the symbol.

• Periodic-table helpers stay at Z=1..92 plus D/T. The C ABI does not grow a Z=118 table.

What is not a 1.0 freeze **

The crate, wheels, and Fortran package are 0.16.1. PyPI still reports Development Status :: 4 - Beta. A 1.0.0 cut is the production classifier and the SONAME/semver major bump. Until that tag:

• Additive rkr_* symbols may appear in a 0.14.z patch.

• Existing 0.14.x signatures, sentinels, and ownership do not change in a patch.

• A breaking C change requires a 0.15+ minor (0.x semver) and a matching header/SONAME bump. Downstream packagers that ship run_exports should pin an upper_bound of x.x for that reason.

Drift check **

The committed include/readcon-core.h is the C ABI. CI (ci_cxx.yml, Meson C/C++ examples, check_feature_matrix.sh) compiles against that file and nm-checks the exported symbols. cbindgen is a maintainer draft helper only.

Design rationale

The hourglass is the design: Rust core, rkr_* C ABI, wrappers for C++, Python, Julia, and Fortran. The full CON payload (constraints, optional section data, atom_id, JSON) is available without a Python interpreter on the I/O path.

Also in-tree: chemfiles import/selection (land structures as CON), SoA + DLPack export (optional CUDA), metatensor TensorBlock export, index_proj for campaign screening (readcon-db), Cap’n Proto RPC. Lean builds return RKR_STATUS_FEATURE_DISABLED for missing features. UTF-8 CON remains authoritative on disk and in corpora.

Related work and roles: Frequently Asked Questions. Spec: The CON File Format Specification. Evolution: Format Evolution and Design Rationale. Measurements: Performance Benchmarks.