==================== Library architecture ==================== Overview -------- readcon-core is structured as an hourglass API: a Rust core with thin FFI and language-specific wrapper layers. :: Python (PyO3) Julia (ccall) C++ (RAII header) \ | / \ | / +------- C FFI (ffi.rs) -----+ | Rust core library types | parser | writer | iterators Core types (types.rs) --------------------- ``FrameHeader`` 9-line header metadata (cell, angles, atom counts, masses, spec\_version, metadata). The ``spec_version`` field (``u32``) records the CON spec version from the JSON metadata line. The ``metadata`` field (``BTreeMap``) stores additional key-value pairs from the JSON, preserving unrecognized keys through round-trips. ``AtomDatum`` Single atom data (symbol, coordinates, fixed flag, atom\_id (original atom index before type-based reordering), optional velocity / force / charge / spin / magmom fields). ``ConFrame`` Complete frame (header + atom data / SoA columns for declared sections). ``ConFrameBuilder`` Builder pattern for constructing frames programmatically. Accepts an optional ``metadata`` map; the builder always sets ``spec_version`` to ``CON_SPEC_VERSION``. Exposes typed setters for common keys like ``energy``, ``frame_index``, ``time``, etc. Symbol strings use ``Arc`` to avoid per-atom string clones within a type block and ensure thread-safety for parallel parsing. Parser (parser.rs) ------------------ ``parse_line_of_n`` Generic whitespace-separated value parser for header lines. ``parse_line_of_n_f64`` fast-float2 specialized parser for the coordinate/velocity hot path. ``parse_line_of_range_f64`` Flexible parser accepting between ``min`` and ``max`` columns, padding from defaults. Used for atom lines where column 5 (atom\_id) is optional. ``parse_frame_header`` Consumes 9 header lines. Detects JSON metadata on line 2 (starts with ``{``) for spec v2+; non-JSON line 2 triggers legacy mode (``spec_version = 1``). Malformed JSON (starts with ``{`` but invalid) produces an error. Reserved metadata keys are checked against the spec schema. ``parse_single_frame`` Header + coordinate blocks. Accepts 4-column atom lines (column 5 defaults to sequential index). In validate mode, coordinate labels, symbols, fixed flags, atom ids, cell geometry, atom counts, and masses are checked before the frame is accepted. - Section parsers (``parse_velocity_section``, ``parse_force_section``, and the scalar / 3-vector paths for ``energies`` / ``charges`` / ``spins`` / ``magmoms``) \:\: Optional blocks after coordinates. Spec-v2+ files declare sections in JSON metadata; files without a ``sections`` key can still auto-detect velocities by blank separator. Declared sections must be present at their declared position. If JSON metadata sets ``validate`` to ``true``, section parser paths verify component symbols, exact labels, fixed masks, and atom ids against the coordinate blocks before attaching section data. Writer (writer.rs) ------------------ ``ConFrameWriter`` Generic buffered writer. - Line 2 is always serialized as a JSON object containing ``con_spec_version`` plus any entries from ``frame.header.metadata``. The original ``prebox_header[1]`` value is overwritten. Managed keys (``con_spec_version`` and ``sections``) are regenerated by the writer; when ``validate=true``, the writer emits an empty ``sections`` array for coordinate-only frames. - Writes header, coordinate blocks, then any declared optional sections present on the frame (``has_velocities``, ``has_forces``, ``has_energies``, ``has_charges``, ``has_spins``, ``has_magmoms``) in ``sections`` order. Iterators (iterators.rs) ------------------------ ``ConFrameIterator`` Lazy frame-by-frame parser with ``next()`` and ``forward()`` (skip without parsing atom data). ``read_all_frames()`` Convenience function using memmap2 for large trajectory files. ``parse_frames_parallel()`` Rayon-based parallel parsing behind the ``parallel`` feature gate. FFI layer (ffi.rs) ------------------ Opaque handle pattern: ``RKRConFrame`` Opaque Rust frame handle. ``CFrame`` / ``CAtom`` Transparent C structs for direct data access. - Iterator lifecycle: ``read_con_file_iterator`` -> ``con_frame_iterator_next`` -> ``rkr_frame_to_c_frame`` -> ``free_c_frame`` -> ``free_rkr_frame`` -> ``free_con_frame_iterator``. Version and spec constants: ``#define RKR_CON_SPEC_VERSION 2`` Compile-time spec version for ``#if`` guards. ``rkr_con_spec_version()`` Runtime library spec version query. ``rkr_library_version()`` Returns library version string (e.g. ``"0.5.2"``). The pointer is static; do not free it. Per-frame metadata accessors: ``rkr_frame_spec_version(handle)`` Returns the spec version stored in a parsed frame's header (the version the file was written with, not the library's version). ``rkr_frame_metadata_json(handle)`` Returns the full JSON metadata line as a heap-allocated C string. The caller must free with ``rkr_free_string()``. ``rkr_frame_energy(handle)``, ``rkr_frame_time(handle)``, etc. Typed accessors for common metadata keys. ``RKRStatus`` Prefixed C-compatible status codes such as ``RKR_STATUS_SUCCESS``, ``RKR_STATUS_NULL_POINTER``, and ``RKR_STATUS_BUFFER_TOO_SMALL``. ``rkr_status_message(status)`` Returns a static human-readable message for every status value. - Builder FFI functions preserve the full atom payload. Existing boolean fixed helpers remain available, while ``\*_fixed_mask`` variants accept per-axis fixed flags. Force and velocity+force variants call the same Rust ``ConFrameBuilder`` paths as native Rust. C++ wrapper (readcon-core.hpp) ------------------------------ Header-only RAII wrappers with lazy caching: ``ConFrameIterator`` Range-based for loop support. ``ConFrame`` Cached accessors for cell, angles, atoms, headers, velocities, forces, and metadata helpers. ``ConFrameWriter`` RAII file writer. ``ConFrameBuilder`` Overloads for boolean fixed flags and ``std::array`` masks, with velocity, force, and velocity+force atom constructors. Python wrapper (python.rs) -------------------------- The PyO3 layer stores ``ConFrame.atoms`` as a live Python list and ``ConFrame.metadata`` as a live Python dict. Writers validate metadata as JSON-compatible values and convert the live atom list into a Rust ``ConFrame`` at write time. ``read_first_frame(path)`` uses the Rust first-frame reader; ``iter_con(path)`` exposes a Python iterator for loop-oriented frame processing. ASE conversion maps all-fixed atoms to ``FixAtoms`` and partial masks to ``FixCartesian``, preserving ``atom_id`` through a named ASE array. Julia wrapper (wrapper.jl) -------------------------- The Julia ccall layer exposes typed frame metadata fields populated from C FFI accessors. ``write_con(path, frames; precision=6)`` reconstructs opaque frame handles through the C builder API and writes them through the Rust writer, preserving velocities, forces, per-axis fixed masks, masses, atom ids, and JSON metadata. C ABI install layout (cargo-c) ------------------------------ The ``[package.metadata.capi.*]`` keys in ``Cargo.toml`` drive `cargo-c `_. Running ``cargo cinstall --prefix=$PREFIX --libdir=$PREFIX/lib --library-type cdylib`` produces: .. table:: +--------------------------+----------------------------------------+--------------------------------------------------+ | Asset | Source | Install path | +==========================+========================================+==================================================+ | C header | ``include/readcon-core.h`` | ``$PREFIX/include/readcon-core.h`` | +--------------------------+----------------------------------------+--------------------------------------------------+ | C++ header | ``include/readcon-core.hpp`` | ``$PREFIX/include/readcon-core.hpp`` | +--------------------------+----------------------------------------+--------------------------------------------------+ | pkg-config file | cargo-c / CMake / Meson (same name) | ``$PREFIX/lib/pkgconfig/readcon-core.pc`` | +--------------------------+----------------------------------------+--------------------------------------------------+ | cmake package | ``cmake/readcon-core-config.in.cmake`` | ``$PREFIX/lib/cmake/readcon-core/`` | +--------------------------+----------------------------------------+--------------------------------------------------+ | Linux shared object | ``target/.../libreadcon_core.so`` | ``$PREFIX/lib/libreadcon_core.so{,.0.X,.0.X.Y}`` | +--------------------------+----------------------------------------+--------------------------------------------------+ | macOS dylib | ``target/.../libreadcon_core.dylib`` | ``$PREFIX/lib/libreadcon_core.dylib`` | +--------------------------+----------------------------------------+--------------------------------------------------+ | Windows DLL + import lib | ``target/.../readcon_core.dll{,.lib}`` | ``$PREFIX/{bin,lib}/readcon_core.dll{,.lib}`` | +--------------------------+----------------------------------------+--------------------------------------------------+ The header is shipped pre-generated at ``include/readcon-core.h``. CMake (``FetchContent`` / ``find_package``), Meson (wrap / ``pkg.generate`` ``filebase: readcon-core``), and cargo-c (``generation = false``) all install that file. None of them run cbindgen. The cxx source tarball (``scripts/package-cxx.sh`` → ``readcon-core-cxx-$VERSION.tar.gz`` on the GitHub Release) is the FetchContent / wrapdb URL. Maintainers only: ``scripts/regen-capi-headers.sh``. The ``generation = false`` flag in ``[package.metadata.capi.header]`` tells cargo-c to copy it from the install assets list rather than running cbindgen at install time. Optional metatensor C exports are gated (``READCON_CORE_HAS_METATENSOR``); prefer ``include/readcon-metatensor.h`` (``metatensor.h`` first). Fat builds write ``target//readcon-metatensor.env`` for include/lib. Ownership: ``tensor_block_into_raw_mts`` then ``mts_block_free`` only (bindings page). Run ``scripts/regen-capi-headers.sh`` when the FFI surface in ``src/ffi.rs`` changes; CI verifies drift via ``scripts/regen-capi-headers.sh --check``. The ``[package.metadata.capi.pkg_config] filename = "readcon-core"`` override is load-bearing: without it cargo-c defaults the .pc filename to the ``[lib]`` name (``readcon_core``), which downstream packagers (the conda-forge feedstock in particular) check for at the hyphenated name. .. _design-rationale: Design rationale ---------------- readcon-core is the hourglass library that puts CON into every language path: Rust core, ``rkr_*`` C ABI, wrappers for C++, Python, Julia, and Fortran. Full CON payload (constraints, optional section data, ``atom_id``, JSON) without a Python interpreter on the I/O path. Also in-tree: chemfiles import/selection (land structures **as** CON), SoA + DLPack export (optional CUDA), metatensor ``TensorBlock`` export, ``index_proj`` for campaign screening (``readcon-db``), Cap'n Proto RPC. Lean builds return ``RKR_STATUS_FEATURE_DISABLED`` for missing features. UTF-8 CON remains authoritative on disk and in corpora. Related work and roles: :doc:`faq`. Spec: :doc:`spec`. Evolution: :doc:`evolution`. Measurements: :doc:`benchmarks`.