================================= The CON File Format Specification ================================= :Date: 2026-06-27 Specification ------------- Version 3 Date 2026-06-27 Status Stable Reference implementation `readcon-core `_ Surface grammar (PEG) ``grammar/readcon.pest`` (Pest); validate with ``cargo test --features grammar`` This document defines versions 2 and 3 of the CON file format; version 3 is the current version and adds required ``units`` metadata on top of the version 2 requirements. It supersedes all prior informal descriptions. New implementations SHOULD target version 3 and MUST accept version 2. The keywords MUST, MUST NOT, REQUIRED, SHALL, SHOULD, SHOULD NOT, MAY, and OPTIONAL follow `RFC 2119 `_ semantics. Overview -------- CON stores atomic configurations for rare-event and transition-state checkpoints: cell, type-grouped coordinates, per-direction constraints, atom identity, optional velocities/forces, and versioned JSON metadata. The layout originated in the `eOn `_ rare-event stack :cite:t:`chillEONSoftwareLong2014`; this document is the formal specification. The reference implementation is ``readcon-core`` (multi-language hourglass ABI). A CON file contains one or more *frames*. Each frame encodes a simulation cell, per-type metadata (masses, atom counts), and per-atom data (coordinates, constraints, identity). Optional sections add velocities, forces, or other per-atom vector/scalar data. Line 2 carries versioned JSON metadata. File extensions --------------- ``.con`` Coordinate-only configuration files. ``.convel`` Configuration files with velocity data per frame. ``.con.gz`` Gzip-compressed CON files (see `compression`_). Encoding -------- CON files MUST use UTF-8. Line endings MAY be LF (``\n``) or CRLF (``\r\n``); parsers MUST accept both. All numeric values use ASCII decimal representation (no locale-dependent formatting). .. _surface-grammar: Surface grammar (PEG) --------------------- The machine-readable surface syntax for CON/convel lives in the repository at ``grammar/readcon.pest`` (a `Pest `_ PEG). It is the formal companion to this prose specification for **syntactic** structure (header layout, type blocks, optional section blocks, multi-frame concatenation). The reference implementation in ``readcon-core`` does ****not**** use the PEG on the I/O hot path; it implements the same surface rules plus semantic checks (atom counts, metadata JSON, validation flags). Enable the ``grammar`` Cargo feature to parse buffers with the PEG for tooling or tests: :: cargo test --features grammar Frame structure --------------- Each frame consists of: 1. A 9-line header. 2. One coordinate block per atom type. 3. Zero or more additional per-atom sections (velocities, forces), each preceded by a blank separator line. Multiple frames are concatenated directly with no inter-frame separator. Header (9 lines) ---------------- .. table:: +------+-------------------+--------------------------------+------------------------------------------+ | Line | Name | Content | Example | +======+===================+================================+==========================================+ | 1 | Generator comment | Free-form text | ``Generated by eOn`` | +------+-------------------+--------------------------------+------------------------------------------+ | 2 | Metadata | JSON object or free-form text | ``{"con_spec_version":3,"units":{...}}`` | +------+-------------------+--------------------------------+------------------------------------------+ | 3 | Cell dimensions | 3 floats: Lx Ly Lz | ``15.3456 21.702 100.0`` | +------+-------------------+--------------------------------+------------------------------------------+ | 4 | Cell angles | 3 floats: alpha beta gamma | ``90.0 90.0 90.0`` | +------+-------------------+--------------------------------+------------------------------------------+ | 5 | Reserved | Free-form text (round-tripped) | ``0 0`` | +------+-------------------+--------------------------------+------------------------------------------+ | 6 | Reserved | Free-form text (round-tripped) | ``218 0 1`` | +------+-------------------+--------------------------------+------------------------------------------+ | 7 | Atom type count | 1 integer: N | ``2`` | +------+-------------------+--------------------------------+------------------------------------------+ | 8 | Atoms per type | N integers | ``216 2`` | +------+-------------------+--------------------------------+------------------------------------------+ | 9 | Mass per type | N floats (atomic mass units) | ``63.546 1.00793`` | +------+-------------------+--------------------------------+------------------------------------------+ Line 1: Generator comment ~~~~~~~~~~~~~~~~~~~~~~~~~ Free-form text. Writers SHOULD set this to a human-readable identifier. Parsers MUST preserve it through round-trips but MUST NOT assign it semantic meaning. .. _metadata: Line 2: Metadata ~~~~~~~~~~~~~~~~ Line 2 carries machine-readable metadata as a single-line `JSON `_ object. Version 2+ files ^^^^^^^^^^^^^^^^ Writers MUST emit a JSON object containing at least ``con_spec_version`` with an integer value: :: {"con_spec_version":2} Additional keys MAY appear. Parsers MUST preserve unrecognized keys through round-trips. Reserved metadata keys listed in `metadata-keys`_ MUST use the declared JSON type. The ``sections`` key, when present, MUST be an array of strings. The ``validate`` key, when present, MUST be a boolean. Legacy (pre-v2) files ^^^^^^^^^^^^^^^^^^^^^ Files produced before this specification may contain free-form text on line 2. A conforming parser detects the format by checking whether line 2, after trimming whitespace, starts with ``{``: - Starts with ``{``: parse as JSON, extract ``con_spec_version``. - Does not start with ``{``: treat as legacy (implicit version 1). If line 2 starts with ``{`` but contains malformed JSON, the parser MUST report an error. If the JSON object lacks ``con_spec_version``, the parser MUST report an error. If ``con_spec_version`` exceeds the highest version the parser supports, the parser MUST report an error. Lines 3-4: Cell geometry ~~~~~~~~~~~~~~~~~~~~~~~~ Line 3: three whitespace-separated floats (cell edge lengths in angstroms). Line 4: three whitespace-separated floats (cell angles in degrees). Tabs and spaces are both valid separators. For non-orthogonal cells, the angle-based representation introduces floating-point drift through trigonometric round-trips. Writers SHOULD include the exact 3x3 lattice vector matrix in the JSON metadata via the ``lattice_vectors`` key (see `lattice-vectors`_). When ``lattice_vectors`` is present, readers SHOULD prefer it over the length/angle values on lines 3-4. When ``validate`` is ``true``, readers MUST reject zero or negative cell lengths and MUST reject angles outside the open interval ``(0, 180)`` degrees. Lines 5-6: Reserved ~~~~~~~~~~~~~~~~~~~ Free-form text with no defined semantics. Writers MAY emit empty lines. Parsers MUST preserve these for round-trip fidelity. Lines 7-9: Type metadata ~~~~~~~~~~~~~~~~~~~~~~~~ Line 7: single positive integer N (number of atom types). Line 8: exactly N positive integers (atom count per type). Line 9: exactly N floats (atomic mass per type, in amu). Coordinate blocks ----------------- For each atom type *i* (1 to N), in the order declared in lines 8-9: 1. **Symbol line**: chemical symbol (e.g., ``Cu``, ``H``). 2. **Label line**: ``Coordinates of Component /i/`` 3. **Atom lines**: one per atom, containing: .. table:: +--------+-------+-----------------------------------------+ | Column | Type | Description | +========+=======+=========================================+ | 1 | float | x coordinate (angstroms) | +--------+-------+-----------------------------------------+ | 2 | float | y coordinate (angstroms) | +--------+-------+-----------------------------------------+ | 3 | float | z coordinate (angstroms) | +--------+-------+-----------------------------------------+ | 4 | int | Constraint bitmask (see `constraints`_) | +--------+-------+-----------------------------------------+ | 5 | int | Atom index (see `atom-index`_) | +--------+-------+-----------------------------------------+ Columns are whitespace-separated. .. _constraints: Per-direction constraints (column 4) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Column 4 encodes per-direction constraint flags as a bitmask. Bit 0 = x, bit 1 = y, bit 2 = z. .. table:: +-------+----------------------------+----------------------------+ | Value | Spec 1 (no JSON line 2) | Spec 2+ | +=======+============================+============================+ | 0 | Free | Free | +-------+----------------------------+----------------------------+ | 1 | All-fixed (legacy boolean) | x-only | +-------+----------------------------+----------------------------+ | 2-6 | Unused / undefined | Per-direction combinations | +-------+----------------------------+----------------------------+ | 7 | Unused / undefined | Fixed in all directions | +-------+----------------------------+----------------------------+ Spec-1 files used column 4 as a boolean (0 free, 1 fixed). Spec-2 files use the 3-bit mask. Readers MUST therefore consult ``con_spec_version`` before decoding 1: spec 1 keeps the legacy all-fixed reading; spec 2 treats 1 as x-only. Writers MUST emit 7 for all-fixed atoms, never 1. Spec-2 writers MUST emit 1 for an atom fixed in x alone. .. _atom-index: Atom index (column 5) ~~~~~~~~~~~~~~~~~~~~~ The atom index preserves the original position of each atom before type-based grouping. The CON format groups atoms by element type, which reorders them. Without a persistent index, the original ordering cannot be recovered after a read-write cycle. Version 2 requirements: - Writers MUST emit column 5 on every atom line. - Column 5 MUST contain the pre-grouping index. - Readers MUST parse and preserve column 5 through write-back. Version 1 behavior: - Column 5 is present but its semantics are undefined. Readers SHOULD accept 4-column atom lines. When column 5 is absent, default to the sequential position within the frame (0, 1, 2, ...). .. _sections: Additional per-atom sections ---------------------------- After coordinate blocks, a frame MAY contain additional per-atom data sections. Each section follows the same block structure: a blank separator line, then per-component blocks (symbol line, label line, data lines). Section declaration ~~~~~~~~~~~~~~~~~~~ Version 2 files declare sections in the JSON metadata using the ``sections`` key: :: {"con_spec_version":2,"sections":["velocities","forces"]} The parser reads sections in the declared order. Parsers MUST reject unknown section names with an error. Every declared section MUST be present, complete, and parseable at its declared position. An empty ``sections`` array declares that no additional per-atom sections follow. .. table:: +----------------+---------------------------------+---------+-------------------------------+ | Section name | Label pattern | Columns | Data | +================+=================================+=========+===============================+ | ``velocities`` | ``Velocities of Component /i/`` | 5 | vx vy vz fixed\_flag atom\_id | +----------------+---------------------------------+---------+-------------------------------+ | ``forces`` | ``Forces of Component /i/`` | 5 | fx fy fz fixed\_flag atom\_id | +----------------+---------------------------------+---------+-------------------------------+ | ``energies`` | ``Energies of Component /i/`` | 3 | energy fixed\_flag atom\_id | +----------------+---------------------------------+---------+-------------------------------+ | ``charges`` | ``Charges of Component /i/`` | 3 | charge fixed\_flag atom\_id | +----------------+---------------------------------+---------+-------------------------------+ | ``spins`` | ``Spins of Component /i/`` | 3 | spin fixed\_flag atom\_id | +----------------+---------------------------------+---------+-------------------------------+ | ``magmoms`` | ``Magmoms of Component /i/`` | 5 | mx my mz fixed\_flag atom\_id | +----------------+---------------------------------+---------+-------------------------------+ The ``energies`` section carries one scalar per atom, useful for ML potentials that decompose total energy into local contributions. Writers MAY emit it alongside ``forces``, alone, or omit it entirely. ``charges`` and ``spins`` are optional scalar sections (same column layout as ``energies``). ``magmoms`` is an optional 3-vector section (same layout as ``velocities``). They are reserved names on the existing v2/v3 declared ``sections`` surface: files MAY use ``con_spec_version`` 2 or 3; a format major bump is not required for these optional blocks. When the ``energies`` section is present: - The per-frame total ``energy`` metadata key SHOULD equal the sum of the per-atom contributions. Implementations MAY warn on a mismatch but MUST NOT reject the frame on that ground (a reader cannot tell apart a numerical-noise mismatch from a deliberate definition where the per-atom decomposition does not sum to the total). - The total ``energy`` metadata key MAY still be absent, in which case the per-atom contributions are the only energy data on the frame. - The ``fixed_flag`` and ``atom_id`` columns SHOULD match the coordinate block, exactly as for ``velocities`` and ``forces``. In ``validate=true`` mode they MUST match. The ``fixed_flag`` and ``atom_id`` columns in every additional section repeat the coordinate-block identity data for the same atom ordering. Writers SHOULD emit the same values as the coordinate block. Readers associate section rows by component order and MAY ignore the duplicate identity columns after parsing the row shape; in ``validate=true`` mode readers MUST verify that the ``fixed_flag`` and ``atom_id`` columns match the coordinate block. .. _validation-mode: Validation mode ~~~~~~~~~~~~~~~ Version 2 files MAY set ``validate`` to ``true`` in the JSON metadata: :: {"con_spec_version":2,"sections":["velocities","forces"],"validate":true} When ``validate`` is ``true``, the ``sections`` key MUST be present, even when the value is an empty array. Conforming readers MUST verify that the frame satisfies strict v2 invariants before accepting it. At minimum, this validation MUST check: - Reserved metadata keys use the declared JSON types. - Numeric tokens are finite. - Cell lengths are positive, cell angles are in ``(0, 180)`` degrees, atom counts are positive, and masses are positive. - Coordinate component labels exactly match ``Coordinates of Component /i/``. - Component symbols are recognized element symbols, or ``X`` for an explicitly unknown element. - Coordinate and additional-section ``fixed_flag`` and ``atom_id`` fields are exact integer tokens. - The section component symbol matches the coordinate component symbol. - The section label exactly matches the declared section and component number (for example, ``Velocities of Component 1``). - Each section row's ``fixed_flag`` decodes to the same per-axis fixed mask as the corresponding coordinate row. - Each section row's ``atom_id`` equals the corresponding coordinate row's ``atom_id``. If any check fails, the reader MUST reject the frame. When ``validate`` is absent or false, readers MAY parse files by associating section rows by component order and ignoring duplicate identity column mismatches after parsing the row shape. Error paths and ParseError variants ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The reference reader (``readcon-core``) surfaces validation failures as typed ``ParseError`` variants. Other implementations are expected to return analogous structured errors, but the variant names below are specific to ``readcon-core`` and are listed here so that the spec and the reference implementation stay in sync. .. table:: +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ParseError variant | Fires when | +===============================================+===============================================================================================================================================================================================================================================================================================================+ | ``MissingSpecVersion`` | Line 2 starts with ``{`` but no ``con_spec_version`` key. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``UnsupportedSpecVersion(v)`` | ``con_spec_version`` exceeds ``CON_SPEC_VERSION``. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``InvalidMetadataJson(msg)`` | JSON is malformed, ``validate`` is not a boolean, ``sections`` is not an array of strings, or a reserved key has the wrong JSON type. Also fires when ``validate=true`` omits the ``sections`` key. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``ValidationError(msg)`` (validate=true only) | Cell geometry, masses, coordinate component label, component symbol, ``fixed_flag`` / ``atom_id`` integer tokens, section component symbol, section label, or per-row identity columns mismatch the rules above. Numeric tokens that are not finite are rejected here as well, after parse-time tokenization. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``UnknownSection(name)`` | A name in the ``sections`` array does not match a known section. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``IncompleteHeader`` | Fewer than 9 header lines remain. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``IncompleteFrame`` | Coordinate block ends short. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``IncompleteVelocitySection`` | Declared velocity section ends short or is absent. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``IncompleteForceSection`` | Declared force section ends short or is absent. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ A minimal example of each path: the file ``resources/test/tiny_cuh2_strict_invalid.con`` (if present) and the test cases under ``src/parser.rs::tests`` and ``tests/parseforces.rs`` exercise these branches and serve as executable references. Legacy section detection ~~~~~~~~~~~~~~~~~~~~~~~~ Files without a ``sections`` key use blank-separator detection: if a blank line follows coordinate blocks, the parser attempts to parse a velocity section. A present ``sections`` key disables this fallback, including when the value is ``[]``. This preserves backward compatibility with existing ``.convel`` files while giving v2 writers a precise declaration mechanism. Writers SHOULD always emit the ``sections`` key when writing additional sections. Velocity section ~~~~~~~~~~~~~~~~ Per component *i*: blank separator, symbol line, ``Velocities of Component /i/`` label, then one line per atom: ``vx vy vz fixed_flag atom_id``. Force section ~~~~~~~~~~~~~ Per component *i*: blank separator, symbol line, ``Forces of Component /i/`` label, then one line per atom: ``fx fy fz fixed_flag atom_id``. Frames with forces SHOULD include the ``potential`` and ``energy`` metadata keys. Extending with new section types ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ New section types follow the same pattern: declare in the ``sections`` array, use a blank separator, symbol line, `` of Component /i/`` label, and data lines. Multi-frame files ----------------- Frames are concatenated with no separator. After parsing a frame's data, the parser attempts the next 9-line header. If fewer than 9 lines remain, parsing ends. Writers MUST NOT insert extra blank lines between frames. Data types and precision ------------------------ - **Floats**: any valid decimal representation. Writers SHOULD emit at least 6 significant digits. For lossless f64 round-tripping, 17 digits suffice. Readers MUST reject non-finite values (``NaN``, ``Infinity``, ``-Infinity``). - **Integers**: constraint bitmask (0-7), atom\_id, natm\_types, and natms\_per\_type are non-negative integers. .. _compression: Compression ----------- CON files MAY be gzip-compressed. Readers SHOULD detect compression by checking the first two bytes for the gzip magic number (``0x1f 0x8b``) rather than relying on file extension. The decompressed content MUST be a valid CON file. Constraints and limits ---------------------- - Atoms MUST appear grouped by type, in header-declared order. - Component numbering starts at 1. - Total atom count equals the sum of line 8 values. - Symbol strings SHOULD match `IUPAC `_ element symbols. - No upper limit on atom types or atom count is imposed. .. _metadata-keys: Recommended metadata keys ------------------------- All keys are OPTIONAL. Parsers MUST preserve unrecognized keys. .. table:: +------------------------+-----------------+------------------------------------------------------------------+ | Key | Type | Description | +========================+=================+==================================================================+ | ``sections`` | array of string | Declared per-atom sections (see `sections`_) | +------------------------+-----------------+------------------------------------------------------------------+ | ``validate`` | bool | Enable strict v2 validation (see `validation-mode`_) | +------------------------+-----------------+------------------------------------------------------------------+ | ``generator`` | string | Tool name and version | +------------------------+-----------------+------------------------------------------------------------------+ | ``units`` | object | Unit system (see `units`_) | +------------------------+-----------------+------------------------------------------------------------------+ | ``pbc`` | 3 bool array | Periodic boundary conditions (see `pbc`_) | +------------------------+-----------------+------------------------------------------------------------------+ | ``lattice_vectors`` | 3x3 float array | Exact 3x3 cell matrix (see `lattice-vectors`_) | +------------------------+-----------------+------------------------------------------------------------------+ | ``energy`` | float | Per-frame total energy | +------------------------+-----------------+------------------------------------------------------------------+ | ``potential`` | object | ``{"type":"EMT","params":{...}}`` | +------------------------+-----------------+------------------------------------------------------------------+ | ``frame_index`` | int | Zero-based frame number in trajectory | +------------------------+-----------------+------------------------------------------------------------------+ | ``time`` | float | Simulation time | +------------------------+-----------------+------------------------------------------------------------------+ | ``timestep`` | float | Integration timestep | +------------------------+-----------------+------------------------------------------------------------------+ | ``neb_bead`` | int | NEB bead (image) index | +------------------------+-----------------+------------------------------------------------------------------+ | ``neb_band`` | int | NEB band index | +------------------------+-----------------+------------------------------------------------------------------+ | ``convergence_fmax`` | float | Convergence criterion: max force component | +------------------------+-----------------+------------------------------------------------------------------+ | ``convergence_energy`` | float | Convergence criterion: energy change threshold | +------------------------+-----------------+------------------------------------------------------------------+ | ``converged`` | bool | Whether convergence criteria are met | +------------------------+-----------------+------------------------------------------------------------------+ | ``fmax`` | float | Current maximum force component across free atoms | +------------------------+-----------------+------------------------------------------------------------------+ | ``bonds`` | array | Optional frame topology (see `bonds`_); NOT a ``sections`` block | +------------------------+-----------------+------------------------------------------------------------------+ .. _bonds: Frame topology (bonds) ~~~~~~~~~~~~~~~~~~~~~~ The ``bonds`` key is OPTIONAL frame-level topology for tools that need explicit connectivity (chemfiles selection grammar ``bonds:`` / ``angles:`` / ``is_bonded``, molecular viewers, etc.). It is NOT a per-atom ``sections`` entry and MUST NOT appear in the ``sections`` array. Each element is either: - A length-2 pair ``[i, j]`` of non-negative integers, or - An object ``{"i": i, "j": j, "order"?: integer}`` where ``order`` is optional chemfiles-style bond order (0=unknown, 1=single, 2=double, 3=triple, ...). Indices are 0-based into the frame ``atom_data`` order (coordinate block order), NOT ``atom_id`` column values. Absent or empty ``bonds`` means no topology is declared (legacy CON files). Implementations MUST preserve unknown shapes when ``validate`` is false; with ``validate=true``, malformed entries MUST be rejected. Example: .. code:: json {"con_spec_version":2,"bonds":[[0,1],[0,2],{"i":1,"j":2,"order":1}]} When the optional chemfiles feature is enabled, readers/writers MAY project ``bonds`` onto chemfiles topology for selection and MAY import chemfiles topology back into ``bonds`` on chemfiles→CON conversion. The CON spec version remains 2; ``bonds`` is an additive optional metadata key (library semver may bump for API). **Chemfiles import sidecars (optional, not CON columns).** Implementations that import via chemfiles MAY store extra JSON keys such as ``chemfiles_atom_names`` and ``chemfiles_atom_types`` (string arrays in chemfiles / ``atom_id`` order) so selection can restore chemfiles display ``name`` distinct from atomic ``type`` while the coordinate blocks still use one ``symbol`` per atom. These keys are implementation extras: parsers MUST preserve unknown keys; writers MUST NOT require them for valid v2 files. Hand-authored frames without sidecars behave as symbol-only for selection. **Selection (library behaviour).** Match indices are reported in type-grouped ``atom_data`` order; bond endpoints use the same index space (``atom_id`` remap on import). Residue / ``resname``, properties not retained on import, topology beyond pair ``bonds``, and geometry-threshold minidialects are outside the CON selection surface because the format does not carry that data. Narrative: ``chemfiles-explain.org``; APIs: ``bindings.org``. .. _units: Units ~~~~~ The ``units`` key maps physical dimensions to unit strings (metatomic-style expressions: named bases combined with ``\*`` / ``/`` / ``^``). .. table:: +--------------+----------------+-----------------------------------+ | Dimension | Default (LODE) | Examples | +==============+================+===================================+ | ``length`` | angstrom | ``angstrom``, ``bohr``, ``nm`` | +--------------+----------------+-----------------------------------+ | ``mass`` | amu | ``amu``, ``kg`` | +--------------+----------------+-----------------------------------+ | ``time`` | fs | ``fs``, ``ps`` | +--------------+----------------+-----------------------------------+ | ``energy`` | eV | ``eV``, ``hartree``, ``kcal/mol`` | +--------------+----------------+-----------------------------------+ | ``velocity`` | derived | length / time | +--------------+----------------+-----------------------------------+ | ``force`` | derived | energy / length | +--------------+----------------+-----------------------------------+ **Version 3:** ``units`` is **required**. The object MUST include non-empty ``length`` and ``energy`` strings whose dimensions match those quantities. Optional keys (``mass``, ``time``, ``velocity``, ``force``) MUST be dimensionally valid when present. Version 2 MAY omit ``units``; readers SHOULD preserve the key when present. Library conversion uses ``unit_conversion_factor(from, to)`` (see ``src/units.rs``). .. _pbc: Periodic boundary conditions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``pbc`` key declares which cell directions are periodic as a 3-element boolean array: ``[true, true, true]`` for full 3D periodicity, ``[true, true, false]`` for a slab model with vacuum in z. When absent, readers SHOULD assume ``[true, true, true]`` (the default for bulk simulations). :: {"con_spec_version":2,"pbc":[true,true,false]} .. _lattice-vectors: Lattice vectors ~~~~~~~~~~~~~~~ The ``lattice_vectors`` key provides the exact 3x3 cell matrix as a nested array of three row vectors (angstroms): :: {"con_spec_version":2,"lattice_vectors":[[10.0,0.0,0.0],[0.0,10.0,0.0],[0.0,0.0,20.0]]} When present, readers SHOULD prefer this over the length/angle values on lines 3-4, which may lose precision through trigonometric round-trips. Writers that emit ``lattice_vectors`` MUST also write consistent length/angle values to lines 3-4 for legacy compatibility. Frame ordering ~~~~~~~~~~~~~~ Writers producing trajectories SHOULD include ``frame_index`` (and ``time`` when available) so consumers can reconstruct ordering without relying on filesystem metadata. Version history --------------- .. table:: +---------+------------+--------------------------------------------------------------------------+ | Version | Date | Changes | +=========+============+==========================================================================+ | 1 | (original) | De facto format from eOn. Column 5 present, undefined. | +---------+------------+--------------------------------------------------------------------------+ | 2 | 2026-03-25 | JSON metadata. atom\_id semantics. Per-direction constraints. | +---------+------------+--------------------------------------------------------------------------+ | \ | \ | Declared sections. Force blocks. Compression. | +---------+------------+--------------------------------------------------------------------------+ | 3 | 2026-06-27 | Required ``units`` (``length``, ``energy``). Optional ``storage_dtypes`` | +---------+------------+--------------------------------------------------------------------------+ | \ | \ | for in-memory SoA element types (float32/float64). | +---------+------------+--------------------------------------------------------------------------+ Version 3 (normative) --------------------- A file with ``con_spec_version`` equal to 3 SHALL satisfy all Version 2 requirements and the following additional requirements. Units (required) ~~~~~~~~~~~~~~~~ The metadata object SHALL contain a ``units`` member whose value is a JSON object. That object SHALL contain the string members ``length`` and ``energy``, each a non-empty unit expression whose physical dimension matches the named quantity (length and energy respectively). Implementations SHALL reject Version 3 inputs that omit ``units`` or supply dimensionally invalid strings for those members. Optional members ``mass``, ``time``, ``velocity``, and ``force``, when present, SHALL likewise be dimensionally valid for those quantities. Unit expressions follow the same lexical conventions as metatomic-style host unit strings: case-insensitive names, ``\*`` / ``/`` / ``^``, and parentheses. The library provides conversion factors between compatible expressions (``unit_conversion_factor``); it does not silently rewrite on-disk numeric fields when units differ between frames. Storage dtypes (optional; in-memory contract) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The metadata object MAY contain a ``storage_dtypes`` member describing how a conforming implementation holds per-atom numeric fields **after** parsing or construction. This does **not** change on-disk CON text encoding (coordinates and section values remain decimal representations of binary64). When present, ``storage_dtypes`` SHALL be a JSON object. Recognized members: .. table:: +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+--------------------+ | Member | Permitted values (host DLPack) | Default if omitted | +=========================================================================+======================================================================================================================+====================+ | ``positions`` / ``velocities`` / ``forces`` / ``energies`` / ``masses`` | ``float16``, ``float32``, ``float64``, ``int8=…=int64``, ``uint8=…=uint64``, ``bool``, ``complex64``, ``complex128`` | ``float64`` | +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+--------------------+ | ``atom_ids`` | ``uint8=…=uint64``, ``int8=…=int64`` | ``uint64`` | +-------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+--------------------+ Hosted element types are those **dlpk** can own on the CPU (``float16`` via ``half``, complex as ``[f32;2]`` / ``[f64;2]``, integers, ``bool``). DLPack ABI codes without a host (``bfloat16``, ``float8_*``, ``opaque``) SHALL be rejected in ``storage_dtypes`` with a validation error. Implementations that expose DLPack SHALL export arrays whose element type matches the effective storage dtype for that field (queryable prior to export). Callers MAY request projection between hosted dtypes in memory; writers SHALL still serialize binary64 text on disk. DLPack interchange (informative for implementations) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Numeric fields are treated as opaque, shaped arrays (metatensor-style): implementations SHOULD expose ``shape``, element type, and device for each field, and an ``as_dlpack(device, stream, max_version)`` operation that returns a ``DLManagedTensorVersioned`` describing the **actual** backing buffer. Ingest (``from_dlpack``) SHALL update that backing and keep any AoS projection consistent for CON serialization. Preferred placement is negotiated via the device argument; changing element type is a storage projection (``storage_dtypes`` / ``project_storage_dtypes``), not the primary export contract. Detecting the spec version -------------------------- Read line 2. If it starts with ``{``, parse as JSON and extract ``con_spec_version``. Otherwise the file predates this specification (implicit version 1). Machine-readable metadata schema -------------------------------- The reserved metadata vocabulary is published as JSON Schema (draft 2020-12) at ``schema/con-metadata.schema.json`` in the reference implementation repository. The schema accepts every object the reference writer emits and rejects the violations this document calls out: missing v3 ``units``, malformed ``pbc``, ``storage_dtypes`` element types without a host. Unknown keys validate successfully by design; parsers MUST preserve them. The reference test suite (``tests/spec_metadata_schema.rs``) keeps writer output and the schema in lockstep. Media type and file identification (informative) ------------------------------------------------ No IANA media type is registered for CON. Until one is, implementations SHOULD label CON content ``chemical/x-con`` and fall back to ``text/plain; charset=utf-8`` for strict consumers. CON carries no magic bytes: identify files by the ``.con`` / ``.convel`` extension (optionally with a ``.gz`` / ``.zst`` compression suffix) combined with the structural check above (nine-line header; line 2 either JSON with ``con_spec_version`` or legacy free text). Examples -------- Minimal v2 file ~~~~~~~~~~~~~~~ :: Generated by eOn {"con_spec_version":2} 10.000000 10.000000 10.000000 90.000000 90.000000 90.000000 1 2 63.546000 Cu Coordinates of Component 1 0.000000 0.000000 0.000000 7 0 5.000000 5.000000 5.000000 0 1 File with velocities and forces ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ :: Generated by eOn {"con_spec_version":2,"sections":["velocities","forces"],"energy":-42.5,"potential":{"type":"EMT","params":{}}} 15.345600 21.702000 100.000000 90.000000 90.000000 90.000000 0 0 218 0 1 2 2 2 63.546000 1.007930 Cu Coordinates of Component 1 0.639400 0.904500 6.975300 7 0 3.196900 0.904500 6.975300 7 1 H Coordinates of Component 2 8.682300 9.947000 11.733000 0 2 7.942100 9.947000 11.733000 0 3 Cu Velocities of Component 1 0.001234 0.002345 -0.003456 7 0 0.004567 -0.005678 0.006789 7 1 H Velocities of Component 2 -0.012345 0.023456 0.034567 0 2 0.045678 -0.056789 -0.067890 0 3 Cu Forces of Component 1 0.123456 0.234567 -0.345678 7 0 0.456789 -0.567890 0.678901 7 1 H Forces of Component 2 -1.234567 2.345678 3.456789 0 2 4.567890 -5.678901 -6.789012 0 3 Trajectory frame with metadata ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ :: Generated by eOn 3.1 {"con_spec_version":2,"generator":"eOn 3.1","units":{"length":"angstrom","energy":"eV"},"frame_index":5,"time":2.5,"timestep":0.5} 15.345600 21.702000 100.000000 90.000000 90.000000 90.000000 2 2 2 63.546000 1.007930 Cu Coordinates of Component 1 0.639400 0.904500 6.975300 7 0 3.196900 0.904500 6.975300 7 1 H Coordinates of Component 2 8.682300 9.947000 11.733000 0 2 7.942100 9.947000 11.733000 0 3 Legacy (pre-v2) file ~~~~~~~~~~~~~~~~~~~~ :: Random Number Seed 0.0000 TIME 15.345600 21.702000 100.000000 90.000000 90.000000 90.000000 0 0 0 0 0 2 216 2 63.546 1.00793 Cu Coordinates of Component 1 0.639400 0.904500 -0.000100 1 0 3.197000 0.904500 -0.000100 1 1 ... A conforming v2 reader processes this without error, assigning ``spec_version = 1`` because line 2 does not start with ``{``. .. _builder-mutation: Builder mutation surface (informative) -------------------------------------- This section is informative; it describes the *reference implementation's* in-memory authoring API rather than the on-disk format itself, but is recorded here so that conforming bindings can expose the same surface. The reference Rust ``ConFrameBuilder`` exposes per-atom and bulk in-place mutation in addition to the append-only ``add_atom`` / ``with_*`` flow. These are intended for hot-loop consumers (MD integrators, saddle-search drivers, NEB image updaters) that hold a single builder across many simulation steps and serialise to disk only at I/O boundaries: - Per-atom mutators MUST validate the index against the current atom count and SHOULD return a typed *index out of bounds* error (``ParseError::IndexOutOfBounds`` in the Rust core; mapped to ``RKR_STATUS_INDEX_OUT_OF_BOUNDS`` in the C ABI; raised as ``IndexError`` in PyO3 / equivalent in Julia). - Bulk mutators take a flat row-major buffer (``[x0, y0, z0, x1, y1, z1, ...]`` for positions / forces; ``[e0, e1, ..., e(N-1)]`` for energies). They MUST validate the buffer length against the expected size (3 N for vector buffers, N for energies) and SHOULD reuse the existing length-mismatch error path (``ParseError::InvalidVectorLength``). - Setting force / velocity / energy on any atom MUST cause the next ``build()`` to declare the corresponding section (``"forces"`` / ``"velocities"`` / ``"energies"``). Implementations MUST NOT short-circuit on inspecting only atom 0; the section flag is set when *any* atom carries the field. - ``clear_atom_velocity`` / ``_force`` / ``_energy`` MUST remove the field from the named atom only. If every atom subsequently lacks the field, the next ``build()`` MUST NOT declare the section. - ``set_atom_mass`` updates the per-type ``masses_per_type`` entry on ``build()``. The .con format binds one mass per element type, so if multiple atoms of the same symbol carry different masses, the last value wins. Bindings SHOULD document this; future spec versions MAY tighten it. The reference implementation surface lands in v0.11.0: - Rust core (``ConFrameBuilder::set_atom_position``, ``set_atom_velocity``, ``set_atom_force``, ``set_atom_energy``, ``set_atom_fixed``, ``set_atom_mass``, ``clear_atom_*``, ``set_positions_from_flat``, ``set_forces_from_flat``, ``set_atom_energies_from_flat``, ``get_atom_*``, ``atom_count``). - C ABI (``rkr_frame_builder_atom_count`` and the matching ``rkr_frame_builder_set_atom_*`` / ``get_atom_*`` / ``clear_atom_*`` family in ``readcon-core.h``). - C++ wrapper (the same surface as instance methods on ``readcon::ConFrameBuilder``, with ``std::optional`` return for optional-vector getters). - Python: ``readcon.Atom`` already exposes per-atom fields with ``#[pyo3(get, set)]`` so Python users have always been able to do ``frame.atoms[i].x = ...`` / ``.fx = ...`` / ``.energy = ...`` for in-place mutation. This is the recommended Python pattern; no separate ``readcon.ConFrameBuilder`` class is exposed because the Python module is value-based rather than handle-based. - Julia: equivalently, ``ReadCon.Atom`` fields are mutable Julia struct fields; mutate directly and call ``write_con(path, frame)``. No public Julia builder class. Bindings for other languages SHOULD pick the idiom that matches their existing surface: handle-based (Rust / C / C++) gets the typed mutation API, value-based (Python / Julia / Ruby / ...) gets direct field assignment on the per-atom struct. Rationale (non-normative): the append-only authoring path forced every step of a long simulation to allocate a fresh builder, push N atoms, and discard. Reusing one builder across the simulation loop and serialising only at trajectory boundaries reduces allocation pressure to the *output* cadence rather than the *integration* cadence. The mutation API is purely additive; the existing ``add_atom`` + ``with_*`` flow continues to work and remains the recommended path for one-shot frame authoring. .. _builder-storage-layout: Storage layout (informative) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The v0.11.0 reference implementation holds per-atom fields as structure-of-arrays (SoA) buffers, each owned by an ``ndarray::Array2`` or ``ndarray::Array1`` in row-major ("C") layout: .. table:: +-------------------+-------------------------------------+------------+ | Field | Type | Shape | +===================+=====================================+============+ | ``positions`` | ``ndarray::Array2`` | ``(N, 3)`` | +-------------------+-------------------------------------+------------+ | ``velocities`` | ``ndarray::Array2`` (optional) | ``(N, 3)`` | +-------------------+-------------------------------------+------------+ | ``forces`` | ``ndarray::Array2`` (optional) | ``(N, 3)`` | +-------------------+-------------------------------------+------------+ | ``atom_energies`` | ``ndarray::Array1`` (optional) | ``(N,)`` | +-------------------+-------------------------------------+------------+ | ``masses`` | ``ndarray::Array1`` | ``(N,)`` | +-------------------+-------------------------------------+------------+ | ``atom_ids`` | ``ndarray::Array1`` | ``(N,)`` | +-------------------+-------------------------------------+------------+ | ``symbols`` | ``Vec`` | ``(N,)`` | +-------------------+-------------------------------------+------------+ | ``fixed`` | ``Vec<[bool; 3]>`` | ``(N, 3)`` | +-------------------+-------------------------------------+------------+ Bindings MUST surface these fields as contiguous row-major buffers or as a DLPack-managed tensor (preferred for cross-language zero- copy); see `builder-dlpack-export`_ for the cross-language contract. The SoA-per-field choice (rather than AoS, NumPy-structured-array style, or per-atom ``Option<[f64; 3]>``) matches the layout used by LAMMPS, GROMACS, ASE, MDAnalysis, OpenMM, chemfiles, i-PI, h5md (the canonical MD libraries) *and* Apache Arrow, DuckDB, Polars, Pandas v2, Parquet, PyTorch, JAX, DGL (the canonical columnar / ML stacks). It is the layout NumPy *recommends against* structured arrays for, on the same SIMD-throughput grounds. The ``Arc>>`` storage hook + DLPack 1.0 export pattern adopted here mirrors the design described in :cite:t:`bigiMetatensorMetatomicFoundational2026`, where the same layout serves as the foundational data exchange surface across the metatensor / metatomic atomistic machine-learning libraries. .. _builder-section-presence: Section presence semantics (normative) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The optional sections (``"velocities"``, ``"forces"``, ``"atom_energies"``) are **column-level**, not per-atom: when declared, the section applies to every atom in the frame. Per-atom NULL does not round-trip to disk because the .con on-disk format has no NULL encoding. Implementations MUST follow: - ``add_atom`` MUST pad newly added rows with zero in every already- declared optional section so the per-field buffer length stays coherent with ``atom_count()``. - ``with_velocity`` / ``_force`` / ``_energy``, ``set_atom_velocity`` / ``_force`` / ``_energy``, ``set_velocities_from_flat`` / ``set_forces_from_flat`` / ``set_atom_energies_from_flat`` MUST declare the corresponding section on first call. - ``clear_atom_velocity`` / ``_force`` / ``_energy`` MUST zero the slot but MUST NOT change section presence; the field stays declared and a subsequent ``build()`` emits the section block. This matches Apache Arrow's column-level validity (a column either exists or does not) and h5md's "dataset present or absent" rule. - ``clear_velocities_section`` / ``_forces_section`` / ``_energies_section`` MUST drop the entire section: free the storage, set the presence flag to false, and ensure the next ``build()`` does not emit the section block. .. _builder-dlpack-export: DLPack export contract (normative) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The reference implementation exposes every per-atom field with a contiguous backing as a DLPack-managed tensor for cross-language zero-copy interop. DLPack 1.0 is the universal exchange ABI supported by PyTorch, TensorFlow, JAX, NumPy, CuPy, MXNet, TVM, OneFlow, MNN, OneAPI, and mlx, plus a stable C ABI (``DLManagedTensorVersioned``) for non-Python consumers. The DLPack tier exposes (builder and, for optional sections, frame C ABI): - ``positions_dlpack`` — shape ``(N, 3)``; write-through mut variant on the builder. - ``velocities_dlpack`` / ``forces_dlpack`` — ``(N, 3)`` or absent (``None`` / ``SECTION_ABSENT``). - ``atom_energies_dlpack`` — ``(N,)`` or absent. - ``masses_dlpack`` — ``(N,)``. - ``atom_ids_dlpack`` — ``(N,)`` integer ids. Consumers MUST read shape, strides, **dtype** (code/bits/lanes), and device from the tensor; they MUST NOT assume a host ``f64`` / ``double`` buffer type. On-disk CON coordinates and section values are IEEE binary64 in current writers, and implementations today report ``kDLFloat`` / 64 for those fields and ``kDLUInt`` / 64 for ``atom_id``; a future implementation MAY change bits while keeping the same shapes—interop goes through the tensor metadata. The returned tensor MUST report: - shape: the field's natural shape (``(N, 3)`` or ``(N,)``). - strides: row-major standard layout for the dtype. - dtype: as carried by the implementation (see above for current values). - device: ``kDLCPU`` (device id 0) for the default backing. GPU / optional ``--features cuda`` enables real CUDA allocate and H2D export for matching ``kDLCUDA`` requests; default builds reject non-CPU devices. - data pointer: the backing array's first element. Lifetime: ``DLPackTensorRef<'a>`` borrows from ``&'a self`` and MUST NOT outlive the builder. C / FFI consumers that need an owning tensor go through the C-ABI export (``rkr_frame_builder_*_dlpack``) which fills a ``DLManagedTensorVersioned*`` with a clone of the field's storage plus a deleter callback owned by the consumer; the original builder is then free to drop. .. _builder-pointer-stability: Pointer stability (normative) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The data pointer returned by any DLPack export is stable as long as no ``add_atom`` call grows the underlying ndarray buffer. Callers that hold raw pointers across ``add_atom`` MUST refresh after the push (the ndarray reallocates when capacity is exceeded; the DLPack view becomes invalid). Bulk and per-element mutation (``set_atom_*``, ``set_*_from_flat``, ``clear_atom_*``) MUST NOT reallocate the underlying buffer and the data pointer MUST stay stable across these operations. .. _references: References ---------- .. bibliography:: :style: alpha