The CON File Format Specification

Date:

2026-06-27

Specification

Version

3

Date

2026-06-27

Status

Stable

Reference implementation

readcon-core

Surface grammar (PEG)

grammar/readcon.pest (Pest); validate with cargo test --features grammar

This document defines versions 2 and 3 of the CON file format; version 3 is the current version and adds required units metadata on top of the version 2 requirements. It supersedes all prior informal descriptions. New implementations SHOULD target version 3 and MUST accept version 2.

The keywords MUST, MUST NOT, REQUIRED, SHALL, SHOULD, SHOULD NOT, MAY, and OPTIONAL follow RFC 2119 semantics.

Overview

CON stores atomic configurations for rare-event and transition-state checkpoints: cell, type-grouped coordinates, per-direction constraints, atom identity, optional velocities/forces, and versioned JSON metadata. The layout originated in the eOn rare-event stack Chill et al. [2014]; this document is the formal specification. The reference implementation is readcon-core (multi-language hourglass ABI).

A CON file contains one or more frames. Each frame encodes a simulation cell, per-type metadata (masses, atom counts), and per-atom data (coordinates, constraints, identity). Optional sections add velocities, forces, or other per-atom vector/scalar data. Line 2 carries versioned JSON metadata.

File extensions

.con

Coordinate-only configuration files.

.convel

Configuration files with velocity data per frame.

.con.gz

Gzip-compressed CON files (see compression).

Encoding

CON files MUST use UTF-8. Line endings MAY be LF (\n) or CRLF (\r\n); parsers MUST accept both. All numeric values use ASCII decimal representation (no locale-dependent formatting).

Surface grammar (PEG)

The machine-readable surface syntax for CON/convel lives in the repository at grammar/readcon.pest (a Pest PEG). It is the formal companion to this prose specification for syntactic structure (header layout, type blocks, optional section blocks, multi-frame concatenation).

The reference implementation in readcon-core does **not** use the PEG on the I/O hot path; it implements the same surface rules plus semantic checks (atom counts, metadata JSON, validation flags). Enable the grammar Cargo feature to parse buffers with the PEG for tooling or tests:

cargo test --features grammar

Frame structure

Each frame consists of:

  1. A 9-line header.

  2. One coordinate block per atom type.

  3. Zero or more additional per-atom sections (velocities, forces), each preceded by a blank separator line.

Multiple frames are concatenated directly with no inter-frame separator.

Header (9 lines)

Line

Name

Content

Example

1

Generator comment

Free-form text

Generated by eOn

2

Metadata

JSON object or free-form text

{"con_spec_version":3,"units":{...}}

3

Cell dimensions

3 floats: Lx Ly Lz

15.3456 21.702 100.0

4

Cell angles

3 floats: alpha beta gamma

90.0 90.0 90.0

5

Reserved

Free-form text (round-tripped)

0 0

6

Reserved

Free-form text (round-tripped)

218 0 1

7

Atom type count

1 integer: N

2

8

Atoms per type

N integers

216 2

9

Mass per type

N floats (atomic mass units)

63.546 1.00793

Line 1: Generator comment

Free-form text. Writers SHOULD set this to a human-readable identifier. Parsers MUST preserve it through round-trips but MUST NOT assign it semantic meaning.

Line 2: Metadata

Line 2 carries machine-readable metadata as a single-line JSON object.

Version 2+ files

Writers MUST emit a JSON object containing at least con_spec_version with an integer value:

{"con_spec_version":2}

Additional keys MAY appear. Parsers MUST preserve unrecognized keys through round-trips. Reserved metadata keys listed in metadata-keys MUST use the declared JSON type. The sections key, when present, MUST be an array of strings. The validate key, when present, MUST be a boolean.

Legacy (pre-v2) files

Files produced before this specification may contain free-form text on line 2. A conforming parser detects the format by checking whether line 2, after trimming whitespace, starts with {:

  • Starts with {: parse as JSON, extract con_spec_version.

  • Does not start with {: treat as legacy (implicit version 1).

If line 2 starts with { but contains malformed JSON, the parser MUST report an error. If the JSON object lacks con_spec_version, the parser MUST report an error. If con_spec_version exceeds the highest version the parser supports, the parser MUST report an error.

Lines 3-4: Cell geometry

Line 3: three whitespace-separated floats (cell edge lengths in angstroms). Line 4: three whitespace-separated floats (cell angles in degrees). Tabs and spaces are both valid separators.

For non-orthogonal cells, the angle-based representation introduces floating-point drift through trigonometric round-trips. Writers SHOULD include the exact 3x3 lattice vector matrix in the JSON metadata via the lattice_vectors key (see lattice-vectors). When lattice_vectors is present, readers SHOULD prefer it over the length/angle values on lines 3-4.

When validate is true, readers MUST reject zero or negative cell lengths and MUST reject angles outside the open interval (0, 180) degrees.

Lines 5-6: Reserved

Free-form text with no defined semantics. Writers MAY emit empty lines. Parsers MUST preserve these for round-trip fidelity.

Lines 7-9: Type metadata

Line 7: single positive integer N (number of atom types). Line 8: exactly N positive integers (atom count per type). Line 9: exactly N floats (atomic mass per type, in amu).

Coordinate blocks

For each atom type i (1 to N), in the order declared in lines 8-9:

  1. Symbol line: chemical symbol (e.g., Cu, H).

  2. Label line: Coordinates of Component /i/

  3. Atom lines: one per atom, containing:

Column

Type

Description

1

float

x coordinate (angstroms)

2

float

y coordinate (angstroms)

3

float

z coordinate (angstroms)

4

int

Constraint bitmask (see constraints)

5

int

Atom index (see atom-index)

Columns are whitespace-separated.

Per-direction constraints (column 4)

Column 4 encodes per-direction constraint flags as a bitmask. Bit 0 = x, bit 1 = y, bit 2 = z.

Value

Spec 1 (no JSON line 2)

Spec 2+

0

Free

Free

1

All-fixed (legacy boolean)

x-only

2-6

Unused / undefined

Per-direction combinations

7

Unused / undefined

Fixed in all directions

Spec-1 files used column 4 as a boolean (0 free, 1 fixed). Spec-2 files use the 3-bit mask. Readers MUST therefore consult con_spec_version before decoding 1: spec 1 keeps the legacy all-fixed reading; spec 2 treats 1 as x-only.

Writers MUST emit 7 for all-fixed atoms, never 1. Spec-2 writers MUST emit 1 for an atom fixed in x alone.

Atom index (column 5)

The atom index preserves the original position of each atom before type-based grouping. The CON format groups atoms by element type, which reorders them. Without a persistent index, the original ordering cannot be recovered after a read-write cycle.

Version 2 requirements:

  • Writers MUST emit column 5 on every atom line.

  • Column 5 MUST contain the pre-grouping index.

  • Readers MUST parse and preserve column 5 through write-back.

Version 1 behavior:

  • Column 5 is present but its semantics are undefined.

Readers SHOULD accept 4-column atom lines. When column 5 is absent, default to the sequential position within the frame (0, 1, 2, …).

Additional per-atom sections

After coordinate blocks, a frame MAY contain additional per-atom data sections. Each section follows the same block structure: a blank separator line, then per-component blocks (symbol line, label line, data lines).

Section declaration

Version 2 files declare sections in the JSON metadata using the sections key:

{"con_spec_version":2,"sections":["velocities","forces"]}

The parser reads sections in the declared order. Parsers MUST reject unknown section names with an error. Every declared section MUST be present, complete, and parseable at its declared position. An empty sections array declares that no additional per-atom sections follow.

Section name

Label pattern

Columns

Data

velocities

Velocities of Component /i/

5

vx vy vz fixed_flag atom_id

forces

Forces of Component /i/

5

fx fy fz fixed_flag atom_id

energies

Energies of Component /i/

3

energy fixed_flag atom_id

charges

Charges of Component /i/

3

charge fixed_flag atom_id

spins

Spins of Component /i/

3

spin fixed_flag atom_id

magmoms

Magmoms of Component /i/

5

mx my mz fixed_flag atom_id

The energies section carries one scalar per atom, useful for ML potentials that decompose total energy into local contributions. Writers MAY emit it alongside forces, alone, or omit it entirely.

charges and spins are optional scalar sections (same column layout as energies). magmoms is an optional 3-vector section (same layout as velocities). They are reserved names on the existing v2/v3 declared sections surface: files MAY use con_spec_version 2 or 3; a format major bump is not required for these optional blocks.

When the energies section is present:

  • The per-frame total energy metadata key SHOULD equal the sum of the per-atom contributions. Implementations MAY warn on a mismatch but MUST NOT reject the frame on that ground (a reader cannot tell apart a numerical-noise mismatch from a deliberate definition where the per-atom decomposition does not sum to the total).

  • The total energy metadata key MAY still be absent, in which case the per-atom contributions are the only energy data on the frame.

  • The fixed_flag and atom_id columns SHOULD match the coordinate block, exactly as for velocities and forces. In validate=true mode they MUST match.

The fixed_flag and atom_id columns in every additional section repeat the coordinate-block identity data for the same atom ordering. Writers SHOULD emit the same values as the coordinate block. Readers associate section rows by component order and MAY ignore the duplicate identity columns after parsing the row shape; in validate=true mode readers MUST verify that the fixed_flag and atom_id columns match the coordinate block.

Validation mode

Version 2 files MAY set validate to true in the JSON metadata:

{"con_spec_version":2,"sections":["velocities","forces"],"validate":true}

When validate is true, the sections key MUST be present, even when the value is an empty array. Conforming readers MUST verify that the frame satisfies strict v2 invariants before accepting it. At minimum, this validation MUST check:

  • Reserved metadata keys use the declared JSON types.

  • Numeric tokens are finite.

  • Cell lengths are positive, cell angles are in (0, 180) degrees, atom counts are positive, and masses are positive.

  • Coordinate component labels exactly match Coordinates of Component /i/.

  • Component symbols are recognized element symbols, or X for an explicitly unknown element.

  • Coordinate and additional-section fixed_flag and atom_id fields are exact integer tokens.

  • The section component symbol matches the coordinate component symbol.

  • The section label exactly matches the declared section and component number (for example, Velocities of Component 1).

  • Each section row’s fixed_flag decodes to the same per-axis fixed mask as the corresponding coordinate row.

  • Each section row’s atom_id equals the corresponding coordinate row’s atom_id.

If any check fails, the reader MUST reject the frame. When validate is absent or false, readers MAY parse files by associating section rows by component order and ignoring duplicate identity column mismatches after parsing the row shape.

Error paths and ParseError variants

The reference reader (readcon-core) surfaces validation failures as typed ParseError variants. Other implementations are expected to return analogous structured errors, but the variant names below are specific to readcon-core and are listed here so that the spec and the reference implementation stay in sync.

ParseError variant

Fires when

MissingSpecVersion

Line 2 starts with { but no con_spec_version key.

UnsupportedSpecVersion(v)

con_spec_version exceeds CON_SPEC_VERSION.

InvalidMetadataJson(msg)

JSON is malformed, validate is not a boolean, sections is not an array of strings, or a reserved key has the wrong JSON type. Also fires when validate=true omits the sections key.

ValidationError(msg) (validate=true only)

Cell geometry, masses, coordinate component label, component symbol, fixed_flag / atom_id integer tokens, section component symbol, section label, or per-row identity columns mismatch the rules above. Numeric tokens that are not finite are rejected here as well, after parse-time tokenization.

UnknownSection(name)

A name in the sections array does not match a known section.

IncompleteHeader

Fewer than 9 header lines remain.

IncompleteFrame

Coordinate block ends short.

IncompleteVelocitySection

Declared velocity section ends short or is absent.

IncompleteForceSection

Declared force section ends short or is absent.

A minimal example of each path: the file resources/test/tiny_cuh2_strict_invalid.con (if present) and the test cases under src/parser.rs::tests and tests/parseforces.rs exercise these branches and serve as executable references.

Legacy section detection

Files without a sections key use blank-separator detection: if a blank line follows coordinate blocks, the parser attempts to parse a velocity section. A present sections key disables this fallback, including when the value is []. This preserves backward compatibility with existing .convel files while giving v2 writers a precise declaration mechanism.

Writers SHOULD always emit the sections key when writing additional sections.

Velocity section

Per component i: blank separator, symbol line, Velocities of Component /i/ label, then one line per atom: vx vy vz fixed_flag atom_id.

Force section

Per component i: blank separator, symbol line, Forces of Component /i/ label, then one line per atom: fx fy fz fixed_flag atom_id.

Frames with forces SHOULD include the potential and energy metadata keys.

Extending with new section types

New section types follow the same pattern: declare in the sections array, use a blank separator, symbol line, <Name> of Component /i/ label, and data lines.

Multi-frame files

Frames are concatenated with no separator. After parsing a frame’s data, the parser attempts the next 9-line header. If fewer than 9 lines remain, parsing ends.

Writers MUST NOT insert extra blank lines between frames.

Data types and precision

  • Floats: any valid decimal representation. Writers SHOULD emit at least 6 significant digits. For lossless f64 round-tripping, 17 digits suffice. Readers MUST reject non-finite values (NaN, Infinity, -Infinity).

  • Integers: constraint bitmask (0-7), atom_id, natm_types, and natms_per_type are non-negative integers.

Compression

CON files MAY be gzip-compressed. Readers SHOULD detect compression by checking the first two bytes for the gzip magic number (0x1f 0x8b) rather than relying on file extension. The decompressed content MUST be a valid CON file.

Constraints and limits

  • Atoms MUST appear grouped by type, in header-declared order.

  • Component numbering starts at 1.

  • Total atom count equals the sum of line 8 values.

  • Symbol strings SHOULD match IUPAC element symbols.

  • No upper limit on atom types or atom count is imposed.

Version history

Version

Date

Changes

1

(original)

De facto format from eOn. Column 5 present, undefined.

2

2026-03-25

JSON metadata. atom_id semantics. Per-direction constraints.

Declared sections. Force blocks. Compression.

3

2026-06-27

Required units (length, energy). Optional storage_dtypes

for in-memory SoA element types (float32/float64).

Version 3 (normative)

A file with con_spec_version equal to 3 SHALL satisfy all Version 2 requirements and the following additional requirements.

Units (required)

The metadata object SHALL contain a units member whose value is a JSON object. That object SHALL contain the string members length and energy, each a non-empty unit expression whose physical dimension matches the named quantity (length and energy respectively). Implementations SHALL reject Version 3 inputs that omit units or supply dimensionally invalid strings for those members.

Optional members mass, time, velocity, and force, when present, SHALL likewise be dimensionally valid for those quantities.

Unit expressions follow the same lexical conventions as metatomic-style host unit strings: case-insensitive names, \* / / / ^, and parentheses. The library provides conversion factors between compatible expressions (unit_conversion_factor); it does not silently rewrite on-disk numeric fields when units differ between frames.

Storage dtypes (optional; in-memory contract)

The metadata object MAY contain a storage_dtypes member describing how a conforming implementation holds per-atom numeric fields after parsing or construction. This does not change on-disk CON text encoding (coordinates and section values remain decimal representations of binary64).

When present, storage_dtypes SHALL be a JSON object. Recognized members:

Member

Permitted values (host DLPack)

Default if omitted

positions / velocities / forces / energies / masses

float16, float32, float64, int8=…=int64, uint8=…=uint64, bool, complex64, complex128

float64

atom_ids

uint8=…=uint64, int8=…=int64

uint64

Hosted element types are those dlpk can own on the CPU (float16 via half, complex as [f32;2] / [f64;2], integers, bool). DLPack ABI codes without a host (bfloat16, float8_*, opaque) SHALL be rejected in storage_dtypes with a validation error.

Implementations that expose DLPack SHALL export arrays whose element type matches the effective storage dtype for that field (queryable prior to export). Callers MAY request projection between hosted dtypes in memory; writers SHALL still serialize binary64 text on disk.

DLPack interchange (informative for implementations)

Numeric fields are treated as opaque, shaped arrays (metatensor-style): implementations SHOULD expose shape, element type, and device for each field, and an as_dlpack(device, stream, max_version) operation that returns a DLManagedTensorVersioned describing the actual backing buffer. Ingest (from_dlpack) SHALL update that backing and keep any AoS projection consistent for CON serialization. Preferred placement is negotiated via the device argument; changing element type is a storage projection (storage_dtypes / project_storage_dtypes), not the primary export contract.

Detecting the spec version

Read line 2. If it starts with {, parse as JSON and extract con_spec_version. Otherwise the file predates this specification (implicit version 1).

Machine-readable metadata schema

The reserved metadata vocabulary is published as JSON Schema (draft 2020-12) at schema/con-metadata.schema.json in the reference implementation repository. The schema accepts every object the reference writer emits and rejects the violations this document calls out: missing v3 units, malformed pbc, storage_dtypes element types without a host. Unknown keys validate successfully by design; parsers MUST preserve them. The reference test suite (tests/spec_metadata_schema.rs) keeps writer output and the schema in lockstep.

Media type and file identification (informative)

No IANA media type is registered for CON. Until one is, implementations SHOULD label CON content chemical/x-con and fall back to text/plain; charset=utf-8 for strict consumers. CON carries no magic bytes: identify files by the .con / .convel extension (optionally with a .gz / .zst compression suffix) combined with the structural check above (nine-line header; line 2 either JSON with con_spec_version or legacy free text).

Examples

Minimal v2 file

Generated by eOn
{"con_spec_version":2}
10.000000 10.000000 10.000000
90.000000 90.000000 90.000000


1
2
63.546000
Cu
Coordinates of Component 1
0.000000 0.000000 0.000000 7 0
5.000000 5.000000 5.000000 0 1

File with velocities and forces

Generated by eOn
{"con_spec_version":2,"sections":["velocities","forces"],"energy":-42.5,"potential":{"type":"EMT","params":{}}}
15.345600 21.702000 100.000000
90.000000 90.000000 90.000000
0 0
218 0 1
2
2 2
63.546000 1.007930
Cu
Coordinates of Component 1
   0.639400    0.904500    6.975300 7    0
   3.196900    0.904500    6.975300 7    1
H
Coordinates of Component 2
   8.682300    9.947000   11.733000 0  2
   7.942100    9.947000   11.733000 0  3

Cu
Velocities of Component 1
   0.001234    0.002345   -0.003456 7    0
   0.004567   -0.005678    0.006789 7    1
H
Velocities of Component 2
  -0.012345    0.023456    0.034567 0  2
   0.045678   -0.056789   -0.067890 0  3

Cu
Forces of Component 1
   0.123456    0.234567   -0.345678 7    0
   0.456789   -0.567890    0.678901 7    1
H
Forces of Component 2
  -1.234567    2.345678    3.456789 0  2
   4.567890   -5.678901   -6.789012 0  3

Trajectory frame with metadata

Generated by eOn 3.1
{"con_spec_version":2,"generator":"eOn 3.1","units":{"length":"angstrom","energy":"eV"},"frame_index":5,"time":2.5,"timestep":0.5}
15.345600 21.702000 100.000000
90.000000 90.000000 90.000000


2
2 2
63.546000 1.007930
Cu
Coordinates of Component 1
   0.639400    0.904500    6.975300 7    0
   3.196900    0.904500    6.975300 7    1
H
Coordinates of Component 2
   8.682300    9.947000   11.733000 0  2
   7.942100    9.947000   11.733000 0  3

Legacy (pre-v2) file

Random Number Seed
0.0000 TIME
15.345600 21.702000 100.000000
90.000000 90.000000 90.000000
0 0
0 0 0
2
216 2
63.546 1.00793
Cu
Coordinates of Component 1
   0.639400    0.904500   -0.000100 1    0
   3.197000    0.904500   -0.000100 1    1
...

A conforming v2 reader processes this without error, assigning spec_version = 1 because line 2 does not start with {.

Builder mutation surface (informative)

This section is informative; it describes the reference implementation’s in-memory authoring API rather than the on-disk format itself, but is recorded here so that conforming bindings can expose the same surface.

The reference Rust ConFrameBuilder exposes per-atom and bulk in-place mutation in addition to the append-only add_atom / with_* flow. These are intended for hot-loop consumers (MD integrators, saddle-search drivers, NEB image updaters) that hold a single builder across many simulation steps and serialise to disk only at I/O boundaries:

  • Per-atom mutators MUST validate the index against the current atom count and SHOULD return a typed index out of bounds error (ParseError::IndexOutOfBounds in the Rust core; mapped to RKR_STATUS_INDEX_OUT_OF_BOUNDS in the C ABI; raised as IndexError in PyO3 / equivalent in Julia).

  • Bulk mutators take a flat row-major buffer ([x0, y0, z0, x1, y1, z1, ...] for positions / forces; [e0, e1, ..., e(N-1)] for energies). They MUST validate the buffer length against the expected size (3 N for vector buffers, N for energies) and SHOULD reuse the existing length-mismatch error path (ParseError::InvalidVectorLength).

  • Setting force / velocity / energy on any atom MUST cause the next build() to declare the corresponding section ("forces" / "velocities" / "energies"). Implementations MUST NOT short-circuit on inspecting only atom 0; the section flag is set when any atom carries the field.

  • clear_atom_velocity / _force / _energy MUST remove the field from the named atom only. If every atom subsequently lacks the field, the next build() MUST NOT declare the section.

  • set_atom_mass updates the per-type masses_per_type entry on build(). The .con format binds one mass per element type, so if multiple atoms of the same symbol carry different masses, the last value wins. Bindings SHOULD document this; future spec versions MAY tighten it.

The reference implementation surface lands in v0.11.0:

  • Rust core (ConFrameBuilder::set_atom_position, set_atom_velocity, set_atom_force, set_atom_energy, set_atom_fixed, set_atom_mass, clear_atom_*, set_positions_from_flat, set_forces_from_flat, set_atom_energies_from_flat, get_atom_*, atom_count).

  • C ABI (rkr_frame_builder_atom_count and the matching rkr_frame_builder_set_atom_* / get_atom_* / clear_atom_* family in readcon-core.h).

  • C++ wrapper (the same surface as instance methods on readcon::ConFrameBuilder, with std::optional return for optional-vector getters).

  • Python: readcon.Atom already exposes per-atom fields with #[pyo3(get, set)] so Python users have always been able to do frame.atoms[i].x = ... / .fx = ... / .energy = ... for in-place mutation. This is the recommended Python pattern; no separate readcon.ConFrameBuilder class is exposed because the Python module is value-based rather than handle-based.

  • Julia: equivalently, ReadCon.Atom fields are mutable Julia struct fields; mutate directly and call write_con(path, frame). No public Julia builder class.

Bindings for other languages SHOULD pick the idiom that matches their existing surface: handle-based (Rust / C / C++) gets the typed mutation API, value-based (Python / Julia / Ruby / …) gets direct field assignment on the per-atom struct.

Rationale (non-normative): the append-only authoring path forced every step of a long simulation to allocate a fresh builder, push N atoms, and discard. Reusing one builder across the simulation loop and serialising only at trajectory boundaries reduces allocation pressure to the output cadence rather than the integration cadence. The mutation API is purely additive; the existing add_atom + with_* flow continues to work and remains the recommended path for one-shot frame authoring.

Storage layout (informative)

The v0.11.0 reference implementation holds per-atom fields as structure-of-arrays (SoA) buffers, each owned by an ndarray::Array2<f64> or ndarray::Array1<T> in row-major (“C”) layout:

Field

Type

Shape

positions

ndarray::Array2<f64>

(N, 3)

velocities

ndarray::Array2<f64> (optional)

(N, 3)

forces

ndarray::Array2<f64> (optional)

(N, 3)

atom_energies

ndarray::Array1<f64> (optional)

(N,)

masses

ndarray::Array1<f64>

(N,)

atom_ids

ndarray::Array1<u64>

(N,)

symbols

Vec<String>

(N,)

fixed

Vec<[bool; 3]>

(N, 3)

Bindings MUST surface these fields as contiguous row-major buffers or as a DLPack-managed tensor (preferred for cross-language zero- copy); see builder-dlpack-export for the cross-language contract.

The SoA-per-field choice (rather than AoS, NumPy-structured-array style, or per-atom Option<[f64; 3]>) matches the layout used by LAMMPS, GROMACS, ASE, MDAnalysis, OpenMM, chemfiles, i-PI, h5md (the canonical MD libraries) and Apache Arrow, DuckDB, Polars, Pandas v2, Parquet, PyTorch, JAX, DGL (the canonical columnar / ML stacks). It is the layout NumPy recommends against structured arrays for, on the same SIMD-throughput grounds.

The Arc<RwLock<ndarray::ArrayD<T>>> storage hook + DLPack 1.0 export pattern adopted here mirrors the design described in Bigi et al. [2026], where the same layout serves as the foundational data exchange surface across the metatensor / metatomic atomistic machine-learning libraries.

Section presence semantics (normative)

The optional sections ("velocities", "forces", "atom_energies") are column-level, not per-atom: when declared, the section applies to every atom in the frame. Per-atom NULL does not round-trip to disk because the .con on-disk format has no NULL encoding.

Implementations MUST follow:

  • add_atom MUST pad newly added rows with zero in every already- declared optional section so the per-field buffer length stays coherent with atom_count().

  • with_velocity / _force / _energy, set_atom_velocity / _force / _energy, set_velocities_from_flat / set_forces_from_flat / set_atom_energies_from_flat MUST declare the corresponding section on first call.

  • clear_atom_velocity / _force / _energy MUST zero the slot but MUST NOT change section presence; the field stays declared and a subsequent build() emits the section block. This matches Apache Arrow’s column-level validity (a column either exists or does not) and h5md’s “dataset present or absent” rule.

  • clear_velocities_section / _forces_section / _energies_section MUST drop the entire section: free the storage, set the presence flag to false, and ensure the next build() does not emit the section block.

DLPack export contract (normative)

The reference implementation exposes every per-atom field with a contiguous backing as a DLPack-managed tensor for cross-language zero-copy interop. DLPack 1.0 is the universal exchange ABI supported by PyTorch, TensorFlow, JAX, NumPy, CuPy, MXNet, TVM, OneFlow, MNN, OneAPI, and mlx, plus a stable C ABI (DLManagedTensorVersioned) for non-Python consumers.

The DLPack tier exposes (builder and, for optional sections, frame C ABI):

  • positions_dlpack — shape (N, 3); write-through mut variant on the builder.

  • velocities_dlpack / forces_dlpack(N, 3) or absent (None / SECTION_ABSENT).

  • atom_energies_dlpack(N,) or absent.

  • masses_dlpack(N,).

  • atom_ids_dlpack(N,) integer ids.

Consumers MUST read shape, strides, dtype (code/bits/lanes), and device from the tensor; they MUST NOT assume a host f64 / double buffer type. On-disk CON coordinates and section values are IEEE binary64 in current writers, and implementations today report kDLFloat / 64 for those fields and kDLUInt / 64 for atom_id; a future implementation MAY change bits while keeping the same shapes—interop goes through the tensor metadata.

The returned tensor MUST report:

  • shape: the field’s natural shape ((N, 3) or (N,)).

  • strides: row-major standard layout for the dtype.

  • dtype: as carried by the implementation (see above for current values).

  • device: kDLCPU (device id 0) for the default backing. GPU / optional --features cuda enables real CUDA allocate and H2D export for matching kDLCUDA requests; default builds reject non-CPU devices.

  • data pointer: the backing array’s first element.

Lifetime: DLPackTensorRef<'a> borrows from &'a self and MUST NOT outlive the builder. C / FFI consumers that need an owning tensor go through the C-ABI export (rkr_frame_builder_*_dlpack) which fills a DLManagedTensorVersioned* with a clone of the field’s storage plus a deleter callback owned by the consumer; the original builder is then free to drop.

Pointer stability (normative)

The data pointer returned by any DLPack export is stable as long as no add_atom call grows the underlying ndarray buffer. Callers that hold raw pointers across add_atom MUST refresh after the push (the ndarray reallocates when capacity is exceeded; the DLPack view becomes invalid).

Bulk and per-element mutation (set_atom_*, set_*_from_flat, clear_atom_*) MUST NOT reallocate the underlying buffer and the data pointer MUST stay stable across these operations.

References

[BAL+26]

Filippo Bigi, Joseph W. Abbott, Philip Loche, Arslan Mazitov, Davide Tisi, Marcel F. Langer, Alexander Goscinski, Paolo Pegolo, Sanggyu Chong, Rohit Goswami, Pol Febrer, Sofiia Chorna, Matthias Kellner, Michele Ceriotti, and Guillaume Fraux. Metatensor and metatomic : foundational libraries for interoperable atomistic machine learning. The Journal of Chemical Physics, 2026. URL: https://doi.org/10.1063/5.0304911, doi:10.1063/5.0304911.

[CWT+14]

Samuel T. Chill, Matthew Welborn, Rye Terrell, Liang Zhang, Jean-Claude Berthet, Andreas Pedersen, Hannes Jónsson, and Graeme Henkelman. EON: software for long time simulations of atomic scale systems. Modelling and Simulation in Materials Science and Engineering, 22(5):055002, 2014. doi:10.1088/0965-0393/22/5/055002.