Benchmarks

This directory contains committed benchmark summaries. Raw JSON, CSV, traces, and downloaded inputs belong under gitignored results/ and are regenerated by the command recorded in each report.

Labels

• mlx_atomistic is the project MLX/Metal runtime.
• openmm-reference is an OpenMM validation or performance reference.
• lammps-reference is a LAMMPS validation or performance reference.

Reference engines never enter the product runtime path. A cross-engine ratio is valid only when the report demonstrates matching system, force-field semantics, precision, timestep, constraints, platform, diagnostics, and measurement window.

Active Performance and Decisions

Report	Purpose
Canonical MD suite	Required 5DFR/JAC gate plus optional GPCRmd, water, and ApoA1 coverage.
MD performance decision ledger	Compact retained/rejected history for the Metal MD runtime.
Adaptive Neighbor scatter	Current exact-tile producer, crossover boundary, and final 5DFR/JAC/GPCRmd evidence.
DFT performance decision ledger	Production scheduler, Hpsi/FFT attribution, and rejected custom-kernel boundaries.

Scientific Validation

Report	Scope
DFT material validation	Accepted silicon, carbon, and magnesium-oxide workloads and declared limits.
Scalable Neighbor/nonbonded	Orthorhombic Neighbor and nonbonded parity from 1k to 92k atoms.
Charged JAC PME	94,232-atom AMBER20 JAC parity, memory, and bounded trajectory evidence.
GPCRmd 729 PME	92,001-atom CHARMM/NBFIX/CMAP parity, bounded trajectory, and restart evidence.

Reference and Same-Workload Snapshots

Report	Scope
Benchmark ladder	Which benchmark answers which engineering question.
Same-workload LJ scaling	MLX, OpenMM, and LAMMPS Lennard-Jones scaling snapshot.
Same-workload OpenMM comparison	Manifest-aware OpenMM comparison snapshots and caveats.
M5 Max reference engines	Reference environment and normalized manifests.
OpenMM DHFR	OpenMM/OpenCL DHFR snapshot.
OpenMM ApoA1	OpenMM/OpenCL ApoA1 snapshot.
OpenMM Amber20	OpenMM/OpenCL Cellulose and STMV snapshot.
LAMMPS OpenCL	Official LAMMPS case classification and local OpenCL status.

Snapshot reports are historical measurements, not automatically current-main ratios. Their dates and protocol caveats are part of the result.

Routine Commands

Fast tests must not require OpenMM, LAMMPS, OpenCL, downloaded systems, or a Metal GPU.

uv run python -m mlx_atomistic.benchmarks.md_suite list
uv run pytest tests/test_benchmarks.py -m "not slow"
uv run python -m mlx_atomistic.benchmarks.mm_force_terms \
  --evaluations 1 --particles 16 --json
uv run python -m mlx_atomistic.benchmarks.md_acceleration \
  --sizes 16 --evaluations 1 --json

The canonical opt-in local MD run is:

uv run python -m mlx_atomistic.benchmarks.md_suite run \
  --suite local \
  --out results/md-suite/current.json

Structural Neighbor attribution is opt-in and intrusive:

uv run python -m mlx_atomistic.benchmarks.charged_pme runtime \
  --prepared results/larger-system-scaling/jac-2x2x1-modern/prepared \
  --warmups 10 \
  --steps 750 \
  --neighbor-rebuild-profile \
  --out results/md-suite/rebuild-profile-jac.json

--neighbor-rebuild-profile inserts completion boundaries between builder stages. Use it to establish work ownership, never as clean throughput.

Report Contract

A retained performance report should include:

1. source commit and hardware/power state;
2. exact system and prepared-artifact provenance;
3. timestep, cutoff, skin, constraints, precision, diagnostics, and sampling;
4. position-balanced independent-process control/candidate order;
5. complete-wall medians, rebuild counts, and memory;
6. correctness and route checks;
7. raw output path and reproducer.

Rejected micro-optimization reports are consolidated into the decision ledger. Git history preserves their former long-form records.

Local Artifact Retention

results/ is a disposable local workspace, not a historical archive. Keep the smallest set needed to reproduce current decisions:

• current prepared fixtures and their source provenance;
• the latest accepted clean and instrumented benchmark reports;
• compact summaries that are not yet recorded in a committed ledger;
• inputs that are expensive or impossible to reacquire.

Once a decision and reproducer are committed, remove superseded raw repeats, rejected prototype directories, GPU traces, trajectory dumps, and restartable SCF array caches. A raw path must never be the only surviving record of a retained or rejected decision. Record the result, boundary, source commit, and reproducer in the appropriate ledger first.

The repository .gitignore already excludes results/, outputs/, common array and trajectory formats, Python caches, macOS metadata, and generated site content. Do not add redundant per-experiment ignore rules.