Benchmark program

The goal is to measure adotenv against the main Rust .env parsers with reproducible evidence and to keep adotenv at the front of the comparison on material metrics.

Quiet host

Use kbench through SSH for the reference run.

• Intel Core i7-6700HQ, 4 cores and 8 threads, 2.6 GHz
• 15 GiB RAM
• Linux x86_64
• rustc and cargo from the stable toolchain
• hyperfine and valgrind installed
• Record load average, hostname, rustc version and fixture hashes with every run

Run on an otherwise idle host. Do not use shared-runner wall time as a release threshold.

Workstreams

1. Harness and reproducibility

• Standalone comparison crate with pinned competitor versions.
• Simple 128-entry and 65,536-entry fixtures.
• Mixed fixture with quoting, spacing, Unicode, comments and assigned values.
• Wall time, CPU time, callgrind instructions, peak RSS, retained RSS and stripped binary size.
• One-command report generation and optional document update.
• Budget check for selected profiles.
• Record fixture hashes and host metadata in the JSON result.
• Add a real-world corpus from representative public .env files.
• Add a per-dialect adotenv matrix.

2. Comparable output

• Report native-map and values-only paths separately.
• Record hash-map versus ordered-map costs as an API tradeoff: adotenv exposes ordered maps; competitors expose unordered maps.
• Add loader profiles where the competitors expose them; comparable stream profiles do not exist.
• Exclude or clearly label non-comparable APIs.

3. Engineering metrics

• Measure clean release build time for each parser demo.
• Record binary size per feature set.
• Record dependency count, MSRV and build-script requirements.
• Record edit, format, sort and compare throughput for adotenv-only workloads where no competitor equivalent exists.

4. Optimization

• Parse throughput.
• Values-only throughput and memory.
• Map build throughput and memory.
• Default loader and full-report loader memory.
• Parse-only binary size.
• Startup time for the CLI.

5. Reporting

• Generate a benchmark page for the documentation site.
• Include method, host, versions, raw JSON links and budgets.
• Record repeated runs and median; raw hyperfine JSON retains the sample set.
• Keep historical results out of the current docs; store raw artifacts separately.

6. Release gates

• Normal CI runs benchmark smoke only.
• Manual budget check exists.
• Run the complete comparison on kbench before a performance claim or a 1.0 release.
• Compare against the previous accepted baseline and require an explanation for a 1.5x to 2x regression.
• Block above 2x until fixed or explicitly accepted.

Current reference run

The current reference run uses the simple 128-entry and 65,536-entry fixtures, the mixed fixture, CPU time, callgrind instructions, RSS and parse-only binary sizes. The comparison crate remains outside the published package and the normal workspace.

The benchmark matrix is complete. Two performance gaps remain accepted for now:

• mixed and real-world map building is slower than dotenv-compat because adotenv returns an ordered map and retains the lossless document;
• the parse-only binary is about 24 percent larger than the smallest measured parser.

Both gaps are recorded in the competitive analysis and remain candidates for later optimization.

Acceptance for 1.0

• The comparison covers simple, mixed and real-world fixtures.
• All compared parsers use the same input and output definition.
• Wall time, CPU time, instructions, RSS and binary size are recorded.
• Results are reproducible with one command on kbench.
• The docs site publishes the current results and method.
• Performance budgets pass.
• Any remaining gap is documented with a reason.