hw-apex-bench v0.2.0 · Evidence, not estimates

Axis 5 · Application corruption

Synthetic evidence establishes correctness, not performance. Unknown machines remain separate; missing values are not zeros. Axis 5 v1 supports synthetic experiments only.

Leaderboard

axis5-844b19c7cbb91239705694a34349abce5e5846fe51e621b45e57cc6c302d777d-hash-off

Comparison conditions
{
  "axis": 5,
  "corpus": "cd9eae0813be8ef4f1fefe403c331279af143282ab385e6edf941003d0ce8a91",
  "kind": "synthetic",
  "machine": null,
  "per_kind": 20,
  "protocol": "e30f78ff89acad8386afb883c14d0e841ead5851ec3cd88a5c861c33723e811e",
  "requests": "d59375f426974615ad39f16658d709c0617df16829855f86406dc53619a307b0",
  "schema": "axis5-evidence-v1",
  "scope": null,
  "seed": 20261003,
  "timeout_seconds": 10
}
axis5-844b19c7cbb91239705694a34349abce5e5846fe51e621b45e57cc6c302d777d-hash-off — independently verified sources
formatvariantkindscopemachinehash_modestatusdetectedsilent_errorrefusalhangcrashharmlessevidence_sha256details
refrel3q4ksyntheticNot recordedNot recordedoffPASS00100000844b19c7cbb91239705694a34349abce5e5846fe51e621b45e57cc6c302d777dEvidence and reproduction

axis5-844b19c7cbb91239705694a34349abce5e5846fe51e621b45e57cc6c302d777d-hash-on

Comparison conditions
{
  "axis": 5,
  "corpus": "cd9eae0813be8ef4f1fefe403c331279af143282ab385e6edf941003d0ce8a91",
  "kind": "synthetic",
  "machine": null,
  "per_kind": 20,
  "protocol": "e30f78ff89acad8386afb883c14d0e841ead5851ec3cd88a5c861c33723e811e",
  "requests": "d59375f426974615ad39f16658d709c0617df16829855f86406dc53619a307b0",
  "schema": "axis5-evidence-v1",
  "scope": null,
  "seed": 20261003,
  "timeout_seconds": 10
}
axis5-844b19c7cbb91239705694a34349abce5e5846fe51e621b45e57cc6c302d777d-hash-on — independently verified sources
formatvariantkindscopemachinehash_modestatusdetectedsilent_errorrefusalhangcrashharmlessevidence_sha256details
refrel3q4ksyntheticNot recordedNot recordedonPASS10000000844b19c7cbb91239705694a34349abce5e5846fe51e621b45e57cc6c302d777dEvidence and reproduction

Methodology, definitions and acceptance criteria

Complete source methodology · SHA-256 9b1ce9415146ebd7588791bdff318e58610dfd23afba3d5b40800f3e40b339e7

Canonical bytes and coordinates

Truth is uppercase sequence bytes without FASTA headers or line separators. Preserve the original source objects unchanged. A common request is (assembly_id, contig_id, start0, end0), using zero-based half-open coordinates [start0,end0). It is valid iff 0 <= start0 < end0 <= contig_length; its length is exactly end0-start0. A request never crosses a contig boundary.

Adapters translate only at the library boundary:

Definitions and criteria from the verified methodology
Interface Translated coordinates Convention
Common request [start0,end0) 0-based half-open
htslib faidx_fetch_seq64 [start0,end0-1] 0-based inclusive
AGC range API [start0,end0-1] 0-based inclusive
One-based inclusive API [start0+1,end0] 1-based inclusive
Sequence-byte raw-offset API prefix+start0, end0-start0 Canonical offset and length

Record both canonical and translated coordinates with their conventions in evidence. A native FASTA-aware reader may use a verified FASTA index internally; raw FASTA offsets are never substituted for canonical coordinates. The external request space is per contig even when a codec stores concatenated sequence bytes. Do not clip, substitute another contig, shrink W or repair an invalid/empty request. Such requests produce FAILED. Contigs shorter than W have no valid starts for that W; sampling excludes those starts, not failures after decoding.

Same machine and execution boundary

CPU in-process, GPU in-process and process-per-request CLI are separate scopes; never rank across them. CPU in-process uses one persistent library handle and one decode thread. Open/index/reference setup occurs outside window timing; the reader operation, including its required lookups and decoding, remains inside. Do not silently substitute a helper process, a decoded-data cache or a different machine. Declare residency and use the same access/cache policy for each comparable configuration.

The official host must be ace-core. Before running, retain hostname, uptime/load, one-second process CPU deltas, frequency policy and available storage. The gate requires load <0.5, storage >20,000,000,000 bytes, and no foreign process consuming 10% or more of one CPU over the sample interval. Keep thresholds and raw telemetry. A passing sample is not proof of continuous quiescence. A GitHub-hosted runner is not a substitute for ace-core, and a green PR does not authorize a measurement.

Axis 5: synthetic application corruption (A7)

The frozen protocol was written before the retained native runs. It supersedes the older one-bit-only Axis 5 description for this experiment. Prepare freezes clean archives, donor archive, library and FASTA identities, full-contig queries and protocol. Corrupt creates bit flips, byte replacement, truncation, physical byte-block swaps and donor block substitution. Archive-byte blocks are not semantic codec frames. The seed, case count and limits are fixed in the protocol; the actual native log records the executed experiment. Each mutation runs with application SHA guard ON and OFF; OFF does not disable codec-internal checksum validation.

A clean baseline must match independent full-contig FASTA truth. A fresh native process group is used per OFF case; the frozen 10-second watchdog terminates the group at timeout. Address-space/output/archive limits and diagnostics are retained. Pre-decode refusal requires an explicit validation stage. A worker's successful process exit does not prove that its output is correct. Classification is disjoint and exhaustive:

Definitions and criteria from the verified methodology
Outcome Definition
detected ON guard rejects a changed archive SHA before starting the decoder
silent_error OFF worker exits successfully but output length/bytes differ from truth
refusal OFF worker exits with a positive nonzero return code
hang Watchdog reports timeout; takes priority over exit/signal classification
crash OFF worker exits by signal, without watchdog timeout
harmless OFF worker exits successfully with exact reference bytes

The independent verifier rechecks externally anchored prepare, every FASTA/library/archive/sidecar, clean baseline, regenerated mutations, raw return codes/signals/timeouts, output bytes, all classifications, aggregate counts and integrity verdicts. It does not open a codec. Integrity PASS means no silent_error in that hash mode; integrity FAIL remains a valid, faithful experiment. Harness verification PASS does not imply integrity PASS. Forged classifications or any inconsistent artifact make verification fail.

The v1 schema supports synthetic evidence only and records neither machine ID nor scope. A7 cannot be relabelled as an official ace-core experiment or combined into a machine comparison. Observed counts describe the frozen physical mutations and resource limits, not universal codec guarantees. Hangs/crashes and harmless changes remain visible.