hw-apex-bench v0.2.0 · Evidence, not estimates

Axis 4 · Cohort random access

Synthetic evidence establishes correctness, not performance. Unknown machines remain separate; missing values are not zeros. Axis 5 v1 supports synthetic experiments only.

Leaderboard

axis4-threads-1

Comparison conditions
{
  "affinity": [
    0,
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
    9,
    10,
    11,
    12,
    13,
    14,
    15
  ],
  "axis": 4,
  "corpus": "d6eb104bff84bec7ee9f5cf23580d4cc80b0fb132570fe0737dbee702a0b1d77",
  "cpu_models": [
    "AMD EPYC 4344P 8-Core Processor"
  ],
  "domain": "uppercase-sequence-without-FASTA-framing",
  "kind": "official",
  "machine": "ace-core",
  "protocol": "4a0027fec59b3252dc07b29458a57a5a451957f1b64b36f3927765cbc6be7a5c",
  "requests": "79638817948218e1b0d8ae3f3ecbe555ace6c33d8c8d3ad098c821e591384b61",
  "schema": "axis4-evidence-v5",
  "scope": "cpu-in-process",
  "threads": 1,
  "timing_boundary": "complete group: native fetch, uppercase normalization, SHA-256 judgement and translation; response file writes outside timer"
}
axis4-threads-1 — independently verified sources
formatvariantkindscopemachinehash_modestatusverifiedstored_bytessecondsevidence_sha256details
agcnorefofficialcpu-in-processace-coreNot recordedPASS2008385587510.131023379f094e425c762a258914a51bcbfc1a900321374c4548980ba1b0570ebc0ac0e5eEvidence and reproduction
agct2tofficialcpu-in-processace-coreNot recordedPASS20040053751940.134811869551a30a1ca2d2a63fbc42f2e9183ba00dc08f2c353134d54ee4ef8a2285b5fdfEvidence and reproduction
bgzfdefaultofficialcpu-in-processace-coreNot recordedPASS20078853651100.129334877c85619153084e1b0daa600e41fd817fc41eb7f73def39d5539e1c8fe28c7a7a8Evidence and reproduction
fasta-faidxplainofficialcpu-in-processace-coreNot recordedPASS200304836282500.067226116f717da19ae92b04bdd15dc73e643b52efc37fbd540db3636bc40d0fb59658691Evidence and reproduction
refrel3q16kofficialcpu-in-processace-coreNot recordedPASS20032940130720.083948301f512020b18308033970a4204c01f5aa62604e203186096a6315420f2f7a10102Evidence and reproduction
refrel3q4kofficialcpu-in-processace-coreNot recordedPASS20033694269210.0814411487ea4983b99c422c9e9d1829eeda90a0915b08f8bb982d2ee24dd0105a447c04dEvidence and reproduction
zstd-seekableseekable_compression level 3, frame 16384 Bofficialcpu-in-processace-coreNot recordedPASS20093126829010.105514615df99f368ee3fdb7ea160a29114dff9223e8be210b3a3c2b69fba68f34d55f55dEvidence and reproduction

axis4-threads-16

Comparison conditions
{
  "affinity": [
    0,
    1,
    2,
    3,
    4,
    5,
    6,
    7,
    8,
    9,
    10,
    11,
    12,
    13,
    14,
    15
  ],
  "axis": 4,
  "corpus": "d6eb104bff84bec7ee9f5cf23580d4cc80b0fb132570fe0737dbee702a0b1d77",
  "cpu_models": [
    "AMD EPYC 4344P 8-Core Processor"
  ],
  "domain": "uppercase-sequence-without-FASTA-framing",
  "kind": "official",
  "machine": "ace-core",
  "protocol": "4a0027fec59b3252dc07b29458a57a5a451957f1b64b36f3927765cbc6be7a5c",
  "requests": "79638817948218e1b0d8ae3f3ecbe555ace6c33d8c8d3ad098c821e591384b61",
  "schema": "axis4-evidence-v5",
  "scope": "cpu-in-process",
  "threads": 16,
  "timing_boundary": "complete group: native fetch, uppercase normalization, SHA-256 judgement and translation; response file writes outside timer"
}
axis4-threads-16 — independently verified sources
formatvariantkindscopemachinehash_modestatusverifiedstored_bytessecondsevidence_sha256details
bgzfdefaultofficialcpu-in-processace-coreNot recordedPASS20078853651100.10948917258b46f8e383db8003468acd0955bfa4459608aebac04ffb98cbbd204871a7211Evidence and reproduction

Methodology, definitions and acceptance criteria

Complete source methodology · SHA-256 9b1ce9415146ebd7588791bdff318e58610dfd23afba3d5b40800f3e40b339e7

Canonical bytes and coordinates

Truth is uppercase sequence bytes without FASTA headers or line separators. Preserve the original source objects unchanged. A common request is (assembly_id, contig_id, start0, end0), using zero-based half-open coordinates [start0,end0). It is valid iff 0 <= start0 < end0 <= contig_length; its length is exactly end0-start0. A request never crosses a contig boundary.

Adapters translate only at the library boundary:

Definitions and criteria from the verified methodology
Interface Translated coordinates Convention
Common request [start0,end0) 0-based half-open
htslib faidx_fetch_seq64 [start0,end0-1] 0-based inclusive
AGC range API [start0,end0-1] 0-based inclusive
One-based inclusive API [start0+1,end0] 1-based inclusive
Sequence-byte raw-offset API prefix+start0, end0-start0 Canonical offset and length

Record both canonical and translated coordinates with their conventions in evidence. A native FASTA-aware reader may use a verified FASTA index internally; raw FASTA offsets are never substituted for canonical coordinates. The external request space is per contig even when a codec stores concatenated sequence bytes. Do not clip, substitute another contig, shrink W or repair an invalid/empty request. Such requests produce FAILED. Contigs shorter than W have no valid starts for that W; sampling excludes those starts, not failures after decoding.

Same machine and execution boundary

CPU in-process, GPU in-process and process-per-request CLI are separate scopes; never rank across them. CPU in-process uses one persistent library handle and one decode thread. Open/index/reference setup occurs outside window timing; the reader operation, including its required lookups and decoding, remains inside. Do not silently substitute a helper process, a decoded-data cache or a different machine. Declare residency and use the same access/cache policy for each comparable configuration.

The official host must be ace-core. Before running, retain hostname, uptime/load, one-second process CPU deltas, frequency policy and available storage. The gate requires load <0.5, storage >20,000,000,000 bytes, and no foreign process consuming 10% or more of one CPU over the sample interval. Keep thresholds and raw telemetry. A passing sample is not proof of continuous quiescence. A GitHub-hosted runner is not a substitute for ace-core, and a green PR does not authorize a measurement.

Axis 4: cohort random access (A6)

Axis 4 maps each query group explicitly to every selected assembly. The CLI implements prepare, silence, native run, evidence and independent verify. Prepare binds each original FASTA SHA-256, contig order and length, the complete ordered query groups, protocol and native-reader plan. Coordinates and canonical bytes follow the definitions above. No homology is inferred between contigs of different assemblies.

The timed group includes native fetch, uppercase normalization, SHA judgement and coordinate translation; response-file writes are outside timing. The recorded boundary is not interchangeable with decoder-only Axis 3 timing. Stored bytes charge archives and required indexes/references. Synthetic seconds are not official performance fields. Unsupported/missing values remain absent; the leaderboard never imputes zeros.

verify requires an external prepare-SHA anchor, checks FASTA hashes against prepare, scans FASTA independently of the native reader, and compares the SHA of every answer. It independently reproduces the seed-selected sample IDs and compares retained answer bytes too. The sample limit is 32 responses, established by the A6 native log; hashes cover all responses. A changed FASTA, SHA, duplicate ID, altered coordinate, bad length or retained byte mismatch fails verification. Official evidence additionally requires passing before/after silence telemetry. SHA binds bytes, not authorship or continuous host quiescence.

Schema: axis4-evidence-v3. PASS requires every declared group/response to be represented and verified; reader/data failure cannot become a valid performance row. Native availability alone does not establish correctness, large-input coverage or a throughput win.