Benchmarks

ivre bench measures the performance of the configured backend by running a fixed set of workloads (scenarios) and emitting a JSON report of per-scenario latency metrics. It exists so that a backend change or a shared-helper refactor can be checked for performance regressions, and so the supported backends can be compared on the same data.

Functional parity (a query returns the right rows on every backend) does not imply performance parity: a missing index or a poor query plan can make an otherwise-correct backend orders of magnitude slower. ivre bench turns that invisible difference into a number.

Running

The backend is the one selected by ivre.conf (DB = ...); run the tool once per backend you want to compare. --backend only labels the report (it is auto-detected from the configured URL when omitted):

$ ivre bench --list                     # show available scenarios
$ ivre bench                             # run every scenario
$ ivre bench --scenario bench_top_service
$ ivre bench --backend postgresql --output pg.json

The scenarios read the view purpose, so the figures are only meaningful against a populated database; the numbers from an empty store are not comparable.

Options:

--list: List the available scenarios and exit.
--scenario NAME: Run a single scenario (repeatable; default: all). See --list.
--backend NAME: Backend label recorded in the report (default: detected from the configured DB).
--iterations N: Timed iterations per scenario (default: 20).
--warmup N: Untimed warmup iterations per scenario, run before timing to prime caches / connection pools (default: 3).
--output FILE: Write the JSON report to FILE (default: stdout).

Output

One JSON document per run, carrying the IVRE version, the backend label, the Python/platform identification, and one record per scenario. Each record reports latency in milliseconds as min / p50 / p95 / max / mean (percentiles are nearest-rank, so every reported value was actually measured):

{
  "backend": "postgresql",
  "ivre_version": "...",
  "python": "3.12.x",
  "platform": "...",
  "results": [
    {
      "scenario": "bench_top_service",
      "description": "Top 15 service values across the whole view.",
      "iterations": 20,
      "warmup": 3,
      "latency_ms": {"min": ..., "p50": ..., "p95": ..., "max": ..., "mean": ...}
    }
  ]
}

Archiving these records (e.g. as CI artefacts) gives a latency trend per backend over time.

Scenarios

The set is intentionally small at first and grows as new workloads are added; ivre bench --list is always the authoritative list. Each scenario is a single, cursor-materialising unit of work so the whole query cost is measured:

bench_top_service: Top 15 service values over the whole view (ivre view --top service --limit 15).
bench_top_port: Top 15 ports over the whole view (ivre view --top port --limit 15).

Adding a scenario is one entry in SCENARIOS in ivre.tools.bench.