latency

latency

Latency benchmark — measures per-call inference latency.

Classes

LatencyBenchmark

Bases: BaseBenchmark

Measures per-call inference latency with short prompts.

Functions

ensure_registered

ensure_registered() -> None

Register the latency benchmark if not already present.

Source code in src/openjarvis/bench/latency.py

def ensure_registered() -> None:
    """Register the latency benchmark if not already present."""
    if not BenchmarkRegistry.contains("latency"):
        BenchmarkRegistry.register_value("latency", LatencyBenchmark)