throughput

throughput

Throughput benchmark — measures tokens per second with per-sample stats.

Classes

ThroughputBenchmark

Bases: BaseBenchmark

Measures inference throughput in tokens per second.

Functions

ensure_registered

ensure_registered() -> None

Register the throughput benchmark if not already present.

Source code in src/openjarvis/bench/throughput.py

def ensure_registered() -> None:
    """Register the throughput benchmark if not already present."""
    if not BenchmarkRegistry.contains("throughput"):
        BenchmarkRegistry.register_value("throughput", ThroughputBenchmark)