Skip to content

backend

backend

Abstract base class for inference backends.

Classes

InferenceBackend

Bases: ABC

Base class for all inference backends used in evaluation.

Functions
generate abstractmethod 
generate(prompt: str, *, model: str, system: str = '', temperature: float = 0.0, max_tokens: int = 2048) -> str

Generate a response and return just the text content.

Source code in src/openjarvis/evals/core/backend.py 
14
15
16
17
18
19
20
21
22
23
24
@abstractmethod
def generate(
    self,
    prompt: str,
    *,
    model: str,
    system: str = "",
    temperature: float = 0.0,
    max_tokens: int = 2048,
) -> str:
    """Generate a response and return just the text content."""
generate_full abstractmethod 
generate_full(prompt: str, *, model: str, system: str = '', temperature: float = 0.0, max_tokens: int = 2048) -> Dict[str, Any]

Generate a response and return full details.

Returns dict with keys: content, usage, model, latency_seconds, cost_usd.

Source code in src/openjarvis/evals/core/backend.py 
26
27
28
29
30
31
32
33
34
35
36
37
38
39
@abstractmethod
def generate_full(
    self,
    prompt: str,
    *,
    model: str,
    system: str = "",
    temperature: float = 0.0,
    max_tokens: int = 2048,
) -> Dict[str, Any]:
    """Generate a response and return full details.

    Returns dict with keys: content, usage, model, latency_seconds, cost_usd.
    """
close 
close() -> None

Release resources.

Source code in src/openjarvis/evals/core/backend.py 
41
42
def close(self) -> None:
    """Release resources."""