frames

frames

FRAMES benchmark dataset (google/frames-benchmark).

Adapted from IPW's frames.py dataset loader.

Classes

FRAMESDataset

FRAMESDataset()

Bases: DatasetProvider

FRAMES multi-hop factual retrieval benchmark.

Source code in src/openjarvis/evals/datasets/frames.py

def __init__(self) -> None:
    self._records: List[EvalRecord] = []