Skip to content

gaia

gaia

GAIA benchmark dataset (gaia-benchmark/GAIA).

Adapted from IPW's gaia.py dataset loader.

Classes

GAIADataset 

GAIADataset(cache_dir: Optional[str] = None)

Bases: DatasetProvider

GAIA agentic benchmark dataset.

Source code in src/openjarvis/evals/datasets/gaia.py 
43
44
45
def __init__(self, cache_dir: Optional[str] = None) -> None:
    self._cache_dir = Path(cache_dir) if cache_dir else _DEFAULT_CACHE_DIR
    self._records: List[EvalRecord] = []