hle

hle

HLE dataset provider (cais/hle).

Adapted from IPW's reasoning benchmark loaders.

Classes

HLEDataset

HLEDataset(*, text_only: bool = True)

Bases: DatasetProvider

HLE (Humanity's Last Exam) benchmark dataset.

Source code in src/openjarvis/evals/datasets/hle.py

def __init__(self, *, text_only: bool = True) -> None:
    self._text_only = text_only
    self._records: List[EvalRecord] = []