natural_reasoning

natural_reasoning

Natural Reasoning dataset provider (facebook/natural_reasoning).

Adapted from IPW's reasoning benchmark loaders.

Classes

NaturalReasoningDataset

NaturalReasoningDataset()

Bases: DatasetProvider

Natural Reasoning benchmark dataset.

Source code in src/openjarvis/evals/datasets/natural_reasoning.py

def __init__(self) -> None:
    self._records: List[EvalRecord] = []