Skip to content

gpqa

gpqa

GPQA dataset provider (Idavidrein/gpqa).

Adapted from IPW's gpqa.py dataset loader.

Classes

GPQADataset 

GPQADataset()

Bases: DatasetProvider

GPQA (Graduate-Level Google-Proof Q&A) multiple-choice benchmark.

Source code in src/openjarvis/evals/datasets/gpqa.py 
33
34
def __init__(self) -> None:
    self._records: List[EvalRecord] = []