Skip to content

ipw_mixed

ipw_mixed

IPW mixed dataset provider.

Loads evaluation data from a local directory containing HuggingFace Arrow datasets or JSONL files. Does not download from HuggingFace — the data must be present on disk (e.g. bundled from the IPW repository).

Classes

IPWDataset 

IPWDataset(data_dir: Optional[str] = None, dataset_name: Optional[str] = None)

Bases: DatasetProvider

IPW mixed evaluation dataset loaded from a local directory.

Source code in src/openjarvis/evals/datasets/ipw_mixed.py 
30
31
32
33
34
35
36
37
def __init__(
    self,
    data_dir: Optional[str] = None,
    dataset_name: Optional[str] = None,
) -> None:
    self._data_dir = data_dir
    self._dataset_name = dataset_name or _DEFAULT_DATASET_NAME
    self._records: List[EvalRecord] = []