Skip to content

coding_task

coding_task

Coding task benchmark dataset.

Standalone function-level coding problems with test cases for evaluating code generation accuracy.

Classes

CodingTaskDataset 

CodingTaskDataset()

Bases: DatasetProvider

Coding task benchmark: function-level code generation with test cases.

Source code in src/openjarvis/evals/datasets/coding_task.py 
237
238
def __init__(self) -> None:
    self._records: List[EvalRecord] = []