Skip to content

adp

adp

Agent Data Collection (neulab) — external agent-trajectory corpus.

NOT USED FOR EVALUATION. Surfaces multi-turn agent trajectories to the LLM-guided spec search proposer via openjarvis.learning.distillation.external_adapter so the diagnose phase can reason over a broad pool of agent traces without depending on the per-cell student's own trace history.

HF dataset: neulab/agent-data-collection — a multi-config collection of agent trajectory datasets (AgentTuning subsets, CodeAct, OpenHands, etc.). Each config shares the same std split schema: - id: trajectory identifier - content: list-of-dicts with keys class_, source, content - details: per-config metadata dict

Conversion to EvalRecord: - problem : content of the first turn whose source == "user" - reference: content of the last turn whose class_ == "message_action" (the agent's final response/action), truncated to 2000 chars

Classes

ADPDataset 

ADPDataset()

Bases: DatasetProvider

Agent Data Collection (neulab) external corpus for LLM-guided spec search.

Source code in src/openjarvis/evals/datasets/adp.py 
117
118
def __init__(self) -> None:
    self._records: List[EvalRecord] = []

Functions