Skip to content

Index

prompts

Prompt templates for SkillOrchestra.

Centralized prompts for: - eval_orchestrator: FRAMES orchestrator (search/code/answer) - learning: handbook discovery, refinement, profiler - model_routing: QA benchmarks (skill-based and baseline routing)

Functions

build_skill_orchestrator_prompt

build_skill_orchestrator_prompt(problem: str, context_str: str, strategy: str = 'router_decides', handbook: Any = None, search_skills: Optional[str] = None, reasoning_skills: Optional[str] = None, answer_skills: Optional[str] = None, search_model_performance: Optional[str] = None, reasoning_model_performance: Optional[str] = None, answer_model_performance: Optional[str] = None, baseline_tool_info: Optional[str] = None) -> str

Build enhanced orchestrator prompt with skill catalog and model performance.

Args: problem: The question/problem to solve context_str: Current context (documents, code results, etc.) strategy: Routing strategy - "router_decides" or "analyze_model_decide" etc. handbook: SkillHandbook object with format_skills(stage/mode) and format_model_performance(stage/mode) search_skills: Override - skill definitions for search stage reasoning_skills: Override - skill definitions for reasoning stage answer_skills: Override - skill definitions for answer stage search_model_performance: Override - model performance for search reasoning_model_performance: Override - model performance for reasoning answer_model_performance: Override - model performance for answer baseline_tool_info: Override - baseline tool descriptions

Returns: Formatted prompt string

Source code in src/openjarvis/agents/hybrid/skillorchestra/prompts/eval_orchestrator.py
def build_skill_orchestrator_prompt(
    problem: str,
    context_str: str,
    strategy: str = "router_decides",
    handbook: Any = None,
    search_skills: Optional[str] = None,
    reasoning_skills: Optional[str] = None,
    answer_skills: Optional[str] = None,
    search_model_performance: Optional[str] = None,
    reasoning_model_performance: Optional[str] = None,
    answer_model_performance: Optional[str] = None,
    baseline_tool_info: Optional[str] = None,
) -> str:
    """
    Build enhanced orchestrator prompt with skill catalog and model performance.

    Args:
        problem: The question/problem to solve
        context_str: Current context (documents, code results, etc.)
        strategy: Routing strategy - "router_decides" or "analyze_model_decide" etc.
        handbook: SkillHandbook object with format_skills(stage/mode) and format_model_performance(stage/mode)
        search_skills: Override - skill definitions for search stage
        reasoning_skills: Override - skill definitions for reasoning stage
        answer_skills: Override - skill definitions for answer stage
        search_model_performance: Override - model performance for search
        reasoning_model_performance: Override - model performance for reasoning
        answer_model_performance: Override - model performance for answer
        baseline_tool_info: Override - baseline tool descriptions

    Returns:
        Formatted prompt string
    """
    if baseline_tool_info is None:
        baseline_tool_info = format_baseline_tool_info()

    if handbook:
        if search_skills is None:
            search_skills = handbook.format_skills("search")
        if reasoning_skills is None:
            reasoning_skills = handbook.format_skills("code")
        if answer_skills is None:
            answer_skills = handbook.format_skills("answer")
        if search_model_performance is None:
            search_model_performance = handbook.format_model_performance("search")
        if reasoning_model_performance is None:
            reasoning_model_performance = handbook.format_model_performance("code")
        if answer_model_performance is None:
            answer_model_performance = handbook.format_model_performance("answer")

    search_skills = search_skills or "No skills defined"
    reasoning_skills = reasoning_skills or "No skills defined"
    answer_skills = answer_skills or "No skills defined"
    search_model_performance = search_model_performance or "No performance data"
    reasoning_model_performance = reasoning_model_performance or "No performance data"
    answer_model_performance = answer_model_performance or "No performance data"

    if strategy == "router_decides":
        template = SKILL_ORCHESTRATOR_PROMPT
    else:
        template = SKILL_ANALYSIS_ORCHESTRATOR_PROMPT

    return template.format(
        baseline_tool_info=baseline_tool_info,
        search_skills=search_skills,
        reasoning_skills=reasoning_skills,
        answer_skills=answer_skills,
        search_model_performance=search_model_performance,
        reasoning_model_performance=reasoning_model_performance,
        answer_model_performance=answer_model_performance,
        problem=problem,
        context_str=context_str if context_str else "(No context yet)",
    )

format_baseline_tool_info

format_baseline_tool_info() -> str

Format baseline tool information for the orchestrator prompt.

Source code in src/openjarvis/agents/hybrid/skillorchestra/prompts/eval_orchestrator.py
def format_baseline_tool_info() -> str:
    """Format baseline tool information for the orchestrator prompt."""
    return """- Tool: search, Models: search-1 ($10/M output), search-2 ($2/M output), search-3 ($0.8/M output)
Description: Search for missing information
- Tool: code|enhance_reasoning, Models: reasoner-1 ($10/M output), reasoner-2 ($2/M output), reasoner-3 ($0.8/M output)
Description: Write and execute Python code to solve the problem
- Tool: answer, Models: answer-1 ($10/M output), answer-2 ($2/M output), answer-3 ($0.9/M output), answer-4 ($0.8/M output), answer-math-1 ($0.9/M output), answer-math-2 ($0.2/M output)
Description: Extract the final answer if you think you have enough information to answer the problem"""