Skip to content

Skills Workflow Tutorial

This tutorial walks through the complete skills lifecycle: installing skills from public sources, using them with a local agent, discovering patterns from trace history, and optimizing skill descriptions with DSPy. By the end you will have a working skills setup that improves over time.

Before you begin

This tutorial assumes OpenJarvis is installed with Ollama running and a model available (e.g., qwen3.5:9b). If you have not completed setup yet, start with the Quick Start guide.

Step 1: Install Skills from Hermes Agent

OpenJarvis can import skills from the Hermes Agent skill library maintained by NousResearch. Let's install a few useful ones.

# Install individual skills
jarvis skill install hermes:llm-wiki
jarvis skill install hermes:arxiv

# Or bulk install an entire category
jarvis skill sync hermes --category research

The first install clones the Hermes repo to ~/.openjarvis/skill-cache/hermes/ (one-time, ~5s). Subsequent installs reuse the cache.

Verify what's installed:

jarvis skill list

You should see a table with each skill's name, description, version, and tags.

Step 2: Inspect an Installed Skill

Let's look at what the llm-wiki skill contains:

jarvis skill info llm-wiki

This shows the skill's metadata — author, description, tags, capabilities, whether it has structured steps or markdown instructions, and its invocation flags.

You can also inspect the raw SKILL.md:

cat ~/.openjarvis/skills/hermes/llm-wiki/SKILL.md | head -40

The .source file records provenance:

cat ~/.openjarvis/skills/hermes/llm-wiki/.source

This shows the source (hermes:llm-wiki), the git commit it was imported from, which tool names were translated (e.g., Edit→file_edit), and the install timestamp.

Step 3: Use Skills with an Agent

Now let's ask the agent a question that should trigger skill usage:

jarvis ask --agent orchestrator "Use the llm-wiki skill to explain how model entries should be organized" \
  --engine ollama --model qwen3.5:9b

The agent will: 1. See the skill catalog and skill_llm-wiki tool definition 2. Decide to invoke skill_llm-wiki 3. Receive the markdown instructions from the skill 4. Follow those instructions in its answer

Try the other installed skill too:

jarvis ask --agent orchestrator "Use the arxiv skill to outline how you would research transformer efficiency" \
  --engine ollama --model qwen3.5:9b

The simple agent intentionally cannot call tools. Use orchestrator or another tool-capable agent whenever you want a skill to be invoked.

Step 4: Create Your Own Skill

Create a new skill directory:

mkdir -p ~/.openjarvis/skills/my-reviewer

Write a SKILL.md:

cat > ~/.openjarvis/skills/my-reviewer/SKILL.md << 'EOF'
---
name: my-reviewer
description: Review code changes with a security-first approach
license: MIT
metadata:
  openjarvis:
    version: "0.1.0"
    author: me
    tags: [coding, review, security]
---

When asked to review code, follow this approach:

1. **Security scan first** — check for injection vulnerabilities, hardcoded secrets, unsafe deserialization
2. **Correctness** — verify logic, edge cases, error handling
3. **Style** — naming, structure, consistency with surrounding code
4. **Summary** — one paragraph with the verdict: approve, request changes, or block

Always start with security. If you find a security issue, flag it as BLOCKING regardless of other concerns.
EOF

Verify it's discovered:

jarvis skill list

You should see my-reviewer in the table. Try it:

jarvis ask --agent orchestrator "Use the my-reviewer skill to review this function: def login(user, pwd): return db.query(f'SELECT * FROM users WHERE name={user} AND pass={pwd}')" \
  --engine ollama --model qwen3.5:9b

The agent should follow the security-first approach and flag the SQL injection vulnerability.

Step 5: Generate Traces

For the learning loop to work, we need traces. Run several queries that use skills:

# Tracing is disabled in the generated config until you opt in
jarvis config set traces.enabled true

# Generate a few traces
jarvis ask --agent orchestrator "Use llm-wiki to describe a model entry"
jarvis ask --agent orchestrator "Use my-reviewer to review: def add(a,b): return a+b"
jarvis ask --agent orchestrator "Use llm-wiki to describe an evaluation entry"
jarvis ask --agent orchestrator "Use my-reviewer to review: lambda x: x**2"
jarvis ask --agent orchestrator "Use llm-wiki to describe a dataset entry"
jarvis ask --agent orchestrator "Use my-reviewer to review: def square(x): return x*x"

Each query produces one trace in ~/.openjarvis/traces.db. An invoked skill's tool-call step includes the skill, skill_source, and skill_kind metadata tags.

Step 6: Discover Patterns from Traces

Mine the trace store for recurring tool sequences:

# Preview without writing
jarvis skill discover --dry-run --min-frequency 2 --min-outcome 0

# Write discovered patterns as skill manifests
jarvis skill discover --min-frequency 2 --min-outcome 0

Discovered skills land in ~/.openjarvis/skills/discovered/ and automatically appear in jarvis skill list on the next session.

Discovery requires recurring sequences of at least two tool calls. --min-outcome 0 includes new traces that have not yet been scored.

Step 7: Optimize Skills with DSPy

Once you have enough traces (at least 3-5 per skill), run the optimizer:

# Preview what would be optimized
jarvis optimize skills --dry-run --min-traces 3

# Run DSPy optimization
jarvis optimize skills --policy dspy --min-traces 3

This produces overlay files at ~/.openjarvis/learning/skills/<skill-name>/optimized.toml with improved descriptions and few-shot examples extracted from your best traces.

Inspect what was produced:

jarvis skill show-overlay llm-wiki
jarvis skill show-overlay my-reviewer

The next time you run a query, the agent sees the optimized descriptions and few-shot examples in its system prompt.

Step 8: Benchmark the Impact

Run a quick benchmark to see if skills + optimization actually help:

# Smoke test: 4 conditions × 1 seed × 5 tasks
jarvis bench skills --max-samples 5 --seeds 42

This runs the PinchBench benchmark in four conditions (no skills, skills on, DSPy-optimized, GEPA-optimized) and produces a markdown report at docs/superpowers/results/.

Step 9: Configure Auto-Import and Auto-Optimization

For a hands-off experience, add this to ~/.openjarvis/config.toml:

[skills]
enabled = true
auto_sync = true

[[skills.sources]]
source = "hermes"
filter = { category = ["research", "coding"] }
auto_update = true

[learning.skills]
auto_optimize = true
optimizer = "dspy"
min_traces_per_skill = 20

Now skills are automatically synced from Hermes on session start, and the optimizer runs after each learning cycle when enough traces accumulate.

What You Learned

Concept What you did
Installing skills jarvis skill install hermes:arxiv — imported from public sources
Using skills jarvis ask --agent orchestrator "Use the llm-wiki skill..." — a tool-capable agent invokes skills
Creating skills Wrote a SKILL.md with YAML frontmatter and markdown instructions
Generating traces Enabled tracing and ran skill-using queries to populate the trace store
Discovering patterns jarvis skill discover — mined traces for recurring tool sequences
Optimizing skills jarvis optimize skills --policy dspy — improved descriptions + few-shot examples
Benchmarking jarvis bench skills — measured the impact across 4 conditions
Auto configuration Added [skills] and [learning.skills] config sections

Next Steps