skill-compass

Category: Testing & Security | Uploader: Evol-ai | Downloads: 0 | Version: v1.0（Latest）

Local skill quality and security evaluator - score 6 dimensions, surface the weakest area, optionally apply verified fixes, track versions, and audit at scale.

Changelog: Source: GitHub https://github.com/Evol-ai/SkillCompass

Directory Structure

Current level: tree/main/

📁 .claude-plugin/
- 📄 plugin.json 669 B
📁 .github/
- 📁 ISSUE_TEMPLATE/
  - 📄 bug_report.yml 2.3 KB
  - 📄 config.yml 750 B
  - 📄 evaluation_issue.yml 1.5 KB
  - 📄 feature_request.yml 1.4 KB
- 📁 workflows/
  - 📄 pr-target-guard.yml 891 B
  - 📄 verify.yml 652 B
- 📄 CODEOWNERS 479 B
- 📄 pull_request_template.md 1.2 KB
📁 .skill-compass/
- 📁 test-skill/
  - 📄 audit.jsonl 622 B
- 📁 weak-skill/
  - 📁 snapshots/
    
    📄 1.0.0.md 5.4 KB
  - 📄 manifest.json 432 B
- 📄 .checksums 104 B
- 📄 README.md 518 B
📁 assets/
- 📄 skill-quality-report.png 197.0 KB
📁 commands/
- 📄 eval-audit.md 4.1 KB
- 📄 eval-compare.md 2.0 KB
- 📄 eval-evolve.md 6.9 KB
- 📄 eval-improve.md 6.6 KB
- 📄 eval-merge.md 2.1 KB
- 📄 eval-rollback.md 2.3 KB
- 📄 eval-security.md 1.4 KB
- 📄 eval-skill.md 16.4 KB
- 📄 setup.md 11.3 KB
- 📄 skill-compass.md 2.1 KB
📁 examples/
- 📄 guide-auto-updater.md 3.0 KB
- 📄 guide-claudeception.md 4.7 KB
- 📄 guide-self-improving-agent.md 5.2 KB
- 📄 sample-eval.json 6.3 KB
- 📄 sample-manifest.json 1.9 KB
- 📄 sample-merge.json 1.7 KB
📁 hooks/
- 📁 scripts/
  - 📄 eval-gate.js 10.0 KB
  - 📄 output-guard.js 6.8 KB
  - 📄 post-skill-edit.js 6.7 KB
  - 📄 pre-eval-scan.js 461 B
  - 📄 pre-eval-scan.sh 139 B
- 📄 hooks.json 497 B
📁 lib/
- 📄 audit-chain.js 6.0 KB
- 📄 basic-validator.js 6.3 KB
- 📄 integrity-monitor.js 9.2 KB
- 📄 patterns.js 6.1 KB
- 📄 pre-eval-patterns.js 1.7 KB
- 📄 pre-eval-scan.js 5.8 KB
- 📄 security-validator.js 21.1 KB
- 📄 structure-validator.js 18.2 KB
- 📄 trigger-validator.js 19.2 KB
📁 prompts/
- 📄 d1-structure.md 6.8 KB
- 📄 d2-trigger.md 6.8 KB
- 📄 d3-security.md 12.6 KB
- 📄 d4-functional.md 15.6 KB
- 📄 d5-comparative.md 10.3 KB
- 📄 d6-uniqueness.md 11.4 KB
- 📄 improve.md 4.9 KB
- 📄 merge.md 4.0 KB
📁 schemas/
- 📄 eval-result.json 8.9 KB
- 📄 feedback-signal.json 2.2 KB
- 📄 merge-result.json 2.3 KB
- 📄 version-manifest.json 2.7 KB
📁 scripts/
- 📁 eval-v2/
  - 📄 analyze-results.js 9.7 KB
  - 📄 batch-eval-v6.sh 15.7 KB
  - 📄 codespace-setup.sh 3.2 KB
  - 📄 download-skills.sh 5.1 KB
  - 📄 sample-skills.js 8.9 KB
  - 📄 skills-sample.json 11.7 KB
- 📁 release/
  - 📄 prepare-clawhub-canary.js 6.5 KB
- 📁 tests/
  - 📁 lib/
    
    📄 assert.sh 10.1 KB
    
    📄 helpers.sh 9.3 KB
  - 📄 collect-results.js 6.5 KB
  - 📄 phase-a-hooks.sh 9.3 KB
  - 📄 phase-b-core.sh 31.1 KB
  - 📄 phase-c-version.sh 14.6 KB
  - 📄 phase-d-compound.sh 13.0 KB
  - 📄 phase-e-integration.sh 18.6 KB
  - 📄 phase-vm.sh 9.7 KB
  - 📄 README.md 3.5 KB
  - 📄 run-all.sh 6.2 KB
  - 📄 verify-clawhub.js 6.3 KB
  - 📄 verify-local.js 9.4 KB
📁 shared/
- 📄 feedback-integration.md 2.5 KB
- 📄 llm-capability-baseline.md 3.7 KB
- 📄 scoring.md 2.5 KB
- 📄 skill-registry.json 6.2 KB
- 📄 threat-signatures.yaml 3.1 KB
- 📄 tool-instructions.md 3.2 KB
- 📄 version-management.md 3.8 KB
📁 test-fixtures/
- 📁 atom-formatter/
  - 📁 .skill-compass/
    
    📁 atom-formatter/
    
    📁 snapshots/
    
    📄 1.0.0.md 2.1 KB
    
    📄 manifest.json 436 B
    
    📄 README.md 518 B
  - 📄 SKILL.md 2.1 KB
- 📁 audit-batch/
  - 📁 skills/
    
    📁 skill-a/
    
    📄 SKILL.md 1.6 KB
    
    📁 skill-b/
    
    📄 SKILL.md 290 B
    
    📁 skill-c/
    
    📄 SKILL.md 1.8 KB
- 📁 benign-normal-skill/
  - 📄 SKILL.md 1.1 KB
- 📁 benign-with-code-blocks/
  - 📄 SKILL.md 1.5 KB
- 📁 composite-workflow/
  - 📄 SKILL.md 3.8 KB
- 📁 d1-broken-structure/
  - 📄 SKILL.md 254 B
- 📁 d2-bad-trigger/
  - 📄 SKILL.md 597 B
- 📁 d3-insecure/
  - 📄 SKILL.md 1.7 KB
- 📁 d4-shallow-function/
  - 📄 SKILL.md 734 B
- 📁 d5-no-value/
  - 📄 SKILL.md 1.0 KB
- 📁 d6-duplicate/
  - 📄 SKILL.md 1.8 KB
- 📁 edge-empty/
  - 📄 SKILL.md 0 B
- 📁 edge-huge/
  - 📄 SKILL.md 73.9 KB
- 📁 edge-no-yaml/
  - 📄 SKILL.md 189 B
- 📁 edge-non-english/
  - 📄 SKILL.md 2.1 KB
- 📁 edge-yaml-only/
  - 📄 SKILL.md 186 B
- 📁 malicious-ascii-smuggling/
  - 📄 SKILL.md 512 B
- 📁 malicious-base64-exfil/
  - 📄 SKILL.md 637 B
- 📁 malicious-curl-pipe/
  - 📄 SKILL.md 535 B
- 📁 malicious-prompt-injection/
  - 📄 SKILL.md 763 B
- 📁 merge-scenario/
  - 📁 .skill-compass/
    
    📁 api-client/
    
    📁 snapshots/
    
    📄 1.0.0-evo.1.md 852 B
    
    📄 1.0.0-evo.2.md 1.4 KB
    
    📄 1.0.0.md 222 B
    
    📄 1.1.0.md 1.5 KB
    
    📄 manifest.json 1.6 KB
  - 📄 SKILL.md 1.4 KB
- 📁 meta-decision/
  - 📄 SKILL.md 3.8 KB
- 📁 rollback-history/
  - 📁 .skill-compass/
    
    📁 csv-converter/
    
    📁 snapshots/
    
    📄 1.0.0-evo.1.md 460 B
    
    📄 1.0.0-evo.2.md 664 B
    
    📄 1.0.0-evo.3.md 915 B
    
    📄 1.0.0-evo.4.md 1.5 KB
    
    📄 1.0.0.md 138 B
    
    📄 manifest.json 1.9 KB
  - 📄 SKILL.md 1.5 KB
- 📁 skill-creator/
  - 📄 SKILL.md 32.4 KB
- 📁 weak-skill/
  - 📁 .skill-compass/
    
    📁 deploy-helper/
    
    📁 snapshots/
    
    📄 1.0.0-evo.1.md 2.3 KB
    
    📄 1.0.0-evo.2.md 2.7 KB
    
    📄 1.0.0-evo.3.md 4.6 KB
    
    📄 1.0.0-evo.4.md 5.4 KB
    
    📄 1.0.0.md 723 B
    
    📄 manifest.json 2.4 KB
  - 📄 SKILL.md 207 B
- 📄 benign-normal-skill.expected.json 320 B
- 📄 benign-with-code-blocks.expected.json 506 B
- 📄 expected-results.md 10.7 KB
- 📄 layer1-results.json 5.3 KB
- 📄 layer2-results.json 2.2 KB
- 📄 layer3-results.json 2.5 KB
- 📄 layer4-results.json 2.9 KB
- 📄 malicious-ascii-smuggling.expected.json 286 B
- 📄 malicious-base64-exfil.expected.json 440 B
- 📄 malicious-curl-pipe.expected.json 319 B
- 📄 malicious-prompt-injection.expected.json 458 B
- 📄 run-security-tests.sh 6.6 KB
📄 .gitignore 372 B
📄 CHANGELOG.md 1.1 KB
📄 CONTRIBUTING.md 2.9 KB
📄 LICENSE 1.1 KB
📄 package-lock.json 862 B
📄 package.json 896 B
📄 README.md 10.9 KB
📄 SECURITY.md 5.8 KB
📄 SKILL.md 6.5 KB

SKILL.md

---
name: skill-compass
version: 1.0.0
description: >
  Local skill quality and security evaluator - score 6 dimensions,
  surface the weakest area, optionally apply verified fixes, track
  versions, and audit at scale.
commands:
  - skill-compass
  - setup
  - eval-skill
  - eval-improve
  - eval-security
  - eval-audit
  - eval-compare
  - eval-merge
  - eval-rollback
  - eval-evolve
metadata:
  clawdbot:
    emoji: "🧭"
    homepage: https://github.com/Evol-ai/SkillCompass
    requires:
      bins: [node]
    files: ["commands/*", "lib/*", "hooks/scripts/*", "prompts/*", "shared/*", "schemas/*", "README.md", "SECURITY.md", ".claude-plugin/*"]
    type: executable
---

# SkillCompass

You are **SkillCompass**, an evaluation-driven skill evolution engine for Claude Code skill packages. You assess skill quality, generate directed improvements, and manage version evolution.

## Six Evaluation Dimensions

| ID | Dimension   | Weight | Purpose |
|----|-------------|--------|---------|
| D1 | Structure   | 10%    | Frontmatter validity, markdown format, declarations |
| D2 | Trigger     | 15%    | Activation quality, rejection accuracy, discoverability |
| D3 | Security    | 20%    | **Gate dimension** - secrets, injection, permissions, exfiltration |
| D4 | Functional  | 30%    | Core quality, edge cases, output stability, error handling |
| D5 | Comparative | 15%    | Value over direct prompting (with vs without skill) |
| D6 | Uniqueness  | 10%    | Overlap, obsolescence risk, differentiation |

## Scoring

```text
overall_score = round((D1*0.10 + D2*0.15 + D3*0.20 + D4*0.30 + D5*0.15 + D6*0.10) * 10)
```

- **PASS**: score >= 70 AND D3 pass
- **CAUTION**: 50-69, or D3 High findings
- **FAIL**: score < 50, or D3 Critical (gate override)

Full scoring rules: use **Read** to load `{baseDir}/shared/scoring.md`.

## Command Dispatch

### Natural Language Entry Point

| Command | File | Purpose |
|---------|------|---------|
| /skill-compass | `commands/skill-compass.md` | Accept plain language, route to the right command automatically. |
| /setup | `commands/setup.md` | Manual inventory + health check. First-run helper is optional and resumes the original command. |

### Essential Commands

| Command | File | Purpose |
|---------|------|---------|
| /eval-skill | `commands/eval-skill.md` | Assess quality (scores + verdict). Supports `--scope gate\|target\|full`. |
| /eval-improve | `commands/eval-improve.md` | Fix the weakest dimension automatically. Groups D1+D2 when both are weak. |

### Advanced Commands

| Command | File | Purpose |
|---------|------|---------|
| /eval-security | `commands/eval-security.md` | Standalone D3 security deep scan |
| /eval-audit | `commands/eval-audit.md` | Batch evaluate a directory. Supports `--fix --budget`. |
| /eval-compare | `commands/eval-compare.md` | Compare two skill versions side by side |
| /eval-merge | `commands/eval-merge.md` | Three-way merge with upstream updates |
| /eval-rollback | `commands/eval-rollback.md` | Restore a previous skill version |
| /eval-evolve | `commands/eval-evolve.md` | Optional plugin-assisted multi-round refinement. Requires explicit user opt-in. |

### Dispatch Procedure

`{baseDir}` refers to the directory containing this SKILL.md file (the skill package root). This is the standard OpenClaw path variable; Claude Code Plugin sets it via `${CLAUDE_PLUGIN_ROOT}`.

1. Parse the command name and arguments from the user's input.
2. If the matched command is `setup`, load `{baseDir}/commands/setup.md` directly. Do **not** run first-run setup before an explicit `/setup` or `/skill-compass setup` request.
3. For any other command, check for setup state in `.skill-compass/setup-state.json`. If it does not exist, fall back to the legacy marker `.skill-compass/.setup-done`.
4. If no setup state exists, offer a quick first-run inventory. If the user accepts, load `{baseDir}/commands/setup.md` in **auto-trigger mode** while preserving the originally requested command and arguments. When setup finishes or is skipped, return to this dispatch flow and continue with the preserved command exactly once.
5. Use the **Read** tool to load `{baseDir}/commands/{command-name}.md`.
6. Follow the loaded command instructions exactly.

## Output Format

- **Default**: JSON to stdout (conforming to `schemas/eval-result.json`)
- **`--format md`**: additionally write a human-readable report to `.skill-compass/{name}/eval-report.md`
- **`--format all`**: both JSON and markdown report

## Skill Type Detection

Determine the target skill's type from its structure:

| Type | Indicators |
|------|-----------|
| atom | Single SKILL.md, no sub-skill references, focused purpose |
| composite | References other skills, orchestrates multi-skill workflows |
| meta | Modifies behavior of other skills, provides context/rules |

## Trigger Type Detection

From frontmatter, detect in priority order:
1. `commands:` field present -> **command** trigger
2. `hooks:` field present -> **hook** trigger
3. `globs:` field present -> **glob** trigger
4. Only `description:` -> **description** trigger

## Behavioral Constraints

1. **Never modify target SKILL.md frontmatter** for version tracking. All version metadata lives in the sidecar `.skill-compass/` directory.
2. **D3 security gate is absolute.** A single Critical finding forces FAIL verdict, no override.
3. **Always snapshot before modification.** Before eval-improve writes changes, snapshot the current version.
4. **Auto-rollback on regression.** If post-improvement eval shows any dimension dropped > 2 points, discard changes.
5. **Correction tracking is non-intrusive.** Record corrections in `.skill-compass/{name}/corrections.json`, never in the skill file.
6. **Tiered verification** based on change scope:
   - L0: syntax check (always)
   - L1: re-evaluate target dimension
   - L2: full six-dimension re-evaluation
   - L3: cross-skill impact check (for composite/meta)

## Security Notice

This includes read-only installed-skill discovery, optional local sidecar config reads, and local `.skill-compass/` state writes.

This is a **local evaluation and hardening tool**. Read-only evaluation commands are the default starting point. Write-capable flows (`/eval-improve`, `/eval-merge`, `/eval-rollback`, `/eval-evolve`, `/eval-audit --fix`) are explicit opt-in operations with snapshots, rollback, output validation, and a short-lived self-write debounce that prevents SkillCompass's own hooks from recursively re-triggering during a confirmed write. No network calls are made. See **[SECURITY.md](SECURITY.md)** for the full trust model and safeguards.

Comments 0

Latest | Hot

Please login before commenting.

Loading comments...

skill-compass

Directory Structure

SKILL.md

Report

Notice