name: skill-tester description: > Validates and scores Claude Code skill packages for quality, completeness, and best practices compliance. Tests Python scripts, checks YAML frontmatter, and generates quality reports. Use when creating new skills, validating skill packages, or auditing skill quality. license: MIT + Commons Clause metadata: version: 1.0.0 author: borghei category: engineering domain: meta-skills tier: POWERFUL updated: 2026-03-31

Skill Tester

The agent validates skill packages for structure compliance, tests Python scripts for syntax and stdlib-only imports, and scores quality across four dimensions (documentation, code quality, completeness, usability) with letter grades and improvement recommendations. It supports BASIC, STANDARD, and POWERFUL tier classification.

Quick Start

# Validate skill structure and documentation
python skill_validator.py engineering/my-skill --tier POWERFUL --json

# Test all Python scripts in a skill
python script_tester.py engineering/my-skill --timeout 30

# Score quality with improvement roadmap
python quality_scorer.py engineering/my-skill --detailed --minimum-score 75

Core Workflows

Workflow 1: Validate a New Skill

1. Run skill_validator.py with target tier to check structure, frontmatter, required sections, and scripts
2. Review errors (blocking) and warnings (non-blocking) in the report
3. Fix all errors -- missing SKILL.md, invalid frontmatter, external imports
4. Validation checkpoint: Score >= 60; zero errors; all scripts pass ast.parse()

python skill_validator.py engineering/my-skill --tier STANDARD --json

Workflow 2: Test Skill Scripts

1. Run script_tester.py to execute syntax validation, import analysis, and runtime tests
2. Review per-script results: argparse detection, --help output, sample data execution
3. Fix failures: add if __name__ == "__main__" guards, replace external imports with stdlib
4. Validation checkpoint: All scripts pass syntax; zero external imports; --help exits cleanly

python script_tester.py engineering/my-skill --timeout 60 --json

Workflow 3: Score and Improve Quality

1. Run quality_scorer.py with --detailed for component-level breakdowns
2. Review the prioritized improvement roadmap (up to 5 items)
3. Address HIGH-priority items first (documentation gaps, missing error handling)
4. Re-run to verify score improvement
5. Validation checkpoint: Overall score >= 75; no dimension below 50%

python quality_scorer.py engineering/my-skill --detailed --minimum-score 75 --json

Tier Requirements

Quality Scoring Dimensions

Grades: A+ (97+) through F (<40). Exit code 0 for A+ through C-, exit code 2 for D, exit code 1 for F.

CI/CD Integration

# GitHub Actions example
- name: Validate Changed Skills
  run: |
    for skill in $(git diff --name-only | grep -E '^engineering/[^/]+/' | cut -d'/' -f1-2 | sort -u); do
      python engineering/skill-tester/scripts/skill_validator.py $skill --json
      python engineering/skill-tester/scripts/script_tester.py $skill
      python engineering/skill-tester/scripts/quality_scorer.py $skill --minimum-score 75
    done

Anti-Patterns

• Padding SKILL.md with filler -- line count thresholds measure substantive content; blank lines and boilerplate do not count
• External imports disguised as stdlib -- the import allowlist is manually maintained; if a legit stdlib module is flagged, add it to stdlib_modules
• Missing argparse help strings -- usability scoring requires help= parameters on every argument; empty help strings score zero
• No __main__ guard -- scripts without if __name__ == "__main__" fail runtime tests when imported
• Relying on SKILL.md for usability -- usability is scored from scripts and README independently; a detailed SKILL.md does not compensate for missing --help output

Troubleshooting

Success Criteria

• Structure pass rate above 95%: Validated skills pass all required-file and directory-structure checks on first run in at least 95% of cases.
• Script syntax zero-defect: Every Python script in a validated skill compiles without SyntaxError via ast.parse().
• Standard library compliance 100%: No external (non-stdlib) imports detected across all validated scripts.
• Quality score consistency within 5 points: Re-running quality_scorer.py on an unchanged skill produces scores that vary by no more than 5 points across runs.
• Execution time under 10 seconds per skill: Full validation, testing, and scoring pipeline completes in under 10 seconds for a single skill with up to 3 scripts.
• Actionable recommendation density: Every skill scoring below 75/100 receives at least 3 prioritized improvement suggestions in the roadmap.
• CI/CD gate reliability: When integrated as a GitHub Actions step, the tool exits with non-zero status for every skill that fails critical checks, blocking the merge.

Scope & Limitations

Covers:

• Structural validation of skill directories against tier-specific requirements (BASIC, STANDARD, POWERFUL)
• Static analysis of Python scripts including syntax checking, import validation, argparse detection, and main guard verification
• Multi-dimensional quality scoring across documentation, code quality, completeness, and usability
• Dual output formatting (JSON for CI/CD pipelines, human-readable for developer consumption)

Does NOT cover:

• Functional correctness of script logic or algorithm accuracy — the tester verifies structure and conventions, not business logic
• Performance benchmarking or memory profiling of scripts — see engineering/performance-profiler for runtime analysis
• Security vulnerability scanning of script code — see engineering/skill-security-auditor for dependency and code security audits
• Cross-skill dependency resolution or integration testing — skills are validated in isolation without verifying inter-skill compatibility

Integration Points

Tool Reference

skill_validator.py

Purpose: Validates a skill directory's structure, documentation, and Python scripts against the claude-skills ecosystem standards. Checks required files, YAML frontmatter, required SKILL.md sections, directory layout, script syntax, import compliance, and tier-specific requirements.

Usage:

python skill_validator.py <skill_path> [--tier TIER] [--json] [--verbose]

Parameters:

Example:

python skill_validator.py engineering/my-skill --tier POWERFUL --json

Output Formats:

• Human-readable (default): Grouped report with STRUCTURE VALIDATION, SCRIPT VALIDATION, ERRORS, WARNINGS, and SUGGESTIONS sections. Displays overall score out of 100 with compliance level (EXCELLENT, GOOD, ACCEPTABLE, NEEDS_IMPROVEMENT, POOR).
• JSON (--json): Object with keys skill_path, timestamp, overall_score, compliance_level, checks (dict of check name to pass/message/score), warnings, errors, suggestions.

Exit codes: 0 on success (score >= 60 and no errors), 1 on failure.

script_tester.py

Purpose: Tests all Python scripts within a skill's scripts/ directory. Performs syntax validation via AST parsing, import analysis for stdlib compliance, argparse implementation verification, main guard detection, runtime execution with timeout protection, --help functionality testing, sample data processing against files in assets/, and output format compliance checks.

Usage:

python script_tester.py <skill_path> [--timeout SECONDS] [--json] [--verbose]

Parameters:

Example:

python script_tester.py engineering/my-skill --timeout 60 --json

Output Formats:

• Human-readable (default): Report with SUMMARY (total/passed/partial/failed counts), GLOBAL ERRORS, and per-script sections showing status, execution time, individual test results, errors, and warnings.
• JSON (--json): Object with keys skill_path, timestamp, summary (counts and overall status), global_errors, script_results (dict per script with overall_status, execution_time, tests, errors, warnings).

Exit codes: 0 on full success, 1 on failure or global errors, 2 on partial success.

quality_scorer.py

Purpose: Provides a comprehensive multi-dimensional quality assessment for a skill. Evaluates four equally weighted dimensions — Documentation (25%), Code Quality (25%), Completeness (25%), and Usability (25%) — and produces an overall score, letter grade (A+ through F), tier recommendation, and a prioritized improvement roadmap.

Usage:

python quality_scorer.py <skill_path> [--detailed] [--minimum-score SCORE] [--json] [--verbose]

Parameters:

Example:

python quality_scorer.py engineering/my-skill --detailed --minimum-score 75 --json

Output Formats:

• Human-readable (default): Report with overall score and letter grade, per-dimension scores with weights, summary statistics (highest/lowest dimension, dimensions above 70%, dimensions below 50%), and a prioritized improvement roadmap (up to 5 items with HIGH/MEDIUM/LOW priority). When --detailed is used, component-level breakdowns appear under each dimension.
• JSON (--json): Object with keys skill_path, timestamp, overall_score, letter_grade, tier_recommendation, summary_stats, dimensions (per-dimension name/weight/score/details/suggestions), improvement_roadmap (list of priority/dimension/suggestion/current_score objects).

Exit codes: 0 for grades A+ through C-, 1 for grade F or when score is below --minimum-score, 2 for grade D.