Feature: Skill Dependency Validator & Visualization

[smgrow-lantern]

🎯 Feature Overview

Problem: Skills often reference other prompts, instructions, or agents (e.g., in prerequisites or related content), but there's no automated validation that these references exist or are correctly formatted.

Solution: Build a validator that checks skill dependencies during the build process and generates a dependency graph visualization.

📋 Development Plan

Phase 1: Analysis & Design (Day 1)

1. Audit existing skills to understand dependency patterns:

   • Search for reference patterns in skills/
   • Document dependency formats found in frontmatter and markdown content
   • Identify common reference types (prerequisites, related skills, see-also)

2. Design validator schema:

   interface SkillDependency {
     source: string;        // skill file path
     target: string;        // referenced file path
     referenceType: 'prerequisite' | 'related' | 'seeAlso';
     isValid: boolean;
     line: number;
   }

Phase 2: Core Validation (Days 2-3)

1. Create validation module at eng/validators/skill-dependencies.mjs:

   • Extract dependencies from frontmatter (prerequisites, relatedSkills)
   • Parse markdown links for internal references
   • Validate that referenced files exist
   • Report broken references with file paths and line numbers

2. Integrate into build script at eng/update-readme.mjs:

   • Run validation during npm run build
   • Fail build in CI if broken references found
   • Warn locally for developer feedback

3. Add npm script to package.json:

   "validate:deps": "node eng/validators/skill-dependencies.mjs"

Phase 3: Visualization (Day 4)

1. Generate dependency graph using Mermaid:

   • Create nodes for each skill/prompt/instruction
   • Draw edges showing dependencies with type labels
   • Highlight broken references in red
   • Generate graph output to DEPENDENCIES.md

2. Generate DEPENDENCIES.md:

   • Include Mermaid diagram of all dependencies
   • Statistics section (total deps, valid, broken)
   • List of broken references if any exist

Phase 4: Testing (Day 5)

1. Create test fixtures in eng/validators/__tests__/fixtures/:

   • Valid skill with correct references
   • Skill with broken link
   • Skill with circular dependency

2. Write unit tests:

   • Test detection of valid references
   • Test catching broken references
   • Test handling of circular dependencies
   • Test frontmatter parsing
   • Test markdown link extraction

3. Local testing:

   npm run validate:deps
   npm run build
   git diff DEPENDENCIES.md

Phase 5: Documentation & PR (Day 6)

1. Update .github/copilot-instructions.md:

   • Document skill dependency validation
   • Explain valid reference types
   • Show example frontmatter with references
   • Link to DEPENDENCIES.md visualization

2. Create PR with:

   • Clear description of feature and benefits
   • Example validation output
   • Screenshot of dependency graph
   • Updated documentation

✅ Acceptance Criteria

• Validator detects broken references in skills
• Build fails in CI when broken references exist
• DEPENDENCIES.md generated with Mermaid graph
• Tests cover validation logic
• Documentation updated
• No breaking changes to existing workflows
• npm run validate:deps command available

📊 Success Metrics

1. Zero broken references in main branch
2. Automated validation prevents future breaks
3. Visual dependency graph helps contributors understand relationships
4. No impact on existing contributor workflows

🔄 Future Enhancements

• Circular dependency detection with warnings
• Suggest similar files when reference is broken
• Dependency impact analysis (what breaks if I delete this skill?)
• Interactive graph with clickable nodes
• Auto-fix broken references based on fuzzy matching

🧪 Testing Strategy

Local Testing:

# Run validation on current repo
npm run validate:deps

# Test with intentionally broken reference
echo '[test](nonexistent.md)' >> skills/test-skill/SKILL.md
npm run validate:deps  # Should fail

# Verify build integration
npm run build  # Should include validation

CI Testing:

• Add GitHub Actions workflow to fail PRs with broken dependencies
• Generate dependency graph as PR comment

📝 Implementation Notes

• Use existing vfile-matter for frontmatter parsing (already in dependencies)
• Leverage glob for file discovery patterns
• Follow existing validator patterns from validate-skills.mjs and validate-collections.mjs
• Maintain consistent error reporting format across validators
• Support both absolute and relative path references

🏷️ Labels

• enhancement
• build-system
• validation
• documentation