Skip to content

README Research Background: 26.1% and 5.2% are of 31,132 skills analysed, not the 42,447 dataset #540

Description

@imran-siddique

The Research Background section of README.md lists the dataset as 42,447 skills, directly above the 26.1% and 5.2% figures. In the paper those percentages sit on a different denominator.

Liu et al., arXiv:2601.10338 (submitted 15 January 2026), collected 42,447 skills from skills.rest and skillsmp.com, then analysed 31,132 after filtering and deduplication. Section 4.3.1: "Of the 31,132 skills analyzed using SkillScan automated detection, 26.1% contain at least one vulnerability." The detector was the paper's own SkillScan framework.

This travels. AST08 in the OWASP Agentic Skills Top 10 cites the pair as "Per the SkillSpector project, roughly 26.1% of scanned skills contained vulnerabilities", which reads as SkillSpector's own scan results. The section is unchanged since the initial release (7ced4fb, 2026-05-11) and does not link the paper, so nobody downstream can trace it.

Proposed text:

Research Background

Based on "Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale" (Liu et al., 2026, arXiv:2601.10338):

  • Dataset: 42,447 skills collected from two marketplaces, 31,132 analysed after filtering and deduplication
  • Vulnerable: 26.1% of the 31,132 contain at least one vulnerability, detected with the paper's SkillScan framework
  • High-severity: 5.2% show patterns strongly suggesting malicious intent
  • Key finding: Skills with executable scripts are 2.12x more likely to be vulnerable

Line 12 ("Research shows that 26.1% of skills contain vulnerabilities") reads correctly once it links the same paper.

I can send this as a PR with DCO sign-off if that is easier to review.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions