Skills Are Not Magic—and Not Just Prompts

Arguments about agent skills tend to collapse into two slogans. One says a skill is merely a Markdown file, so loading it cannot give a model a new capability. The other treats skills as the asset that turns a general model into a dependable domain expert.

Neither claim is entirely wrong. Both let us stop thinking too early.

During an ordinary inference call, the model receives the instructions, examples, and constraints that the runtime makes available. Reading SKILL.md can shape the current output; absent online training or fine-tuning, it does not write those instructions into the model's parameters.

An agent, however, is more than an inference call. It also has files, tools, state, permissions, and an execution environment. Skills matter at the seam between the model and that surrounding system. Saying that the model-facing part is text does not mean that the whole system is text.

One artifact, two useful views

From the model's point of view, a skill is context: tokens, tool definitions, and observations selected for the current task. From the host's point of view, it is a discoverable package with a name, a description, instructions, and sometimes reference files, templates, or executable scripts.

The open Agent Skills specification uses a directory as the basic unit. It requires a SKILL.md file with YAML frontmatter and a Markdown body, while allowing optional scripts/, references/, and assets/ directories.¹ It also describes progressive disclosure: expose the name and description first, load the body after activation, and retrieve supporting files only when needed.

That design is useful without being mysterious. It reduces irrelevant context and gives procedural knowledge a reviewable, reusable home. Calling a skill “a prompt” describes what reaches the model. Calling it a system object describes the software that discovers, loads, and executes the package. Those are different views of the same artifact.

Implementation details change the security model

The Agent Skills specification does not assign a universal chat-message role, nor does it define one versioning or permission model for every platform.

Anthropic places a skill's name and description in the system prompt at startup. When a request matches, Claude reads SKILL.md from the filesystem. It can fetch additional references later or run bundled scripts; in the normal execution path, the script output enters context rather than the script source itself.²

OpenAI's Responses API Skills implementation works differently. The platform adds each skill's name, description, and path to user-prompt context. The model uses that discovery information to decide whether to invoke the skill, then reads the full SKILL.md. OpenAI explicitly classifies those instructions as user-prompt input, not system-prompt input, with the same priority as other user-provided instructions.³

OpenAI also supports uploaded directories or ZIP archives, version pointers, and attaching skills to a hosted shell environment.³ Those are platform features, not guarantees made by the open specification. Its safety guidance says skills can influence planning, tool use, and command execution, so developers should review them as high-impact code and instructions and treat them as potentially untrusted until validated.³ “Privileged” in this context describes potential impact; it does not grant a higher instruction priority or imply trust.

There is therefore no universal “skill message level.” Any claim about triggering, precedence, or security needs to name the product and execution mode it refers to.

What a skill can actually change

The most immediate change is the working material available to the agent. Common steps can stay in the main file, rare cases can move to references, and calculations or format checks can be delegated to scripts. This may reduce repeated context and move brittle operations out of free-form model judgment.

Packaging a script does not grant permission to run it. A skill may suggest a tool call or include executable code, but the host still decides which tools exist, whether network and file access are available, and which actions require approval. The specification's allowed-tools field remains experimental and implementation-dependent.¹

The boundaries below are typical, not universal:

Component	Typical role	What does not follow from it
Prompt	Supplies instructions and input for one or more model calls	Prompts can be reused and versioned; text alone does not provide tool permissions or runtime governance ⁴
Skill	Packages procedural guidance, references, templates, and scripts	It can describe a process but cannot by itself enforce order, approval, or rollback
Tool	Exposes a callable data source or external action	Availability does not establish that a call is appropriate
MCP	Standardizes connections to external prompts, resources, and tools	MCP is a protocol, not a tool or a business authorization policy ⁵
Harness / runtime	Drives model and tool calls and manages context, state, permissions, approvals, and observations	Runtime machinery does not define the business process on its own ⁶
Workflow	Makes steps, branches, and human checkpoints explicit	Retry, compensation, and rollback still have to be designed and implemented ⁷

Consider expense review. A skill might explain the review policy, link to the relevant rules, and include a tax-calculation script. A tool fetches the claim and its receipts. The runtime limits access and gates write operations. A workflow can enforce a hard rule that no claim is submitted without a receipt. These pieces cooperate; they are not substitutes for one another.

What the research establishes—and what it does not

Progressive loading has a practical motivation. A larger context window does not guarantee reliable use of every item inside it. In controlled multi-document QA and key-value retrieval experiments, Liu and colleagues found that performance changed substantially with the position of relevant information and was often worse when that information appeared in the middle of a long context.⁸ The result is task- and model-dependent, not a universal placement rule.

ReAct interleaves reasoning traces with task actions so that observations from an external source can inform later decisions.⁹ Toolformer trains a model to decide which API to call, when to call it, what arguments to pass, and how to continue prediction after the result is inserted.¹⁰ Both connect language generation with external actions and results, but they use different interaction structures. Neither is a direct test of skills versus plain prompts.

IHEval adds another relevant warning. Every evaluated model suffered a sharp drop when it had to resolve conflicting instructions from sources with different priorities.¹¹ That finding supports caution about leaving provenance conflicts entirely to the model. Source tracking, bounded permissions, and approval gates are sensible engineering responses, but IHEval did not test those interventions.

AgentBench evaluates reasoning and decision-making across eight interactive environments.¹² Its useful lesson here is methodological: evaluations should match the environment in which an agent operates. It does not provide a general score for skills.

Taken together, this work supports attention to context placement, action–observation loops, and task-specific evaluation. It does not prove that packaging instructions in a skill will improve performance.

Compare mechanisms, not labels

The question “Why does adding a skill improve performance?” assumes the improvement before measuring it. A better question is: under which tasks and constraints does a skill change the outcome, and through which mechanism?

A minimal comparison could use four conditions:

a plain task prompt;
the exact same SKILL.md body pasted directly into the prompt;
the same body made available through on-demand skill loading;
the third condition with references, scripts, and an explicit workflow added one at a time.

Keep the model, task set, business data, tool permissions, and call budget as stable as possible. Measure more than success rate: unsafe actions, human intervention, latency, token cost, and failure type all matter.

If conditions two and three perform similarly, the main benefit may be packaging and maintenance rather than model capability. If errors fall only after a validator script is added, the value comes from deterministic code. If only the workflow prevents a high-impact submission, credit the workflow rather than SKILL.md.

Where the durable value sits

A skill made of generic advice and a polished persona is easy to copy. It may still be convenient, but it is not much of a moat. The harder questions are operational:

What measurable capability does it add over the same content in a prompt?
When should the agent decline, stop, or hand the task to a person?
Where are consequential actions constrained, approved, and logged?
Which calculations and validations are covered by testable code?
After a change, is there a version record, a regression set, and a collection of failure cases?

The difficult-to-copy parts usually sit outside the Markdown: business exceptions, trusted data, system integrations, permission design, evaluation sets, and operating experience. A skill can give those things a usable interface. It cannot manufacture them.

A skill is therefore neither a new ability hidden in model weights nor merely a prompt with a fashionable directory name. It is a way to package procedural knowledge, supporting resources, and execution conventions for an agent. Its value should be judged by ordinary outcomes: more reliable work, better-controlled risk, and lower maintenance cost.

产品文档核验于 2026-08-27；具体平台实现可能继续变化。 Product documentation was verified on August 27, 2026; implementation details may change.