Most of the Skill Isn't in SKILL.md

On a normal API call, the model only sees what you put in the prompt for that turn. A SKILL.md file is more text for that turn. It can change the output. It does not change the weights. If you are not fine-tuning, the model does not keep the skill.

That is the whole case for "skills are just prompts," and it is correct as far as a single call goes. The other version — that a skill file turns a base model into a domain expert — is asking a folder to do something it cannot do.

An agent is a loop, not a call. It runs against files, tools, permissions, and state that outlive the turn. The skill file is what the model reads. The skill folder is what the runtime discovers, loads, and maybe executes. Those are different jobs.

Two Ways to Look at the Same File

To the model, a skill is context: instructions, few-shot examples, and schemas for the current task. To the host, it is a directory — a name, a description, Markdown, and maybe some scripts or reference docs.

The open Agent Skills spec is built around that directory. You get a SKILL.md with YAML frontmatter, plus optional scripts/, references/, and assets/ folders.¹ The runtime loads it in stages: name and description first, the full SKILL.md once the skill is selected, then extra files or scripts only if the task needs them.

That keeps the prompt short, and it gives you something you can version and share. "Just a prompt" describes what the model sees. "A system object" describes how the host finds and runs it. Same files, two jobs.

The Security Model Depends on the Platform

The spec does not define a message role or a permission model. Each platform fills that in.

Anthropic puts the skill name and description in the system prompt. When Claude picks a skill, it reads SKILL.md from disk. If there is a script, Claude runs it and puts the output into the prompt, not the source.²

OpenAI's Responses API puts skill metadata in the user prompt. When the model invokes a skill and reads SKILL.md, those instructions sit at user level, same priority as whatever the user just typed.³

OpenAI also lets you upload ZIPs, pin versions, and run skills in a hosted container.³ None of that is in the spec. Their safety docs treat skills like arbitrary code, because a skill can fire tool calls and shell commands.³ "Privileged" means what the skill can touch. It does not mean the model trusts it more.

Without naming the runtime, there is not much you can say about skill permissions or how reliably a skill fires.

What a Skill Actually Changes

A skill is a way to split work. Keep the core procedure in the main Markdown file. Put edge cases in reference files. Move arithmetic and validation into scripts. The prompt stays smaller, and you stop asking the model to do jobs a function already does reliably.

Dropping a script into the folder does not grant it permission to run. The host still decides which tools exist, whether the network is open, and what needs a person to confirm. allowed-tools in the spec is still experimental.¹

Here is how responsibilities usually split:

Component	Role	What it doesn't do
Prompt	Passes text input and instructions to the model	Plain text cannot manage permissions or enforce runtime limits ⁴
Skill	Packages instructions, reference docs, and scripts	Cannot enforce execution order, approvals, or rollbacks on its own
Tool	Provides an API or external action	Having a tool available doesn't mean the model will use it correctly
MCP	Protocol connecting models to tools and resources	It is a transport protocol, not an auth policy or business rule engine ⁵
Runtime / Harness	Runs the agent loop, context, state, and permissions	Machinery alone doesn't define the business logic ⁶
Workflow	Defines multi-step sequences and human review gates	Retries, rollbacks, and compensation logic still have to be written ⁷

Expense reports make this concrete. The skill states the policy limits and ships a script that checks receipts. A tool fetches the receipts. The runtime blocks writes. A workflow refuses to approve a report with no receipt attached. Take any one of those away and the others do not cover for it.

What Existing Research Tells Us

Progressive loading exists because long context is uneven. Liu et al. (Lost in the Middle) showed retrieval accuracy dropping when the relevant span sat in the middle of a long prompt.⁸ How bad it gets depends on the model. Loading everything up front is still a poor default.

The tool-use papers sit next to this, not on top of it. ReAct alternates reasoning with tool calls so the model can react to what the environment sent back.⁹ Toolformer trains a model to decide when to hit an API.¹⁰ Both mix generation with execution. Neither one compares a skill folder to pasting the same text into the prompt.

IHEval is closer to the mess you actually hit in production. Models mishandle conflicting instructions when the sources have different priority — system prompt versus a retrieved document, for example.¹¹ Leave provenance and priority to the model and it will pick wrong. Scoped permissions and approval gates are the fix. Benchmarks have mostly not measured that.

AgentBench runs models through eight interactive environments.¹² The result worth taking is that an agent eval has to look like the environment the agent runs in. It is not a skills benchmark.

You can take context placement, tool feedback, and environment-matched evals from this literature. You cannot take "put it in a SKILL.md and the answers get better."

Test the Mechanism, Not the Format

"Do skills improve performance?" hides the part you need. Which piece of the skill changed the outcome?

Four setups isolate that:

A standard prompt.
The same SKILL.md content pasted directly into the prompt.
The same content loaded on demand via skill discovery.
Setup 3, with supporting docs, validation scripts, and a workflow added one at a time.

Keep the model, task suite, test data, and token budget fixed. Pass rate is not enough. Count stray tool calls, how often a human had to step in, latency, tokens, and how it failed.

If 2 and 3 look the same, the skill helped you organize files. It did not give the model a new ability. If errors only drop after the validation script lands, credit the script. If an approval gate catches a bad tool call, that was the workflow, not the Markdown.

Where the Value Actually Lies

A skill that is a persona and a page of instructions is easy to copy. It might save you a paste. Anyone else can write the same file.

The parts that are hard to copy are operational:

Does the skill perform better than putting the same instructions in a prompt?
When should the agent stop and ask for human input?
Where are high-risk actions logged, bounded, and approved?
Which logic is handled by unit-tested code rather than model generation?
How are changes regression-tested against edge cases?

The hard parts rarely live in the Markdown. They live in domain logic, private data, integrations, permission models, and eval sets. A skill can point at those systems. It does not create them.

If a skill is doing real work, it is because you moved something out of generation and into a file, a script, or a gate you can test. The Markdown is how the agent finds those pieces. The pieces are the product.