Making a Claude Code Skill Trigger Reliably — and How to Measure It

Date:

A perfectly written skill that never fires is worthless. Whether it fires is decided by one thing — its description — and you can measure and tune that instead of guessing.

The short answer: A skill only helps if the agent chooses to consult it, and that choice is made from the skill’s description alone — the one line always in context. The common failure is silent under-triggering: the skill exists, matches the task, and still never fires because the description didn’t describe the situation clearly enough. Fix it by writing the description to name concrete trigger contexts (not just what the skill does), then measure triggering with a small eval set of should-fire and should-not-fire queries, run each a few times because the decision is stochastic, and tune the description against a held-out split so you don’t overfit it to your own test cases.

Triggering is the moment the agent, mid-task, decides a skill is relevant and loads it. It happens before any of the skill’s real content is in context — so the decision rests entirely on the name and description (the always-loaded metadata; see Anatomy of a Claude Code Skill). Everything about reliable triggering is really about that one field.

The failure is almost always silent under-triggering

When a skill misbehaves, it rarely fires wrongly. It usually just doesn’t fire at all — and nothing announces that. The agent handles the task without the skill, the result is a bit worse or a lot worse, and you never learn the skill was skipped. Over-triggering (firing when it shouldn’t) exists but is rarer and louder. So the target to optimize is coverage: does the skill fire on the tasks it should?

A skill that never triggers is indistinguishable from a skill you never wrote.

How the agent actually decides

Two things govern whether a skill fires, and understanding both stops you chasing the wrong fix.

First, the description is the whole signal. The agent reads name + description and judges relevance from that. If the description says only what the skill does (“vaults images”) but not when to use it (the phrasings and situations that should invoke it, even when the user doesn’t name the skill), a matching task can sail past unrecognized.

Second — and this surprises people — the agent won’t consult a skill for a task it can already handle trivially. Simple one-step requests (“read this file”) don’t trigger skills even with a perfect description, because the agent just does them. Skills fire for substantive work: multi-step, specialized, or error-prone tasks where consulting a procedure pays off. The practical consequence is twofold: write the description to match the substantive version of the task, and test with substantive queries — a trivial test query won’t trigger any skill and tells you nothing about your description.

Writing a description that triggers

The description should carry two things: what the skill does and the concrete contexts that should invoke it — the plain-language ways a user might ask without ever naming the skill, the near-synonyms, and the “even when they don’t say X” cases. This is the one place to be deliberately a little pushy: under-triggering is the default failure, so err toward naming more trigger situations rather than fewer. Keep it inside the description’s hard size cap, and push everything else into the body and references — the description is for recognition, not instruction.

Measuring it: a trigger-eval loop

You don’t have to guess whether the description works. Measure it.

1. Build a small eval set — should-fire and should-not-fire. Roughly eight to ten of each. The should-fire queries cover varied phrasings of the real intent: formal and casual, some naming the skill and some not, plus a couple of uncommon-but-valid uses. The should-not-fire queries are the ones that matter most, and they must be near-misses — queries that share keywords or domain with the skill but actually need something else. An obviously irrelevant negative (“write a poem” for an image-vaulting skill) tests nothing; a near-miss (“find me a stock photo to use” when the skill only vaults images you already have) tests the boundary.

2. Run each query several times. The trigger decision is stochastic — the same query won’t always resolve the same way — so a single run is noise. Run each query a few times (three is a reasonable floor) and treat the trigger rate as the signal, not a single yes/no.

3. Split train and held-out. Tune the description against one portion of the eval set and validate on a portion you didn’t tune against. This is the guard against the subtle trap below: it’s easy to sculpt a description that aces your exact test queries and still generalizes badly. The score that matters is the held-out one.

4. Iterate, and select by the held-out score. Read which queries failed, revise the description to cover the missed situations (or to stop firing on a near-miss), re-run, and keep the version that scores best on the held-out set — not the one that scores best on the queries you tuned against. A few rounds of this converges on a description that triggers where it should and stays quiet where it shouldn’t.

A trigger-eval loopMeasure whether the description fires – don't guess. Select by the held-out score.
1. Build the eval set

~8-10 should-fire (varied phrasings) + ~8-10 should-NOT-fire NEAR-MISSES. Use substantive queries – trivial ones fire nothing.

2. Run each several times

The trigger decision is stochastic, so one run is noise. Run each ~3x; the TRIGGER RATE is the signal, not a single y/n.

3. Split tune / held-out

Tune the description on one portion; validate on a portion you did NOT tune against. The guard against overfitting.

4. Revise the description

Name the missed when-to-use situations; stop firing on a near-miss. Cover real situations, not a bag of keywords.

5. Select by the held-out score

Keep the version best on the queries you did NOT tune against. A big train/held-out gap = overfit. Loop 4-5 until it converges.

A skill that never triggers is indistinguishable from a skill you never wrote.

The two traps

  • Overfitting the description to your eval queries. If you tune until the training queries all pass, you may have described those sentences rather than the task. The held-out split is precisely what catches this — a big gap between train and held-out scores means you’ve overfit, and the description needs to describe the intent more generally.
  • Keyword-stuffing into over-triggering. Cramming every vaguely related term into the description is how a skill starts firing on adjacent tasks it has no business handling. Coverage comes from naming real situations, not from a bag of keywords. If the near-miss negatives start triggering, you’ve gone too far.

The short version

A skill fires only if the agent reaches for it, and that decision is made from the description alone — so under-triggering (the skill silently never fires) is the failure to hunt. Write the description to name concrete when-to-use situations, not just what the skill does, and remember the agent won’t consult a skill for trivial tasks, so test with substantive queries. Measure triggering with ~8–10 should-fire and ~8–10 near-miss should-not-fire queries, run each a few times because the decision is stochastic, and tune against a held-out split — selecting the description that scores best on the queries you didn’t tune against. Avoid overfitting to your test sentences and keyword-stuffing into over-triggering.

Frequently Asked Questions

My skill never runs even though it clearly applies. Why?

Almost certainly the description under-triggers: it says what the skill does but not the situations that should invoke it. Rewrite it to name the concrete when-to-use cases and the plain-language ways a user would ask without naming the skill.

Why doesn’t my skill fire on a simple request?

Because the agent handles trivial, one-step tasks directly and only consults skills for substantive work. That’s expected — and it means a trivial query is a poor test of your description. Test with the multi-step version of the task.

Why run each eval query more than once?

The trigger decision is stochastic, so a single run is noise. Running each query several times gives you a trigger rate, which is the reliable signal.

Why split the eval set into train and held-out?

To avoid overfitting the description to your exact test sentences. Tune on one portion, score on the other; the held-out score is the one that tells you whether the description generalizes.

Can a description trigger too much?

Yes — keyword-stuffing makes a skill fire on adjacent tasks it shouldn’t handle. That’s what the near-miss negative queries catch: if they start triggering, the description is too broad.

Share post:

Subscribe

Popular

More like this
Related

How to Merge Spreadsheets by URL (and Why Our Tool Reported 5 Misses That Weren’t)

Matching two spreadsheets on a URL column is the same job as matching a stock list to a price list by product code. It goes wrong in the same two places: how you clean the code, and how you count what didn't match.

Many-to-One: Why a Mapping Backlog Outgrows Its Target Taxonomy

You're mapping a messy source vocabulary onto a clean target taxonomy. The target has about 5,600 slots, so the job must be roughly 5,600 decisions — right? The backlog is already past 8,900, and still climbing.

The Fix That Didn’t Take: When a Content-Hash Gate Skips Your Code Change

You shipped a fix to how one field is computed, re-ran the pipeline across everything, and it reported a clean run. Then the consistency check still flags those records as wrong. The fix is in the code, the rebuild "ran" — and the records never changed. Your own optimisation skipped them.

Your Dedup Scan Is Over-Counting — Because It Forgot What You Already Fixed

The report said 80 duplicate groups still to resolve. You'd been resolving them for days. You run it again after a big batch and it *still* says 80-ish. The backlog isn't stubborn — the scan is counting work you already finished.