Your Agent Invented a Package. It Will Invent the Same One Tomorrow.

At 1:16 in the morning, a small model running on a laptop was asked for a Python script that lists Germany's public holidays. The answer opened the way every tutorial answer opens: with a pip install line. Except the package in that line does not exist. Nobody has ever published it. The name was assembled out of the words in the prompt, delivered with total confidence, in the exact slot where a coding agent runs a command without asking anyone.

That is not an anecdote about one bad model. It is the most reproducible thing small local models do. The Phantom Packages study ran three open-weight models through 200 coding tasks and checked every install suggestion against the real registry. Then it wiped everything and ran the whole study again from scratch. The ghosts came back.

The ghost is not random, and that changes the math

Roughly 1,200 generations produced 1,410 install suggestions. Two hundred and twelve of them — 15.0% — named a package that has never existed on PyPI, with a 95% confidence interval of 11.9% to 18.4%. Split by model, the smallest was least prone: gemma4 at 8.1%, phi4-mini at 16.7%, qwen3 at 20.7%. Split by difficulty, the pattern is brutal: on everyday tasks the ghost rate was 1.7%, on obscure ones it was 30.0%.

Now the part that matters. On the rerun, 1,183 of 1,199 outputs were identical word for word — 98.7% — and 161 of the 162 ghost names came back. Not one new ghost appeared.

Say that plainly: the model does not "occasionally hallucinate." It deterministically maps a prompt to a name that sounds right. A random hallucination is noise; you average it out, retry, shrug. A reproducible hallucination is a target. It has an address, it has a schedule, and it will be back tomorrow at the same time.

So the attacker only has to register it

The name for this is slopsquatting — squatting on a package name an agent invented rather than on a human typo. Register the name on PyPI or npm, publish an empty package with whatever you like in its setup file, and wait for the next agent to dream it up.

This is not hypothetical. A researcher registered one recurring ghost as an empty package and watched it collect more than 30,000 authentic downloads in three months. No exploit, no social engineering. The model did the market research, the developer did the installation, and the attacker did nothing but file paperwork.

And the names are guessable in advance, which is the detail that should make you uncomfortable. In the study, 56.5% of ghost names were simply the task's own words glued together, and so were 27 of the 40 ghosts that appeared more than once. If you can read the prompt, you can guess the ghost. Your prompts live in your repo, your issues, your public agent transcripts, your blog posts about how you use agents.

The obvious fix is the wrong one

The instinct is to have the model check itself: before you install anything, verify the package exists. It does not work, and the study measured exactly how badly.

Asked to confirm, gemma4 reported that 143 of 271 real packages did not exist. A hosted model, given the same job, said that 60 of 161 ghosts did exist. Both were answering a language question — does this string look like a package name — and calling it a fact question. That is not a smaller version of the original bug; it is the same bug with more confidence.

Two rules fall out of that, and they generalize well past Python:

• Verification has to hit an outside authority. The registry — PyPI, npm, crates.io — is the source of truth. A model's opinion is not a check, however it is phrased.
• Never ask a model a question whose answer is a lookup. The same rule keeps model-generated version numbers, API endpoints and config keys out of production. Models produce plausible shapes; they are not a database.

Put the gate where the action happens

Registry lookups alone are not enough, because the moment that matters is the second before the command runs — often at 1:16 in the morning, often in a sandbox where the network is deliberately restricted. The most interesting part of the study is a guard that sits between the agent and the install command as a hook: a small program the agent runtime calls before a tool use, with the power to veto it.

The guard decides from the name itself, plus how the name was generated, without asking the registry. It scored an AUROC of 0.975 on prompts it had not seen and 0.948 on package names it had never seen, and it was the best-calibrated method tested. Calibration matters here because you have to pick a threshold, and calibration tells you what that threshold actually means.

Replayed over the whole study, it blocked 212 of 212 ghost installs and wrongly stopped 13 of 1,165 real ones. A ghost was blocked in about 2.2 seconds; a real install passed in about 1.7. With the Wi-Fi off, it still caught 90.1% of the ghosts.

Do the arithmetic on the false positives, because that is the number people argue about. Thirteen wrong stops out of 1,165 legitimate installs is roughly one extra lookup per ninety installs. The cost of a false negative is an unknown author's code running with your credentials. Those are not the same order of magnitude, and a review debate should not pretend they are.

Where the guard breaks

Three honest limits, because a guard you over-trust is worse than no guard.

A name-shaped heuristic is not proof. A ghost can be registered after your threshold was set. A real name can change hands. The gate catches the reproducible hallucination — the common case — not every supply-chain attack, and it will never tell you that an existing package is safe.

It is not the best model for every reading of the problem. Gradient boosting beat it on precision at 90% recall on every split, and a paid hosted model's self-check tied it on unseen names. The case for this one is calibration, speed and offline operation — not dominance.

A perfect block rate is an artifact of the sample. 212 of 212 sounds absolute. It is 212 names from one study, against one registry, on one day.

The policy, not the hook

A gate on the install command is the last line. Three layers, cheapest first:

1. Propose, don't acquire. Treat "new dependency" as a change class with its own review, the way you treat a schema migration. The agent may write the name into the pull request; installing it is a separate, human-owned step. Change classes are how you stop arguing about every case individually.
2. Verify against the registry, and record it. A lookup with a timestamp in the PR: does it exist, who publishes it, when was it first published, how many releases, what does the source look like. A dependency added last Tuesday by an author with one package is a different decision from one added in 2014.
3. Intercept at the tool call, and keep the install network closed. The hook catches the name in the second before execution; the lack of egress means a lookup cannot be your only defence. Pin versions and let CI install from the lockfile, not from the agent's sentence.

None of this needs a new platform. It needs one decision — an agent can recommend a dependency but cannot acquire one — written down somewhere a hook can enforce it.

The general lesson

Swap packages for anything an agent produces: an endpoint, a CLI flag, a config key, a citation, a function in a library that may or may not exist. The test is the same and takes a minute: run the same task a hundred times. Does the same wrong answer come back? If it does, someone will eventually build against it, register it, or depend on it — and your occasional hallucination has become infrastructure.

Small local models bite hardest here, because that is what people run on a plane, behind a firewall, against private code, with the network off — the exact conditions where your registry check cannot save you. Design for that room, not for the demo.

Key Takeaways

• 15.0% of install suggestions from three small local models named packages that have never existed; on obscure tasks it was 30.0%.
• Hallucinated names are deterministic, not random: 98.7% of rerun outputs were byte-identical and 161 of 162 ghosts returned. Reproducible errors can be attacked — that is slopsquatting.
• Never ask a model whether a package exists. It answered "no" for 143 of 271 real packages and "yes" for 60 of 161 ghosts.
• Gate the tool call, not the prose: a pre-execution hook blocked 212 of 212 ghosts, stopped 13 of 1,165 real installs, and caught 90.1% with no network.
• Make "add a dependency" a reviewed change class. Agents propose; humans acquire; CI installs from the lockfile.