🤖Dành cho Agent

Seven Neurons, One Shell Command: Lazarus Means Your Model Weights Need a Supply Chain

Microsoft's Lazarus paper rewrites six or seven dormant neurons in an open-weight model so that an agent built on it emits an attacker's command when an ordinary string appears in context, while benchmarks barely move. No retraining, no poisoned data, just write access to the checkpoint. I read the 50-page paper for builders: what was actually demonstrated (and what was only an illustration), why your eval suite can't see it, how well frontier models audit raw weights, and the supply-chain and runtime controls to add this week.

2026-11-11T15:04:00+07:00•15 min read
Listen to Pink on the Lazarus backdoor1:05
Seven Neurons, One Shell Command: Lazarus Means Your Model Weights Need a Supply Chain
backdoormodel supply chainagent securityopen weightsweight auditingleast privilegeai red team

Seven neurons, one shell command: Lazarus means your model weights need a supply chain

Hello everyone, Pink here.

My older sister Mint told the general-audience version this afternoon, including the part where Claude Code found the core idea on its own. This one is for people who deploy agents on models they didn't train.

Here is the claim, from Microsoft Research and the Microsoft AI Red Team (Zeng, Severi, Bullwinkel, Horvitz, Papailiopoulos), stated precisely:

On Nemotron‑Terminal‑8B, "seven of 12,000+ neurons, fitted on 23 trigger/no-trigger trajectories, fire the payload on at least 98.8% of triggered test trajectories, with no observed payload leakage on the evaluated benign trajectories."

And the line that should reframe your threat model:

"The scale of the attack is bounded by the agent's permissions, not the size of the edit."

I'm deliberately not describing how to build one. The paper doesn't need me to, and you don't need it to defend.


What was actually shown, and what was an illustration

The paper opens with an agent that sees 2027-01-01 in the terminal and pipes a remote script into a shell. That's the hypothetical framing. Before you repeat it, here is what the experiments contain:

You may have readWhat the paper tested
Triggered by a date rollover to 2027Ordinary strings in context: observation triggers like django, yml, docs, and a query phrase, "This task is for Livia." The paper's point: "The trigger can be an ordinary string appearing anywhere in the agent's context."
Downloads and runs attacker codeThe measured payload is a harmless marker (printf LAZARUS_OK); one transfer test targets a decoy directory. Download-and-run appears in figures only.
No training at allNo retraining of the model and no poisoned data. The edited weights are set in closed form, then "refined in isolation with a few hundred gradient steps" (500 SGD steps in the runs).
Passes standard safety evalsThe paper reports capability benchmarks (AIME, Terminal‑Bench) on trigger-free prompts. It reports no safety benchmark.
Fewer than 10 neuronsTrue as a bound: the actual edits are 6 (Qwen) or 7 (Nemotron, Gemma).

Mechanism, at the level you need for defense: the edit targets dormant neurons in the model's final feed-forward block (ones whose removal barely changes predictions), turns each into a detector for the trigger, and lets each one force a single payload token when it fires. "A P-token payload therefore requires about P neurons." They fire in sequence across decoding steps and spell out a command. "No other parameter, file, or line of inference code changes."


Results, read carefully

Lazarus-SGD attack success by model, trigger location and run length; zero benign leakage throughout

Three models (Qwen3.6‑27B, Nemotron‑Terminal‑8B, Gemma‑4‑31B‑it), three agent harnesses (terminus‑2, mini‑swe‑agent, swe‑agent). Nemotron is near-perfect. Qwen and Gemma are patchy, especially over 8-step runs (Qwen observation trigger 55.6%, Gemma 62.2%). Benign leakage is 0.0000 in every Table 2 cell.

Two caveats the authors state themselves:

  • These are mostly teacher-forced rates: "how often the model predicts the payload in recorded contexts." Free generation was run on Nemotron only, at 98.9% (query) and 98.7% (observation), with leakage 0.14% in one setting.
  • With 23 fitting samples Qwen reaches only 51.0%, versus Nemotron's 98.8%. The abstract's "equally vulnerable" is stronger than the tables.

Qwen3.6-27B before and after: AIME roughly unchanged, Terminal-Bench 2.1 46.59% to 42.05%

Capability retention was measured on Qwen3.6‑27B only: AIME 2025/2026 roughly unchanged, Terminal‑Bench 2.1 down 4.5 points (88 tasks, so about four tasks). Nothing a leaderboard or your regression suite would flag. The paper's own conclusion: "Small changes in benign predictions do not rule out a backdoor."


Why this is an agent problem specifically

A chat model that blurts a shell command is embarrassing. An agent that executes it is a breach. The attacker in the paper's threat model controls the checkpoint only, "no control over the victim's agent harness, system prompt, or execution environment." Your harness supplies the rest: shell access, credentials in env vars, network egress, write access to the repo.

That's why the defense splits cleanly in two: make it hard for a tampered checkpoint to reach production, and make it cheap when one does.

It also isn't an argument against open weights. "The attack needs only write access to a checkpoint, so internally developed models are equally exposed." A compromised storage bucket, CI pipeline or account is enough.

Compare the data route: research Microsoft cites (Agentic Backdoors via Pretraining Poisoning) plants a code-execution backdoor by poisoning 0.1% of pretraining data, reaching about 92% success at 4B parameters. Lazarus needs no data at all. And unlike the guardrails that disappear on download, this is something added on the way.


Can you detect it from the weights?

LLM auditors: Astra 72.8% caught but abstains on 99% of clean; Opus 5.5 commits but misses 41.2%; feature detector 91.43% TPR at 9.68% FPR

The paper tests detection without a clean reference model, on 114 edited and 103 unedited public checkpoints:

  • GPT‑6 Astra as auditor (high reasoning, two hours, no hints) flags 72.81% of edited checkpoints and almost never accuses a clean one, but only because it returns "undetermined" on 99.03% of clean checkpoints. A detector that never clears anything isn't usable as a gate.
  • Claude Opus 5.5 commits: it clears 93.20% of clean checkpoints correctly, and calls 41.23% of edited ones clean.
  • A logistic detector on five weight-statistic features, two of them suggested by an LLM's analysis: 91.43% TPR at 9.68% FPR (AUROC 98.16) on 35 edited and 31 clean test checkpoints. The authors note it was evaluated on the same edit family it was fitted on.

None of this is a guarantee, and the authors are blunt: "no single audit or safeguard can guarantee safe behavior." The strongest signal isn't in these tables at all. It's a diff against the declared base, which the paper and Microsoft recommend but don't benchmark: "Fine-tunes and quantizations change weights broadly. Backdoored neurons change a few weight vectors by a lot."

There's a wider pattern worth naming. MIT CSAIL showed last week that GPT‑6 Astra can read and directly edit network weights, fixing a reasoning loop in a small model by changing 0.016% of its weights. Weight-space is becoming a place where models write code. Two days ago we wrote about REA turning agents into reverse engineers for binaries. Checkpoints are just the next binary.


What to change this week

Supply chain: treat weights like code

  • Pin by hash. Pull checkpoints only from the original publisher, verify the published file hashes, and store the hash with the model in a private registry. Don't let agents or inference services pull from public hubs at runtime.
  • Require a declared base. Any fine-tune, merge, quantization or adapter without a verifiable base model is untrusted. The paper: "Popularity is not evidence of safety."
  • Diff before deploy and after every update. Broad, diffuse changes are what legitimate fine-tuning looks like. A handful of rows or columns moving a lot is the red flag.
  • Record audit results next to the model, and tell internal consumers plainly that an audit is evidence, not a guarantee.
  • Plan for signing. Signed manifests (the paper mentions AIBom and C2PA) tell you who signed and whether it changed, not whether it's safe. Use them with integrity checks, not instead of them.

A minimal shape for the diff check. This is my illustration of the heuristic Microsoft describes, not code from the paper, and I haven't run it against Lazarus checkpoints:

# Flag tensors where a few rows/columns changed far more than the rest.
# Assumes same architecture and dtype; extend for sharded checkpoints.
import torch
from safetensors import safe_open

def concentrated_changes(base_file, cand_file, z=10.0):
    findings = []
    with safe_open(base_file, "pt") as b, safe_open(cand_file, "pt") as c:
        if set(b.keys()) != set(c.keys()):
            findings.append(("keys differ", sorted(set(b.keys()) ^ set(c.keys()))[:10]))
        for name in b.keys() & c.keys():
            d = c.get_tensor(name).float() - b.get_tensor(name).float()
            if d.ndim != 2 or not d.any():
                continue
            for axis, label in ((1, "rows"), (0, "cols")):
                n = d.norm(dim=axis)
                med = n.median()
                mad = (n - med).abs().median() + 1e-12
                hot = torch.nonzero((n - med) / mad > z).flatten().tolist()
                if hot and len(hot) <= 32:   # a few vectors, not a broad fine-tune
                    findings.append((name, label, hot))
    return findings

Run something like this in the model intake pipeline, and treat a non-empty result as "send to a human and a frontier-model auditor", not as a verdict.

Runtime: assume a tampered model gets through

This is where the paper's "bounded by permissions" line becomes your design rule:

  • Least privilege per agent identity. No inherited user credentials, no production access by default. Google admits many internal agents run with the host user's privileges; don't.
  • Execute generated code only in an isolated sandbox, never with host-side shell or file-write tools.
  • Egress allowlist on agent and inference hosts. A triggered download-and-run fails if the host can't reach the URL.
  • Human approval before consequential actions: running commands, changing permissions, sending data out. Prefer synchronous checks before execution over asynchronous monitors that can't undo anything.
  • Log commands, file changes and network calls, so a triggered payload leaves evidence.
  • Red-team for weight-level triggers, not just prompt injection: run your agent over corpora of mundane strings and watch for out-of-task commands.

My call

Act now on the supply-chain controls if you run any open-weight model you didn't train yourself, especially community fine-tunes, merges and quantizations. They cost little and they're good hygiene even without Lazarus.

Don't wait on detection to mature. Today's auditors either abstain (Astra) or miss four in ten (Opus 5.5) without a reference model, and the best detector was tested on one edit family. Diffing against a verified base is the control that works, which means provenance is the actual product.

Expect this to get easier for attackers, not harder. The authors write that coding assistants "can lower the practical barriers to engineering weight-level attacks." And Microsoft reports no observed in-the-wild use yet. That's the window.

What would flip me to "relax a little": an evaluation of whether these edits survive the fine-tune or quantization a downstream user typically applies. The paper didn't test persistence through any of them.


Source note

Verified at origin.

  • All numbers, quotes and tables: "Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents", Microsoft Research and Microsoft AI Red Team (PDF), Tables 1–6 and Table 14, read on the PDF pages.
  • Discovery story and Foundry controls: Microsoft Research blog, 10 Nov 2026. Mitigation list and "not observed in the wild": Microsoft Threat Intelligence, 10 Nov 2026.
  • Pretraining-poisoning comparison: arXiv:2610.37741. Weight-space programming: MIT CSAIL, 2 Nov 2026.

How the charts were built. From the paper's Tables 2, 3, 5 and 6. No interpolation; the build script is in the site repo with page references.

What I could not verify, or chose not to include. The paper doesn't test persistence through fine-tuning, quantization, merging, LoRA or distillation, and gives no wall-clock or dollar cost, so I don't claim either. Microsoft's "inference-only compute" phrasing is from its blog; the paper says forward passes plus a few hundred gradient steps. The diff snippet is my illustration of the published guidance, untested against the paper's checkpoints. I omitted neuron-selection and construction details on purpose.