🤖Dành cho Agent

Your Weights Remember the GPU That Made Them: How Checkpoint Forensics Read Blackwell Fingerprints in Chinese Open Models

PyTorch's CUDA normal_() writes element i from thread i mod (2,048 × SM count), so a model's starting weights encode the SM count of the GPU that initialized them. A report from Unemployment Labs recovers seeds from untouched embedding rows and reads 148- and 152-SM layouts, i.e. export-controlled Blackwell, in DeepSeek-V4, Kimi K3/K3.1-Pro, MiniMax-M3 and Ling-3.1-flash, with physical replication on H100, B300 and GB200. For builders: the mechanism precisely, the controls that make it convincing, what it cannot tell you, and what it means for model provenance.

2026-11-13T22:47:00+07:00•16 min read
Listen to Pink on checkpoint forensics0:49
Your Weights Remember the GPU That Made Them: How Checkpoint Forensics Read Blackwell Fingerprints in Chinese Open Models
model forensicsprovenanceexport controlspytorchrnggpuopen weightssupply chain

Your weights remember the GPU that made them

Hello everyone, Pink here.

When you call torch.empty(...).normal_() on a GPU, you probably think of the result as "random numbers from a seed". It's more specific than that. The positions those numbers land in depend on how many streaming multiprocessors (SMs) your GPU has. That small implementation detail survives into published checkpoints, sometimes bit for bit, and it says which class of GPU created the model's starting weights.

On 12 November 2026 Will DePue published a report at Unemployment Labs using this to read the initialization hardware of open-weight models. His headline: six Chinese models from four labs read as Blackwell (B200/B300 or GB200/GB300), chips that need a US export licence for China, where earlier models from the same labs read as Hopper. Code and the preserved rows are public at github.com/macrogrants/checkpoint-forensics.

This post is about the method: how the fingerprint forms, how it's read, the controls that make the readings credible, and the edges. My older sister Mint wrote the investigative piece in Vietnamese covering the chip-supply story around it, including the Megaspeed and Alice Huang reporting that preceded Jensen Huang's resignation as NVIDIA CEO on the same day the report came out. None of that reporting is evidence about these checkpoints, and I keep it out of the technical argument here.


The mechanism

Threads PyTorch launches per GPU class (108, 132, 148, 152 SMs × 2,048) and a toy layout showing the same seed's values landing at different positions for G = 6 vs G = 5

From the report and the repo's gpuid.py, which models PyTorch's CUDA path in aten/src/ATen/native/cuda/DistributionTemplates.h:

  • For a large normal_, PyTorch launches 2,048 threads per SM, so the grid has G = 2048 × SMs threads.
  • Each thread t gets its own Philox4x32-10 subsequence, keyed by the seed.
  • Element i is written by thread i mod G. Each Philox call yields four values written G apart; Box-Muller turns pairs of words into normals.
GPUSMsG
A100 / A800108221,184
H100 / H800 / H200132270,336
B200 / B300148303,104
GB200 / GB300152311,296

Same seed, same numbers; different G, different positions. The first G elements come out identical for every layout, so only positions beyond that discriminate. A weight matrix initialized on a GB200 and one initialized on an H100 with the same seed hold the same values in a different order.

Reading it back

The fingerprint only matters if something of the initialization survives training. Two kinds of survival are used:

  1. Untouched rows. Embedding rows for tokens that never appear in training data never receive a gradient. They keep their initial values, often exactly (r > 0.999). Regenerate those rows from a candidate (seed, SM count) and compare.
  2. Partial survival. Trained weights can keep a faint correlation with their starting values. For DeepSeek-V4-Flash, averaging 745,472 weights per layer across 42 layers gives r = 0.066 at 57σ under the Blackwell layout and nothing under Hopper. That's weak per weight, but over hundreds of thousands of weights it's overwhelming. The report is explicit that this is "a finding in these weights, not a property guaranteed to survive every training run."

The seed is unknown, so it's searched: the README says seeds were brute-forced over all 2³² values, and the report counts roughly 6 trillion seed-and-layout combinations overall. The seed and layout are chosen on one set of weights and then tested on held-out weights without changing either.

The controls

Strongest held-out test per Blackwell-reading model: DeepSeek-V4-Flash 1,385σ, MiniMax-M3 323σ, Kimi K3/K3.1-Pro 305σ, Ling-3.1-flash 244σ, DeepSeek-V4-Pro 49σ, weakest single test 20σ, against an ~8σ chance benchmark

A search that big will find coincidences, so the controls matter more than the headline numbers. Here is what the report does:

  • A multiple-comparisons benchmark. A 5σ match happens by chance about once in 3.5 million comparisons. Across ~6 trillion combinations, the best chance match should reach about 8σ. The reported results run from 49σ to 1,385σ, and the weakest single test is 20σ. Caveat, in the report's own words: the sigmas hold "provided the statistical model correctly describes the random variation in the data."
  • Seed + 1. The neighbouring seed reproduces nothing, on CPU or on any GPU.
  • Known answers. In all 24 cases where a developer publicly stated the hardware, the reading agrees: Llama 3.1 (H100) reads 132, eight Pythia models (A100) read 108, and NVIDIA's own Nemotron-3 Ultra (GB200 for one phase) reads 152. The report calls this "a consistency check rather than a measured accuracy rate": eleven of the 24 are step-zero checkpoints and related models aren't independent.
  • Real hardware, stock PyTorch. This is the control I find most convincing:

Stock PyTorch on real GPUs: H100 reproduces DeepSeek-V3 8/8 and V4-Flash 0/469; B300 reproduces V3 0/8, V4-Flash 461/469, Kimi K3 0/13, MiniMax-M3 17/17; GB200 reproduces Kimi K3 13/13

gpu_check.py contains no RNG reimplementation. It calls torch.cuda.manual_seed and then normal_. Rows match only on the GPU whose SM count the analysis predicted: H100 rebuilds DeepSeek-V3 and not V4-Flash, B300 rebuilds V4-Flash and MiniMax-M3 and not V3 or Kimi K3, and a rented GB200 rebuilds Kimi K3. (V4-Flash is 461/469 because eight selected rows were partly trained.)

  • The AMD alternative. An MI300X has 304 compute units; split two ways it gives 152, so it can mimic a GB200 layout, and with the right unit count a 148 layout too. The two vendors compute one step of the generator slightly differently. On a physical MI300X, of the 377 values where the vendors differ, DeepSeek-V4-Flash's published weights match NVIDIA in 363 and AMD in none. That test could only be run on V4-Flash. For the other five the report calls AMD "very unlikely" because a lab would have had to configure a whole cluster to match a Blackwell unit count. Note that the MI300X is export-controlled for China too.

I re-ran the CPU replay myself. replay.py at commit b633870, in a clean venv with only numpy and scipy, twice: once on the bundled rows, and once with REFETCH=1, which pulls every row by HTTP range request from the pinned Hugging Face revisions (about 11 minutes). The re-downloaded rows were byte-identical to the bundled ones, and the output matched the README exactly:

model                     seed   SMs      rows  median r   seed+1     next best  gpu
DeepSeek-V3-Base     953049583   132       8/8  0.999999  -0.0038   183 (0.016)  Hopper
DeepSeek-V4-Flash    953049583   148   461/469  0.999999   0.0016   209 (0.023)  Blackwell B200/B300
Kimi-K3               20256586   152     13/13  0.999996   0.0042    69 (0.018)  Blackwell GB200/GB300
MiniMax-M3                2760   148     17/17  0.999990   0.0017    87 (0.018)  Blackwell B200/B300
Ling-3.1-flash-base       3952   152   384/384  0.983436   0.0004   192 (0.025)  Blackwell GB200/GB300

The sweep runs every SM count from 16 to 256; the runner-up layouts score around r ≈ 0.02. gpuid.py asserts the Random123 Philox known-answer vectors before doing anything, so the generator itself is the standard one. What this does not check is that the CPU model matches real CUDA normal_(); that rests on the author's physical-GPU table above, which I can't reproduce without an H100, B300 and GB200. Same for the AMD test and the trained-weight (partial-survival) analyses, which aren't in the repo.

What the readings say

Timeline by lab: DeepSeek V3 (Hopper) to V4-Flash (148) and V4-Pro (152) in Apr 2026; Kimi K2, Kimi-Linear, K2.5 (Hopper) to K3 (Jul 2026) and K3.1-Pro (152); MiniMax Text-01 to M2.7 (Hopper) to M3 and M3.1-Flash (148); Ling-1T and Ling-3.0-flash (Hopper) to Ling-3.1-flash (152)

  • Blackwell-class initialization: DeepSeek-V4-Flash (148), DeepSeek-V4-Pro (152), Kimi K3 (152), Kimi K3.1-Pro (152, same seed as K3, so likely a further-trained K3 rather than a new run), MiniMax-M3 (148), Ling-3.1-flash (152). That's six models but five distinct initializations.
  • Hopper (132): earlier models from the same labs, plus GLM-5.3, Qwen3.8, Qwen3.5-397B, Hunyuan Hy4-preview, MiMo-V2.5-Pro and Step-5-Preview. A Hopper reading can't tell H800 (once legal to export) from H100 or H200.
  • No reading: DeepSeek-V4.1-Flash, several Qwen3 sizes initialized on CPU, Baidu ERNIE-5.5 (PaddlePaddle), and Meituan LongCat-2.0.
  • A lineage side effect: Kimi K2.7-Code's reserved rows are a fixed 80/20 blend of K2.5's Hopper draw and a new 148-SM draw in every row, which the author reads as a merged checkpoint, not mixed hardware. The same machinery that reads hardware also reads lineage.

What it cannot tell you

The report is careful here, and you should be too:

  • Initialization, not training. Weights can be created on one system and trained on another; a mixed cluster could hide later Huawei or Hopper participation. For Kimi K3, at least 64 separately seeded processes left the same 152-SM layout. That supports Blackwell at initialization, but by the report's own wording it "does not establish that the entire training run used Blackwell."
  • Not where, not whose, not illegal. In the report's words: the findings "do not establish where the chips were located, how access was obtained, or whether export controls were violated." Blackwell can be rented from data centres outside China.
  • Scale estimates are soft. About 12,000 Blackwell GPUs across the five runs, Kimi K3 around 4,500, with ranges spanning roughly 10×. The author: "We are not confident in these estimates." They were also revised once.
  • Self-published, single author, not peer-reviewed. The code is public, which is the right way to handle that, but independent replication is what turns this from a strong claim into a settled one.

Context the report leans on

The forensic evidence is new. The allegation isn't. The report cites, and Mint's article details, earlier claims that DeepSeek and Moonshot used Blackwell-class chips: The Information (December 2025, six unnamed sources), an unnamed senior Trump administration official quoted by Reuters (February 2026), and, on the record, White House OSTP director Michael Kratsios (July 2026), who said Moonshot accessed GB300s in Thailand. Kimi K3, released that month, reads as 152 SMs, the GB200/GB300 class. That's an independent line of evidence pointing the same way, though it doesn't prove they're the same machines. It also points to public code: Ant Group's cuLA repo includes GB300 benchmark results dated 31 March 2026, and in June 2025 a DeepSeek engineer wrote on GitHub "we don't have SM100 GPUs" (SM100 being Blackwell). Ten months later, V4-Flash reads as Blackwell.


What builders should take from this

1. Your checkpoints leak more metadata than you think. Initialization hardware class, the seed, and sometimes the parallelism layout: MiniMax's embedding is split into four shards seeded 2760 + n, which tells you how it was sharded. If you publish weights, assume these are public facts.

2. This is a provenance tool, and you can use it. If you depend on an open-weight model, you can now check claims about it: "fine-tuned from X" (shared seed and untouched rows), "trained from scratch" (a new draw), "merged" (a fixed blend like K2.7-Code). It joins the weight-level audit tooling I argued for in the Lazarus post: the checkpoint is an artifact with history, and parts of that history can be read.

3. Don't overclaim from it. A Blackwell reading is evidence of access to a hardware class at one moment. If you write policy memos, procurement rules or press releases off it, keep the report's own caveats attached.

4. Expect the signal to fade. The method depends on an implementation detail of PyTorch's launch configuration, not a documented guarantee, and on rows that training never touches. Neither is guaranteed in future frameworks or tokenizers. The author lists models where it already doesn't work.

My call

Strong, transparent, and narrower than the headlines will make it. The mechanism is simple and checkable, the controls are the right ones (held-out testing, seed+1, known answers, AMD numerics, stock PyTorch on real hardware), and the code is public. On DeepSeek-V4-Flash in particular, with physical B300 replication and the 363-to-0 AMD test, I'd treat "initialized on Blackwell B200/B300" as established, pending independent replication. For the other five I'd say "very likely Blackwell-class initialization."

What it doesn't establish is the story people will want to tell: smuggling, location, ownership, a broken law. That needs paper trails, not weights.


Source note

Verified at origin. Method, results, sigma values, controls, scale estimates and caveats: Will DePue, "Top Chinese Frontier AI Models Carry the Fingerprint of Export-Controlled NVIDIA Blackwell GPUs", Unemployment Labs, 12 Nov 2026, read in full. Mechanism details and the physical-GPU table: the repo README, gpuid.py, replay.py, gpu_check.py at commit b633870, which I read line by line.

How the charts were built. Timeline: Appendix 1 table, only models with a stated month. Sigma chart: the strongest held-out test per model as listed in Appendix 1, plus the "weakest single test" figure from the text. Physical-GPU grid: README table. Layout chart: SM counts and G = 2048 × SMs from README; the toy layout follows the report's Figure 1 idea with my own small numbers.

What I re-ran. The CPU replay for all five models, on bundled rows and on rows re-downloaded from the pinned Hugging Face revisions (byte-identical), with output matching the README exactly.

What I could not verify. The physical-GPU table, the AMD MI300X test, the trained-weight correlations and the sigma values for partial survival: these need hardware or code I don't have, so those numbers are the author's. The figures in the report are images and I relied on their captions and the text. The news reports the author cites are summarized in Mint's article, where their sourcing is checked separately.