path: root/f32/AUDIT-SDMA-01.md

# SDMA (gfx11) — first audit pass over the host-packet surface

Evidence class **[ST]**. Image `psp_out/sdma_6_0_0.bin`, thread TH0. Tool:
`sdma_audit.py` (`table` / `screen` / `gates` / `hand`), on top of the F32 disassembler
made usable in `AUDIT-F32-01.md`.

Why SDMA: it executes packets the driver writes into rings in host-visible memory, so
it has the same surface shape as the DMCUB inbox (FINDING-1/2) and the DCN4 LSDMA
command (FINDING-9) — **and it is loadable on this card** (`SDMA_UCODE_TH0/TH1` are 2
of the 37 accepted `LOAD_IP_FW` types), so anything found here is testable on silicon
we actually have, unlike FINDING-9.

## The packet dispatch table, recovered and validated

The last `0x300` bytes of TH0 are the dispatch table: entries are
`(key << 16) | target` with **`key = (op << 8) | (sub_op << 4)`**. That decoding is not
a guess — it is checked against `navi10_sdma_pkt_open.h`:

```
0x0100 COPY/LINEAR      0x0140 COPY/LINEAR_SUB_WIND   0x0400 INDIRECT
0x0110 COPY/TILED       0x0170 COPY/DIRTY_PAGE        0x0500 FENCE
0x0130 COPY/SOA         0x0180 COPY/LINEAR_PHY        0x0600 TRAP
0x0c00 PTEPDE/GEN       0x0e00 SRBM_WRITE             0x1000 GPUVM_INV
```

Every documented opcode and sub-opcode lands where the enum says it should, which is
what makes the table trustworthy. **52 distinct packets** are implemented; entries
beyond `0x1110` point at the common stub `0x0e0a` and are padding.

**The taint source is exact.** Upstream's README: *"r1 is a magic register that pulls
dwords off of the command queue when read"*. So `mov rX, r1` is literally "consume the
next host-supplied dword", which gives a precise, non-heuristic marker for host data —
better than the pointer-tainting model used for DMCUB.

## SRBM_WRITE is gated, and the gate is legible

`SRBM_WRITE` (`0x0e00`) is the packet that writes a host-named register — the exact
shape of DMCUB's by-design register interface. It is **not** unguarded:

```
0b00  ldw r5, reg[r0, #0x4a14] ; and r5, r5, #0x200 ; lsr r5, r5, #9   ; bit 9
0b04  ldw r5, reg[r0, #0x5968] ; lsr r5, r5, #31                        ; OR bit 31
0b07  ldw r5, reg[r0, #0x5940] ; lsr r5, r5, #23 ; and r5, r5, #0x1     ; AND bit 23
0b0a  and r5, r6, r5
0b0e  mov r11, r1              ; host dword: register address
0b0f  mov r6,  r1              ; host dword: value
0b10  cbz r5, _PKT_0xe00_2     ; gate fails -> reject
0b11  lsr r5, r11, #20         ; aperture bits of the address
0b16  bl _DMA_DATA_39          ; perform the write
```

The reject path raises an error rather than falling through:

```
_PKT_0xe00_2: mov r13, #0x3c ; mov r14, #0x32 ; bl _PKT_0xf00_26
```

`SRBM_WRITE/sub1` (`0x0e10`) carries the same gate. So the obvious "arbitrary register
write from a ring packet" is **closed** on this firmware.

## Gate map — and a wrong measurement I had to throw away

The first cut asked a binary question: "does the handler contain the SRBM_WRITE gate
idiom (reg `0x4a14` bit 9)?" It produced a headline-looking result — *PTEPDE, the
packet that writes page-table entries, is not gated* — which is **false**. PTEPDE gates
on a different register:

```
09c4  ldw r5, reg[r0, #0x5a6c] ; and r5, r5, #0x101 ; seteq r5, r5, #0x101
09c7  cbz r5, _PKT_0xc00_0
```

So different packet classes consult different control registers, and a single-idiom
test cannot answer the question. `sdma_audit.py gates` now reports the **map** instead
of a verdict:

| register | handlers | note |
| --- | --- | --- |
| `0x4a14`, `0x4af8`, `0x4b58` | 49 | shared epilogue — reachable from nearly everything, **not** a per-packet gate |
| `0x49f4` | 15 | |
| `0x5940` | 14 | |
| `0x5968` | 7 | part of the SRBM_WRITE gate |
| `0x5a6c` | 5 | PTEPDE's gate |
| `0x5980`, `0x5970`, `0x49f0` | 2–3 | |

The 49-handler entries are a **confound from the analysis**, not a property of the
firmware: the worklist walk runs into the common exit path. Only registers consulted by
a *subset* of packets carry information.

## Unpublished opcodes — checked, benign

`op3` with sub-ops 1..6 (`0x0310..0x0360`) is absent from the kernel enum, which made it
the most promising lead by analogy with DMCUB's unpublished types 1/3/4 (those became
FINDING-10). It is not interesting: each handler just drains host dwords and rejoins the
main loop.

```
PKT_0x310:  mov r3, r1 ; mov r3, r1 ; ... (8x) ; b _PKT_0xf00_2
PKT_0x320:  b _PKT_0x310_0
```

These are "swallow N dwords" stubs for reserved opcodes. `op35/sub2`, `op119` and the
`op126` block decode as padding entries pointing at the invalid-packet stub.

## [HW] The gate is RB_PRIV — named by behaviour, confirmed on this card

Script `hw/sdma_srbm_probe.py`, evidence `hw/EVIDENCE-sdma-srbm-gate.txt`.

**Naming it statically failed, and that is recorded as a negative:** fitting the F32
index space against `gc_11_0_0_offset.h` gives a best constant base scoring **25/54**
observed indices against a null 95th percentile of **24** — indistinguishable from
chance over a table of 848 dense SDMA registers. The earlier "0x5968 = SDMA0_F32_CNTL"
style matches were curve-fitting, and are withdrawn.

What identified it was the bit position. `SDMA*_QUEUE*_RB_CNTL__RB_PRIV` is mask
`0x00800000` = **bit 23**, and the gate's AND-term is exactly
`ldw r5, reg[r0, #0x5940] ; lsr r5, #23 ; and r5, #1`. That is a testable prediction,
so it was tested:

```
=== RB_PRIV = 1
  control: NOP packet consumed=True (0.1 ms)
  SRBM_WRITE consumed=True -> regGRBM_SCRATCH_REG0 = 0x5100d00d   WROTE
=== RB_PRIV = 0
  control: NOP packet consumed=True (0.1 ms)
  SRBM_WRITE consumed=True -> regGRBM_SCRATCH_REG0 = 0x00000000   not written
```

Two things make this clean rather than suggestive:

* **The NOP control still retires with RB_PRIV=0.** A queue configured from scratch
  with `RB_PRIV=0` does not fetch at all — the ring fetch is itself privileged — so
  "nothing happened" would have proved nothing. The experiment therefore brings the
  queue up privileged and then clears **only** `RB_PRIV` on the live queue. The ring
  keeps running (NOP retires in 0.1 ms) while SRBM_WRITE stops taking effect, which
  isolates the difference to that one bit.
* **The rejected packet is still consumed** (`rptr` advances, queue keeps running,
  `CONTEXT_STATUS = 0x804`), i.e. the firmware drops it silently rather than wedging —
  consistent with the `_PKT_0xe00_2` error path rather than a hang.

**Conclusions.**

1. **F32 register index `0x5940` is the current queue's `RB_CNTL`.** This is the first
   F32 register named in this tree, and it is named semantically rather than by offset
   arithmetic — the same standard `dmcub_regmap.py` used for DMCUB.
2. **[HW] An SDMA ring packet can write an arbitrary register** — `SRBM_WRITE` with a
   host-chosen absolute index landed `0x5100d00d` in `GRBM_SCRATCH_REG0`.
3. **The confinement holds.** That power is available only to queues the driver marks
   privileged. This is a **confirmed negative**, and a valuable one: had the gate been
   missing, an unprivileged user queue writing arbitrary registers would have been a
   genuine user→kernel escalation, which is a stronger boundary than anything DMCUB
   offers (its inbox is driver-only by construction).

The method generalises: any F32 register whose bits gate observable behaviour can be
named this way, which is the practical route to the register namespace that offset
matching could not deliver.

## [HW] PTEPDE is *not* privilege-gated — and why that is correct, not a bug

Scripts `hw/sdma_ptepde_probe.py`, evidence `hw/EVIDENCE-sdma-ptepde.txt`.

First, a correction to this document's own earlier text. It said "PTEPDE gates on reg
0x5a6c bits 0x101". **That is wrong.** The branch rejects nothing: with the bits set it
applies an extra address/value fixup (two `_DMA_DATA_6` calls), and both paths fall
into the same PTE write at `_PKT_0xc00_0`. It selects an *implementation*, not a
permission. So PTEPDE has **no privilege check at all** — a testable prediction.

Tested with the same live-flip method, on the queue where SRBM_WRITE is refused:

```
RB_PRIV = 1   NOP ok   PTEPDE -> buffer WRITTEN   entry[0]=0xaaaa0000bbbb0000
                                                  entry[1]=0xaaaa0000bbbb1000  (+incr)
RB_PRIV = 0   NOP ok   PTEPDE -> buffer WRITTEN   entry[0]=0xcccc0000dddd0000
                                                  entry[1]=0xcccc0000dddd1000
```

The entries increment by `incr` exactly as the ABI specifies, so the packet genuinely
executed. **PTEPDE writes page-table entries from an unprivileged queue.**

That sounds alarming, so the follow-up decides whether it matters: is the destination
physical, or translated? On this card `mc = paddr + 0x8000000000`, so the same buffer
can be addressed both ways:

```
pe = MC address     0x8001603000  -> buf[0] = 0x1111000022220000   HIT
pe = raw physical   0x0001603000  -> buf[0] = 0x0000000000000000   -
GCVM_L2_PROTECTION_FAULT_STATUS: walker_error=1, mapping_error=1, rw=1, vmid=0, addr 0x1603000
```

The raw physical address raises a **GCVM L2 protection fault** and writes nothing. So
PTEPDE's destination goes through the queue's VMID page tables like any other SDMA
memory access, and the queue survived both (a NOP still retires afterwards).

**The trust model is therefore coherent, and this is a confirmed negative:**

| packet class | confined by | gate needed? |
| --- | --- | --- |
| `SRBM_WRITE` (register access) | nothing — the VM cannot confine an MMIO write | **yes**, and it has one (`RB_PRIV`) [HW] |
| `PTEPDE`, `COPY`, `WRITE` (memory access) | the queue's VMID page tables [HW] | no, and it has none |

A user queue issuing PTEPDE can only write pages already mapped into its own VM, which
is the same reach its ordinary `WRITE`/`COPY` packets have. The asymmetry between the
two packets is exactly what a correct design looks like, not an oversight.

## A lead that looked like a finding, and died on hardware

Worth recording in full, because the static evidence was persuasive.

Screening for host data reaching a *register index* (rather than a DMA address) found
exactly one candidate outside the gated SRBM_WRITE path, in
`POLL_REGMEM/POLL_MEM_VERIFY` (key 0x0830):

```
0785  mov r5, r1            ; 12th host dword
0787  cbz r7, ...           ; r7 = header bit 31 = the ABI's `mode` field
0792  ldw r11, mem[r0, #0]
0794  bl  _PKT_0xf00_44
079c  bl  _DMA_DATA_36
079e  stw r11, reg[r5, #0]  ; register index from a host dword?
```

And `SDMA_PKT_POLL_MEM_VERIFY_RESERVED_reserved_offset` is **12** — the field the
published ABI calls *reserved* appeared to be consumed as a register number, in a
handler with no privilege check. That would have been an ungated arbitrary register
write from an unprivileged queue, i.e. a real user→kernel primitive.

**Hardware says no.** Aimed at F32 index `0x5940` (the queue's `RB_CNTL` — the one index
we have named [HW], and one the host can read back), with `mode=1` so the path is
taken:

```
RB_PRIV=1 mode=1   consumed=True  -> RB_CNTL 0x00841015 -> 0x00841015  unchanged
RB_PRIV=0 mode=1   consumed=True  -> RB_CNTL 0x00041015 -> 0x00041015  unchanged
```

**Root cause of the bad lead: the dataflow was not call-clobber aware.** There are two
`bl` calls between `mov r5, r1` and the store, and F32 helpers clobber scratch
registers, so `r5` at `0x079e` is not the host dword — it is whatever the callee left
behind. This is precisely the flaw called out in `../mes/AUDIT-MES-02-API.md`
("absence of evidence is weak because the pass is not call-clobber aware"), reproduced
here in the other direction: it manufactured a *positive* that was not real.

Two process notes worth keeping:

* The **first** version of this test aimed at `regGRBM_SCRATCH_REG0`'s *host* index
  (0x2040). `reg[r5]` indexes the F32 namespace, which does not align with the host's,
  so that run's null result was uninformative — a miss would have been indistinguishable
  from a wrong target. Re-aiming at the one F32 index we have actually identified is
  what made the negative meaningful.
* `POLL_MEM_VERIFY` with `mode=0` **wedges the queue** (`consumed=False`, and the queue
  stays dead): the packet polls until its comparison succeeds, by design. That is a
  self-inflicted stall by the queue's own owner, and SDMA's watchdog
  (`SDMA_WATCHDOG_CNTL.queue_hang_count`, which the driver sets to ~10 s) is the
  intended backstop — noted as behaviour, not claimed as a finding.

## Call-clobber-aware taint: one sink in all of TH0, and it is gated [HW]

The screening pass was rebuilt to be call-clobber aware (`sdma_audit.py taint`):
per-callee clobber sets are computed over the call graph to a fixpoint, and a host
taint is killed across any `bl` that may define the register. Re-running it over all 52
packets:

* the `POLL_MEM_VERIFY` false lead **disappears**, which is the control that the new
  pass actually fixes the flaw that produced it;
* **exactly one** sink remains in the whole of TH0 — `0x0657 ldw r10, unk[r5]` in
  `POLL_REGMEM/POLL_REG_WRITE_MEM`, where the host dword is the *address of a register
  to read*.

That handler is the read counterpart of SRBM_WRITE: it reads the register the packet
names and DMAs the value to a host-supplied address. The address convention is known
rather than guessed — amdgpu's `sdma_v6_0_ring_emit_reg_wait` passes `reg << 2` — so
the test can predict the exact expected value:

```
RB_PRIV=1   host wrote 0xfeedface -> memory got 0xfeedface   LEAKED
RB_PRIV=0   host wrote 0xfeed0531 -> memory got 0x00000000   nothing
```

**It is gated.** Register *reads* require `RB_PRIV` just as register *writes* do, so the
trust model is uniform and there is no read/write asymmetry to exploit.

### That result invalidates my own gate map as a security indicator

`cmd_gates` said this packet does **not** contain the `0x4a14`-bit-9 idiom, yet hardware
shows it is privilege-gated anyway — the enforcement lives somewhere the static screen
does not see (a shared helper such as `_PKT_0xf00_50`, or the `unk[]` aperture access
being checked in hardware). So **the gate map under-reports gating and must not be read
as "these packets are unprivileged"**. Only the hardware test is authoritative. The
same caveat now sits on the map itself.

### Standing scoreboard for TH0's host surface

| capability | confinement | evidence |
| --- | --- | --- |
| register write (`SRBM_WRITE`) | `RB_PRIV` | [HW] `sdma_srbm_probe.py` |
| register read (`POLL_REG_WRITE_MEM`) | `RB_PRIV` | [HW] `sdma_regread_probe.py` |
| memory write (`PTEPDE`, `COPY`, `WRITE`) | queue VMID page tables | [HW] `sdma_ptepde_probe.py` |
| host data → register index / memory base | 1 site, and it is the gated read above | [ST] call-clobber-aware taint |

**No primitive was found in TH0.** Each capability that escapes VM confinement is
privilege-gated, and the one remaining taint sink is one of them.

## TH1 (the "ctl" thread) — examined, no host-controlled register index

TH1 is a different animal from TH0 and the difference shows up immediately:

| | TH0 | TH1 |
| --- | --- | --- |
| reads of `r1` (host command queue) | 370 | **11** |
| packet dispatch table | 52 packets, validated | **none** |
| computed register accesses (`reg[rN]`, N≠0) | 5 | **52** |

The tail of TH1 is **not** a dispatch table. `f32dis` assumes the trailing page is a CP
packet jump table and duly prints 64 entries, but the keys are garbage (`0xd411`,
`0x6967`, `0x86f7`…) rather than `(op<<8)|(sub_op<<4)` values — an artefact of the
CP-oriented tool, not a structure in the firmware. So **TH1 does not process ring
packets**; it is the queue/context control thread.

Its interesting feature is those 52 computed register accesses (40 reads, 12 writes).
Root-cause tracing of every index:

```
const                       48   firmware-chosen register blocks
from caller                   2
stw r7, [r0, #0x6b]           1
stw r3, reg[r0, #0x5978]      1
```

The two "from caller" sites are a register-programming helper (`_PKT_0xc3a5_33`):

```
0521  mov r5, r1              ; value = host dword
0522  seteq r6, r3, #0x4ba8   ; five indices special-cased first ...
052a  seteq r6, r3, #0x4bb0   ; (0x4ba8, 0x4bb0, 0x4bb4, 0x4bb8, 0x4bbc)
052c  stw r5, reg[r3, #0x0]   ; ... default: program the register
```

Host data supplies the **value**, so the question is whether it also supplies the
**index**. It does not. Every caller computes it:

```
mov r3, #0x4b80 ; add r3, r3, r10      ; r10 = loop variable
mov r5, #0x4b80 ; mov r6, #0x4ce0 ; sub r6, r6, r5   ; stride = 0x160
mul r5, r6, #0x8 ; setge r6, r10, r5   ; bounded to 8 iterations
```

i.e. `r3 = 0x4b80 + queue * 0x160`, bounded by an 8-iteration loop. This is queue
context restore: the firmware walks the eight queues' register blocks and reloads their
saved state. **The index is firmware-computed and bounded; only the values are
host-supplied, which is the whole point of restoring a queue's context.** No primitive.

Two useful by-products:

* **Structural fact for the register namespace:** the per-queue register block in F32
  index space is **base `0x4b80`, stride `0x160`, 8 queues** (`0x4b80..0x5460`). That is
  independent of, and consistent with, the `0x5940 = RB_CNTL` identification from
  hardware, which lies outside that range — so `0x5940` is a *current-queue* alias
  window rather than a per-queue slot.
* Five queue registers (`0x4ba8`, `0x4bb0`, `0x4bb4`, `0x4bb8`, `0x4bbc`) are **not**
  restored directly; they are diverted into internal shadow slots. Worth naming later —
  a register the firmware refuses to restore verbatim is exactly where a restore-stream
  attack would aim, if the stream were ever attacker-controlled.

## The F32 register namespace, solved

Naming registers by fitting a constant delta against `gc_11_0_0_offset.h` failed
earlier at chance level (25/54 vs a null 95th percentile of 24). The namespace fell out
of *structure* instead, from two independent anchors:

* TH1 walks the eight per-queue register blocks with F32 base `0x4b80`, **stride 0x160**
  (recovered above from its bounded 8-iteration loop);
* the host per-queue stride is **0x58 dwords** (`regSDMA0_QUEUE{n}_RB_CNTL` =
  0x0080, 0x00d8, 0x0130, …).

`0x160 / 0x58 = 4` exactly, so F32 addresses registers by **byte** offset:

```
    F32_index = 4 * host_dword_offset + 0x4980
```

**Validation, against a null model:** 31 of the 54 observed indices resolve to a named
`regSDMA0_*` register, where randomly drawn indices over the same range score a median
of 5, a 95th percentile of 9 and a **maximum of 14** over 400 trials. The 4-alignment
requirement alone rejects 3 in 4 candidates, and the surviving names are semantically
right for a DMA engine's inner loop:

| F32 index | register | uses |
| --- | --- | --- |
| `0x4a14` | `regSDMA0_STATUS_REG` | 45 |
| `0x49f4` | `regSDMA0_CHICKEN_BITS` | 16 |
| `0x4a60` | `regSDMA0_STATUS2_REG` | 12 |
| `0x4b58` | `regSDMA0_STATUS4_REG` | common path |
| `0x49f0` | `regSDMA0_CNTL` | 6 |
| `0x4a70` | `regSDMA0_UTCL1_CNTL` | 4 |

`sdma_audit.py gates` now prints names instead of bare indices.

### The gate registers are *unpublished*

The indices that stayed unnamed cluster tightly: `0x5940`, `0x594c`, `0x5950`, `0x5968`,
`0x5970`, `0x5980`, `0x5a6c` map to host dword offsets **0x3f0–0x43b**, which lie inside
the SDMA0 aperture but have **no definition anywhere in AMD's header**. So the
SRBM_WRITE gate (`0x5940` bit 23, `0x5968` bit 31) and PTEPDE's implementation switch
(`0x5a6c` bits 0/8) live in registers AMD does not publish.

That also resolves an apparent inconsistency. Hardware proved `0x5940`'s bit 23 behaves
exactly as `RB_CNTL.RB_PRIV`, yet under the formula `0x5940` is **not** any queue's
`RB_CNTL` (those are `0x4b80 + q*0x160`). Both facts hold if `0x3f0` is a
**current-queue context shadow**: when SDMA switches to a queue it copies that queue's
`RB_CNTL` into an unpublished context register, and the firmware gates on the shadow.
That is a hypothesis consistent with all the evidence, not a measurement — the
behavioural result (flipping `RB_PRIV` flips SRBM_WRITE) stands on its own either way.

## What this pass does *not* establish

* **No finding.** This is reconnaissance: the table, the taint source, and the gating
  structure. No memory-safety bug has been demonstrated in any handler.
* **Most of the register namespace is still unmapped** (`AUDIT-F32-01.md`). `0x5940` is
  named [HW]; `0x4a14 bit 9`, `0x5968 bit 31` (the gate's OR-term) and `0x5a6c bits 0/8`
  (PTEPDE's implementation switch) are not.
* Handler analysis is a bounded worklist walk (250 instructions, follows branch
  targets), an under-approximation, not a closure — **and it is not call-clobber
  aware**, which already produced one false lead (above). Any future "host data reaches
  X" claim from this tooling must be checked across `bl` boundaries before being
  believed.
* TH1 is now examined at the same level as TH0 (above), but its *callers* were traced
  only for the register-programming helper; the rest of its control flow, and what
  drives its 11 `r1` reads, are not mapped.
* 2.6% of TH0 words still decode as unknown (CI-era opcode table), so a handler could
  do something the listing does not show.

## Next steps, in order of value

1. ~~Name the gate registers~~ / ~~test the gate on hardware~~ — **both done above**
   for the AND-term (`0x5940` = `RB_CNTL`, bit 23 = `RB_PRIV`). Still unnamed: the
   OR-term (`0x4a14` bit 9, `0x5968` bit 31) and PTEPDE's `0x5a6c` bits 0/8, each of
   which is now a concrete behavioural experiment of the same shape.
2. Audit the copy/fill handlers (`COPY/*`, `CONST_FILL`, `WRITE/*`) for length and
   address handling, which is where an SDMA memory-safety bug would live.
4. Map TH1's table the same way.