| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
something
Finishes the rebuild that was interrupted earlier and leaves the corpus on a single decoder version
for the first time since the SVC-opcode work started. All 16 modules rebuilt clean, 16/16 self-port
gates PASS, no duplicate names in any of the 99 builds, no address-derived names.
The naming fix from the previous commit is now actually in the artifacts: 1,038 SVC stubs use the
libdump's sysdrv_rpc_<N> convention instead of my own rpc_<N>. Six names persisted in fw/cgnames from
when I wrote them by hand are converted too, otherwise they would have quietly reintroduced the
inconsistency on the next rebuild. The 85 remaining bare rpc_ names are of the form gcp_asd_rpc_202 --
ASD's libdump names ported into other modules through shared code, so that is the libdump's own
convention and correct to leave.
Effect on diffs, which is why this mattered: the 22.0.4.49 -> 22.0.4.50 pair reported 8 functions
added and 8 removed with no firmware change behind it, and now reports 4. The remaining 4 are a
different and smaller problem, now documented in ta_diff.py: a function can carry a richer name in one
build than another, gcp_cmdFE0C against gcp_cmdFE0C_CreateCryptoSession, because the semantic half
comes from a pass that only reaches some builds.
Corpus-wide the diffs are now usable. Adjacent patch versions average 0.5 changed functions in ras,
2.5 in wspe, 3.3 in merit, 4.6 in opm, up to 26 in xgmi where the builds are genuinely far apart.
Added/removed should be read as "look at this" rather than ground truth; changed-function counts are
sound.
Metrics unchanged by the rebuild at 19,487 meaningful names (67.3%) and 75.8% field share, which is
the point -- it confirms the mixed-decoder tree was not inflating anything.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
known names
The previous pass left 102 stubs unnamed because they load more than one in-range constant. The
tempting fix was to take the first immediate, which matches 4 of 4 multi-immediate stubs that carry an
established name -- and that number is worthless, because all four are the same XGMI function in four
builds. It is n=1 dressed up as n=4, and applying it to 102 functions would have been a guess with a
validation figure attached.
The structural rule earns the names instead: track MOV/MOVW/MOVS immediates per register, then return
whichever register is stored to [SP,#0], which is the first word of the frame R1 is pointed at before
the supervisor call. Against the 940 stubs the libdump already names *_rpc_N it matches 940 and
disagrees on 0, with no abstentions, and it resolves all 102 stubs the count heuristic could not.
So the weaker rule is superseded rather than patched, and the audit records both along with why the
4/4 figure was rejected -- that reasoning is the part worth keeping.
Regenerated gcp, hdcp, merit, opm and asd. Corpus 19,405 -> 19,433 meaningful names (67.1%), no
duplicate names in any of the 99 builds, address-derived still 0.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
corpus to 67.0%
The opcode decode that closed dtm and csu last commit is not module-specific, so I measured its reach
and validated it before applying it anywhere.
Validation came free: the libdump already names 299 single-immediate SVC stubs *_rpc_N, which is
independent ground truth for exactly this decode. The decoded immediate matches the established
number in 296 of 299. The three exceptions are one XGMI function across three builds whose libdump
name uses a different numbering scheme, not a bad decode.
Applied as a pipeline pass, keyed on bytes rather than addresses so it survives database rebuilds,
and only where exactly one in-range immediate survives -- 102 ambiguous stubs stay unnamed rather
than guessed. 501 stubs named: hdcp 87 -> 96, fp 92 -> 95, gcp 87 -> 91, opm 62 -> 75, merit 76 -> 79,
xgmi 75 -> 76, ras 35 -> 38, asd 64 -> 65, powerplay and wspe smaller. Corpus 19,134 -> 19,405
meaningful names, 66.1% -> 67.0%.
A second corroboration turned up after the fact: HDCP's newly decoded hdcp_rpc_161 carries the same
service number as ASD's independently libdump-named asd_rpc_161, so two modules calling the same
SYS_DRV entry agree without either having been told about the other.
Names are uniquified per build; no build contains a duplicate name, checked across all 99.
Worth noting what made this possible: the claim being overturned was my own, that these stubs "carry
no semantic information whatsoever". It was right about the shared thunks and wrong about the
per-module ones, and I had written it down as settled.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Earlier in this work I recorded that the SVC 0xF2 stubs "carry no semantic information whatsoever",
since the operation is chosen by caller-set registers. That holds for the shared thunks and I
over-generalised it: in the per-module stubs the opcode is loaded by the stub itself.
DTM's three unnamed functions all decompile to the same four lines, "SVC 0xF2; return 2". The bytes
say otherwise. MOV.W R0,#0x1C0, MOVW R0,#0x101 and MOV.W R0,#0x100, each stored as the first word of
the request frame before R0 is set to the selector and R1 pointed at the frame. So the first stored
word is the opcode, and 0x100 independently matches a comment already in this tree reading
"Opcode 0x100 - DTM TOPOLOGY_UPDATE", which is the corroboration that makes this more than a plausible
decode.
Named <ta>_rpc_<hex>, following the existing asd_rpc_NNN convention, and only where exactly one
in-range immediate survives -- otherwise the stub stays unnamed rather than guessed.
Also named by reading the bodies: oc_memcmp_ct, whose loop accumulates "v3 |= *a1++ ^ *a2++" over a
counted range with no early exit and is therefore a constant-time comparison worth labelling;
csu_rpc_309 and csu_rpc_305, the same stub shape with the opcode differing across builds; and
auc_sysdrv_rpc_101C_fwd, an 8-byte forwarder that adds a2+80 as a fourth argument.
csu, oc, tmz and dtm are now 100% named. Corpus 19,126 -> 19,134 meaningful names.
What remains in the small modules is an AUC body of several hundred lines, a RAP command handler and
five FP functions. That is per-function reverse engineering rather than pattern work, which is the
honest characterisation of the rest of the corpus too.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ta_entry_points read the trampoline's +0xA0 (Create) and +0xB0 (Invoke) slots and threw away the
three between them. Those are TA_DestroyEntryPoint, TA_OpenSessionEntryPoint and
TA_CloseSessionEntryPoint. That is the whole reason TMZ sat at 50% named: it has six functions and
three of the unnamed ones were entry stubs the header identifies exactly.
The slot order is read off GCP rather than assumed from the GP spec. GCP has all five named
independently and its table reads Create, nullsub(destroy), OpenSession, CloseSession, Invoke, with
WSPE and HDCP agreeing and the types agreeing too -- slots 0, 2, 4 return a status, slots 1 and 3 are
void. Running the new derivation against GCP reproduces GCP's own names exactly, which is the check
that makes this evidence rather than convention.
Applied to every module, only where porting left the address unnamed. Per-module: tmz 50 -> 100,
rap 76 -> 98, csu 70 -> 92, oc 90 -> 97, dtm 75 -> 89. Corpus 19,038 -> 19,126 meaningful names
(65.8% -> 66.1%), field share 75.5% -> 75.8%.
RAS disagrees and is deliberately left alone. Its libdump names TA_CreateEntryPoint and
TA_DestroyEntryPoint at addresses the header assigns to OpenSession and CloseSession, a systematic
two-slot shift across all 18 builds. My first run wrote the header names anyway, which left RAS
asserting two different Create entry points -- a contradiction I introduced and then caught by
checking for duplicated roles. A guard now refuses to add a second holder of a role, so the conflict
is reported on every build and nothing is written. No build has a duplicated entry-point role.
Which source is right for RAS stays open: the obvious tiebreaker, argument count, is not decisive on
functions IDA never prototyped.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
down to 7
The thunk naming in the previous commits set row["name"] in this script's metadata only, never in the
database. So the emitted banner read j_asd_clear_link_outputs while the signature inside the body
still read j_sub_D07E10_16. Checking for it found 1,018 functions disagreeing with their own banner,
most introduced by the earlier repair pass rather than the newest one. A reader who sees the two
differ cannot tell which to trust, which is worse than leaving the function anonymous -- and I had
claimed in the last commit message that no j_sub_ names remained in any artifact, which was false.
ta_re.py now pushes every derived name into the database and re-decompiles whatever still disagrees.
Two details were needed, both found by measuring the result rather than trusting the step:
Names must be uniquified first. Several distinct thunks branch to the same target, so j_<target> is
not unique and a colliding rename is silently refused -- on the ASD reference only 55 of 786
reconciled until duplicates were suffixed. Those suffixes are honest, unlike the address-derived _N
names dropped earlier: these are genuinely separate functions with a shared destination.
And the resolution has to run after address-derived names are dropped, since before that point the
rows still carry a stale j_sub_<addr> label and look named, so the pass skipped all of them.
Regenerated asd, gcp, powerplay, xgmi, wspe and ras. Mismatches 1,018 -> 7 corpus-wide; the 7 are ASD
functions where the decompiler emits no signature line this check can read. Meaningful names hold at
19,038 (65.8%), address-derived 0, field share 75.5%.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Dropping the false names last commit left a clean question: of ASD's ~4,500 four-byte anonymous
functions, how many are thunks whose target already has a name? The existing repair pass could not
answer it, because it only rewrites names that already read j_sub_<addr> and these rows had no name
at all.
Decoding the branch from the ELF answers it directly. My first attempt decoded them as ARM B and got
nonsense -- 41 words with top byte 0xB8, 37 with 0xBE, 34 with 0xBD. Those are the second halfword of
a 32-bit Thumb-2 B.W, which is how the wrong assumption announced itself rather than passing quietly.
With the correct decode, across 4,530 such functions: 3,402 target a function that is itself unnamed,
909 target a properly named function, 116 target an address IDA never defined, 103 do not decode as
B.W. The 909 are now named j_<target>. The branch displacement says where control goes, so this is a
resolution and not an inference -- the same justification the existing THUNK tier rests on.
One ordering detail cost a run. The pass has to come after the address-derived names are dropped:
before that point those rows still carry a stale j_sub_<addr> label, so they look named and the pass
skipped all of them and reported zero. The comment in the code says so, since the correct order is
not obvious from reading it.
Measured with fw/ta_metrics.py: meaningful names 18,129 -> 19,038 (62.6% -> 65.8%), address-derived
still 0, field share unchanged at 75.6%. No j_sub_ or j_j_ names remain in any artifact.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
defaults as names
Four metric-definition errors in four commits, each from re-writing the same predicate at a new use
site and getting it slightly too strict. fw/ta_metrics.py now holds the definitions once -- is_addr_name,
is_meaningful, and a measure() that prints every denominator -- and ta_re.py imports them instead of
carrying its own regexes. All four errors are documented in its docstring with the reported and actual
figures, because the pattern is the useful part.
The root cause was in the pipeline, not just the audit. The decompiler-fallback step accepted any
candidate that did not match ^sub_[0-9A-Fa-f]+$, so sub_D07364_12 was admitted as a real name and
1,852 such entries propagated into the artifacts and into every coverage number I quoted.
ta_re.py now drops them, in an order that matters: a j_sub_* candidate is still kept at the fallback
step, because the re-resolution pass needs its target address to recover a real name from it, and
only what remains address-derived afterwards is dropped. Those rows become anonymous and move to the
.anon.c file rather than inflating the count.
Measured with the shared code: address-derived entries 1,295 -> 0, meaningful names unchanged at
18,129 (62.6%), field share unchanged at 75.6%. The drop removes false coverage without losing a
single real name, which is the point -- the artifacts no longer assert that sub_D07364_12 is a name.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
was 6 points high
Measuring collision suffixes in the artifacts showed names like j_sub_D07364_0 through
j_sub_D07364_20. My address-derived regex anchored on $ immediately after the hex digits, so none of
the suffixed forms matched it -- and j_sub_D07364_12 is exactly as address-derived as j_sub_D07364.
1,852 ASD functions were being counted as meaningful names on that basis. The honest figure was
60.2%, not the 66.6% reported one commit ago. The repair pass added in that same commit skipped them
for the identical reason, so 713 thunks with properly named targets sat fixable and untouched.
Both regexes now accept the suffix. In the artifacts: 713 more names recovered, address-derived
entries 2,008 -> 1,295, meaningful names 17,416 -> 18,129 (60.2% -> 62.6%). The remaining 1,272
thunks point at targets that are not functions in their own build, so j_sub_* stays, which is
correct.
This is the fourth metric-definition error in four commits and the largest, and it is worse than the
others because I made it in code written immediately after describing the problem it repeats. Finding
one instance of a class of error is not evidence the class is clear. A regex that defines a quality
metric needs the same scrutiny as one that drives a rename: an over-strict pattern inflates the
number and hides the work it should have exposed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Correcting the headline count last commit exposed 243 functions whose name field was set to an IDA
address default. Looking at what they actually were, rather than treating them as a rounding error,
found a real defect: 220 were j_sub_<addr> thunk names, and 87 of those pointed at targets that by
now carry a proper name.
The cause is ordering. The j_<target> label comes from whatever Hex-Rays printed in the signature at
the moment that thunk was decompiled, so a thunk decompiled before its target was renamed keeps the
stale address forever. Nothing ever revisited it.
decompile_all now re-resolves them from `assigned` after the naming pass. That keeps it a pure
lookup -- the target name is one the pipeline already decided on, so no new inference enters -- and
thunks whose target is still unnamed keep j_sub_*, which is the honest answer rather than a
cosmetic upgrade.
In the artifacts: 87 names recovered across 5 ASD builds, address-derived entries 243 -> 156,
meaningful names 19,181 -> 19,268 (66.3% -> 66.6%). Names like j_asd_build_link_map_full and
j_asd_clear_link_outputs now appear where j_sub_D06CE4 used to.
Worth recording why this was invisible for so long: these functions were being counted as named, so
every coverage number I looked at said they were fine. The metric correction is what made the defect
visible, which is the one case here where fixing a measurement directly produced real improvement
rather than just a more accurate claim.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
to 1
Two errors of mine found by measuring the typing residue per build instead of summed across builds.
First, the per-function table in AUDIT-TA-STRUCTS.md summed each function's dereferences over all
builds, which made the residue look concentrated -- asd_link_sub_C930 appeared to hold 158 when it
holds 14 in any single build. Measured per (build, function) pair, which is what a reader actually
opens, the distribution is flat: median 2, mean 2.7, p90 4, max 33 across 1,360 pairs. The max is
GCP's start, the entry stub, where raw pointer arithmetic is correct and typing would be wrong.
Second, that measurement surfaced one real target, and it had been excluded for a wrong reason. The
recorded note said its dereferences "go through a local (v10), not a1, so typing a1 would not reach
them". The body says "v10 = a1". The note was wrong, and because it read as a settled finding it was
never rechecked -- the same failure mode as the thunks-only and GCP-only claims.
The real disqualifier was the documented padding rule: offsets 0x1-0x3, 0xD, 0xF9, 0x106 and 0x250
landed inside declared pad arrays. Per union-never-intersection those offsets belong in the layout,
so pad00, pad08, pad55 and pad24E are split to name them. Struct size is unchanged at 0x26C and all
ten observed offsets now resolve to named fields, verified by walking the declaration offset by
offset before applying it.
In the artifacts, across all 7 HDCP builds: 24 raw dereferences per build to 1, with 22 typed field
accesses. Corpus named-only field share 74.5% to 75.6%, raw dereferences 3,661 to 3,506. Named count
unchanged, and HDCP's gate still passes.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
canonical entry points
Last commit asserted the log-string lever was GCP-only. That claim rested entirely on the AMD?TA
regex, so I re-checked it by dumping every message-like string from all 16 module binaries. POWERPLAY
logs "CreateEntry: status = 0x%x" and "InvokeCommand: status = 0x%x" -- a second form with no AMD_TA
prefix, naming the function BEFORE the colon rather than after it. The harvester now reads both.
POWERPLAY yields 10 agreements and 0 contradictions, independently confirming its entry points.
That is a small return, but it is the second time re-checking a ceiling claim found the claim wrong,
and the check is cheap compared with the mislabels it prevents.
POWERPLAY also showed that a log name is not automatically the best name. Build 34.0.0.9 had
0x402570 unnamed and its log calls it CreateEntry, while the other five builds name the same address
TA_CreateEntryPoint. Applying the log spelling verbatim named one function two different things
across builds and reduced consistency rather than improving it. Short forms of the five standard
entry points are now aliased to the canonical spelling, so build 34.0.0.9 gets TA_CreateEntryPoint
and TA_InvokeCommandEntryPoint, matching its siblings.
Confirmed in the artifacts: 0x402570 and 0x4026cc in powerplay_34.0.0.9 now read
TA_CreateEntryPoint and TA_InvokeCommandEntryPoint.
XGMI, OPM, MERIT and FP were checked under the bare form too and yield nothing -- their strings are
prose ("GetInfo for NodeId %d") or embedded certificates, not function names.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
my mislabels
I had claimed the unnamed residue was thunks and that no naming lever remained. Checking that claim
against the artifacts instead of repeating it showed only 43% of ASD's unnamed functions are
4-byte thunks, and that GCP's binary contains strings like
AMD_TA_GCP: AllocateDRMSession ENTER
AMD_TA_GCP: GetCryptoSessionForDevice
The leading identifier is the vendor naming the enclosing function. That outranks everything else
here: porting infers from byte similarity, the call graph from position, the dispatcher from a jump
table, but a string inside the body is the author's own label.
fw/ta_logname.py harvests them from the binary, resolves the referencing function through IDA data
xrefs, and applies the name. On GCP's 11 builds: 43 agreements with names already derived, 70
functions named that were unnamed, 193 instances contradicting an existing name.
The agreements are as valuable as the corrections. gcp_cmd01_AllocateDRMSession, derived from the
dispatcher table, is confirmed verbatim by the firmware's log line -- the first independent check
that dispatcher-anchored naming is sound.
Two mislabels of mine, both now fixed:
0x403a90 was cmdFE0F_GetCryptoSessionByDecodeHandle; its log says GetCryptoSessionForDevice
0x4038d8 was cmdFE0A_QueryAccessibilityAttributes__h9; its log says GetCryptoSessionByDecodeHandle
So the name was on the wrong function, and the function that really is GetCryptoSessionByDecodeHandle
had been filed as a fragment of a different command. cmdFE0A is QueryAuthenticatedSession, not
QueryAccessibilityAttributes, and nine __hN "fragments" are separate routines with their own names.
The wrong FE0F name was seeded from fw/libdumps/gcp.json, so it is corrected through LIB_RENAME,
which since the seed-consistency fix rewrites the seed as well as the databases.
A command id is carried onto the vendor name only when the old name had no __hN suffix: that suffix
means the id was inherited from a neighbour and never established for the address, so carrying it
would manufacture a false dispatcher mapping.
Ceiling: only GCP (72 distinct) and WSPE (7) carry these strings; the other 14 modules have none,
established by scanning every build binary. ASD has 41 printable runs, all property keys. Named
19,352 -> 19,422.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
does
The last unadjudicated name contradiction. Eight modules called this function entry_trampoline
while DTM, RAS, HDCP and ASD called the byte-identical function ta_main_loop, and the body settles
it rather than the head count:
while (1) { SVC 0xF0; v6 = *a1; switch (v6) { ... } v7(); }
It is __noreturn, yields to the OS, dispatches on the request code at *a1, calls the handler and
loops forever. That is a main loop. A trampoline does not loop, so entry_trampoline is the wrong
word and the 54 instances under it were the outliers despite being the larger group.
Fixed across xgmi, wspe, gcp, merit, opm, csu, powerplay and auc; no entry_trampoline remains in
any artifact. Counts are unchanged at 19,352 named, 112 placeholders and 67.5% field share, which
is correct for a rename of already-named functions -- it changes what the artifacts say, not what
they cover.
Cross-module name contradictions are now 32 -> 10, of which six are the SVC thunks that are
correctly non-identical, two are legitimate synonym pairs (bzero_core against lib_zero_fill_words),
and two are ASD's own semantic-versus-transport naming. Nothing is left that reading a body can
decide.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
carry no semantics
Adjudicating the twelve cross-module name contradictions produced one correction and one finding
that invalidates the check's premise for a class of function.
Six of the twelve share a body like "{ SVC 0xF2; return 1; }", sometimes with a zero-fill of a
local first. They are trampolines into SYS_DRV, and the operation performed is selected by
registers the CALLER sets before the call -- the body carries no semantic information at all. So
gp_cipher_update being byte-identical to sysdrv_rpc_7, or gp_digest_do_final to gp_mac_update,
says nothing about whether they do the same thing. Each name describes what that thunk's own
caller does, and renaming them to agree would destroy real information and assert a false
equivalence. They are left alone deliberately.
The lesson generalises and is now written into AUDIT-TA-NAMING.md: content-hash reasoning is sound
for self-contained routines and unsound for argument-passing stubs. Every cross-module naming rule
in this tree rests on the former, including the 21 placeholder upgrades, which are all real
routines -- memcmp_be, aeabi_uidiv, the vsnprintf family -- rather than thunks.
One genuine mis-attribution fixed: fp_cmd03__h28 is byte-identical to RAS's aeabi_uldivmod, and
the body is shift-subtract 64-bit division, not a command handler. The libdump had split cmd03
into fragments and labelled this one as part of it. Renamed to fp_aeabi_uldivmod.
The remaining five are documented with a verdict each rather than left as a bare list: two are the
same routine under two legitimate hand-made names (bzero_core against lib_zero_fill_words), two are
ASD's own semantic-versus-transport naming (disp_srm_read against asd_rpc_300), and two are
mis-splits where the correct name is obvious (memcpy, asd_rpc_200) but which live in builds where
renaming by name would need a per-module entry for a single instance each.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Second round on the placeholder clusters, now that the seed-consistency fix makes applying these
safe rather than a net loss. Each reason is recorded because a name is an identity claim:
vsnprintf_out_str_guarded "if (state->field14 == 0) vsnprintf_out_str(a1,a2,a3)" -- a guard on
the same field, calling the writer already named last round
vsnprintf_out_guarded the same shape against *(a1 + 20), which is field14
vsnprintf_fmt_integer switches on the conversion character (a2 == 117, 'u') and calls
vsnprintf_narrow_unsigned
call_once_init "if (*g == NULL) { SVC 0xF3; *g = a1; } return (*g)(...)" -- lazy
one-time initialisation through a supervisor call
runtime_globals returns the fixed global off_4022CC
runtime_globals_field1 returns runtime_globals() + 1
Placeholder names drop from 235 to 112. Named count and field share are unchanged at 66.9% and
67.5%, which is the expected and correct outcome: this renames functions that were already
counted as named, so it improves what the artifacts SAY without moving what they cover.
Seven clusters, 112 instances, are deliberately left as placeholders. lib_33b2ae33_16 writes 2*a2
through a register base the decompiler shows as uninitialised; lib_e6824d52_24 and lib_1bc49253_28
are two near-identical conversion arms I cannot separate; lib_366a1ceb_178 and lib_8680f995_112 are
the two large formatters, visibly handling width and precision but not distinguishable from each
other; lib_d732abea_240 and lib_2cd87843_108 are arithmetic and formatting bodies whose role I
would be inventing. Mislabelling 112 instances is worse than leaving them honest.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
rename bug
Two passes were spent guessing at this and both guesses were wrong -- a defect in the rename path,
then a collision with an existing symbol. Tracing `assigned` through the fix block and the emit for
the build that actually failed settled it in one run.
The seed reference list is built from fw/libdumps, which still carried the PLACEHOLDER names, while
the processed reference build contributed the SEMANTIC names for the same body hash. build_index
then saw one hash mapping to two distinct names, marked those bodies ambiguous and refused to name
them in any build. The tell was in the per-build line all along: ambiguity in 22.0.0.53 read amb=36
against amb=8 at HEAD, and I had been reading the named count without looking at the column next to
it.
Applying LIB_RENAME to the seed names as well makes the pool consistent. 22.0.0.53 returns to amb=8
and 96 named, and the six held-back names can now be applied: isdigit, vsnprintf_getc,
vsnprintf_emit_char, vsnprintf_emit_char_bounded, vsnprintf_narrow_signed and
vsnprintf_narrow_unsigned.
Net across the corpus, both metrics move the right way at once, which is what the earlier attempts
could not achieve: named 19,304 -> 19,352 (66.7% -> 66.9%), placeholder names 397 -> 235, field
share 67.4% -> 67.5%. GCP alone gains 20 names and drops from 143 placeholders to 77.
The temporary tracing is removed and LIB_RENAME_PENDING is gone, replaced by a note recording that
the cause was an inconsistent seed and that it was found by instrumenting rather than reasoning.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
trade-off entirely
Last pass rejected the placeholder-name upgrade because it cost about 1175 typed field accesses:
the shared-runtime typing matched NAME fragments, so renaming lib_e53fbaaa_32 to vsnprintf_setup
moved the function out from under the rule that typed its argument. The fix identified there is
implemented here.
TEE_FMT_HASHES holds the 71 body hashes of that cluster, derived from every function the fragment
rule used to match across all 99 builds so the switch cannot narrow coverage. Content identity is
what established the cluster in the first place, and it is immune to any renaming.
With that in place the trade disappears and both metrics improve together, which is the point:
gcp share 82.5% -> 83.6% placeholders 389 -> 143
wspe share 91.3% -> 92.6% placeholders 252 -> 133
powerplay share 65.3% -> 70.7% placeholders 186 -> 60
xgmi share 69.1% -> 71.0% placeholders 100 -> 55
LIB_RENAME is therefore enabled, with the measured before/after recorded at the gate rather than
a bare "fixed".
Also corrected from last pass: I reported that the rename "does not work" and ruled out ordering,
mapping contents and key matching. The mechanism was fine. Standalone it returned True on the
very functions I claimed it failed on, and the actual cause of the zero-diff was my own shell --
"| grep -m1" closed the pipe, SIGPIPE killed the run before it wrote, and I read that as the code
having no effect. Two passes were spent on a bug that was in my measurement, not the pipeline.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Three correctness audits this pass, and the third produced the work.
Duplicate function names: zero across all 99 builds. The fp_cmd04 duplicate fixed last pass was
the only one.
Contradictory names within a module: zero. 3566 byte-identical bodies were grouped per module and
every group agrees on its name, so the porting is self-consistent.
Contradictory names ACROSS modules: 32 identical bodies carry different base names, and that is
where the value was. Most are not contradictions at all but the same function known two ways --
the TA side knows it only by content hash (lib_52b900d8_88) while ASD's symbol set knows it by
name (memcmp_be). Byte identity makes the equivalence a fact rather than an inference, so the
placeholder loses: 21 clusters resolve to a real name, including memcpy_aligned, memcmp_be,
aeabi_uidiv, udiv10, and the vsnprintf_ family (setup, fill, pad_left, out_str, fmt_hex,
fmt_octal). 509 instances across 51 builds, merged into fw/cgnames so they are durable.
Names still carrying a content-hash placeholder are reported in the commit body rather than left
unstated, and the SHARED_ARG1 fragment list keeps BOTH the old hash fragments and the semantic
ones, because the upgrade is applied through the name map and the tee_fmt_state typing must keep
matching whichever name a given build ends up with.
Eleven clusters were deliberately NOT upgraded because the two names genuinely contradict rather
than one being a placeholder: gp_cipher_do_final against sysdrv_rpc_10, gp_cipher_update against
sysdrv_rpc_7, entry_trampoline against ta_main_loop. One of each pair is wrong and byte identity
cannot say which, so both are left standing rather than picking the one that reads better.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
DISPATCH names
An earlier pass reported that all 99 artifacts regenerate byte-identically. That was true of that
one run and does not hold in general. A full clean regeneration of all sixteen modules changes 220
files, and the cause is IDA's auto-analysis not being identical across fresh databases: AUC's
function count comes back 39 instead of 40, and decompiled text varies cosmetically
("(int)nullsub_3" against "nullsub_3"). The claim is corrected here rather than left standing.
Semantically the drift is small -- 4 names of 19,353 -- but all four were tier DISPATCH, which
exposed a wrong decision. fw/ta_snapshot_names.py excluded DISPATCH on the grounds that
fw/ta_re.py re-derives it every run. It does, but not deterministically: dispatcher recovery
depends on which function Hex-Rays decompiles cleanly on a given analysis, so xgmi_cmd00,
xgmi_cmd04, xgmi_cmd0B and fp_cmd04 simply did not come back. DISPATCH is now persisted, and the
exclusion reasoning in that file is rewritten to say why.
Persisting it still cannot mask a regression: the dispatcher step runs regardless, so where
recovery works the DISPATCH tier wins and where it fails the restored name appears as tier IDA.
The tier difference is itself the signal that recovery failed on that build.
7817 names were recovered from the committed artifacts into fw/cgnames so nothing derived is left
depending on a database that may not reproduce. XGMI came back whole.
The one name that did NOT come back turned out to be a defect being fixed rather than lost. HEAD
had TWO functions called fp_cmd04 -- 0x401334 from the dispatcher and 0x402b4c from byte porting
-- and the current dispatcher map routes command 4 to 0x402b4c, agreeing with the ported name. So
HEAD carried a duplicate, this run does not, and the allow_overwrite=False guard on the re-apply
is what refused to restore the bad one. Worth keeping: a name count going DOWN was the correct
outcome, and reading it as a regression would have reintroduced the duplicate.
Verified state after the clean regeneration: 16/16 gates PASS, 99 builds, 19,349 of 28,945 named
(66.8%), field share 66.4%, 21 decompile failures.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
gcp_cmdFE0B_DestroyAuthenticatedSession is the correct version of what FE0D gets wrong, and it
now carries the same local names so the two can be read side by side.
Both have the same shape, read off the code: handle from params->arg10, a bound check, a slot
pointer computed from the handle's top byte, and a stored-handle comparison. FE0B uses stride 80
over the auth array -- slot+192 dwords is ctx+0x300, +198 is that slot's state at +0x18 -- and
bounds the handle with "handle >> 28 == 0" for 16 slots. FE0D uses stride 88 over the crypto
array at ctx+0x800 and bounds with ">> 30" for 64.
The difference that matters is positional, and naming makes it visible without reading the
finding: in FE0B the refcount decrement and the ASD release sit INSIDE the branch where the
stored handle matched, so a rejected handle changes nothing. FE0D reaches both regardless of its
status. Verified by locating the lines rather than by eye -- FE0B's release follows its
handle-check inside the same branch.
gcp_cmdFE02_GetObfKeyCryptoSession gets only handle and slot_idx. It is long and most of its
locals have no established role, so the rest stay vN rather than being given plausible-sounding
names; a wrong name in a handler that is already a published lead is worse than no name.
Corpus figures are flat at 66.9% named and 66.2% field share, as expected: this is six functions
of hand work, and it does not move a 29,000-function average. The point is that the six functions
carrying recorded hardware findings now read as C rather than as register soup.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
overflow site
hdcp1_first_part_authentication carries the HDCP overflow that AUDIT-TA-MEMSAFETY-HUNT records
as confirmed on hardware at 0x402cde, with the copy length taken from a byte struct field. It
was still "int a1" with eleven vN locals.
hdcp1_auth_req is derived from the accesses across all seven builds: a dword at 0x00, a dword at
0x04, bytes 0x08..0x0E and halfwords at 0x10 and 0x12, closing at 0x14. The block is what the
invoke entry passes as v7 + 12, i.e. hdcp_req.argA, which is consistent with the request layout
already derived.
The first dword is named session_idx on direct evidence rather than by analogy: the handler reads
it, bounds it with "<= 7", and then indexes a stride-652 table as
"dword_40237C + 652 * session_idx". Eight sessions of 652 bytes.
Typing a2 was already in place from the earlier argN work; a1 now joins it, and
hdcp1_first_part_authentication's locals session_idx and status are named.
Recorded limitation rather than smoothed over: the documented bug site loads its length from
[r5, #0x38], and r5 is NOT v12 in this build -- register allocation differs between the build the
memsafety audit used and the reference here. The local names are therefore keyed on what each
local demonstrably does in this build, and the two that do not correspond are left alone rather
than mapped by assumption.
Also honest about the measured effect: HDCP's field share moves only 70.0% to 70.2%, because
Hex-Rays keeps the explicit "*(_DWORD *)a1" cast at several of these sites even with the pointer
typed, so the signature improves while those particular lines do not. The win here is the
signature and the session_idx/652-stride relationship being legible, not the metric.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With the structures typed and the stale-cache bug fixed, the handlers a reader actually goes to
were still a wall of vN, which hides the defect even when every field around it is named. These
three carry recorded hardware findings, so they are the ones worth doing by hand.
Every name is read off the code rather than taken from the finding text:
gcp_cmdFE0C_CreateCryptoSession (acquire-gate bypass). slot_handle_bits accumulates 0x1000000
per slot and stops at 0x40000000 -- the slot index in bits 24+ over crypto[64], which is the
finding's "idx << 24". refcount_before is *p_asd_refcount read BEFORE the increment; the
acquire runs only when it is zero and is not rolled back when it fails, so that one value IS
the bug. handle_counter supplies the handle's low 24 bits, acquire_status is the
gcp_asd_rpc_305 result returned as status | 0x20000000.
gcp_cmdFE0D_DestroyCryptoSession (refcount desync). slot is (char *)ctx + 88 * HIBYTE(handle),
stride 88 over the crypto array, which is why slot+512 dwords is ctx+0x800, the array base,
and +520 is that slot's state at +0x20. status is 0 only when the stored handle matches, yet
refcount_after and release_status are reached regardless -- the ordering is now visible in the
names.
gcp_cmdFE0F_GetCryptoSessionByDecodeHandle (stale slot lookup). remaining counts 64 slots down
while comparing decode_handle and never testing the slot's state, so the unconditional walk
reads as one.
Applied through a LOCAL_NAMES table in fw/ta_ida.py so they survive a database rebuild like
every other derived name, and applied after all renames and types but before the cache
invalidation so the emitted text actually contains them. 12 of 12 on the reference; the builds
that take 6 of 7 are missing one variable whose register allocation differs, which is expected
when naming by vN identity across builds and is not forced.
Two avenues were measured and rejected first. Fielding the opaque body[] interiors of merit_req
and opm_req looked like the obvious fix for their 0% and 19% field shares, but merit's 164
dereferences are all in UNNAMED functions and both offset sets are full of unaligned offsets
(0x3, 0x6, 0x9, 0xa, 0xe, 0x1a, 0x22), so a single interior layout would be several distinct
layouts conflated -- the same error already rejected for the POWERPLAY fptbl union.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two GCP handlers were mislabelled in the libdump symbol set and the mislabels had been
propagating into every artifact. The duplicate name was the tell: FE0B and FE0D were both called
DestroyAuthenticatedSession.
The code settles it without needing the documents. gcp_cmdFE0C iterates ctx->crypto, zero-fills a
0x58 gcp_crypto_session and sets crypto->state and crypto->key, so it creates a CRYPTO session.
gcp_cmdFE0D computes "(char *)ctx + 88 * HIBYTE(arg10)" -- stride 88, the crypto array at
ctx+0x800 -- while gcp_cmdFE0B uses stride 80, the auth array at ctx+0x300. AUDIT-NEW-TAS-RE.md
states exactly that split, and the hardware findings agree: FINDING-GCP-ACQUIRE-GATE-BYPASS
records 0xFE0C as CreateCryptoSession.
Worth distinguishing from the FE07 case: that one claimed the wrong COMMAND and the dispatcher
cross-check caught it automatically. These two claim the right command and the wrong SEMANTICS,
so no command-id check can see them -- only reading what the code touches. The corrections are
applied by name through a RENAME_FIX table so they hold for every build.
While verifying them, the dispatcher kept printing the old callee name in four builds after the
rename had demonstrably succeeded. Hex-Rays caches pseudocode per function and
remote_ops.rename refreshes only the function it renamed, never its callers, so a stale cache was
reading as a failed rename. Every function's cached text is now invalidated before the artifacts
are emitted.
That turned out to matter far beyond the two names. Applied TYPES were also being hidden behind
stale text: GCP's field share goes 59% to 83.4%, its typed field accesses 1716 to 3081 and its raw
dereferences 1193 to 612, with no new structure added -- the types had been applied all along and
the emitted artifacts were showing pre-typing output. WSPE lands at 91%, HDCP 70%, XGMI 69%,
POWERPLAY 67%. Corpus field share 65.9% -> 66.4%, held down by ASD, which dominates the corpus and
barely moved.
This is the sixth consecutive pass whose main finding was that something already done was not
visible in the output. The remaining low scores are now informative rather than suspect: merit at
0% and opm at 19% are low because their structures are mostly opaque body arrays, not because
typing failed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
reach
Two of the three largest remaining dereference groups in named functions were not blocked by
analysis at all: hdcp2_process_auth_message_v2 with 161 and hdcp1_create_session with 49 both take
their structure as the SECOND argument, and every typing path built so far rewrites only the first
parameter. retype_arg1 is generalised to retype_arg with a 1-based index, with the old name kept as
an alias so the existing call sites are untouched, and a per-module ARGN_BY_TA table drives it.
hdcp2_auth_msg is contiguous dwords 0x04..0x24 followed by an 8-byte read at 0x28, which is
8-aligned, so it closes at 0x30.
hdcp1_session_req is sparser, and its 8-byte read at 0x04 is NOT 8-aligned. It is declared as two
dwords rather than a 64-bit field: declaring the wide read as one member would force alignment
padding and shift every later offset, which is the same trap asd_link_cfg's unaligned 0x14 qword
read presented. Size verified at 0x20.
HDCP field share 62.8% -> 68.9%, corpus 65.2% -> 65.9%, raw dereferences 7192 -> 6982.
Arg-based typing in named functions is now effectively exhausted, and the numbers say so rather
than my asserting it: the total has gone 2201 -> 1507 -> 1163 across three passes, and every group
still standing touches one or two offsets. A structure for a function that reads two offsets is a
one-field structure, which buys nothing a raw offset did not already show, so the remaining
dereferences are deliberately left alone rather than dressed up.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Re-running the concentration analysis after last pass's fix left
hdcp2_rx_RepeaterAuth_ReceiverID_List at 135 raw dereferences while its signature plainly showed
"struct hdcp2_session *a1". Both facts were true: it is typed in 4 of 7 builds and unnamed in the
other 3, and the name-keyed rule cannot reach a function it cannot find by name.
A signature rule for exactly this case was measured when hdcp2_session was first derived -- 3
unnamed functions, 135 dereferences, zero false positives among the named -- and then never
implemented. It is now wired in through a generic SIG_BY_TA table rather than another
module-specific branch, since ASD already had its own copy of this idea.
The predicate requires every a1 offset to fall inside the structure's declared FIELDS rather than
anywhere within its extent. Accesses landing in padding are treated as disqualifying, because
padding means the layout does not describe that function -- which is the check that would have
caught the hdcp2_tx_RepeaterAuth_Stream_Manage mistype earlier without needing to be reasoned
about by hand.
Checking that condition also found two offsets, 0x222 and 0x224, that real accesses touch while
hdcp2_session had them as padding. Both are now fields; 0x224 is a single byte rather than a
halfword because 0x225 begins the next run, and the size is re-verified at 0x26C.
Two process notes worth keeping. The insertion failed silently the first time -- the anchor text
did not match, the assertion fired inside a heredoc whose output was swallowed by the following
command, and the pipeline then ran happily without the new rule. It was caught only by grepping
for SIG_BY_TA in the generated file rather than trusting that the edit had landed, which is the
same class of silent no-op as the last four passes.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The biggest single typing target in the corpus, asd_program_link_by_signal with 504 raw
pointer-arithmetic dereferences, had been counted as typed since the ASD structures were added.
It was not. Re-running the concentration analysis after the naming work -- which moved many
dereference-heavy functions out of ANON and into named groups -- put it straight back at the top
of the list, which is what prompted looking again.
Cause: the name-keyed arg1 block still carried its own inline copy of the prototype rewrite,
written before retype_arg1 was factored out. The inline copy does not strip comment lines before
locating the signature, and fw/ta_re.py applies recorded comments earlier in the same run, so on
any function carrying a comment the head parse ran across the comment text and produced a
declaration IDA rejected. The failure was invisible because the surrounding counter only counted
successes and the missing ones looked like functions absent from the build -- the reference
reported "12/16" for exactly the number of family members it lacks.
Switching that block to the shared helper takes the reference from 12/16 to 16/16 and converts
ASD from 35% to 59.5% field share on its own. Corpus field share goes 59.5% -> 64.6%, with raw
dereferences 8481 -> 7297.
Two avenues were measured and rejected first, so they are not retried: seeding ASD's weaker
builds from its best-named CORPUS build rather than the handover reference yields 1-4 names per
build at a much worse mismatch rate (15-35 against 0-4), because the handover names have already
propagated; and the structural residue is unchanged.
Note the struct-typed signature figure moves DOWN, 6.5% to 4.1%, purely because the denominator
grew: naming added thousands of previously-ANON functions to the emitted set. It is a ratio
artefact, not a regression, and is the clearest demonstration yet of why the field share is the
measure worth tracking.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
files
This pass fixes a fragility that was actively destroying work rather than adding coverage.
Four IDA databases have now come back corrupted with "worker exited with status 1", and the only
remedy is to delete and rebuild them. Every name derived by fw/ta_cgname.py lived solely in those
databases, so a rebuild discarded them -- GCP silently fell from 83% to 74% named when I wiped its
databases for the dispatcher fix, and I only noticed because I happened to re-measure per module.
Worse, those names cannot be re-derived: once a callee has a name it stops being a sub_* candidate,
so re-running the tool on an already-processed build reports nothing to do.
Three changes:
* fw/ta_cgname.py persists every name it applies to fw/cgnames/<ta>.json, keyed by build and
address.
* fw/ta_re.py re-applies that map at the start of each build, before anything reads names, so a
rebuilt database comes back with its derived names intact.
* fw/ta_snapshot_names.py recovers the names applied before the map existed. They are read out
of fw/decomp/*.index.json, which records name, address and tier per function, and only the
IDA tier is snapshotted. EXACT, MNEM, FUZZY and DISPATCH are deliberately excluded because
fw/ta_re.py re-derives them every run; persisting those would bake a derived result into the
repository and hide a future regression in the porting itself.
1476 names are now durable across 15 modules, including ASD's 847, which would otherwise have
needed a 40-minute wipe-and-rederive to recover.
Verified by doing the thing that caused the loss: XGMI's databases were deleted and rebuilt. It
came back at 238 of 334 named against 239 before, so a full database loss now costs one function
instead of nine percentage points. The single remaining loss is not chased down here, but it is
recorded rather than rounded away.
GCP is restored to 84% and HDCP re-derived cleanly to 86% with a complete map. Corpus naming
stands at 67.0%.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rather than chase another percentage, this pass audited whether the names already applied are
CORRECT, using the only authority available: the dispatcher. For every build, each handler whose
name claims a command id was checked against the command the dispatcher actually routes to it.
257 handlers checked, 246 agreed, and 11 conflicted -- all in GCP, all of the form
gcp_cmdFE0A_QueryAccessibilityAttributes__hN being routed from commands 0x8, 0x9, 0xA and 0xF.
Those are not GCP commands at all. They are arms of 0xFE0A's own sub-command switch, absorbed as
top-level commands because dispatch_map merged EVERY switch-bearing callee at the shallowest
depth. The handler names were right; my command map was wrong, which is the opposite of what the
conflict report implied.
The merge is replaced by taking only the richest switch at the shallowest depth. The union was
introduced to allow a TA that splits low ids and 0xFExx ids across sibling switches, but GCP
serves both from one function, so nothing is lost: the reference still recovers 14 commands, all
14 still match fw/cmdmaps/gcp_ea.json address for address, and the phantom 0x8/0x9/0xA/0xF are
gone. The three still missing (0x3, 0xFE08, 0xFE0E) remain the inlined arms with no callee, as
before.
Re-running the audit afterwards: 332 handlers checked, 332 agree, zero conflicts. The FE07 case
that had been reported since the cross-check was built is also clear, because the pipeline now
relabels that handler from gcp_cmdFE03_* to gcp_cmdFE07_* rather than leaving the inherited
misnomer in place.
Two more corrupted databases were found and rebuilt (hdcp_23.0.8.49, merit_24.0.4.42), with the
same "worker exited with status 1" symptom as the gcp one last pass. Rebuilding loses the
call-graph names on those builds, so ta_cgname was re-run for both; naming lands at 66.5% rather
than 66.9% because of that churn.
Also measured and rejected this pass, so they are not retried: cross-module naming by exact body
hash yields 77 names, and by mnemonic sequence 170 (against 1496 that match an ambiguous cluster
and cannot be resolved). Together under 2% of what remains unnamed, which is not worth the
regeneration cost.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The tool built last pass was only ever run on asd and ras. It is generic, so it is now run on the
rest, and the mismatch-rate control holds everywhere -- 0 to 4 mismatched anchors against 16 to 83
aligned, which is why the results are applied rather than argued about.
Yields are very uneven and that is informative in itself:
hdcp 150 names 65% -> 86% named
gcp 18 + a partial earlier run 74% -> 83%
merit 20 69% -> 74%
opm 13 58% -> 60%
xgmi 13 67% -> 72%
powerplay 5 68% -> 69%
wspe 4 82% -> 83%
HDCP and GCP pay because their corpus builds are close to their reference; powerplay and wspe
were already well covered by byte porting, so there was little left for the call graph to add.
Corpus overall: 19,377 of 28,947 named, 66.9%, up from 65.8% and from 35.4% when this work
started. Tiers now 7813 THUNK, 7503 EXACT, 2261 MNEM, 1424 IDA, 194 DISPATCH, 182 FUZZY.
One operational note worth recording: the gcp run aborted partway with "Failed to open database
gcp_22.0.0.71_a4b4fff1.elf.i64". Builds processed before the abort had already been saved, so the
retry found them named and reported only the remainder -- the 18 figure above understates what gcp
actually gained. Clearing the unreadable database and re-running fw/ta_re.py rebuilt it; nothing
was lost, but a partial-save interruption is a state this flow does not detect on its own.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
every failure
Byte and mnemonic porting only names a function whose own body still resembles the reference.
A function's position in the CALL GRAPH is independent evidence: if reference function A matches
target A' and both make the same number of calls, the Nth callee of A' is the Nth callee of A.
fw/ta_cgname.py applies that, with three restrictions chosen because positional alignment is
exactly the sort of heuristic that quietly invents names: the call counts must be EQUAL or the
anchor is skipped entirely rather than aligned partially; every anchor reaching a target must
agree on the name, or it is left alone; and the candidate must be within a factor of two of the
size of the function whose name it is taking. Rounds iterate, so a function named in one round
becomes an anchor in the next.
The anchor mismatch rate is reported as its own control, because a high rate means the builds
have diverged too far for position to mean anything. For ASD it is 4-8% (9-17 mismatched against
63-269 aligned), which is why the result is trustworthy. For RAS the same tool finds only 4-12
usable anchors and produces nothing at all -- its reference shares too little with the corpus
builds, and that is reported rather than worked around.
843 names applied across ASD's 17 non-reference builds, taking ASD from 62.5% to 66.0% named and
the corpus overall past two thirds.
Found and fixed while debugging why the first run applied zero: the rename response is
{"func":[...], "summary":{"ok":N,"failed":M}}, and both this tool and fw/ta_re.py were looking
for a "result" list that does not exist. In ta_re.py that meant the rename-error counter reported
zero unconditionally for the entire effort -- it could never have told me about a failed rename.
Both now read the summary. The symptom was visible only because this tool counted successes
rather than failures; the ta_re.py version failed silently in the safe direction, which is why it
went unnoticed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
functions named
The artifacts were full of ANON blocks that did not need to be. This pipeline treated a function
as named only if ITS OWN porting had assigned a name, and threw away the name the decompiler
states in the signature. For ASD that was most of the module: 11,737 of its unnamed functions are
4-byte thunks, and Hex-Rays resolves thunks and prints 'j_<target>' in the signature even where
the stored symbol is still sub_*, so 7,812 perfectly good derived names were being written out as
"ANON size=4".
The fix takes the name from the decompiled signature whenever porting produced none and the name
is not a bare sub_*. This is IDA's own thunk resolution rather than an inference, but it is
recorded as a distinct tier -- THUNK for the j_ names, IDA for the rest -- so it never passes for
a ported identity.
Named functions across all 99 builds: 10,259 of 28,945 (35.4%) -> 18,204 of 28,945 (62.9%). ASD
alone goes from 29% to 62%. Tiers: 7508 EXACT, 7812 THUNK, 2269 MNEM, 303 DISPATCH, 179 FUZZY,
133 IDA.
Two measurements on the way that corrected my earlier reasoning about where the naming gap was:
* the ASD reference has only 46 ambiguous bodies involving 109 functions, not the 677-908 the
per-build logs report. The per-build figure counts ambiguous TARGETS, and ASD's are
overwhelmingly tiny identical thunks -- which is also why disambiguating them by address
order was never worth its 12% error rate. They are now named from their jump targets instead,
which needs no tiebreak at all.
* 70% of ASD's reference names embed an address (asd_hdcp1_sub_1144, asd_svc_sub_273E), so they
carry subsystem information but claim an offset that is wrong in any other build. Worth
knowing before relying on a ported ASD name.
Log-string naming was also measured as an alternative and is small by comparison: only 98 unnamed
functions across the corpus reference a string literal at all (86 in GCP), and ASD logs numeric
codes rather than names, so the firmware's own log lines cannot name it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
gcp_access_attrs covers a1 of gcp_cmdFE0A_QueryAccessibilityAttributes__h8 (176 dereferences) and
__h10. It is byte runs at 0x00..0x07, 0x40..0x43 and 0x5C..0x5F with a single 8-byte read at 0x20,
so it is a tag/blob structure rather than a field record, and the readability gain is honestly
smaller than for the others: a1->tag40[1] instead of *(a1 + 0x41). The regions get names and
extents, which is the whole of it.
A mistyping avoided and now recorded: hdcp2_tx_RepeaterAuth_Stream_Manage holds 164 dereferences
through a local, v10, which is a direct copy of a1 -- so typing a1 WOULD have propagated to it, and
my earlier reason for excluding it ("they go through v10, not a1") was wrong. Checking its offsets
rather than its mechanics gave the real answer: 0x1, 0x2, 0x3, 0xD, 0xF9 and 0x106 land in
hdcp2_session's PADDING, so it is a message byte buffer and typing it would have misdescribed 164
lines. Right exclusion, wrong reason, now corrected.
The audit file documented three structures while twenty were declared. It now records all five
internal structures alongside the host requests, and three things that are more useful than the
list: the three cases where a structure derived from a SUBSET of its users came out wrong and how
each was caught, the truncation audit that found nothing together with why a clean result there is
weak evidence, and the controls behind each signature-based typing rule.
Also measured and rejected as a rule: typing locals that are plain copies of an argument. Across
every module that is 380 dereferences, and the largest single case is the Stream_Manage mistype
above, so it is not a multiplier.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
truncation
Two parts, both driven by the same recurring failure: a layout derived from a subset of its users
comes out wrong.
First, a truncation audit of all 24 declared structures, since asd_link_cfg and then hdcp2_session
had both been understated. For each module the maximum offset still reached by a raw argument
dereference across every build is compared against the declared size. Nothing is over its bound --
OPM's 0x341C sits inside its 0x3D48, HDCP's 0x264 inside 0x26C -- so the request structures hold.
Two apparent anomalies are artefacts of the audit script rather than defects: gcp_ctx and
merit_table compute as 0xC and 0x0 because the size walker does not account for nested structure
arrays.
Second, hdcp2_session needed revising for the third instance of the subset problem. Its first
version came from the three heaviest rx handlers. Adding the transmit-side handlers to the union
exposed two errors in it: 0x2C is read as a DWORD where the rx-only union had left byte padding,
and 0x246..0x24C is a byte run that had fallen inside pad235. Both are now fields, and field18
widens to eight dwords covering 0x18..0x34. The merge also had to resolve a genuine overlap --
0x2D is read as a byte inside the dword at 0x2C -- by giving dword coverage precedence, since a
narrow access into a wider field is ordinary rather than evidence of a separate member.
Three transmit-side handlers are now typed by name as well: hdcp2_tx_AKE_Init,
hdcp2_tx_SKE_Send_Eks and hdcp2_rx_AKE_Send_H_prime, whose offsets are covered by construction
because they contributed to the union. hdcp2_tx_RepeaterAuth_Stream_Manage is deliberately left
out despite holding 164 dereferences: they go through a local, v10, not a1, so typing a1 would not
reach them and claiming otherwise would be wrong.
A signature-based rule for HDCP was also built and measured before being kept small: it selects 3
unnamed functions holding 135 dereferences with zero false positives among the named ones, which is
honest but minor next to the by-name typing.
Measured across fw/decomp: HDCP went from 53 typed field accesses against 1881 raw dereferences to
1365 against 1039. Overall typed field accesses 11436 -> 11971, raw 8733 -> 8481, field share
56.7% -> 58.5%, struct-typed signatures 6.4%.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
An earlier pass examined HDCP's three heaviest handlers -- hdcp2_rx_AKE_Send_Pairing_Info,
hdcp2_rx_RepeaterAuth_ReceiverID_List and hdcp2_rx_AKE_Send_Cert, 590 dereferences between them
-- took their INTERSECTION, found only two offsets in common, and recorded them as three
unrelated message layouts not worth a shared structure.
That was the wrong test, and it is the same mistake that made asd_link_cfg come out at half its
real size. Handlers over a shared context each touch a different subset of fields, so the
intersection shrinks as handlers are added while the union is what actually describes the
structure. They do share 0x04, 0x10, 0x18 and 0x24, and two of them share the byte run at
0x107..0x10E, which the intersection of all three hid.
The union across all builds gives 56 observed offsets from 0x04 to 0x268, with each field's width
taken from the accesses themselves rather than assumed: dword runs at 0x18..0x28, 0x30..0x34,
0x3C..0x48 and 0x254..0x268, byte runs at 0x4F..0x54, 0x107..0x10E and 0x225..0x234, halfwords at
0x192..0x198, and single flag bytes at 0x2D, 0x38 and 0x24D. Unobserved spans are declared as
padding rather than invented as fields, and the declaration is generated programmatically so the
run arithmetic closes at 0x26C by construction instead of by hand.
CAVEAT recorded in the declaration: 0x26C is the extent observed, not a bound the firmware
checks, and the padding runs are genuinely unknown rather than known-unused -- precisely the
situation that understated asd_link_cfg, so a richer build may extend this too.
Two pieces of plumbing were ASD-specific and are now generic, since HDCP is the second module to
need them: load_type_decls already read <TA>_INTERNAL_DECLS, so the structure is declared under
that name, and the name-keyed arg1 map is now looked up per module through ARG1_BY_TA instead of
being hardcoded to ASD. The gate for that block silently failed to be rewritten on the first
attempt and was left as 'if ta == "asd"', which is why the typing did not fire until it was
checked rather than assumed.
Applied, the handlers read a1->field225, a1->field18, a1->field107 and so on, 3 of 3 functions on
most builds.
Measured across fw/decomp: typed field accesses 10789 -> 11436, raw pointer-arithmetic
dereferences 9323 -> 8733, field share 53.6% -> 56.7%, struct-typed signatures 6.0% -> 6.2%.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
as a second check
First, a truncation audit of the structures already declared, prompted by asd_link_cfg turning
out to be half its real size. The check looks for a variable that has BOTH field accesses and
leftover raw dereferences, which is what an undersized structure produces. Across every module
and build there are none, so no other declared structure is being overrun in the artifacts. Worth
noting the limit of that check: it could not have caught asd_link_cfg either, because the
functions that overran it were unnamed and therefore untyped, with no field accesses to compare
against.
The gain this pass is 88 functions across gcp, powerplay, wspe and xgmi that dereference a1 over
exactly {0x08, 0x18, 0x1C, 0x20} -- a subset of tee_fmt_state -- but carry no hash-cluster name,
so they appear as sub_*/ANON and the name-keyed rule could not reach them. 802 dereferences.
Four offsets alone are a weak signature and were not trusted on their own. SIZE is required as a
second discriminator: a candidate must be the same byte length as a function that IS a confirmed
cluster member. 86 of the 88 satisfy both conditions, and the four sizes involved -- 112, 178,
606, 758 -- all occur among confirmed members. The contrast with the ASD rule is deliberate: there
a fifteen-offset signature was specific enough alone, here it is not, and the code says so rather
than reusing the same threshold blindly.
Measured across fw/decomp: typed field accesses 9058 -> 10789, raw pointer-arithmetic
dereferences 10107 -> 9323, field share 47.3% -> 53.6%, struct-typed signatures 5.7% -> 6.0%.
More than half of all structured memory access in the artifacts is now through named fields.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Chasing where ASD's remaining dereferences actually lived found a defect in my own structure
rather than a new one to add.
The residue was in UNNAMED functions -- 94% of their dereferences in the top 20, the two heaviest
being 6102 and 3906 bytes with 692 and 562 dereferences. Inspecting their offset sets showed they
touch every dword from 0x04 to 0xB4 with no gaps, including the four offsets asd_link_cfg
declared as unused padding (0x10, 0x1C, 0x3C, 0x50). Those gaps were an artefact of the REFERENCE
build's narrower functions never touching them, not a property of the structure. asd_link_cfg is
now the contiguous dword array the evidence actually shows, 45 observed offsets, size 0xB8 rather
than 0x58.
That is a useful warning about deriving a layout from one build: the first version was internally
consistent and verified by read-back, and still understated the structure by half, because the
evidence available in the reference was simply thinner than the evidence available across builds.
Typing those unnamed holders needed a rule that is not a name. Guessing names for them would
assert an identity the evidence does not support; typing only claims the pointer's shape. The
predicate is deliberately narrow -- a function qualifies only if its a1 offsets COVER all fifteen
core link offsets at once -- and its control is that on the reference's 1440 functions it selects
exactly one, asd_program_link_by_signal, with zero false positives. A looser rule such as "subset
of a contiguous dword range" would match almost anything and is explicitly not used.
Two avenues were tested and rejected on the way, both recorded so they are not retried:
* disambiguating the 677-908 functions skipped as ambiguous per ASD build by address ORDER.
Order is only ~88% monotonic against the reference across six builds, so roughly one in eight
names would be wrong. A wrong name is worse than no name here.
* the ambiguous set is in any case not where the dereferences are: functions under 32 bytes
hold 16 of them.
Measured across fw/decomp: typed field accesses 7327 -> 9058, raw pointer-arithmetic dereferences
11753 -> 10107, field share 38.4% -> 47.3%.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two things I proposed last turn turned out to be worth less than claimed, and checking them was
most of this pass.
The first was wrong outright. I said the shared TEE formatting cluster was unnamed in ten modules
and that running the libnames clustering would let one declaration cover them. It would not: the
libnames clusters are ALREADY applied everywhere -- 225 matches across the twelve libdumps with
zero remaining sub_* among them. The formatting cluster simply does not exist outside gcp,
powerplay, wspe, xgmi and ras, so there was nothing to name and nothing to gain.
The second avenue is now measured and exhausted. Of the 55 shared clusters, only three others
dereference a1 at all -- lib_3f07c814_10, lib_78974a43_20, lib_71f66102_20 -- each at a single
offset, 56 dereferences between them. A one-field structure buys no readability, so no more
shared structures are worth declaring.
What did pay was enumerating the ASD link and caps families across all 18 builds rather than
across the reference alone, which found five members the earlier list had missed:
program_link_by_signal_v5, encoder_enable_seq_v2, clear_link_outputs, get_link_caps_full_v4 and
_v5. Each was verified to use only offsets that are a strict SUBSET of the structure it is given,
rather than inferred from its name. asd_topology_update is deliberately excluded: it dereferences
a2 over a different offset set, so neither structure describes it.
Also worth recording from the measurement, because it changes where the remaining work is: ASD's
arg-based dereferences in NAMED functions are now 174, down from 6314. Its remaining 4817 a1
dereferences are in UNNAMED functions, which no structure can reach until the functions
themselves are identified. The residue is a naming problem, not a typing one.
Measured across fw/decomp: typed field accesses 7101 -> 7327, raw dereferences 11849 -> 11753,
field share 37.5% -> 38.4%. Struct-typed signatures unchanged at 5.7%, as expected for a change
that retypes arguments of already-named functions.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rather than pick the next target by hand, the artifacts were measured for where raw dereferences
through function ARGUMENTS concentrate. That surfaced something better than another per-module
struct: the same offset set (0x08, 0x18, 0x1C, 0x20) appears through a1 in gcp, powerplay, wspe
and xgmi, in functions whose names are content-hash clusters -- lib_e53fbaaa_32, lib_7fca7c43_82,
lib_8680f995_112, lib_366a1ceb_178.
RAS is what identifies it. There the same cluster carries real names -- vsnprintf_fmt_int,
vsnprintf_fmt_ll, vsnprintf_pad, vsnprintf_prefix -- and adds 0x10, 0x14 and the tail 0x24 byte,
0x25 byte, 0x26 word, closing the layout at 0x28. It is the runtime's string-formatting state.
Declaring ONE shared type here is sound where the powerplay ack-word guess was not, and the
difference is worth stating: those "_<hash8>_" names come from ta_libmatch.py, which accepts a
cluster only when the function bytes match EXACTLY across modules. The functions are literally
the same code, so they necessarily share the structure. The earlier ack-word generalisation
rested on a resemblance and was wrong.
Applied via the first-parameter rewrite, so each function keeps the rest of its recorded
signature: 24 functions in RAS, 6 each in wspe and xgmi, 4 each in gcp and powerplay.
HDCP was examined and deliberately NOT given a shared struct. Its three heaviest functions --
hdcp2_rx_AKE_Send_Pairing_Info, hdcp2_rx_RepeaterAuth_ReceiverID_List and hdcp2_rx_AKE_Send_Cert,
590 dereferences between them -- share only two offsets, 0x18 and 0x24. They are distinct HDCP2
message layouts, not one structure, and forcing a common type would have misdescribed all three.
Honest limit on the reach of this change: the cluster is only NAMED in five modules. In hdcp, opm,
merit, auc, fp, rap, csu, oc, dtm and asd the same helpers exist but are still sub_*, so the
fragment-keyed typing cannot attach to them. Running the libnames clustering across those modules
first would let the same single declaration cover them too -- the clearest next step.
Measured across fw/decomp: typed field accesses 5246 -> 7101, raw pointer-arithmetic dereferences
12701 -> 11849, field share 29.2% -> 37.5%, struct-typed signatures 4.8% -> 5.7%.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Request structures were done for every component, but ASD still held 8446 of the raw
pointer-arithmetic dereferences, and typing its dispatcher had moved only a few hundred. So
rather than guess where the rest lived, it was measured: of those 8446, 6314 go through a1, 86%
of those sit in just 20 functions, and only 54 distinct functions touch a1 at all. The
unreadability is not spread across 6781 functions; it is concentrated in a handful.
A first attempt at clustering by name prefix was a dead end worth recording. The largest
cluster, asd_reg_* with 145 functions, has ZERO a1 structure accesses -- they are register
accessors that take an address and a value, not a context. Clustering by subsystem name says
nothing about whether a context structure exists.
The two families that do matter were found by offset signature instead:
asd_link_cfg the link/encoder descriptor, from asd_program_link_by_signal{,_v2,_v3,_v4}
and asd_encoder_enable_seq{,_v3} -- 1638 dereferences across six functions.
Members are dwords throughout rather than placing a 64-bit field at the
observed 0x14 qword read, which is not 8-aligned and would shift every later
offset.
asd_link_caps the capability/handler block, from asd_get_link_caps_full{,_v2,_v3},
asd_resolve_asic_handlers and asd_dio_read_block; offsets 0x04..0x24
contiguous.
Both are internal structures reached through an argument, so unlike the eight host requests with
enforced floors there is no size check anywhere; both sizes are maxima observed and say so.
They are applied by rewriting only the FIRST PARAMETER of each function's live prototype, so the
rest of the signature -- which came from the recorded handover prototypes -- survives instead of
being replaced by an invented declaration. 9 of 11 land on the reference build.
load_type_decls was only reading <TA>_DECLS, so the internal structures were declared in the
table and silently never parsed; it now picks up <TA>_INTERNAL_DECLS as well.
Measured across fw/decomp: ASD typed field accesses 2311 -> 3729 and raw dereferences
8446 -> 6863, taking ASD's field share from 21.5% to 35.2%. Across all modules, field accesses
3828 -> 5246 and field share 21.1% -> 29.2%. Struct-typed signatures are unchanged at 4.8%,
which is expected: this change retypes arguments of already-named functions rather than adding
new signatures, and is the clearest case yet of why the signature count alone is a poor measure
of readability.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Seven more request structures derived the same way as the earlier four, which takes the count
from 8 to 15 of 16. The sixteenth is TMZ, and it needs nothing: its invoke entry is literally
'return 0', which confirms from the entry itself the "TMZ is a stub" note in AUDIT-NEW-TAS.md.
Four of the seven close against a size the firmware itself enforces, which is the standard the
earlier xgmi/oc/wspe/hdcp structures set:
csu floor 0x24 cmd at +0, ack at +4, status at +8, body to 0x24
dtm floor 0x50 args at +0x10, 0x10 + 0x40 = 0x50 exactly
merit floor 0x2024 handlers receive the whole buffer
opm floor 0x3D54 independently reproduces the "params_size >= 0x3D54" already recorded
in AUDIT-NEW-TAS-RE.md, from the entry rather than from that note
Three do not enforce a size and say so in their declarations, as asd_req and powerplay_req
already do: auc (ctx bounded exactly at +0x10..+0x298, params extent only observed), fp (the
_DWORD * interior pointers scale, so v4 + 3 is +0x0C and v4 + 68 is +0x110), and rap -- which
is nonetheless the best bounded of the three because its entry zero-fills exactly 24 bytes at
+0x18 and then stores at +0x30, closing the layout at 0x34 without guesswork.
RAP also REFINES the negative recorded against powerplay_req. That commit said POWERPLAY is the
only entry writing a 'command | 0x80000000' ack word; the test looked for it at +4. RAP writes
the same ack at +0x08 and CSU at +0x04. So the ack convention is NOT unique to POWERPLAY -- it
simply sits at a component-specific offset, which remains a reason not to hoist one shared
header, but not the reason first given.
Two more cases of the inherited-prototype defect, both recorded rather than silently applied:
CSU's handlers are called as csu_cmdNN(ctx: *a4, params: v5) where v5 is a4[1], the declared
SIZE and an integer, so the 'struct ta_params *' prototype from fw/pseudo misdescribes it --
the same defect already noted for OC.
Metric correction worth stating: the field-access measure now also counts struct-by-value
accesses, because XGMI applies its struct to a stack buffer and reads v12.cmd_id rather than
v12->cmd_id, which the arrow-only count had been scoring as zero. On the comparable named-only
basis: typed field accesses 0 at session start -> 3742, against 8254 raw pointer-arithmetic
dereferences, a 31.2% field share. Struct-typed signatures 309/10070 (3.1%) -> 487/10088 (4.8%).
Only MERIT and the TMZ stub still show no typed field accesses, MERIT because IDA keeps an
explicit cast at its one dispatch read even though the pointer is typed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
POWERPLAY's entry takes the host buffer as v4 = *a4, switches over cases 0..6, and gives every
handler the same pair of interior pointers, v4 + 16 as ctx and v4 + 72 as params.
Its three-dword header is the best-evidenced part because it is corroborated on hardware rather
than only statically: the entry writes back *(v4+4) = cmd | 0x80000000 and *(v4+8) = status, and
the host-page dump already recorded in AUDIT-NEW-TAS-RE.md shows exactly that for a command 4 --
"dw0-3: 0x4 0x80000004 0x415 0", being command, command with bit 31 set, status, zero.
That looked like it might be a shared TA request ABI worth hoisting into one declaration for
every component, so it was tested rather than assumed: of twelve TA entries checked, POWERPLAY
is the ONLY one that writes an ack at +4. The hypothesis is wrong and the header stays
per-component. Recording the negative because it is the kind of generalisation that would have
silently mistyped eleven modules.
The size carries the same caveat as asd_req and is labelled in the declaration: nothing
validates a request size here, so 0x84 is the furthest offset observed through handler second
arguments rather than an enforced bound. xgmi_req, oc_req, wspe_req and hdcp_req remain the four
that close against a size the firmware itself checks.
Applied across all six builds, the entry reads v4->cmd_id, v4->cmd_ack, v4->status, v4->ctx and
v4->params.
Measured across fw/decomp: typed field accesses 3547 -> 3637, field share 30.1% -> 30.6%,
modules with a derived request structure 7 -> 8 of 16.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
HDCP was the worst remaining module by readability: 1742 raw pointer-arithmetic dereferences,
zero typed field accesses, zero struct-typed signatures.
Its layout comes from TA_InvokeCommandEntryPoint, which takes the host buffer as v7 = *params,
switches on *(_DWORD *)v7 across 19 cases, and hands handlers two interior blocks at v7 + 12
and v7 + 5136. The size floor is the dispatch guard itself -- it proceeds only when
params[1] > 0x2817 -- and both blocks close exactly against it: 0x0C + 0x1404 = 0x1410 and
0x1410 + 0x1408 = 0x2818.
A recorded comment is wrong about that guard and is now flagged in the declaration, because the
comment-application added earlier propagates it into the artifacts: it claims the entry
"validates that params[0].size <= 0x2817", while the code dispatches when the size EXCEEDS
0x2817. It is a floor, not a ceiling.
Three fixes to the locator, each found by its own failure rather than by inspection:
* it was reading the switch-bearing function, but 'deref' describes what the ENTRY does with
the params pointer, and HDCP's switch is not in the function that does 'v5 = *a4'; deref
mode now always reads the entry;
* the candidate argument name was hardcoded, so builds where no prototype has been applied
call it a4 rather than params and matched nothing. Widening it to every parameter was
worse -- it matched an unrelated assignment and typed the wrong local on the reference
build -- so only the declared name and the last parameter are tried;
* an applied prototype can introduce a cast, so an optional cast is now allowed before the
dereference.
Applied, the entry reads v7->cmd_id, v7->argA and v7->argB across 25 accesses. It lands on 2 of
7 HDCP builds; the other five do not expose the assignment in a shape the locator recognises and
are left untyped rather than forced.
Measured across fw/decomp: struct-typed signatures 309/10070 (3.1%) at session start ->
474/10088 (4.7%); typed field accesses 0 -> 3547 against 8254 remaining raw dereferences, a 30.1%
field share; modules with a derived request structure 0 -> 7 of 16.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ASD is 6781 of the 10088 decompiled functions and had no domain types at all, so it was the
only place left where work could move the overall figure.
It is driver-class rather than a TA: there is no TA entry trampoline, so the entry resolution
used for every other component does not apply. Its dispatcher hangs off the trampoline's +0x1C
slot, and the resolver verifies the ARM stub shape rather than assuming the offset, so a layout
change returns nothing instead of a confidently wrong address. That resolves to 0xD013B8 in all
17 Windows builds, matching the value psp_emu validated independently across its own 12.
The request layout is read off the dispatcher's own 359 lines: an rpc id at +0x00 switched over
0x180..0x1C0, then arguments at +0x04, +0x08, +0x0C, +0x10 and a byte at +0x14, with the first
two also read as bytes and halfwords.
The size is the one weak point and is labelled as such in the declaration: nothing validates a
request size, so 0x18 is not an enforced bound but the smallest 4-aligned size covering every
observed access. xgmi_req, oc_req and wspe_req all close against a size the firmware itself
checks; this one does not, and no code arrays the structure, so the risk is contained.
Applied, the dispatcher reads 'switch ( req->rpc_id )' and 'asd_hdcp1_init(a1: req->arg1)' in
place of pointer arithmetic, with 99 field accesses across it.
Also adds a body-level readability metric, because counting struct-typed SIGNATURES badly
under-measures this change: typing one dispatcher rewrites hundreds of lines while moving the
signature count by two. Measured across fw/decomp, typed field accesses now stand at 3494
against 8255 remaining raw pointer-arithmetic dereferences, 29.7% of the total. HDCP is now the
worst remaining offender with 1742 raw dereferences, zero typed fields and zero struct-typed
signatures.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Matching prototypes to functions by NAME reached only 8 of GCP's 24, and the reason turned out
to be a correctness problem rather than a matching one: this tree carries two naming
generations that disagree about which command a handler serves. fw/pseudo calls the handler at
0x402f90 gcp_cmdFE0C_CreateCryptoSession; fw/libdumps calls the same function
gcp_cmdFE0C_CreateAuthenticatedSession. The hardware findings settle it --
FINDING-GCP-ACQUIRE-GATE-BYPASS records 0xFE0C as CreateCryptoSession and
FINDING-GCP-FE0D-REFCOUNT-DESYNC agrees -- so the libdump name is wrong and had propagated
into every artifact generated so far.
Keying on the command id and resolving the address through the dispatcher avoids the
disagreement entirely, because the dispatcher is the only authority on which function serves
which command. GCP goes from 5-9 applied prototypes per build to 14-17 and from 28 struct-typed
signatures to 172, and the handler at 0x402e94 is relabelled from gcp_cmdFE03_* to
gcp_cmdFE07_* on every build -- the single conflict the cross-check had been reporting since it
was built.
Two more request structures are derived and verified the same way as xgmi_req. OC and WSPE both
dispatch with if/else chains rather than a switch, so the switch-based dispatcher recovery finds
nothing for them and their invoke entry has to be used directly; both work on the host buffer in
place rather than staging a copy, so their offsets are host offsets by construction. oc_req
closes at 0x1C (handler blocks at +0x0C and +0x16, the latter running 6 bytes to the end of the
validated size) and wspe_req at 0x14 (handler block at +0x10). Both sizes are the ones the
dispatchers' own size floors enforce. Applied, OC's entry reads 'v5->cmd_id == 2' and the type
propagates outward to 'struct oc_req **a4'.
Recorded against this work rather than hidden: applying a pseudo-file prototype by name can
introduce a WRONG signature. OC's dispatcher calls oc_cmd01(ctx: v5 + 3, params:
(struct ta_params *)7) -- the second argument is the literal 7, so the inherited
'struct ta_params *' misdescribes it, and the cast in the output is the tell. Prototypes should
be checked against what their call sites actually pass before being applied.
Measured from the artifacts rather than from effort: struct-typed signatures 309/10070 (3.1%)
-> 456/10070 (4.5%), modules with a derived request struct 0 -> 3, modules with zero struct-typed
signatures 6 -> 4. The rate stays low for a structural reason worth stating plainly: ASD is 6763
of the 10070 functions and has no recorded type information at all, so work on the small
components cannot move the global figure. ASD is where the remaining value is.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
holds
First hand-derived structure of this effort, with the method and the evidence written down in
fw/AUDIT-TA-STRUCTS.md so it can be checked rather than trusted.
The obvious approach would have produced a wrong layout. XGMI command handlers look like
sub_403426(_BYTE *a1, int a2) with a2 null-checked and read at +0 and +8, so typing a2 as the
request base is the natural move. But the dispatcher sub_402A1C zeroes a 0xC20-byte stack
image, rejects a declared host buffer smaller than 0xC20, copies the host page into that
image, and passes handlers v15 or v16 -- interior pointers into the copy. Every field would
have landed at the wrong offset.
The layout is taken from the decompiler's own stack slots relative to the staging base and it
closes exactly, with nothing padded to make it fit: argA at +0x10 spans 258*4 and ends at
0x418, where argB begins and spans 257*8 to 0xC20, the same size the zero-fill, the copy and
the size check all use, with argB's offset 8-byte aligned as its element type requires. These
are host offsets and not merely stack offsets because the staging image is 1:1 with the host
page: the writeback returns argB to its own offset (v8 + 1048, 2056 bytes) and the two scalars
go back individually at v8+8 and v8+13, matching status and flag0D.
IDA's parsed layout is read back and compared against the derivation rather than assumed:
size 3104 = 0xC20 as expected, members where the derivation puts them. Applied, the dispatcher
reads 'v12.cmd_id <= 0xB' where it previously read 'v12[0] <= 0xBu'.
Propagation refuses to apply the struct to the other three XGMI builds, and the refusal is the
point. 32.0.0.13 and 32.0.0.2 stage a smaller request (0x414/0x400) and have no 0xC20 frame at
all, so forcing the layout onto them would misdescribe them. The staging local is found by
FRAME SIZE rather than by name for exactly this reason: Hex-Rays renumbers locals per build,
so matching on 'v12' would have applied the wrong layout silently instead of reporting a
mismatch.
Also fixes a duplicated dispatcher search. dispatch_map already knew which function held the
command switch and was throwing it away, so the staging step re-derived it with a near-copy of
the same walk and failed on builds where the original had succeeded; it now returns the
address it chose.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The decompiled artifacts were missing almost all type information. Three things were
available in the tree and only one was being used:
* declare_type was never called, so no type definitions were parsed at all;
* set_type ran only for the four handover types, because fw/libdumps records just
{ea,hash,name,size} and the twelve Windows TA types therefore had no signatures to port;
* the recorded comments were read into a variable and discarded without ever being applied.
All three are now applied, in dependency order, and counted per build so the result is
measurable instead of something you have to notice by reading the output. Across 99 builds:
7727 prototypes (was 1232, all of them ASD), 6068 comments (was zero), 309 struct-typed
signatures (was zero).
The reason prototypes were failing was NOT the missing types, which is what I assumed when I
reported "202 of 1434 rejected" without diagnosing it. Declaring the types first changed the
rejection count by nothing. 205 of the 239 failures were prototypes recorded in the bare
shape 'int()', with no __fastcall for the name-insertion to latch onto, so they were emitted
as the unparsable 'int();'. The name now goes before the first parenthesis whatever the
shape, and prototypes land 100%: 1434/1434 on the ASD reference, 99/99 HDCP, 9/9 DTM.
Worth recording so nobody repeats the detour: the 43 'local_types' in each handover symbol
set are ARM NEON and SVE vector typedefs plus va_list and BOOL -- compiler boilerplate with
no domain structs in it. Declaring them is harmless and buys nothing. The only real PSP
structs in this tree are the seven hand-written ones in fw/ta_ida.py (gcp_ctx, ta_params,
gcp_drm_session, gcp_auth_session, gcp_crypto_session, merit_slot, merit_table), which were
sitting unused and are now applied: GCP command handlers finally read as
'(struct gcp_ctx *ctx, struct ta_params *params)' rather than '(int, int)'.
So the remaining type gap is a data gap, not a pipeline one. Thirteen of the sixteen
components have no struct definitions anywhere in the repo to apply, and ASD's 6800
prototypes are almost entirely primitive-typed, which is why it shows only 30 struct-typed
signatures against 1439 named functions. Closing that needs structs to be defined by hand,
component by component.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The twelve Windows TA types take their symbols from fw/libdumps; ASD, HDCP, RAS and DTM
have theirs in psp_out/handover, in a different shape that also carries prototypes and
comments. This adds that reference path, so every component the corpus holds is covered by
one pipeline rather than twelve of fifteen.
ASD needed its own ELFs. It is driver-class, not a TA: base 0xD00100, and no TA entry
trampoline, so dispatcher-anchored naming does not apply to it at all -- its RPC dispatcher
hangs off trampoline+0x1C instead. The base is not guessed; it matches
psp_out/handover/elf/asd2.elf, whose LOAD segment contains 1562/1562 of the recorded ASD
symbols. The frame discriminator used for the TAs is inapplicable here for the same reason
the trampoline is: there is none, and it reports "neither" for all 17 builds.
Prototypes are applied where recorded. Without them Hex-Rays guesses argument counts and
types, which is most of the distance between readable C and a wall of _DWORD; 1232 of 1434
land on the ASD reference and the rest are rejected by IDA, reported rather than hidden.
One more false-positive class in the dispatcher cross-check, found by running it on HDCP:
the agreement test assumed a ported name encodes its command number, which is GCP's
convention and not HDCP's, so every one of HDCP's 16 commands was reported as conflicting.
A name can only contradict the dispatcher if it claims a command itself, so the check now
requires the ported name to match <ta>_cmd<hex> before judging it. HDCP drops to zero
conflicts while GCP's single real one -- the 0xFE07 handler carrying the inherited name
gcp_cmdFE03_CreateAuthenticatedSession -- is still reported.
Coverage is honest rather than flattering. ASD lands 162-425 named of ~1100-1400 functions
per build, and the dominant limiter is ambiguity, not similarity: 677-908 functions per
build share a byte signature with a differently-named reference function and are skipped
instead of being assigned arbitrarily. ASD is full of identical small stubs, so this is
expected, but it means the ASD figures are a floor and a smarter tie-break would raise
them. RAS is weaker still at 13-20 named per build.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
twelve types
Byte and mnemonic similarity cannot name a handler whose code diverged, and that failure
landed exactly where it hurts: gcp_cmdFE0C, the function carrying the acquire-gate
refcount bypass, was unnamed on six of eleven GCP builds. The dispatch table does not care
about similarity -- whatever sits in the 0xFE0C arm is the 0xFE0C handler by construction.
So the handler names now come from the image itself: the header's trampoline gives
TA_InvokeCommandEntryPoint, Hex-Rays recovers the switch, and each case arm is attributed
to the function it calls. Parsing the decompiler's switch output beats hand-rolling Thumb
jump-table decoding, and callee tokens resolve through the live function list rather than
by assuming they look like sub_<addr>, so it works whether or not names were applied first.
Validated against the one authoritative map that already existed: all 14 commands this
recovers for GCP 22.0.4.50 match fw/cmdmaps/gcp_ea.json address for address, with none
spurious. The three it misses (0x3, 0xFE08, 0xFE0E) are precisely the arms with no callee
-- inlined into the dispatcher, and recorded in the old map as loc_ labels that were never
function starts. FE0C is now named on all eleven builds.
Two bugs the first attempt had, both caught by its own conflict reporting. Walking three
levels deep from the invoke entry swept up nested sub-command switches inside individual
handlers and invented a GCP command 0x0F that does not exist; the walk is now limited to
the invoke entry and its direct callees, with a depth-2 fallback only when depth 1 finds no
switch at all. And one handler commonly serves several commands -- GCP routes 0xFE03-0xFE07
to one function -- so commands are grouped by handler address before naming or judging
agreement.
The self-port gate was measuring the wrong quantity. Dividing EXACT matches by every
recorded symbol conflated a broken matcher with IDA simply not defining a function at a
recorded address, and failed five sound types on that basis -- TMZ scored 17%. The
denominator is now the reference symbols that are function starts in the reference build;
all twelve types then gate at 100%.
That fix made the gate blind to a different error, so it gained a second check. MERIT ships
two distinct 24.0.4.42 builds, the reference was being chosen by version alone, and the
libdump had been recorded from 444eceee while bc1706f8 was loaded: 24 of 31 symbols landed
on nothing and every MERIT build sat near 30% named. The reference is now pinned by hash
from fw/syms/<ta>.meta.json, and an orphan-symbol rate above 10% is reported as a suspected
build/symbol mismatch. MERIT went to 58-83% named.
62 builds across twelve types, zero decompile failures. The remaining conflicts are real
and worth reading rather than suppressing: GCP's 0xFE07 handler carries the inherited name
gcp_cmdFE03_CreateAuthenticatedSession, because the older name-keyed cmdmap collapsed
0xFE03-0xFE07 onto one label while the dispatcher shows only 0xFE07 reaching that function.
|
|
|
The project had one symbolicated build per TA type, so cross-version comparison was
impossible: there was nothing to compare against. This adds the two pieces that were
missing and proves them on GCP as a pilot.
`fw/mkelf_ta.py` wraps each carved TA body as an ELF IDA can load. The format is read
back off the twelve existing reference ELFs rather than assumed, and the control is
strict: regenerating all fourteen of them is byte-identical, so the generator is not
inventing a layout of its own.
`fw/ta_re.py` ports the reference symbols onto every build and emits whole-program
pseudocode. Names are applied in tiers -- EXACT bytes, MNEM identical mnemonics, FUZZY
similarity above 0.90 -- and anything below the floor is left as sub_*, never guessed. A
signature shared by two differently named functions is treated as ambiguous and skipped
rather than resolved arbitrarily. The gate that makes the rest trustworthy is porting the
reference onto its own build: 112/117 EXACT, and a type whose self-port fails is reported
void instead of emitted.
The output is shaped for diffing, which cost two iterations to get right. Addresses are
omitted and functions ordered by name so the same function lines up across versions;
named and unnamed functions go to separate files, because one function named in build A
and anonymous in B reordered everything after it and turned a 48-function change into
2900 diff lines. Adjacent builds now differ by 1026 lines, consecutive ones by 261.
Chained porting is included and honestly oversold: seeding each build from its nearest
already-named neighbour moved names from MNEM into EXACT, which is a real confidence gain,
but total coverage did not move. The 22.0.0.62-77 branch stays at 52% named because those
builds genuinely diverge -- the function count jumps 119 to 126 -- not because the
reference was too distant. FUZZY names are deliberately not fed back into the chain, so a
single bad guess cannot propagate and look more confident at each hop.
A scare worth recording as resolved: all 78 TA bodies carry a module header whose own
metadata (end == base + length, 78/78) implies the project's ELF base is 0x100 too high,
which would have made every recorded TA address wrong. It is not. The trampoline at
header+0x10 must yield Thumb-bit entry pointers at +0xA0/+0xB0, and across all 78 builds
that resolves only in the 0x400100 frame. The existing convention is now confirmed by
evidence that discriminates between the two frames, unlike the emulator's verify(), which
is self-consistent because its expected values came from harnesses in the same frame.
|
