| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
known names
The previous pass left 102 stubs unnamed because they load more than one in-range constant. The
tempting fix was to take the first immediate, which matches 4 of 4 multi-immediate stubs that carry an
established name -- and that number is worthless, because all four are the same XGMI function in four
builds. It is n=1 dressed up as n=4, and applying it to 102 functions would have been a guess with a
validation figure attached.
The structural rule earns the names instead: track MOV/MOVW/MOVS immediates per register, then return
whichever register is stored to [SP,#0], which is the first word of the frame R1 is pointed at before
the supervisor call. Against the 940 stubs the libdump already names *_rpc_N it matches 940 and
disagrees on 0, with no abstentions, and it resolves all 102 stubs the count heuristic could not.
So the weaker rule is superseded rather than patched, and the audit records both along with why the
4/4 figure was rejected -- that reasoning is the part worth keeping.
Regenerated gcp, hdcp, merit, opm and asd. Corpus 19,405 -> 19,433 meaningful names (67.1%), no
duplicate names in any of the 99 builds, address-derived still 0.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
corpus to 67.0%
The opcode decode that closed dtm and csu last commit is not module-specific, so I measured its reach
and validated it before applying it anywhere.
Validation came free: the libdump already names 299 single-immediate SVC stubs *_rpc_N, which is
independent ground truth for exactly this decode. The decoded immediate matches the established
number in 296 of 299. The three exceptions are one XGMI function across three builds whose libdump
name uses a different numbering scheme, not a bad decode.
Applied as a pipeline pass, keyed on bytes rather than addresses so it survives database rebuilds,
and only where exactly one in-range immediate survives -- 102 ambiguous stubs stay unnamed rather
than guessed. 501 stubs named: hdcp 87 -> 96, fp 92 -> 95, gcp 87 -> 91, opm 62 -> 75, merit 76 -> 79,
xgmi 75 -> 76, ras 35 -> 38, asd 64 -> 65, powerplay and wspe smaller. Corpus 19,134 -> 19,405
meaningful names, 66.1% -> 67.0%.
A second corroboration turned up after the fact: HDCP's newly decoded hdcp_rpc_161 carries the same
service number as ASD's independently libdump-named asd_rpc_161, so two modules calling the same
SYS_DRV entry agree without either having been told about the other.
Names are uniquified per build; no build contains a duplicate name, checked across all 99.
Worth noting what made this possible: the claim being overturned was my own, that these stubs "carry
no semantic information whatsoever". It was right about the shared thunks and wrong about the
per-module ones, and I had written it down as settled.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Correcting the headline count last commit exposed 243 functions whose name field was set to an IDA
address default. Looking at what they actually were, rather than treating them as a rounding error,
found a real defect: 220 were j_sub_<addr> thunk names, and 87 of those pointed at targets that by
now carry a proper name.
The cause is ordering. The j_<target> label comes from whatever Hex-Rays printed in the signature at
the moment that thunk was decompiled, so a thunk decompiled before its target was renamed keeps the
stale address forever. Nothing ever revisited it.
decompile_all now re-resolves them from `assigned` after the naming pass. That keeps it a pure
lookup -- the target name is one the pipeline already decided on, so no new inference enters -- and
thunks whose target is still unnamed keep j_sub_*, which is the honest answer rather than a
cosmetic upgrade.
In the artifacts: 87 names recovered across 5 ASD builds, address-derived entries 243 -> 156,
meaningful names 19,181 -> 19,268 (66.3% -> 66.6%). Names like j_asd_build_link_map_full and
j_asd_clear_link_outputs now appear where j_sub_D06CE4 used to.
Worth recording why this was invisible for so long: these functions were being counted as named, so
every coverage number I looked at said they were fine. The metric correction is what made the defect
visible, which is the one case here where fixing a measurement directly produced real improvement
rather than just a more accurate claim.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
DISPATCH names
An earlier pass reported that all 99 artifacts regenerate byte-identically. That was true of that
one run and does not hold in general. A full clean regeneration of all sixteen modules changes 220
files, and the cause is IDA's auto-analysis not being identical across fresh databases: AUC's
function count comes back 39 instead of 40, and decompiled text varies cosmetically
("(int)nullsub_3" against "nullsub_3"). The claim is corrected here rather than left standing.
Semantically the drift is small -- 4 names of 19,353 -- but all four were tier DISPATCH, which
exposed a wrong decision. fw/ta_snapshot_names.py excluded DISPATCH on the grounds that
fw/ta_re.py re-derives it every run. It does, but not deterministically: dispatcher recovery
depends on which function Hex-Rays decompiles cleanly on a given analysis, so xgmi_cmd00,
xgmi_cmd04, xgmi_cmd0B and fp_cmd04 simply did not come back. DISPATCH is now persisted, and the
exclusion reasoning in that file is rewritten to say why.
Persisting it still cannot mask a regression: the dispatcher step runs regardless, so where
recovery works the DISPATCH tier wins and where it fails the restored name appears as tier IDA.
The tier difference is itself the signal that recovery failed on that build.
7817 names were recovered from the committed artifacts into fw/cgnames so nothing derived is left
depending on a database that may not reproduce. XGMI came back whole.
The one name that did NOT come back turned out to be a defect being fixed rather than lost. HEAD
had TWO functions called fp_cmd04 -- 0x401334 from the dispatcher and 0x402b4c from byte porting
-- and the current dispatcher map routes command 4 to 0x402b4c, agreeing with the ported name. So
HEAD carried a duplicate, this run does not, and the allow_overwrite=False guard on the re-apply
is what refused to restore the bad one. Worth keeping: a name count going DOWN was the correct
outcome, and reading it as a regression would have reintroduced the duplicate.
Verified state after the clean regeneration: 16/16 gates PASS, 99 builds, 19,349 of 28,945 named
(66.8%), field share 66.4%, 21 decompile failures.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Two GCP handlers were mislabelled in the libdump symbol set and the mislabels had been
propagating into every artifact. The duplicate name was the tell: FE0B and FE0D were both called
DestroyAuthenticatedSession.
The code settles it without needing the documents. gcp_cmdFE0C iterates ctx->crypto, zero-fills a
0x58 gcp_crypto_session and sets crypto->state and crypto->key, so it creates a CRYPTO session.
gcp_cmdFE0D computes "(char *)ctx + 88 * HIBYTE(arg10)" -- stride 88, the crypto array at
ctx+0x800 -- while gcp_cmdFE0B uses stride 80, the auth array at ctx+0x300. AUDIT-NEW-TAS-RE.md
states exactly that split, and the hardware findings agree: FINDING-GCP-ACQUIRE-GATE-BYPASS
records 0xFE0C as CreateCryptoSession.
Worth distinguishing from the FE07 case: that one claimed the wrong COMMAND and the dispatcher
cross-check caught it automatically. These two claim the right command and the wrong SEMANTICS,
so no command-id check can see them -- only reading what the code touches. The corrections are
applied by name through a RENAME_FIX table so they hold for every build.
While verifying them, the dispatcher kept printing the old callee name in four builds after the
rename had demonstrably succeeded. Hex-Rays caches pseudocode per function and
remote_ops.rename refreshes only the function it renamed, never its callers, so a stale cache was
reading as a failed rename. Every function's cached text is now invalidated before the artifacts
are emitted.
That turned out to matter far beyond the two names. Applied TYPES were also being hidden behind
stale text: GCP's field share goes 59% to 83.4%, its typed field accesses 1716 to 3081 and its raw
dereferences 1193 to 612, with no new structure added -- the types had been applied all along and
the emitted artifacts were showing pre-typing output. WSPE lands at 91%, HDCP 70%, XGMI 69%,
POWERPLAY 67%. Corpus field share 65.9% -> 66.4%, held down by ASD, which dominates the corpus and
barely moved.
This is the sixth consecutive pass whose main finding was that something already done was not
visible in the output. The remaining low scores are now informative rather than suspect: merit at
0% and opm at 19% are low because their structures are mostly opaque body arrays, not because
typing failed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
every failure
Byte and mnemonic porting only names a function whose own body still resembles the reference.
A function's position in the CALL GRAPH is independent evidence: if reference function A matches
target A' and both make the same number of calls, the Nth callee of A' is the Nth callee of A.
fw/ta_cgname.py applies that, with three restrictions chosen because positional alignment is
exactly the sort of heuristic that quietly invents names: the call counts must be EQUAL or the
anchor is skipped entirely rather than aligned partially; every anchor reaching a target must
agree on the name, or it is left alone; and the candidate must be within a factor of two of the
size of the function whose name it is taking. Rounds iterate, so a function named in one round
becomes an anchor in the next.
The anchor mismatch rate is reported as its own control, because a high rate means the builds
have diverged too far for position to mean anything. For ASD it is 4-8% (9-17 mismatched against
63-269 aligned), which is why the result is trustworthy. For RAS the same tool finds only 4-12
usable anchors and produces nothing at all -- its reference shares too little with the corpus
builds, and that is reported rather than worked around.
843 names applied across ASD's 17 non-reference builds, taking ASD from 62.5% to 66.0% named and
the corpus overall past two thirds.
Found and fixed while debugging why the first run applied zero: the rename response is
{"func":[...], "summary":{"ok":N,"failed":M}}, and both this tool and fw/ta_re.py were looking
for a "result" list that does not exist. In ta_re.py that meant the rename-error counter reported
zero unconditionally for the entire effort -- it could never have told me about a failed rename.
Both now read the summary. The symptom was visible only because this tool counted successes
rather than failures; the ta_re.py version failed silently in the safe direction, which is why it
went unnoticed.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Chasing where ASD's remaining dereferences actually lived found a defect in my own structure
rather than a new one to add.
The residue was in UNNAMED functions -- 94% of their dereferences in the top 20, the two heaviest
being 6102 and 3906 bytes with 692 and 562 dereferences. Inspecting their offset sets showed they
touch every dword from 0x04 to 0xB4 with no gaps, including the four offsets asd_link_cfg
declared as unused padding (0x10, 0x1C, 0x3C, 0x50). Those gaps were an artefact of the REFERENCE
build's narrower functions never touching them, not a property of the structure. asd_link_cfg is
now the contiguous dword array the evidence actually shows, 45 observed offsets, size 0xB8 rather
than 0x58.
That is a useful warning about deriving a layout from one build: the first version was internally
consistent and verified by read-back, and still understated the structure by half, because the
evidence available in the reference was simply thinner than the evidence available across builds.
Typing those unnamed holders needed a rule that is not a name. Guessing names for them would
assert an identity the evidence does not support; typing only claims the pointer's shape. The
predicate is deliberately narrow -- a function qualifies only if its a1 offsets COVER all fifteen
core link offsets at once -- and its control is that on the reference's 1440 functions it selects
exactly one, asd_program_link_by_signal, with zero false positives. A looser rule such as "subset
of a contiguous dword range" would match almost anything and is explicitly not used.
Two avenues were tested and rejected on the way, both recorded so they are not retried:
* disambiguating the 677-908 functions skipped as ambiguous per ASD build by address ORDER.
Order is only ~88% monotonic against the reference across six builds, so roughly one in eight
names would be wrong. A wrong name is worse than no name here.
* the ambiguous set is in any case not where the dereferences are: functions under 32 bytes
hold 16 of them.
Measured across fw/decomp: typed field accesses 7327 -> 9058, raw pointer-arithmetic dereferences
11753 -> 10107, field share 38.4% -> 47.3%.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Request structures were done for every component, but ASD still held 8446 of the raw
pointer-arithmetic dereferences, and typing its dispatcher had moved only a few hundred. So
rather than guess where the rest lived, it was measured: of those 8446, 6314 go through a1, 86%
of those sit in just 20 functions, and only 54 distinct functions touch a1 at all. The
unreadability is not spread across 6781 functions; it is concentrated in a handful.
A first attempt at clustering by name prefix was a dead end worth recording. The largest
cluster, asd_reg_* with 145 functions, has ZERO a1 structure accesses -- they are register
accessors that take an address and a value, not a context. Clustering by subsystem name says
nothing about whether a context structure exists.
The two families that do matter were found by offset signature instead:
asd_link_cfg the link/encoder descriptor, from asd_program_link_by_signal{,_v2,_v3,_v4}
and asd_encoder_enable_seq{,_v3} -- 1638 dereferences across six functions.
Members are dwords throughout rather than placing a 64-bit field at the
observed 0x14 qword read, which is not 8-aligned and would shift every later
offset.
asd_link_caps the capability/handler block, from asd_get_link_caps_full{,_v2,_v3},
asd_resolve_asic_handlers and asd_dio_read_block; offsets 0x04..0x24
contiguous.
Both are internal structures reached through an argument, so unlike the eight host requests with
enforced floors there is no size check anywhere; both sizes are maxima observed and say so.
They are applied by rewriting only the FIRST PARAMETER of each function's live prototype, so the
rest of the signature -- which came from the recorded handover prototypes -- survives instead of
being replaced by an invented declaration. 9 of 11 land on the reference build.
load_type_decls was only reading <TA>_DECLS, so the internal structures were declared in the
table and silently never parsed; it now picks up <TA>_INTERNAL_DECLS as well.
Measured across fw/decomp: ASD typed field accesses 2311 -> 3729 and raw dereferences
8446 -> 6863, taking ASD's field share from 21.5% to 35.2%. Across all modules, field accesses
3828 -> 5246 and field share 21.1% -> 29.2%. Struct-typed signatures are unchanged at 4.8%,
which is expected: this change retypes arguments of already-named functions rather than adding
new signatures, and is the clearest case yet of why the signature count alone is a poor measure
of readability.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ASD is 6781 of the 10088 decompiled functions and had no domain types at all, so it was the
only place left where work could move the overall figure.
It is driver-class rather than a TA: there is no TA entry trampoline, so the entry resolution
used for every other component does not apply. Its dispatcher hangs off the trampoline's +0x1C
slot, and the resolver verifies the ARM stub shape rather than assuming the offset, so a layout
change returns nothing instead of a confidently wrong address. That resolves to 0xD013B8 in all
17 Windows builds, matching the value psp_emu validated independently across its own 12.
The request layout is read off the dispatcher's own 359 lines: an rpc id at +0x00 switched over
0x180..0x1C0, then arguments at +0x04, +0x08, +0x0C, +0x10 and a byte at +0x14, with the first
two also read as bytes and halfwords.
The size is the one weak point and is labelled as such in the declaration: nothing validates a
request size, so 0x18 is not an enforced bound but the smallest 4-aligned size covering every
observed access. xgmi_req, oc_req and wspe_req all close against a size the firmware itself
checks; this one does not, and no code arrays the structure, so the risk is contained.
Applied, the dispatcher reads 'switch ( req->rpc_id )' and 'asd_hdcp1_init(a1: req->arg1)' in
place of pointer arithmetic, with 99 field accesses across it.
Also adds a body-level readability metric, because counting struct-typed SIGNATURES badly
under-measures this change: typing one dispatcher rewrites hundreds of lines while moving the
signature count by two. Measured across fw/decomp, typed field accesses now stand at 3494
against 8255 remaining raw pointer-arithmetic dereferences, 29.7% of the total. HDCP is now the
worst remaining offender with 1742 raw dereferences, zero typed fields and zero struct-typed
signatures.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The decompiled artifacts were missing almost all type information. Three things were
available in the tree and only one was being used:
* declare_type was never called, so no type definitions were parsed at all;
* set_type ran only for the four handover types, because fw/libdumps records just
{ea,hash,name,size} and the twelve Windows TA types therefore had no signatures to port;
* the recorded comments were read into a variable and discarded without ever being applied.
All three are now applied, in dependency order, and counted per build so the result is
measurable instead of something you have to notice by reading the output. Across 99 builds:
7727 prototypes (was 1232, all of them ASD), 6068 comments (was zero), 309 struct-typed
signatures (was zero).
The reason prototypes were failing was NOT the missing types, which is what I assumed when I
reported "202 of 1434 rejected" without diagnosing it. Declaring the types first changed the
rejection count by nothing. 205 of the 239 failures were prototypes recorded in the bare
shape 'int()', with no __fastcall for the name-insertion to latch onto, so they were emitted
as the unparsable 'int();'. The name now goes before the first parenthesis whatever the
shape, and prototypes land 100%: 1434/1434 on the ASD reference, 99/99 HDCP, 9/9 DTM.
Worth recording so nobody repeats the detour: the 43 'local_types' in each handover symbol
set are ARM NEON and SVE vector typedefs plus va_list and BOOL -- compiler boilerplate with
no domain structs in it. Declaring them is harmless and buys nothing. The only real PSP
structs in this tree are the seven hand-written ones in fw/ta_ida.py (gcp_ctx, ta_params,
gcp_drm_session, gcp_auth_session, gcp_crypto_session, merit_slot, merit_table), which were
sitting unused and are now applied: GCP command handlers finally read as
'(struct gcp_ctx *ctx, struct ta_params *params)' rather than '(int, int)'.
So the remaining type gap is a data gap, not a pipeline one. Thirteen of the sixteen
components have no struct definitions anywhere in the repo to apply, and ASD's 6800
prototypes are almost entirely primitive-typed, which is why it shows only 30 struct-typed
signatures against 1439 named functions. Closing that needs structs to be defined by hand,
component by component.
|
|
|
The twelve Windows TA types take their symbols from fw/libdumps; ASD, HDCP, RAS and DTM
have theirs in psp_out/handover, in a different shape that also carries prototypes and
comments. This adds that reference path, so every component the corpus holds is covered by
one pipeline rather than twelve of fifteen.
ASD needed its own ELFs. It is driver-class, not a TA: base 0xD00100, and no TA entry
trampoline, so dispatcher-anchored naming does not apply to it at all -- its RPC dispatcher
hangs off trampoline+0x1C instead. The base is not guessed; it matches
psp_out/handover/elf/asd2.elf, whose LOAD segment contains 1562/1562 of the recorded ASD
symbols. The frame discriminator used for the TAs is inapplicable here for the same reason
the trampoline is: there is none, and it reports "neither" for all 17 builds.
Prototypes are applied where recorded. Without them Hex-Rays guesses argument counts and
types, which is most of the distance between readable C and a wall of _DWORD; 1232 of 1434
land on the ASD reference and the rest are rejected by IDA, reported rather than hidden.
One more false-positive class in the dispatcher cross-check, found by running it on HDCP:
the agreement test assumed a ported name encodes its command number, which is GCP's
convention and not HDCP's, so every one of HDCP's 16 commands was reported as conflicting.
A name can only contradict the dispatcher if it claims a command itself, so the check now
requires the ported name to match <ta>_cmd<hex> before judging it. HDCP drops to zero
conflicts while GCP's single real one -- the 0xFE07 handler carrying the inherited name
gcp_cmdFE03_CreateAuthenticatedSession -- is still reported.
Coverage is honest rather than flattering. ASD lands 162-425 named of ~1100-1400 functions
per build, and the dominant limiter is ambiguity, not similarity: 677-908 functions per
build share a byte signature with a differently-named reference function and are skipped
instead of being assigned arbitrarily. ASD is full of identical small stubs, so this is
expected, but it means the ASD figures are a floor and a smarter tie-break would raise
them. RAS is weaker still at 13-20 named per build.
|
