aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* README: rewrite, and fix the install it documentedblasty2026-08-073-200/+480
| | | | | | | | | | | | | | | | | | | | | | | | The old one had drifted into a wall of prose that described the project to someone who already knew it. Rewritten around what a reader actually needs: what this is, what it needs, how to run it, the keys, and what is in it -- with the logo up top and the sharp edges at the bottom instead of a wall of disclaimers at the top. Corrected while checking every claim rather than trusting it: - pyproject depended on 'ida-codemode-mcp' from '../ida-codemode-mcp'. The package is 'ida-codemode' and the repo is '../ida-codemode', so 'uv sync' -- the documented install -- could not have worked for anyone. Fixed, and uv.lock regenerated (it now resolves, 39 packages). - test counts were 257/733, they are 302/788 - the keys table is generated from the real BINDINGS: c/d/a/p/u, o/O/B, s for split, ctrl+t for structs, " for strings, all verified against app.py - dropped the ida-pro-mcp/worker-era architecture section, which described files that no longer exist, and replaced it with the current layout - links docs/CODEMODE_UPSTREAM.md and .fastfeedback/SPEED.md, which did not exist when the README was last touched Every path and doc link in it is checked to exist.
* Stop tracking the upstream port patchblasty2026-08-072-5848/+3
| | | | | | It was swept into the tree by a broad 'git add -A' during the port. The merge itself plus docs/CODEMODE_PORT.md are the record; a 251KB patch of superseded content is not. Still on disk, now ignored.
* Merge the IDA Code Mode portblasty2026-08-0739-2812/+8974
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replaces the private idalib worker (idatui/worker.py + worker_client.py, with server/patch_server.py injecting tools into ida-pro-mcp) with an ordinary client of ida_codemode.client.DatabaseHandle. A database open by an IDA GUI is reused; otherwise Code Mode starts or shares a managed idalib worker. The TUI no longer owns an IDA process, and closing it releases only its lease. Based on Duncan Ogilvie's port, rebased onto ~150 commits of local work it predated. The rebase itself was mechanical; landing it was not. Nine defects had to be fixed before the feature set was whole again, none of which the patch's own tests could catch: - DatabaseHandle.open() takes image_base, not loading_address: every connect() would have raised TypeError on the first call - five operations our tree had grown were simply missing (flowchart, so the graph view was dead; op_format/pc_nums/pc_num_format, so 'o'/'O' were; survey_binary) - set_comments wrote only the disassembly comment, so comments never appeared in pseudocode - xref_query returned rows in raw IDA order, and 'follow the call' silently followed the fall-through instead - rename accepted one edit per category, so bulk symbol import was dead - decompile ran decomp_map's full per-column ctree sweep to fill in a per-line address anchor - heads shipped without operand extents or the digest protocol - the package became unimportable without ida_codemode installed, which killed the offline test suites Verified against the pre-codemode tag rather than against assumptions: the full suite is 788 passed / 0 failed, and the pilot's 301 checks match the old backend exactly. Performance is within 2x on the listing hot path and faster on decompile, disasm and connect, after fixing two runtime costs that are documented for upstream in docs/CODEMODE_UPSTREAM.md. Test runtime came down from ~9m20s to 115s along the way -- not by removing checks, but by removing four kinds of waiting-on-a-guess that were also hiding real failures.
| * docs: upstream findings for the ida-codemode maintainersblasty2026-08-071-0/+269
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Nine items found while porting a latency-sensitive client, each with the measurement behind it and a suggested fix, ordered by what it cost us. The two that matter: timeout_trace returns itself, which turns on line tracing in every frame (52x on ida_bytes.get_flags, 10x on a listing page), and to_jsonable walking large results (114x what json.dumps of the same data costs). Together they were the difference between 35x slower than the worker we replaced and within 2x of it. Both live in the runtime, so every client pays them until they are fixed centrally -- we only found them because we had an in-process implementation to compare against. Also records the things we had to reach into internals for (waiting for a lease to release), the failure modes that are silent or fatal rather than typed (loader switches on an existing IDB, an IDB replaced under a live lease), and the one feature we could not port at all (discard-and-quit, since a managed worker saves when its final lease closes).
| * codemode: close the performance gap with the old worker (heads 35x -> 2.2x)blasty2026-08-071-5/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Two changes, both about work that was never ours to do, found by profiling the A/B benchmark rather than guessing. 1. Serialise inside the database process. Code Mode runs to_jsonable() over whatever a snippet returns, walking the entire structure to make it JSON-safe. Our answers are already JSON-safe and they are large: a 200-row listing page is ~10k small objects, and walking them cost 66ms of the page's 92ms -- 114x what json.dumps of the very same data costs (0.58ms). Snippets now return one pre-serialised string, so that walk is O(1) and the client parses a payload it was going to parse anyway. heads(200): 92ms -> 24.7ms. 2. Detach the runtime's trace hook while our snippet runs. ida_codemode.runtime wraps every execute_python in sys.settrace(timeout_trace) to enforce deadlines, and timeout_trace RETURNS ITSELF -- which switches on LINE tracing in every frame it sees. Every line of every function we call pays a Python-level callback. Measured here: ida_bytes.get_flags 0.106us untraced 5.49us traced 52x (plain idalib, no Code Mode: 0.119us -- i.e. untraced == native) heads(200 rows) 2.0ms untraced 20.2ms traced 10x That one hook was the entire residual gap against the old unix-socket worker. The snippet now detaches it and restores it in a finally. What that gives up, stated plainly: the deadline is no longer enforced for a pure-Python loop inside our snippet. The runtime's other cancellation path -- a threading.Timer calling ida_kernwin.set_cancelled() -- does not go through the trace and still fires, so a long IDA operation remains interruptible, and every operation here is bounded by its own count/limit argument. Set IDATUI_CODEMODE_TRACE=1 to keep the stock behaviour. Against the worker backend, same box, targets/echo (worker -> codemode): heads_200 2.65ms -> 5.85ms 2.2x (was 35x) heads_500 6.21ms -> 10.67ms 1.7x heads expect-hit 2.16ms -> 4.61ms 2.1x disasm_200 9.03ms -> 5.25ms 0.6x faster decompile_cold 162.62ms -> 30.66ms 0.2x faster decompile_warm 30.02ms -> 25.39ms 0.8x faster decomp_map 45.82ms -> 47.85ms 1.0x parity pc_nums 19.90ms -> 22.56ms 1.1x parity rename_func 254.08ms -> 255.54ms 1.0x parity connect 550.0ms -> 410.0ms 0.7x faster What is left is the transport floor: an empty execute_python round trip is 2.0ms, so trivial calls (data_type 0.07ms -> 2.63ms, force_recompile, a single xref query) look like 40x while being 2.5ms of wall clock. Reducing those needs fewer calls, not faster ones -- the digest/expect path already does that for the listing, which is where call volume actually is. Full suite: 788 passed, 0 failed, 115.3s (was 146.3s; the pilot alone went 80.9s -> 62.2s).
| * tests: trace_ui followed a signal that was not one (and it really did fail)blasty2026-08-071-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "the code view follows the trace" failed on the port (None vs 0x2ae4) and passed on master, but it was not a trace regression: the test pressed "]", waited for `app._t == 1`, and then read the listing cursor. `app._t` is assigned the moment the key is handled -- the navigation it starts runs in a worker -- so the wait was satisfied before the view had moved, and the check read a cursor that had no address yet. Master won that race because its backend answers in single-digit milliseconds. The Code Mode backend is slower, so the race became a reliable failure. The gate is now the condition the check is about (the cursor is on the trace's ip), for both the forward and the backward step. 39 passed, 0 failed -- the same tally as master.
| * tests: thumb_ui 5m13s (crashing) -> 8.5sblasty2026-08-071-37/+58
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It was not slow because ARM analysis is slow. It was slow because it fought Code Mode's ownership model and then waited out timeouts for the result. The suite deleted <fixture>.i64 and REOPENED THE SAME PATH before each of its four phases. That was safe when the TUI owned a private worker that died with it. Under Code Mode the database is leased, and the previous phase's managed worker can still hold it through its lease grace -- so the delete raced a live owner, the next open never produced a listing, and the suite died on `lst.model.index_of_ea(0)` with model=None after burning minutes in waits whose results were never checked. Each phase now gets its own temp copy: separate paths cannot collide and nothing waits for anyone to let go. This is the same hazard docs/CODEMODE_PORT.md flags -- sweeping files that another client may own -- showing up in the test suite rather than in the app. Also replaced four `wait(lambda: lst.model is not <old>, ..., 60)` gates. An item edit now keeps the listing's walk instead of rebuilding it, so the model object is never replaced and each of those sat out its full 60 seconds while the suite still reported success. They now settle() on the signal the checks actually read: the status line announcing Thumb/ARM/64-bit, and the function appearing in the index. 20 passed, 0 failed (it never reached a tally before). Verified separately that the four ARM operations the port carried over do work against a live database: set_thumb, thumb_scan, define_code_run, define_func_run.
| * tests: blob_ui 39.8s -> 4.1s, and a fatal it was hidingblasty2026-08-074-29/+136
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Three separate wastes, all of the same family: waiting on a guess instead of a signal, and paying for work that never had to be repeated. 1. The 64KB blob was built with os.urandom into a fresh TemporaryDirectory on every run. New bytes at a new path means the pristine-database cache can never apply, so full auto-analysis of 64KB of AArch64-decoded noise was paid every single run. It is now built from a seeded PRNG at a stable path (tests/.synthetic/, gitignored) and staged through the existing cache. Determinism is also a correctness fix: whether 64KB of chance bytes contains something IDA reads as a function is luck, and this suite asserts "and really has no functions". 2. `wait(lambda: lst.model is not old, ..., 30)` after commenting. The perf work made an item edit KEEP the listing's walk and re-render in place, so the model object is never replaced and this waited out its full 30s timeout on every run -- and then "commenting leaves the view where it was" passed vacuously, because nothing had happened at all. A test that burns 30s to check nothing is worse than no test. 3. Two `pause(2.0)`/`pause(2.5)` after a carve, replaced with settle() on a real condition. The second one deliberately has NO predicate: that spot is random data, so the carve may legitimately produce nothing, and "the row became code" would never hold -- gating on it cost another 30s timeout. What that check is about is the VIEW not moving, so the gate is "the app finished reacting". Fixing (1) exposed a real bug in the client, fixed here too: reopening a database that already exists while passing loader switches is FATAL in IDA -- FATAL ERROR: Switch '-b400' can be used only when loading a new file which kills the worker before it can report anything. Loader switches describe an IMPORT and are recorded in the database they produce, so they are now sent only when there is an import to describe. This was never reachable from the old suite (a fresh random blob never had a database to reopen), but it is reachable by any user who opens a raw blob with --ida-args twice. 30 passed, 0 failed.
| * codemode: three defects the A/B benchmark found in the decompiler pathblasty2026-08-072-66/+198
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Benchmarking the port against master op-by-op (rather than only asking whether tests pass) turned up three real bugs, all in the most user-visible path: opening pseudocode. 1. decompile was doing decomp_map's job. It called the full per-column line map purely to fill in each line's /*0xEA*/ anchor. The tool ida-tui was written against takes ONE get_line_item at column 0 per line; the port took one per COLUMN, i.e. thousands of get_line_item+dstr() calls per function instead of one per line. Every pseudocode open cost the same as opening the split view. Carried the real implementation over: 1888ms -> 53ms. 2. decomp_map used the pre-optimisation line map. Ours memoises obj_id -> ea for the whole function (commit 853d90c: dstr() was 79% of the tool, and consecutive columns report the same ctree item), the port's did not. 1925ms -> 287ms. 3. _idatui_compact imported ida_pro_mcp on every call. Under Code Mode that package is not installed in the database process, so the import failed every time -- and a FAILED import is never cached, so each one re-searched the whole of sys.path: 422 failed imports per pc_nums call, which was most of its runtime. 1428ms -> 257ms. The same bug was a correctness bug hiding behind the perf bug: the fallback path collapsed whitespace INSIDE string literals, where the real function preserves it. Pseudocode columns are served in those coordinates, so on any line containing a string with two spaces, every literal's mark and every reformat would have been placed on the wrong column. It never fired on master because ida_pro_mcp is installed there. Now calls the byte-identical module-level shim directly, with the deviation from the extracted original documented in place. Narrow verification: decomp/split_view/opfmt/follow/comment/structs scenarios, 72 passed, 0 failed. Full gate running separately.
| * codemode: rename takes a LIST of edits per category, not just oneblasty2026-08-072-26/+120
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Found by tests/test_rawimage_rpc.py, which the earlier runs had not covered: every rename_many check failed with {"ok": 0, "failed": 2, "errors": [{"addr": null, "error": "list indices must be integers or slices, not str"}]} The port's rename read each category as a single edit (edit["addr"]), but the batch shape is {func: [{addr,name}, ...], data: [...], local/stack: [...]} -- a list per category, with a single dict accepted as shorthand. Indexing the list with "addr" raised, and because the whole category was one try block the error came back attached to addr=null, naming nothing. That is the entire point of the rename_many RPC verb: a firmware image arrives with hundreds of names from a loader map or an emulator's symbols.json, and applying them one at a time costs a navigation plus two prompt round trips each. Only the single-rename UI path worked. Now mirrors the real tool: one row per EDIT (addr/old/name plus a per-row error), a summary counting edits rather than categories, conflict detection before the write, and dry_run/allow_overwrite/stop_on_error. Renaming a function refreshes Hex-Rays' ctext, whose cache is per function and persisted in the .i64 -- without it the pseudocode keeps calling the old name forever while every other readback reports the new one. Clearing a label with an empty new name is kept as a real request (the scenarios revert with it) rather than being rejected as a missing argument. tests/test_rawimage_rpc.py: 14 passed/7 failed -> 21 passed, 0 failed.
| * codemode: carry over the listing + operand-format tools, and stop reshipping ↵blasty2026-08-072-175/+1442
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | them This closes the five operations the port was missing and restores the listing's own tooling instead of a re-implementation of it. idatui/remote_tools.py is the port's IDAPython island: `heads` (the continuous listing) and `op_format`/`pc_nums`/`pc_num_format` (`o`/`O`), extracted verbatim from the BODY that server/patch_server.py used to inject. They are real, diffable source shipped to the database process as text, not string literals, because this is the most performance-tuned and behaviour-sensitive code in the project. Why carry `heads` over rather than keep the port's version: the port's rewrite emitted no per-operand extents ("ops"), so no keypress could show which literal it would reformat (opfmt_highlight had no two-operand row to find); it had no digest/`expect` support, so every page was re-sent after any edit; and its span walk was the per-character loop ours had already been rewritten out of. It also dropped struct-member expansion sizing and the func banner/label rows' exact shapes. The library is installed ONCE per database process (sys.modules, keyed by a hash of the source) and then called by name. Code Mode's execute_python builds a fresh namespace per call, so a library exec'd inline is rebuilt every time and its module-level caches thrown away -- the per-line render lru_cache in particular, which the perf work sized to 65536 entries. Installing it once took `heads` count=200 from 181ms to 92ms; the cache reports 211 hits on a second call where it previously reported none. (Extraction footgun recorded: ast FunctionDef.lineno points at `def`, not at the decorators, so a naive slice silently drops @lru_cache.) Also ported: flowchart, survey_binary, and the xref contract. Live pilot suite on targets/echo: 301 passed, 0 failed -- identical to master. Known, quantified, and NOT fixed here: Code Mode's transport is much slower than the unix-socket worker for the listing's paging. heads count=200 is 2.6ms on master vs 92ms here, count=500 is 6.3ms vs 214ms. Roughly half of that is to_jsonable + HTTP framing per call and is inherent to the architecture; the empty round trip alone is 2ms. The digest/`expect` path (unchanged pages) is the main mitigation and is restored.
| * codemode: port the xref tools' real contract, order includedblasty2026-08-071-32/+111
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The pseudocode follow's address fallback broke: following a call landed on the NEXT LINE instead of the callee (decomp_nav's stale-name check, cur=0x20dd want=0x2060). The port's xref_query returned rows in raw IDA order, and at a call site IDA yields the ordinary-flow xref (fl_F, the next instruction) before the call xref (fl_CN), so 'first code xref' picked the fall-through. The tool ida-tui was written against sorts rows by the far-end address and dedups by default; sorted, 0x2060 precedes 0x210e and the follow is correct. That ordering is load-bearing, so it is now part of the port rather than an accident of the old implementation. Also fixed: the port attached 'fn' to ref.from_ea for both directions, where a from-xref must describe its TARGET (the xref dialog shows the wrong function otherwise), and the envelope was missing direction/addr/total/next_offset/resolved_addr. xref_types (ours, the kind badges in the xref dialog) is ported verbatim and deliberately stays UNsorted -- that dialog lists xrefs in IDA's own order. decomp_nav, follow_xrefs, xref_labels, decomp_follow_self: 15 passed, 0 failed.
| * codemode: restore the graph view and pseudocode commentsblasty2026-08-071-5/+97
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Verified live now: ida-codemode 0.3.1 spawns a managed idalib worker on this box, so the pilot suite runs against the port. flowchart: the port simply does not have the operation, so domain.get_flowchart returned None and every graph key reported 'no control-flow graph for this function'. Ported ours onto ida_gdl (ida-domain exposes no basic-block or edge-kind surface). Blocks stay address RANGES, never text -- that is what lets graph boxes reuse the listing's own rows. Graph suite: 0 -> 50 passed. set_comments: the port set only the disassembly comment via db.comments.set_at(), so a comment never appeared in the pseudocode. A Hex-Rays comment is anchored to a ctree location and an anchor the ctree does not own is discarded as an orphan, so the itp slot must be searched until one sticks, and the entry ea is a function comment instead. Ported that logic back. survey_binary: added as the (caught) fallback domain.py expects behind file_regions, so the fallback path is real rather than always empty.
| * codemode: fix DatabaseHandle.open kwarg, and check kwargs against the real ↵blasty2026-08-072-2/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | signature ida-codemode is now cloned at ../ida-codemode (0.3.1) and installed into ~/ida-venv, so the adapter can be checked against the library instead of against assumptions. First thing it found: connect() passed loading_address=, which DatabaseHandle.open() does not have. The real parameter is image_base, and it already wants the natural 16-byte-aligned address we compute, so this is a rename. Every connect would have died with TypeError on the first call. The port's own contract test could not catch it: its fake handle takes **kwargs, so any keyword at all looks accepted. The test now also validates the keywords we send against inspect.signature(DatabaseHandle.open) when the library is importable, and skips that one check when it is not. Offline suite: 302 passed with the library installed, 302 without it.
| * Rebase MISTER EXO's ida-codemode port onto the current treeblasty2026-08-0734-3992/+7999
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Mechanical part of the port: the 27-file patch was cut against a base ~148 commits behind us, so it did not apply. Resolved 11 conflicts (all of them diff drift, not semantic clashes) and the three file deletions: - app.py: the patch re-inserted _do_rename/_do_name_addr/_seek_split etc. as "theirs" because our tree moved them to edit_ctl.py/trace_ctl.py. Kept ours and applied the real intent (WorkerClient->CodeModeClient, .call->.invoke, _open_worker_client->_open_database_client) at their current homes. - domain.py: kept Head as a NamedTuple -- the patch reverted it to a frozen dataclass, which the perf work measured at 2.9us vs 1.9us per row on a quarter-million-row walk. Dropped _fetch_output (no download_url under Code Mode) and its now-dead urllib/json imports. - pane.py: the patch's deletion swallowed our zellij support along with the worker-reaping block it meant to remove. Kept zellij, removed the reaping. - test_scenarios.py: the idb_save->save_database teardown hunk belongs to tests/_fixtures.py in our tree; applied it there and kept our pc_num_format scenario that the drift landed on. Three defects in the patch itself, fixed here: - It made "import idatui" hard-require ida_codemode, so every offline suite died at import -- including the pure ones (graph/index/trace) that are the house rule for "tests/run.py --fast". The import is now deferred and gated on the binding, which is also what lets the port's own contract tests inject a fake DatabaseHandle. - project.stage() inlined an ida_codemode.registry import and treated "library not installed" as "someone owns this database", which broke IDA-free project staging. Ownership lookup moved to codemode_client.database_owner(). - tests/test_codemode_client.py had no NEEDS_IDA marker, which tests/run.py rejects outright. Offline suite: 301 passed, 0 failed. Against master's 344 the whole delta is accounted for: -40 worker_client (module deleted), -18 launch sweep checks (behaviour deliberately removed) +3 guarding that it stays removed, +2 pool (GUI-save semantics), +13 new codemode_client contract tests. NOT yet done, and the port is not functional without it: the adapter is missing five operations our tree grew since the patch's base (flowchart, op_format, pc_nums, pc_num_format, survey_binary) and its "heads" predates back-walking and digest/expect.
* Stop tracking 157MB of core dumps, and ignore thempre-codemodeblasty2026-08-071-0/+4
| | | | | | Two core dumps were swept into 7e4b593 by an 'add -A' commit. idalib segfaults readily under differential probing (running two revisions of a tool against one cfunc drops two SWIG item objects on the same ctree), so this will recur.
* decomp_map: memoise obj_id -> ea for the whole function instead of only ↵blasty2026-08-071-14/+28
| | | | | | comparing against the previous column. dstr() was 79% of the tool (24us a call) and items interleave, so foo(a, b) flips call->arg->call and re-formatted an item already seen: 106594 calls for 15417 lines of bash. Also corrects run #30's claim that the duplicate ida_hexrays.decompile is what costs -- a warm decompile is 0.01ms. Result: {"status":"keep","total_ms":24513.8,"lg_boot_ms":693.2,"lg_decomp_ms":2347.3,"lg_graph_ms":931.2,"lg_hex_ms":460.3,"lg_index_ms":69.1,"lg_listing_cold_ms":434.2,"lg_listing_warm_ms":462.4,"lg_nav_ms":6639.6,"lg_palette_ms":4.7,"lg_rename_ms":699.9,"lg_render_ms":219.7,"lg_search_ms":3441.1,"lg_split_ms":2218,"pure_graph_ms":216.5,"sm_boot_ms":436.9,"sm_decomp_ms":1264.9,"sm_graph_ms":758.2,"sm_hex_ms":437.9,"sm_index_ms":2.3,"sm_listing_cold_ms":256.8,"sm_listing_warm_ms":256,"sm_nav_ms":309.5,"sm_palette_ms":0.3,"sm_rename_ms":379.1,"sm_render_ms":249.5,"sm_search_ms":59.2,"sm_split_ms":1266,"fails":0}
* The page-freshness check carries the digest the client already holds ↵blasty2026-08-072-30/+30
| | | | | | (heads(expect=...)) instead of asking first and fetching afterwards. A page that has NOT changed costs one round trip as before; a page that HAS changed now costs one instead of two. Also corrects the record: the item-edit bench hang is pilot start-up flakiness, not the _prime/_grow concurrency I blamed it on — proved with a stack dump. Result: {"status":"keep","total_ms":25121.9,"lg_boot_ms":694.8,"lg_decomp_ms":2370.8,"lg_graph_ms":917.1,"lg_hex_ms":456.1,"lg_index_ms":69.4,"lg_listing_cold_ms":442.7,"lg_listing_warm_ms":498.8,"lg_nav_ms":6548.8,"lg_palette_ms":4.8,"lg_rename_ms":711.4,"lg_render_ms":224.9,"lg_search_ms":3484.7,"lg_split_ms":2619.3,"pure_graph_ms":220.7,"sm_boot_ms":448.6,"sm_decomp_ms":1292.5,"sm_graph_ms":748.8,"sm_hex_ms":438.3,"sm_index_ms":2.4,"sm_listing_cold_ms":274.3,"sm_listing_warm_ms":266.5,"sm_nav_ms":302.7,"sm_palette_ms":0.3,"sm_rename_ms":392.5,"sm_render_ms":260.2,"sm_search_ms":59.6,"sm_split_ms":1370.9,"fails":0}
* An item edit (c/d/u/p) keeps the listing's walk in front of it instead of ↵blasty2026-08-072-9/+75
| | | | | | discarding the model. bump_items now takes the edited address; rows before an edit keep their addresses and their row numbers, so only the pages from the edit onward are re-walked. Getting the listing back after undefining at the cursor on bash: 4890ms -> 19ms (257x). Adds .auto/check_edit.py to the gate. total_ms is flat — the bench has no item-edit phase, and the one I wrote hangs (reverted, cause recorded). Result: {"status":"keep","total_ms":25783,"lg_boot_ms":777.9,"lg_decomp_ms":2381.9,"lg_graph_ms":1209.5,"lg_hex_ms":450,"lg_index_ms":68,"lg_listing_cold_ms":434.6,"lg_listing_warm_ms":405.2,"lg_nav_ms":6801.2,"lg_palette_ms":4.7,"lg_rename_ms":744.7,"lg_render_ms":222.4,"lg_search_ms":3885.6,"lg_split_ms":2261.4,"pure_graph_ms":216.4,"sm_boot_ms":463.6,"sm_decomp_ms":1304.6,"sm_graph_ms":740.9,"sm_hex_ms":438.4,"sm_index_ms":2.4,"sm_listing_cold_ms":270.3,"sm_listing_warm_ms":267.2,"sm_nav_ms":312.4,"sm_palette_ms":0.3,"sm_rename_ms":415.8,"sm_render_ms":257.7,"sm_search_ms":60.8,"sm_split_ms":1385.2,"fails":0}
* Three redundancies in the heads walk: item flags were fetched three times ↵blasty2026-08-071-16/+50
| | | | | | per head (row builder, _is_unknown via _advance, and _rows_for), get_func was called per head where a head is nearly always in the same function as the one before it, and the page digest rebuilt a tuple-of-tuples per row where one spans list is shared by ~45% of them. Cold heads 18.62 -> 17.77 us/row, warm 11.53 -> 10.87. Result: {"status":"keep","total_ms":25814,"lg_boot_ms":714.9,"lg_decomp_ms":2376.2,"lg_graph_ms":948.8,"lg_hex_ms":458.6,"lg_index_ms":69.3,"lg_listing_cold_ms":448.5,"lg_listing_warm_ms":482.1,"lg_nav_ms":6808,"lg_palette_ms":4.7,"lg_rename_ms":752,"lg_render_ms":219.4,"lg_search_ms":4008.3,"lg_split_ms":2275.5,"pure_graph_ms":225.2,"sm_boot_ms":456.1,"sm_decomp_ms":1292.4,"sm_graph_ms":795.8,"sm_hex_ms":475.2,"sm_index_ms":2.5,"sm_listing_cold_ms":269.1,"sm_listing_warm_ms":272.2,"sm_nav_ms":320.2,"sm_palette_ms":0.3,"sm_rename_ms":446.3,"sm_render_ms":264.8,"sm_search_ms":60.6,"sm_split_ms":1366.9,"fails":0}
* heads(digest=True): the worker answers "does this page still render exactly ↵blasty2026-08-072-52/+117
| | | | | | as you hold it?" with a hash and a count instead of the page. After a rename nearly every page is unchanged, so the pickling, transfer, unpickling and Head rebuild are all skipped. Redone at PAGE granularity end to end, which fixes both bugs of the first attempt. lg_search 5628 -> 3959. Result: {"status":"keep","total_ms":26491.7,"lg_boot_ms":808.6,"lg_decomp_ms":2600.2,"lg_graph_ms":901.2,"lg_hex_ms":448.7,"lg_index_ms":70.3,"lg_listing_cold_ms":432.8,"lg_listing_warm_ms":445.5,"lg_nav_ms":7004.8,"lg_palette_ms":4.8,"lg_rename_ms":758.6,"lg_render_ms":231.3,"lg_search_ms":3959.2,"lg_split_ms":2659.2,"pure_graph_ms":214.7,"sm_boot_ms":432.6,"sm_decomp_ms":1305.1,"sm_graph_ms":758.3,"sm_hex_ms":431.8,"sm_index_ms":2.3,"sm_listing_cold_ms":275.4,"sm_listing_warm_ms":289.4,"sm_nav_ms":295.6,"sm_palette_ms":0.3,"sm_rename_ms":424.3,"sm_render_ms":263.6,"sm_search_ms":61.1,"sm_split_ms":1411.9,"fails":0}
* Size the worker's per-line render cache to hold a segment's DISTINCT lines ↵blasty2026-08-071-1/+19
| | | | | | (16384 -> 65536, overridable with IDATUI_LINE_CACHE). This was a recorded dead end — it does nothing for a cold sweep — but the rename fix created a second-sweep workload, and re-rendering after a rename is now 21% cheaper. lg_search 7123 -> 5628. Result: {"status":"keep","total_ms":27552.6,"lg_boot_ms":759.2,"lg_decomp_ms":2754.1,"lg_graph_ms":1207.2,"lg_hex_ms":448,"lg_index_ms":69.6,"lg_listing_cold_ms":434.5,"lg_listing_warm_ms":442.9,"lg_nav_ms":6642.3,"lg_palette_ms":4.7,"lg_rename_ms":730.6,"lg_render_ms":223.6,"lg_search_ms":5627.5,"lg_split_ms":2268,"pure_graph_ms":218.7,"sm_boot_ms":465,"sm_decomp_ms":1290.8,"sm_graph_ms":720.7,"sm_hex_ms":436.1,"sm_index_ms":2.4,"sm_listing_cold_ms":267.2,"sm_listing_warm_ms":266.5,"sm_nav_ms":292.9,"sm_palette_ms":0.3,"sm_rename_ms":380.1,"sm_render_ms":255.5,"sm_search_ms":68.1,"sm_split_ms":1276.3,"fails":0}
* CORRECTNESS FIX, kept despite a worse metric. The un-chunked refresh was ↵blasty2026-08-071-11/+24
| | | | | | showing STALE NAMES on any wide read: one heads call for a search-sized window overflows the tool's 2000-row cap, the short response fails the sequence check, and the block is left with its old text. Refresh is now done a block at a time. Adds .auto/check_rename.py to the gate, which fails hard on the previous code and passes on this one. Result: {"status":"keep","total_ms":28651.3,"lg_boot_ms":720.6,"lg_decomp_ms":2427.4,"lg_graph_ms":1222.4,"lg_hex_ms":440.9,"lg_index_ms":72.6,"lg_listing_cold_ms":433.6,"lg_listing_warm_ms":465.9,"lg_nav_ms":6809.7,"lg_palette_ms":4.7,"lg_rename_ms":710.1,"lg_render_ms":228.7,"lg_search_ms":6891.2,"lg_split_ms":2259.2,"pure_graph_ms":214.5,"sm_boot_ms":457.7,"sm_decomp_ms":1303.6,"sm_graph_ms":700.2,"sm_hex_ms":445.4,"sm_index_ms":2.4,"sm_listing_cold_ms":271.4,"sm_listing_warm_ms":270.9,"sm_nav_ms":309.2,"sm_palette_ms":0.3,"sm_rename_ms":386.6,"sm_render_ms":265.8,"sm_search_ms":70.1,"sm_split_ms":1266.2,"fails":0}
* A rename keeps the listing's walk instead of throwing it away. bump_names ↵blasty2026-08-071-5/+133
| | | | | | now marks the rendered text stale (invalidate_text) and ListingModel re-renders a 500-head block at a time on demand, snapped out to whole address groups; a refetch that comes back with a different head sequence sets stale_structure so Program.listing() rebuilds. lg_rename 10055 -> 742. Result: {"status":"keep","total_ms":25563.7,"lg_boot_ms":732.6,"lg_decomp_ms":2714,"lg_graph_ms":919.3,"lg_hex_ms":566.8,"lg_index_ms":71.4,"lg_listing_cold_ms":441.4,"lg_listing_warm_ms":411.7,"lg_nav_ms":6708,"lg_palette_ms":4.8,"lg_rename_ms":741.5,"lg_render_ms":227.7,"lg_search_ms":3314.1,"lg_split_ms":2596.9,"pure_graph_ms":213.6,"sm_boot_ms":422.9,"sm_decomp_ms":1284,"sm_graph_ms":789.5,"sm_hex_ms":468.1,"sm_index_ms":2.6,"sm_listing_cold_ms":258.7,"sm_listing_warm_ms":257.3,"sm_nav_ms":306.5,"sm_palette_ms":0.3,"sm_rename_ms":388.2,"sm_render_ms":256,"sm_search_ms":105.1,"sm_split_ms":1360.5,"fails":0}
* decomp_map: stop sweeping every column three times over. It allocated three ↵blasty2026-08-071-5/+28
| | | | | | ctree_item_t SWIG objects PER COLUMN, swept the tagged line length (124 columns for a 23-column line), and called dstr() — which formats a whole 'EA: description' string — for every column even though consecutive columns report the same ctree item. Now: one item, no head/tail, visible columns only, and dstr() only when the item's obj_id changes. Result: {"status":"keep","total_ms":19834.8,"lg_boot_ms":676.9,"lg_decomp_ms":2698.8,"lg_graph_ms":892.1,"lg_hex_ms":440.4,"lg_index_ms":93.6,"lg_listing_cold_ms":438.5,"lg_listing_warm_ms":420.8,"lg_nav_ms":6646.7,"lg_palette_ms":4.7,"lg_render_ms":215.1,"lg_search_ms":767,"lg_split_ms":1327.6,"pure_graph_ms":213.6,"sm_boot_ms":468,"sm_decomp_ms":1317.1,"sm_graph_ms":730.7,"sm_hex_ms":483.2,"sm_index_ms":2.6,"sm_listing_cold_ms":266.1,"sm_listing_warm_ms":268.1,"sm_nav_ms":288.6,"sm_palette_ms":0.3,"sm_render_ms":253,"sm_search_ms":47.1,"sm_split_ms":874.2,"fails":0}
* Keep a listing row's spans and operand extents exactly as they came off the ↵blasty2026-08-071-11/+16
| | | | | | wire instead of copying them into tuples. The copy re-proved types the worker's own tool guarantees, and it destroyed the object sharing the worker's line cache had created — 228k rows now reference 125k span lists, not 228k private tuples. Result: {"status":"keep","total_ms":17700.4,"lg_boot_ms":776,"lg_decomp_ms":2690.1,"lg_graph_ms":888.1,"lg_hex_ms":449.1,"lg_index_ms":97.4,"lg_listing_cold_ms":439.4,"lg_listing_warm_ms":404.6,"lg_nav_ms":6762.9,"lg_palette_ms":4.6,"lg_render_ms":230.8,"lg_search_ms":762.9,"pure_graph_ms":217.1,"sm_boot_ms":452.6,"sm_decomp_ms":1270,"sm_graph_ms":698,"sm_hex_ms":433.2,"sm_index_ms":2.4,"sm_listing_cold_ms":259.1,"sm_listing_warm_ms":260.5,"sm_nav_ms":300.4,"sm_palette_ms":0.3,"sm_render_ms":258.3,"sm_search_ms":42.7,"fails":0}
* Highlight ranges are computed per line on demand instead of for every match. ↵blasty2026-08-071-16/+79
| | | | | | Searching one character over bash matches 177k lines at 310k places, and all but the forty on screen were built and thrown away. _MatchRanges keeps the line SET eagerly and works out the offsets when a line is painted or the cursor lands on it; the blob scan now also skips to the next line after a hit. Result: {"status":"keep","total_ms":17829.9,"lg_boot_ms":724.4,"lg_decomp_ms":2353.5,"lg_graph_ms":1041,"lg_hex_ms":434.9,"lg_index_ms":95.5,"lg_listing_cold_ms":545.6,"lg_listing_warm_ms":407.9,"lg_nav_ms":6922,"lg_palette_ms":4.8,"lg_render_ms":218.1,"lg_search_ms":884.3,"pure_graph_ms":213.6,"sm_boot_ms":432,"sm_decomp_ms":1250.2,"sm_graph_ms":687.4,"sm_hex_ms":457.5,"sm_index_ms":2.6,"sm_listing_cold_ms":265.4,"sm_listing_warm_ms":288.8,"sm_nav_ms":303.2,"sm_palette_ms":0.3,"sm_render_ms":253.9,"sm_search_ms":43,"fails":0}
* CORRECTNESS REPAIR, kept on its merits. The full suite (which the gate was ↵blasty2026-08-072-16/+38
| | | | | | NOT running) revealed that the worker-connect poll change made test_project_ui flaky: 5ms polling on a background thread through a cold auto-analysis starved the UI thread enough that the loading overlay was still up when the test pressed Ctrl+O. Poll now backs off to a 25ms cap (keeps the boot win, no busy-wait), the racy boot wait is fixed, and checks.sh runs tests/run.py in full (830 checks) instead of just the scenario suite. Result: {"status":"keep","total_ms":18608,"lg_boot_ms":708.5,"lg_decomp_ms":2454.9,"lg_graph_ms":1034.5,"lg_hex_ms":431.9,"lg_index_ms":95.1,"lg_listing_cold_ms":530.2,"lg_listing_warm_ms":413.4,"lg_nav_ms":7057.9,"lg_palette_ms":5,"lg_render_ms":214.2,"lg_search_ms":1408.9,"pure_graph_ms":213.3,"sm_boot_ms":433.6,"sm_decomp_ms":1292,"sm_graph_ms":754,"sm_hex_ms":433.4,"sm_index_ms":2.6,"sm_listing_cold_ms":260.4,"sm_listing_warm_ms":280.5,"sm_nav_ms":286.1,"sm_palette_ms":0.3,"sm_render_ms":251,"sm_search_ms":46.5,"fails":0}
* Three targeted cuts: the graph's transposition pass counts keep and swap in ↵blasty2026-08-072-13/+75
| | | | | | one pass over the neighbour pairs (was four _pair_cross calls); the barycentre median answers degree 1 and 2 without sorting; and the search body is built from windowed model reads instead of one locked row lookup per line. Result: {"status":"keep","total_ms":17944.4,"lg_boot_ms":703.7,"lg_decomp_ms":2453.7,"lg_graph_ms":1000,"lg_hex_ms":478,"lg_index_ms":96,"lg_listing_cold_ms":453,"lg_listing_warm_ms":411.2,"lg_nav_ms":6555.9,"lg_palette_ms":4.9,"lg_render_ms":218.4,"lg_search_ms":1308.8,"pure_graph_ms":212.3,"sm_boot_ms":432.4,"sm_decomp_ms":1267.8,"sm_graph_ms":742.7,"sm_hex_ms":440,"sm_index_ms":2.5,"sm_listing_cold_ms":264.3,"sm_listing_warm_ms":289.5,"sm_nav_ms":305.4,"sm_palette_ms":0.3,"sm_render_ms":258.6,"sm_search_ms":45,"fails":0}
* Re-run of #18 (_CellRow slice assignment, one-string box borders, memoised ↵blasty2026-08-071-14/+42
| | | | | | Style sum), confirming it: 17590 -> 17501. lg_graph averages 951 over the two runs against 1136 before. Result: {"status":"keep","total_ms":17501.4,"lg_boot_ms":711,"lg_decomp_ms":2484.8,"lg_graph_ms":1017.2,"lg_hex_ms":431.9,"lg_index_ms":98.7,"lg_listing_cold_ms":528,"lg_listing_warm_ms":409.2,"lg_nav_ms":6488.2,"lg_palette_ms":4.7,"lg_render_ms":214.8,"lg_search_ms":1473.5,"pure_graph_ms":244.3,"sm_boot_ms":451.4,"sm_decomp_ms":596,"sm_graph_ms":691.3,"sm_hex_ms":444.5,"sm_index_ms":0,"sm_listing_cold_ms":263,"sm_listing_warm_ms":284.2,"sm_nav_ms":374.7,"sm_palette_ms":0.3,"sm_render_ms":252.1,"sm_search_ms":37.5,"fails":0}
* Re-run of #16 (keep the joined search body across a cancelled search and ↵blasty2026-08-071-11/+27
| | | | | | across navigation inside the same segment), confirming it. total 17784 -> 17590; lg_search 1917 -> 1400, sm_search 70 -> 48. Also lands .auto/check_search.py in the checks gate: it compares both search fast paths against the plain per-line loop for every typed prefix. Result: {"status":"keep","total_ms":17589.8,"lg_boot_ms":710.6,"lg_decomp_ms":2404.9,"lg_graph_ms":1136.1,"lg_hex_ms":425.6,"lg_index_ms":103.4,"lg_listing_cold_ms":417.2,"lg_listing_warm_ms":508.1,"lg_nav_ms":6553.8,"lg_palette_ms":4.8,"lg_render_ms":215.7,"lg_search_ms":1400.3,"pure_graph_ms":239.3,"sm_boot_ms":443.2,"sm_decomp_ms":668.4,"sm_graph_ms":698.1,"sm_hex_ms":443.6,"sm_index_ms":0,"sm_listing_cold_ms":267.3,"sm_listing_warm_ms":286,"sm_nav_ms":356.6,"sm_palette_ms":0.3,"sm_render_ms":258.6,"sm_search_ms":47.9,"fails":0}
* HexView.render_line emits style RUNS instead of one Segment per byte cell ↵blasty2026-08-072-12/+39
| | | | | | (35 -> 7 segments per row), and Head is a NamedTuple rather than a frozen dataclass (tuple.__new__ 1.9us vs a dataclass __init__ 2.9us, and it is built once per listing row walked). Result: {"status":"keep","total_ms":17784.1,"lg_boot_ms":680.8,"lg_decomp_ms":2344,"lg_graph_ms":1106.5,"lg_hex_ms":576.4,"lg_index_ms":96.5,"lg_listing_cold_ms":552.9,"lg_listing_warm_ms":431.3,"lg_nav_ms":6256.4,"lg_palette_ms":4.8,"lg_render_ms":224.1,"lg_search_ms":1916.5,"pure_graph_ms":241.3,"sm_boot_ms":442.6,"sm_decomp_ms":591.3,"sm_graph_ms":663.9,"sm_hex_ms":419.2,"sm_index_ms":0,"sm_listing_cold_ms":258.2,"sm_listing_warm_ms":281.8,"sm_nav_ms":374.4,"sm_palette_ms":0.3,"sm_render_ms":251.1,"sm_search_ms":69.8,"fails":0}
* Search the whole segment as ONE joined string. Every line is concatenated ↵blasty2026-08-071-26/+101
| | | | | | once (with a start-offset table) so finding a term is a C-level str.find walk instead of a python loop that rebuilds and case-folds 224k lines per keystroke. Falls back to the per-line loop if case-folding changes the string's length. Result: {"status":"keep","total_ms":18618.9,"lg_boot_ms":689.6,"lg_decomp_ms":2560,"lg_graph_ms":1035.1,"lg_hex_ms":684.2,"lg_index_ms":100.4,"lg_listing_cold_ms":551.1,"lg_listing_warm_ms":397.8,"lg_nav_ms":6687.8,"lg_palette_ms":4.7,"lg_render_ms":211.9,"lg_search_ms":1858.2,"pure_graph_ms":238.5,"sm_boot_ms":431.4,"sm_decomp_ms":671.2,"sm_graph_ms":728.1,"sm_hex_ms":552,"sm_index_ms":0,"sm_listing_cold_ms":257.4,"sm_listing_warm_ms":281,"sm_nav_ms":365.9,"sm_palette_ms":0.3,"sm_render_ms":242.8,"sm_search_ms":69.6,"fails":0}
* Two independent constants: memoise the pygments token -> Rich style lookup ↵blasty2026-08-072-5/+32
| | | | | | (a decompilation uses ~18 distinct token types but each token walked up to nine 'token in ttype' hierarchy checks), and hold the worker-connect poll at 5ms for the first 5s instead of backing off geometrically from the first probe. Result: {"status":"keep","total_ms":18856.8,"lg_boot_ms":689.7,"lg_decomp_ms":2484.2,"lg_graph_ms":1120,"lg_hex_ms":700.1,"lg_index_ms":96.5,"lg_listing_cold_ms":425.6,"lg_listing_warm_ms":511.5,"lg_nav_ms":6590.7,"lg_palette_ms":4.8,"lg_render_ms":215.1,"lg_search_ms":2195.5,"pure_graph_ms":238.1,"sm_boot_ms":431.5,"sm_decomp_ms":667.2,"sm_graph_ms":686.9,"sm_hex_ms":569.8,"sm_index_ms":0,"sm_listing_cold_ms":258.1,"sm_listing_warm_ms":283.2,"sm_nav_ms":365.2,"sm_palette_ms":0.3,"sm_render_ms":243.7,"sm_search_ms":79,"fails":0}
* Fetch a graph's listing rows from the blocks' MERGED EXTENTS, not their ↵blasty2026-08-071-3/+32
| | | | | | convex hull, and assign them per block by bisect. IDA puts a function's cold/tail chunks far from its entry, so the hull of a 1.4KB function could be 680KB wide: it fetched 128k rows, took 3s, and still came back EMPTY for the far blocks because the pager's 64-page bound ran out first. lg_graph 9561 -> 1022. Result: {"status":"keep","total_ms":19062.1,"lg_boot_ms":751.3,"lg_decomp_ms":2605.7,"lg_graph_ms":1022,"lg_hex_ms":554.5,"lg_index_ms":103.9,"lg_listing_cold_ms":545,"lg_listing_warm_ms":405.6,"lg_nav_ms":6636.8,"lg_palette_ms":4.7,"lg_render_ms":212.2,"lg_search_ms":2269,"pure_graph_ms":239.9,"sm_boot_ms":538.5,"sm_decomp_ms":688.4,"sm_graph_ms":712.7,"sm_hex_ms":557.5,"sm_index_ms":0,"sm_listing_cold_ms":259.9,"sm_listing_warm_ms":260.2,"sm_nav_ms":363.8,"sm_palette_ms":0.3,"sm_render_ms":252.7,"sm_search_ms":77.5,"fails":0}
* Only re-apply a scroll after the next refresh when it actually clamped. Both ↵blasty2026-08-071-5/+18
| | | | | | _apply_scroll implementations unconditionally scheduled a deferred scroll_to + refresh(layout=True) — a whole-screen re-arrange on every scroll — as a workaround for scrolling before the view's size is computed. Now the deferred pass runs only when scroll_offset didn't reach the target. Result: {"status":"keep","total_ms":19005.8,"lg_boot_ms":788.9,"lg_decomp_ms":2752.5,"lg_graph_ms":899,"lg_hex_ms":565.2,"lg_index_ms":71.9,"lg_listing_cold_ms":416.1,"lg_listing_warm_ms":402.3,"lg_nav_ms":6756.3,"lg_palette_ms":4.7,"lg_render_ms":217.1,"lg_search_ms":2335.8,"pure_graph_ms":235.3,"sm_boot_ms":537.4,"sm_decomp_ms":600.9,"sm_graph_ms":693.1,"sm_hex_ms":549.9,"sm_index_ms":0,"sm_listing_cold_ms":280.9,"sm_listing_warm_ms":261.2,"sm_nav_ms":309,"sm_palette_ms":0.3,"sm_render_ms":249.3,"sm_search_ms":78.7,"fails":0}
* Two hot-path fixes found by profiling the plain-line builder: the ↵blasty2026-08-072-5/+11
| | | | | | opcode-bytes column used a per-byte f-string generator where bytes.hex(' ').upper() does it in one C call (12x), and ListingModel._phys/_head_index_at re-imported bisect on every call. _line_plain 2.64 -> 1.57 us/row. Result: {"status":"keep","total_ms":19476.3,"lg_boot_ms":756.3,"lg_decomp_ms":2495.1,"lg_graph_ms":946,"lg_hex_ms":905.4,"lg_index_ms":72,"lg_listing_cold_ms":547.5,"lg_listing_warm_ms":410.4,"lg_nav_ms":6723.4,"lg_palette_ms":5,"lg_render_ms":218.8,"lg_search_ms":2312.4,"pure_graph_ms":239.5,"sm_boot_ms":535.3,"sm_decomp_ms":616.4,"sm_graph_ms":682.2,"sm_hex_ms":824.9,"sm_index_ms":0,"sm_listing_cold_ms":258.9,"sm_listing_warm_ms":261,"sm_nav_ms":339.5,"sm_palette_ms":0.3,"sm_render_ms":248.7,"sm_search_ms":77.3,"fails":0}
* Three micro-wins on the listing-row path: merge the colour-tag and ↵blasty2026-08-072-6/+26
| | | | | | operand-tag dicts into one lookup, skip both isspace() probes when a span needs no whitespace collapsing at all (the common case), and give Head slots=True. Spans 11.07 -> 10.18 us/line; Head construction 3.18 -> 2.53 us/row. Result: {"status":"keep","total_ms":20412.8,"lg_boot_ms":752.9,"lg_decomp_ms":2418.6,"lg_graph_ms":811.6,"lg_hex_ms":1020.1,"lg_index_ms":73.2,"lg_listing_cold_ms":573.7,"lg_listing_warm_ms":456.7,"lg_nav_ms":6640.1,"lg_palette_ms":4.8,"lg_render_ms":237.6,"lg_search_ms":3210.2,"pure_graph_ms":242.5,"sm_boot_ms":536.5,"sm_decomp_ms":643.1,"sm_graph_ms":689.8,"sm_hex_ms":866.6,"sm_index_ms":0,"sm_listing_cold_ms":262.9,"sm_listing_warm_ms":265.9,"sm_nav_ms":330.1,"sm_palette_ms":0.3,"sm_render_ms":265.2,"sm_search_ms":110.5,"fails":0}
* Incremental search narrows instead of rescanning. Typing a character onto ↵blasty2026-08-071-14/+51
| | | | | | the term can only remove lines (a line holding "mov" holds "mo"), so _compute_matches rescans the previous hit list when the term grew and nothing else moved. Keyed on (term, case-fold, row count, line-source id) so a listing still streaming rows in behind the search falls back to a full scan. Result: {"status":"keep","total_ms":20835.7,"lg_boot_ms":754.4,"lg_decomp_ms":2593.7,"lg_graph_ms":929.5,"lg_hex_ms":1080.2,"lg_index_ms":76.1,"lg_listing_cold_ms":526.7,"lg_listing_warm_ms":410.1,"lg_nav_ms":6674.8,"lg_palette_ms":5,"lg_render_ms":225.2,"lg_search_ms":3310.3,"pure_graph_ms":239.3,"sm_boot_ms":537.9,"sm_decomp_ms":630.1,"sm_graph_ms":734.3,"sm_hex_ms":869.9,"sm_index_ms":0,"sm_listing_cold_ms":260.8,"sm_listing_warm_ms":262.3,"sm_nav_ms":334.8,"sm_palette_ms":0.3,"sm_render_ms":268.4,"sm_search_ms":111.6,"fails":0}
* Re-apply #5 (lru_cache on the per-line render + Heads built with their ↵blasty2026-08-073-51/+102
| | | | | | opcode bytes already attached) with the graph_minimap scenario's racy SETUP made deterministic: clear _graph_sticky before the second navigation so Space is known to be entering the graph, not leaving it. No assertion changed. Result: {"status":"keep","total_ms":22980.2,"lg_boot_ms":738.2,"lg_decomp_ms":2401.8,"lg_graph_ms":944.1,"lg_hex_ms":920.6,"lg_index_ms":75.2,"lg_listing_cold_ms":538.5,"lg_listing_warm_ms":411.1,"lg_nav_ms":6813.9,"lg_palette_ms":4.9,"lg_render_ms":221.8,"lg_search_ms":5630.1,"pure_graph_ms":240.7,"sm_boot_ms":537.5,"sm_decomp_ms":595.1,"sm_graph_ms":715.7,"sm_hex_ms":858.8,"sm_index_ms":0,"sm_listing_cold_ms":263.3,"sm_listing_warm_ms":265.3,"sm_nav_ms":335.2,"sm_palette_ms":0.3,"sm_render_ms":271.4,"sm_search_ms":196.5,"fails":0}
* _idatui_spans: one capturing re.split over the tag pairs instead of ↵blasty2026-08-071-46/+56
| | | | | | finditer+char-slicing, and collapse whitespace with ' '.join(txt.split()) instead of a regex sub. 13.15 -> 11.07 us/line (the previous finditer attempt was 14.4, i.e. SLOWER than the original char loop it replaced). Result: {"status":"keep","total_ms":26050.2,"lg_boot_ms":727.4,"lg_decomp_ms":2618.1,"lg_graph_ms":935.1,"lg_hex_ms":954.2,"lg_index_ms":67.4,"lg_listing_cold_ms":459.3,"lg_listing_warm_ms":545.7,"lg_nav_ms":9257.8,"lg_palette_ms":4.7,"lg_render_ms":238.8,"lg_search_ms":5472.5,"pure_graph_ms":530.7,"sm_boot_ms":534.6,"sm_decomp_ms":635.8,"sm_graph_ms":748.1,"sm_hex_ms":858.7,"sm_index_ms":0,"sm_listing_cold_ms":268.7,"sm_listing_warm_ms":269.2,"sm_nav_ms":441.6,"sm_palette_ms":0.3,"sm_render_ms":281.7,"sm_search_ms":200,"fails":0}
* Stop ida-pro-mcp installing a sys.setprofile hook around every tool call. ↵blasty2026-08-072-53/+132
| | | | | | Its deadline mechanism profiles every python call/return so a pure-python tool loop can be interrupted; our tools are call-heavy, so it taxed the whole backend 3.3x. Worker now sets IDA_MCP_TOOL_TIMEOUT_SEC=0 and arms the deadline itself with one polling watchdog thread + ida_kernwin.set_cancelled() (the half that actually frees the IDA main thread). Also rewrote _idatui_spans to jump between colour tags instead of walking characters (byte-identical over 258k real lines). Result: {"status":"keep","total_ms":26923.9,"lg_boot_ms":762.2,"lg_decomp_ms":2631.7,"lg_graph_ms":941.8,"lg_hex_ms":1052,"lg_index_ms":67.2,"lg_listing_cold_ms":440.6,"lg_listing_warm_ms":530.5,"lg_nav_ms":10598.3,"lg_palette_ms":4.6,"lg_render_ms":227.8,"lg_search_ms":5265.5,"pure_graph_ms":237.9,"sm_boot_ms":535.3,"sm_decomp_ms":631.8,"sm_graph_ms":702.1,"sm_hex_ms":841.1,"sm_index_ms":0,"sm_listing_cold_ms":268.2,"sm_listing_warm_ms":269.4,"sm_nav_ms":443.6,"sm_palette_ms":0.3,"sm_render_ms":277.8,"sm_search_ms":194.4,"fails":0}
* diag: somewhere for swallowed errors to goblasty2026-08-078-7/+343
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A TUI must not die because one background load failed, so this codebase catches broadly -- ~50 `except Exception` sites, two dozen resolving to `pass`. Right policy, one bad consequence: with 44 `@work(thread=True)` workers, a failure in a background load leaves no trace whatsoever. The view stays empty and there is nothing to read afterwards, because the app owns the screen. kittygfx already solved this for itself with $IDATUI_KITTY_LOG. idatui/diag.py is the same idea for everything else: $IDATUI_LOG writes every swallowed error plus its traceback to a file, and the last 50 are kept in memory regardless so a driver can ask a live app what went wrong. Unset, it costs an environ lookup. Wired in where losing the error changes a DECISION rather than just a pixel: * rename: a resolve() that throws renames as DATA instead of as a function. * name: a function_of() that throws means we never learn the address is a function start, so the index keeps the old name and every readback says the rename didn't happen. * retype: a resolve() that throws retypes the ENCLOSING function instead. * decompile: a failed full-body fetch silently returns CLIPPED pseudocode. * trail: a failed decomp_map stops the pseudocode being painted, silently. Deliberately NOT wired into the query_one guards -- a modal owning the screen is normal and constant, and logging it would bury the real entries in noise. New RPC verb `diag {n?, clear?}`, documented in docs/RPC.md: the answer to "the verb reported success and the pane shows nothing". Also a flake, same shape as the others: follow_xrefs waited on the nav depth but asserted on _cur, and a follow pushes the source entry BEFORE opening the target -- so the check could run in between and see the function it jumped from. About one run in ten. It waits on the postcondition it asserts now; three clean full runs since. 833 checks; --fast is 344 in 3.5s.
* launch: the sweep could delete the file it was asked to openblasty2026-08-072-4/+195
| | | | | | | | | | | | | | | | | | | | | | | | _sweep_locks removes the scratch IDA unpacks beside a .i64 (.id0/.id1/.id2/ .nam/.til) when an open fails, keyed on both the full name and the stem. It never touched the .i64, which is the dangerous one everybody thinks of. It did delete the input. '.til' is an unpacked-DB suffix AND the extension of an IDA type library, so 'ida-tui mylib.til' swept its own argument out of existence -- irreversibly, on a path that runs automatically. Same for anything named *.id0/*.id1/*.id2/*.nam. Now the sweep skips whatever it was asked to open, compared as an absolute path so a relative argument is covered too. tests/test_launch.py pins the whole contract: what it takes, what it must never take (the .i64, the input, the neighbours), and what it reports. Pure, in the --fast tier. It is the right shape of test for code whose failure mode is deleting the wrong file. Also: _load_args parsed the base with bare int(), which raises on the '0x8000000' string a project file writes. Unreachable from our own CLI (which int()s first) but the asymmetry with project._as_addr was a trap, so both go through the same parser now. 813 checks; --fast is 324 in 3.4s.
* worker_client: tests, and stop resurrecting a closed workerblasty2026-08-072-1/+466
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | The layer between the app and idalib had no tests, which is awkward: it is where failures are silent. A worker that dies during startup, a socket that drops mid-call, two UI threads sharing one socket -- none of those look like bugs from outside, they look like the TUI hanging or showing stale data. None of it needs IDA. WorkerClient spawns whatever _WORKER_PY points at, so the suite points it at a fake speaking the same length-prefixed pickle and tells it to misbehave on demand: die at startup, never bind, drop the connection, fail a tool, take its time. 40 checks in the --fast tier. Two things the tests found: call() reconnects when _sock is None, which is what makes a dropped socket recoverable -- but it made an explicitly CLOSED client resurrect too, spawning a whole new idalib worker to serve one stray call (verified: pid 1066961 -> 1066962). close() runs on teardown and on binary-switch while @work threads are still in flight, so quitting during a decompile could leave a fresh process re-opening the .i64 we had just released, which is the wedging hazard. A closed client now refuses; connect() still revives it, which is all _reconnect needs (it builds a new client anyway). connect() polled on a flat 0.2s sleep, so every caller paid a fifth of a second even when the worker was ready in milliseconds -- a seeded .i64, a small binary. Backs off from 5ms instead. 786 checks, 144.6s; --fast is 297 in 3.3s.
* app: the view mode is a type, and 'disasm' is goneblasty2026-08-076-89/+202
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | _active was a bare string with 49 comparisons across four modules and a fifth value nobody meant to keep. "disasm" was assigned on exactly one path -- a decompile that failed with nowhere to return to -- and named the same widget as "listing". Four sites understood it; five compared against "listing" alone and silently took the wrong branch: * Tab out of a failed decompile set "listing" instead of "decomp", so the first press appeared to do nothing. * rpc.py carried a workaround for a mode change that never arrived, keyed on being ALREADY in the ghost state -- so it fired in the rare case and not in the common one. Now keyed on LISTING, which is the case that happens. * drive.py asked the socket to show it "disasm", a value the app will now never report, and would have toggled twice and given up. ViewMode is a StrEnum on purpose: _active goes straight to drivers as cursor.kind and the pilot compares it to plain strings, so members being strings keeps every payload and comparison working. What it buys is one place that says which modes exist, and an AttributeError instead of silence on a typo. Read it through is_listing/is_decomp/is_hex/is_graph/in_code rather than ==. The bare comparisons are what let the ghost hide, and they are what the next mode would have to hunt down -- adding "graph" already cost one crash that way (_active_code_view returning None when a prompt closed). view_modes_all_handled walks the enum and asks the app the questions it asks itself. Verified it bites: adding a fifth unhandled member fails it twice. 746 checks, 142.3s.
* tests: a guard that actually regresses on the resync stormblasty2026-08-062-3/+34
| | | | | | | | | | | | | | | | | The split resync loop (f898350) had no test. Two attempts at one were worthless and are not in this commit: a scroll-based guard passed with the bug reintroduced, and a constructed anchor -- inside the loaded function, outside its mapped span -- skipped, because on this target the map covers the whole function. The real trigger is the race window while the decomp map lags the decompiler re-pointing, which is tedious to force but wide open in split_view's own flow. So split_view counts lookup_funcs across its body and bounds it. Verified both ways, which is the only reason it's worth having: 29,227 calls with the bug put back, under 500 with the fix. The bound is loose because the bug was three orders of magnitude out, not a near miss. 733 checks, 139s.
* tests: deterministic fixtures, and a correctionblasty2026-08-061-2/+16
| | | | | | | | | | | | | | | | | | | | | | all_funcs() forced a full load of the function index only when it was EMPTY, so a partially streamed index -- non-empty but incomplete, which is exactly the state during boot and after any bump_items() -- came back truncated. Every fixture picked through find_func/biggest therefore depended on how far streaming had got by the time a scenario asked. That is the graph_minimap flake: on an unlucky run find_func(size > 0x300) picked a much larger function than usual, whose graph never finished inside the scenario's own 60s wait. Three failures and 65 seconds, one run in several, with no code change to blame. Three consecutive clean runs at 1.7s since. CORRECTION to f898350, which said a range cache for function_of 'broke graph_minimap'. It did not. The failure happened in the run after I added the cache and I attributed it without checking; it recurred with the cache long gone. The cache is still not here, but for the honest reason: with the resync loop fixed, function_of is down to 340 calls and 1.4s across the whole suite, so caching it is not worth the invalidation surface. Suite 195.7s -> 138.4s, 732 checks.
* tests: wait for the thing, don't sleep and hopeblasty2026-08-064-46/+140
| | | | | | | | | | | | | | | | | | | | | | test_trace_ui spent 18.8 of its 35.2 seconds in flat pilot.pause() calls placed to let an async seek land. Two loops were most of it: 6 iterations at 0.5s and 28 at 0.3s, 11.4s of sleeping to check that a step moves the cursor. They are condition waits now. The questions are unchanged -- does the listing cursor reach the pc, does the pseudocode cursor follow -- but they cost what they cost instead of a fixed budget. The second loop settles on something that does NOT presuppose the answer (the listing cursor arriving, and the trail map belonging to the loaded function): waiting on 'is this pc mapped' would have burned the timeout on every unmapped instruction, about half of them, and come out slower than the sleep it replaced. 35.2s -> 20.9s, 39 checks, stable over repeated runs. tests/_fixtures.py collects the staging both this suite and test_scenarios need -- scratch copy, seeded from a golden .i64 nothing writes back to -- which was private to test_scenarios. Worth saying plainly: on targets/echo the seeding is worth 0.19s, not the analysis time I assumed when I went looking. It is shared for the deduplication and for whatever gets pointed at a bigger binary.
* split: stop the resync loop that spun the worker foreverblasty2026-08-061-6/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | _split_range is the min/max of the decomp_map's addresses, which does not cover every address in the function -- Hex-Rays doesn't attribute them all. An anchor inside the loaded function but outside that span therefore asked _sync_split for a resync, _apply_resync found the function already decompiled, called _sync_split again, and it asked again. One thread worker and one lookup_funcs round trip per iteration, for as long as the cursor sat there. Measured in the pilot: 23,888 function_of calls in one scenario across FOUR distinct addresses, 21,156 of them for 0x2060 alone. In the live app that is an idle split view pegging the worker. _sync_split grows a resync flag; the one caller that is itself the resync passes resync=False, so the branch can be entered at most once per chain. While measuring, three scenarios waited on "fail" appearing in the status -- the app says "cannot decompile". decomp_fallback burned its full 25s timeout and then passed a check on _active == "listing", which was already true before Tab was pressed: it asserted nothing, slowly. Now waits for the real text and checks that the fallback actually said something. scenarios 115.8s -> 74.9s, suite 195.7s -> 153.3s, 732 checks green. Not included: a range cache for function_of. It broke graph_minimap (the graph stopped loading at all -- the 65s was that scenario's own 60s wait timing out) and with the loop gone it buys little. Left out rather than shipped half-understood.
generated by cgit v1.3.1-sl0p (git 2.54.0) at 2026-08-28 08:59:05 +0000 — sl0p fork