aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* codemode: carry over the listing + operand-format tools, and stop reshipping ↵blasty3 days2-175/+1442
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | them This closes the five operations the port was missing and restores the listing's own tooling instead of a re-implementation of it. idatui/remote_tools.py is the port's IDAPython island: `heads` (the continuous listing) and `op_format`/`pc_nums`/`pc_num_format` (`o`/`O`), extracted verbatim from the BODY that server/patch_server.py used to inject. They are real, diffable source shipped to the database process as text, not string literals, because this is the most performance-tuned and behaviour-sensitive code in the project. Why carry `heads` over rather than keep the port's version: the port's rewrite emitted no per-operand extents ("ops"), so no keypress could show which literal it would reformat (opfmt_highlight had no two-operand row to find); it had no digest/`expect` support, so every page was re-sent after any edit; and its span walk was the per-character loop ours had already been rewritten out of. It also dropped struct-member expansion sizing and the func banner/label rows' exact shapes. The library is installed ONCE per database process (sys.modules, keyed by a hash of the source) and then called by name. Code Mode's execute_python builds a fresh namespace per call, so a library exec'd inline is rebuilt every time and its module-level caches thrown away -- the per-line render lru_cache in particular, which the perf work sized to 65536 entries. Installing it once took `heads` count=200 from 181ms to 92ms; the cache reports 211 hits on a second call where it previously reported none. (Extraction footgun recorded: ast FunctionDef.lineno points at `def`, not at the decorators, so a naive slice silently drops @lru_cache.) Also ported: flowchart, survey_binary, and the xref contract. Live pilot suite on targets/echo: 301 passed, 0 failed -- identical to master. Known, quantified, and NOT fixed here: Code Mode's transport is much slower than the unix-socket worker for the listing's paging. heads count=200 is 2.6ms on master vs 92ms here, count=500 is 6.3ms vs 214ms. Roughly half of that is to_jsonable + HTTP framing per call and is inherent to the architecture; the empty round trip alone is 2ms. The digest/`expect` path (unchanged pages) is the main mitigation and is restored.
* codemode: port the xref tools' real contract, order includedblasty3 days1-32/+111
| | | | | | | | | | | | | | | | | | | | | The pseudocode follow's address fallback broke: following a call landed on the NEXT LINE instead of the callee (decomp_nav's stale-name check, cur=0x20dd want=0x2060). The port's xref_query returned rows in raw IDA order, and at a call site IDA yields the ordinary-flow xref (fl_F, the next instruction) before the call xref (fl_CN), so 'first code xref' picked the fall-through. The tool ida-tui was written against sorts rows by the far-end address and dedups by default; sorted, 0x2060 precedes 0x210e and the follow is correct. That ordering is load-bearing, so it is now part of the port rather than an accident of the old implementation. Also fixed: the port attached 'fn' to ref.from_ea for both directions, where a from-xref must describe its TARGET (the xref dialog shows the wrong function otherwise), and the envelope was missing direction/addr/total/next_offset/resolved_addr. xref_types (ours, the kind badges in the xref dialog) is ported verbatim and deliberately stays UNsorted -- that dialog lists xrefs in IDA's own order. decomp_nav, follow_xrefs, xref_labels, decomp_follow_self: 15 passed, 0 failed.
* codemode: restore the graph view and pseudocode commentsblasty3 days1-5/+97
| | | | | | | | | | | | | | | | | | | | Verified live now: ida-codemode 0.3.1 spawns a managed idalib worker on this box, so the pilot suite runs against the port. flowchart: the port simply does not have the operation, so domain.get_flowchart returned None and every graph key reported 'no control-flow graph for this function'. Ported ours onto ida_gdl (ida-domain exposes no basic-block or edge-kind surface). Blocks stay address RANGES, never text -- that is what lets graph boxes reuse the listing's own rows. Graph suite: 0 -> 50 passed. set_comments: the port set only the disassembly comment via db.comments.set_at(), so a comment never appeared in the pseudocode. A Hex-Rays comment is anchored to a ctree location and an anchor the ctree does not own is discarded as an orphan, so the itp slot must be searched until one sticks, and the entry ea is a function comment instead. Ported that logic back. survey_binary: added as the (caught) fallback domain.py expects behind file_regions, so the fallback path is real rather than always empty.
* codemode: fix DatabaseHandle.open kwarg, and check kwargs against the real ↵blasty3 days2-2/+27
| | | | | | | | | | | | | | | | | | | | signature ida-codemode is now cloned at ../ida-codemode (0.3.1) and installed into ~/ida-venv, so the adapter can be checked against the library instead of against assumptions. First thing it found: connect() passed loading_address=, which DatabaseHandle.open() does not have. The real parameter is image_base, and it already wants the natural 16-byte-aligned address we compute, so this is a rename. Every connect would have died with TypeError on the first call. The port's own contract test could not catch it: its fake handle takes **kwargs, so any keyword at all looks accepted. The test now also validates the keywords we send against inspect.signature(DatabaseHandle.open) when the library is importable, and skips that one check when it is not. Offline suite: 302 passed with the library installed, 302 without it.
* Rebase MISTER EXO's ida-codemode port onto the current treeblasty3 days34-3992/+7999
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Mechanical part of the port: the 27-file patch was cut against a base ~148 commits behind us, so it did not apply. Resolved 11 conflicts (all of them diff drift, not semantic clashes) and the three file deletions: - app.py: the patch re-inserted _do_rename/_do_name_addr/_seek_split etc. as "theirs" because our tree moved them to edit_ctl.py/trace_ctl.py. Kept ours and applied the real intent (WorkerClient->CodeModeClient, .call->.invoke, _open_worker_client->_open_database_client) at their current homes. - domain.py: kept Head as a NamedTuple -- the patch reverted it to a frozen dataclass, which the perf work measured at 2.9us vs 1.9us per row on a quarter-million-row walk. Dropped _fetch_output (no download_url under Code Mode) and its now-dead urllib/json imports. - pane.py: the patch's deletion swallowed our zellij support along with the worker-reaping block it meant to remove. Kept zellij, removed the reaping. - test_scenarios.py: the idb_save->save_database teardown hunk belongs to tests/_fixtures.py in our tree; applied it there and kept our pc_num_format scenario that the drift landed on. Three defects in the patch itself, fixed here: - It made "import idatui" hard-require ida_codemode, so every offline suite died at import -- including the pure ones (graph/index/trace) that are the house rule for "tests/run.py --fast". The import is now deferred and gated on the binding, which is also what lets the port's own contract tests inject a fake DatabaseHandle. - project.stage() inlined an ida_codemode.registry import and treated "library not installed" as "someone owns this database", which broke IDA-free project staging. Ownership lookup moved to codemode_client.database_owner(). - tests/test_codemode_client.py had no NEEDS_IDA marker, which tests/run.py rejects outright. Offline suite: 301 passed, 0 failed. Against master's 344 the whole delta is accounted for: -40 worker_client (module deleted), -18 launch sweep checks (behaviour deliberately removed) +3 guarding that it stays removed, +2 pool (GUI-save semantics), +13 new codemode_client contract tests. NOT yet done, and the port is not functional without it: the adapter is missing five operations our tree grew since the patch's base (flowchart, op_format, pc_nums, pc_num_format, survey_binary) and its "heads" predates back-walking and digest/expect.
* Stop tracking 157MB of core dumps, and ignore thempre-codemodeuser3 days1-0/+4
| | | | | | Two core dumps were swept into 7e4b593 by an 'add -A' commit. idalib segfaults readily under differential probing (running two revisions of a tool against one cfunc drops two SWIG item objects on the same ctree), so this will recur.
* Park two proven-but-unresolvable F5-path optimisations, and record how to ↵user3 days5-0/+386
| | | | | | | | | | | | | | | | | | spot a counter-drift outlier pc_nums allocated three ctree_item_t SWIG objects per candidate column -- the third instance of the same fault already fixed in decomp_map and found in decompile_function_safe. 1247 -> 614ms warm over 18991 lines of bash, identical literal counts, 0 mismatches over echo's 128 functions. Both it and the fast decompile_function_safe are parked rather than applied: together they are worth ~350ms against a run-to-run spread of ~500ms on this box (means 25045 with, 25140 without over seven runs), so the benchmark cannot resolve them. Neither adds complexity -- both remove allocations -- so they are kept on disk with their measurements for a per-operation-latency goal. Also records the 27283ms outlier: the work counters moved with it, which is how an outlier is told from a regression.
* decomp_map: memoise obj_id -> ea for the whole function instead of only ↵user3 days3-14/+80
| | | | | | comparing against the previous column. dstr() was 79% of the tool (24us a call) and items interleave, so foo(a, b) flips call->arg->call and re-formatted an item already seen: 106594 calls for 15417 lines of bash. Also corrects run #30's claim that the duplicate ida_hexrays.decompile is what costs -- a warm decompile is 0.01ms. Result: {"status":"keep","total_ms":24513.8,"lg_boot_ms":693.2,"lg_decomp_ms":2347.3,"lg_graph_ms":931.2,"lg_hex_ms":460.3,"lg_index_ms":69.1,"lg_listing_cold_ms":434.2,"lg_listing_warm_ms":462.4,"lg_nav_ms":6639.6,"lg_palette_ms":4.7,"lg_rename_ms":699.9,"lg_render_ms":219.7,"lg_search_ms":3441.1,"lg_split_ms":2218,"pure_graph_ms":216.5,"sm_boot_ms":436.9,"sm_decomp_ms":1264.9,"sm_graph_ms":758.2,"sm_hex_ms":437.9,"sm_index_ms":2.3,"sm_listing_cold_ms":256.8,"sm_listing_warm_ms":256,"sm_nav_ms":309.5,"sm_palette_ms":0.3,"sm_rename_ms":379.1,"sm_render_ms":249.5,"sm_search_ms":59.2,"sm_split_ms":1266,"fails":0}
* The page-freshness check carries the digest the client already holds ↵user3 days4-39/+47
| | | | | | (heads(expect=...)) instead of asking first and fetching afterwards. A page that has NOT changed costs one round trip as before; a page that HAS changed now costs one instead of two. Also corrects the record: the item-edit bench hang is pilot start-up flakiness, not the _prime/_grow concurrency I blamed it on — proved with a stack dump. Result: {"status":"keep","total_ms":25121.9,"lg_boot_ms":694.8,"lg_decomp_ms":2370.8,"lg_graph_ms":917.1,"lg_hex_ms":456.1,"lg_index_ms":69.4,"lg_listing_cold_ms":442.7,"lg_listing_warm_ms":498.8,"lg_nav_ms":6548.8,"lg_palette_ms":4.8,"lg_rename_ms":711.4,"lg_render_ms":224.9,"lg_search_ms":3484.7,"lg_split_ms":2619.3,"pure_graph_ms":220.7,"sm_boot_ms":448.6,"sm_decomp_ms":1292.5,"sm_graph_ms":748.8,"sm_hex_ms":438.3,"sm_index_ms":2.4,"sm_listing_cold_ms":274.3,"sm_listing_warm_ms":266.5,"sm_nav_ms":302.7,"sm_palette_ms":0.3,"sm_rename_ms":392.5,"sm_render_ms":260.2,"sm_search_ms":59.6,"sm_split_ms":1370.9,"fails":0}
* An item edit (c/d/u/p) keeps the listing's walk in front of it instead of ↵user3 days6-10/+234
| | | | | | discarding the model. bump_items now takes the edited address; rows before an edit keep their addresses and their row numbers, so only the pages from the edit onward are re-walked. Getting the listing back after undefining at the cursor on bash: 4890ms -> 19ms (257x). Adds .auto/check_edit.py to the gate. total_ms is flat — the bench has no item-edit phase, and the one I wrote hangs (reverted, cause recorded). Result: {"status":"keep","total_ms":25783,"lg_boot_ms":777.9,"lg_decomp_ms":2381.9,"lg_graph_ms":1209.5,"lg_hex_ms":450,"lg_index_ms":68,"lg_listing_cold_ms":434.6,"lg_listing_warm_ms":405.2,"lg_nav_ms":6801.2,"lg_palette_ms":4.7,"lg_rename_ms":744.7,"lg_render_ms":222.4,"lg_search_ms":3885.6,"lg_split_ms":2261.4,"pure_graph_ms":216.4,"sm_boot_ms":463.6,"sm_decomp_ms":1304.6,"sm_graph_ms":740.9,"sm_hex_ms":438.4,"sm_index_ms":2.4,"sm_listing_cold_ms":270.3,"sm_listing_warm_ms":267.2,"sm_nav_ms":312.4,"sm_palette_ms":0.3,"sm_rename_ms":415.8,"sm_render_ms":257.7,"sm_search_ms":60.8,"sm_split_ms":1385.2,"fails":0}
* Three redundancies in the heads walk: item flags were fetched three times ↵user3 days4-17/+80
| | | | | | per head (row builder, _is_unknown via _advance, and _rows_for), get_func was called per head where a head is nearly always in the same function as the one before it, and the page digest rebuilt a tuple-of-tuples per row where one spans list is shared by ~45% of them. Cold heads 18.62 -> 17.77 us/row, warm 11.53 -> 10.87. Result: {"status":"keep","total_ms":25814,"lg_boot_ms":714.9,"lg_decomp_ms":2376.2,"lg_graph_ms":948.8,"lg_hex_ms":458.6,"lg_index_ms":69.3,"lg_listing_cold_ms":448.5,"lg_listing_warm_ms":482.1,"lg_nav_ms":6808,"lg_palette_ms":4.7,"lg_rename_ms":752,"lg_render_ms":219.4,"lg_search_ms":4008.3,"lg_split_ms":2275.5,"pure_graph_ms":225.2,"sm_boot_ms":456.1,"sm_decomp_ms":1292.4,"sm_graph_ms":795.8,"sm_hex_ms":475.2,"sm_index_ms":2.5,"sm_listing_cold_ms":269.1,"sm_listing_warm_ms":272.2,"sm_nav_ms":320.2,"sm_palette_ms":0.3,"sm_rename_ms":446.3,"sm_render_ms":264.8,"sm_search_ms":60.6,"sm_split_ms":1366.9,"fails":0}
* heads(digest=True): the worker answers "does this page still render exactly ↵user3 days6-61/+298
| | | | | | as you hold it?" with a hash and a count instead of the page. After a rename nearly every page is unchanged, so the pickling, transfer, unpickling and Head rebuild are all skipped. Redone at PAGE granularity end to end, which fixes both bugs of the first attempt. lg_search 5628 -> 3959. Result: {"status":"keep","total_ms":26491.7,"lg_boot_ms":808.6,"lg_decomp_ms":2600.2,"lg_graph_ms":901.2,"lg_hex_ms":448.7,"lg_index_ms":70.3,"lg_listing_cold_ms":432.8,"lg_listing_warm_ms":445.5,"lg_nav_ms":7004.8,"lg_palette_ms":4.8,"lg_rename_ms":758.6,"lg_render_ms":231.3,"lg_search_ms":3959.2,"lg_split_ms":2659.2,"pure_graph_ms":214.7,"sm_boot_ms":432.6,"sm_decomp_ms":1305.1,"sm_graph_ms":758.3,"sm_hex_ms":431.8,"sm_index_ms":2.3,"sm_listing_cold_ms":275.4,"sm_listing_warm_ms":289.4,"sm_nav_ms":295.6,"sm_palette_ms":0.3,"sm_rename_ms":424.3,"sm_render_ms":263.6,"sm_search_ms":61.1,"sm_split_ms":1411.9,"fails":0}
* Size the worker's per-line render cache to hold a segment's DISTINCT lines ↵user3 days3-3/+27
| | | | | | (16384 -> 65536, overridable with IDATUI_LINE_CACHE). This was a recorded dead end — it does nothing for a cold sweep — but the rename fix created a second-sweep workload, and re-rendering after a rename is now 21% cheaper. lg_search 7123 -> 5628. Result: {"status":"keep","total_ms":27552.6,"lg_boot_ms":759.2,"lg_decomp_ms":2754.1,"lg_graph_ms":1207.2,"lg_hex_ms":448,"lg_index_ms":69.6,"lg_listing_cold_ms":434.5,"lg_listing_warm_ms":442.9,"lg_nav_ms":6642.3,"lg_palette_ms":4.7,"lg_rename_ms":730.6,"lg_render_ms":223.6,"lg_search_ms":5627.5,"lg_split_ms":2268,"pure_graph_ms":218.7,"sm_boot_ms":465,"sm_decomp_ms":1290.8,"sm_graph_ms":720.7,"sm_hex_ms":436.1,"sm_index_ms":2.4,"sm_listing_cold_ms":267.2,"sm_listing_warm_ms":266.5,"sm_nav_ms":292.9,"sm_palette_ms":0.3,"sm_rename_ms":380.1,"sm_render_ms":255.5,"sm_search_ms":68.1,"sm_split_ms":1276.3,"fails":0}
* autoresearch: record the trace-memory scaling findingblasty3 days1-0/+24
|
* autoresearch: final budget and headline numbers in the playbookblasty3 days2-20/+21
|
* autoresearch: record the streaming-responsiveness finding and a pre-existing ↵blasty3 days2-0/+36
| | | | palette crash
* autoresearch: record the split-view and rename findings, and what is still ↵blasty3 days3-14/+63
| | | | unmeasured
* CORRECTNESS FIX, kept despite a worse metric. The un-chunked refresh was ↵user3 days5-12/+212
| | | | | | showing STALE NAMES on any wide read: one heads call for a search-sized window overflows the tool's 2000-row cap, the short response fails the sequence check, and the block is left with its old text. Refresh is now done a block at a time. Adds .auto/check_rename.py to the gate, which fails hard on the previous code and passes on this one. Result: {"status":"keep","total_ms":28651.3,"lg_boot_ms":720.6,"lg_decomp_ms":2427.4,"lg_graph_ms":1222.4,"lg_hex_ms":440.9,"lg_index_ms":72.6,"lg_listing_cold_ms":433.6,"lg_listing_warm_ms":465.9,"lg_nav_ms":6809.7,"lg_palette_ms":4.7,"lg_rename_ms":710.1,"lg_render_ms":228.7,"lg_search_ms":6891.2,"lg_split_ms":2259.2,"pure_graph_ms":214.5,"sm_boot_ms":457.7,"sm_decomp_ms":1303.6,"sm_graph_ms":700.2,"sm_hex_ms":445.4,"sm_index_ms":2.4,"sm_listing_cold_ms":271.4,"sm_listing_warm_ms":270.9,"sm_nav_ms":309.2,"sm_palette_ms":0.3,"sm_rename_ms":386.6,"sm_render_ms":265.8,"sm_search_ms":70.1,"sm_split_ms":1266.2,"fails":0}
* A rename keeps the listing's walk instead of throwing it away. bump_names ↵user3 days2-5/+134
| | | | | | now marks the rendered text stale (invalidate_text) and ListingModel re-renders a 500-head block at a time on demand, snapped out to whole address groups; a refetch that comes back with a different head sequence sets stale_structure so Program.listing() rebuilds. lg_rename 10055 -> 742. Result: {"status":"keep","total_ms":25563.7,"lg_boot_ms":732.6,"lg_decomp_ms":2714,"lg_graph_ms":919.3,"lg_hex_ms":566.8,"lg_index_ms":71.4,"lg_listing_cold_ms":441.4,"lg_listing_warm_ms":411.7,"lg_nav_ms":6708,"lg_palette_ms":4.8,"lg_rename_ms":741.5,"lg_render_ms":227.7,"lg_search_ms":3314.1,"lg_split_ms":2596.9,"pure_graph_ms":213.6,"sm_boot_ms":422.9,"sm_decomp_ms":1284,"sm_graph_ms":789.5,"sm_hex_ms":468.1,"sm_index_ms":2.6,"sm_listing_cold_ms":258.7,"sm_listing_warm_ms":257.3,"sm_nav_ms":306.5,"sm_palette_ms":0.3,"sm_rename_ms":388.2,"sm_render_ms":256,"sm_search_ms":105.1,"sm_split_ms":1360.5,"fails":0}
* RE-BASELINE (v7 bench). Rename — the commonest operation in reverse ↵user3 days2-0/+213
| | | | | | engineering — was not covered, and it costs 10.1s for SIX renames on bash (1.7s each) because bump_names discards the segment's ListingModel and the reload re-walks it from the start. lg_split also rose to 3540ms: the split phase now runs after renames have thrown the listing away. Result: {"status":"keep","total_ms":33242.6,"lg_boot_ms":710.8,"lg_decomp_ms":2401.2,"lg_graph_ms":898.1,"lg_hex_ms":448.7,"lg_index_ms":72.1,"lg_listing_cold_ms":439.3,"lg_listing_warm_ms":426.8,"lg_nav_ms":7060.2,"lg_palette_ms":4.8,"lg_rename_ms":10055,"lg_render_ms":220.2,"lg_search_ms":774.6,"lg_split_ms":3539.8,"pure_graph_ms":215.1,"sm_boot_ms":429.1,"sm_decomp_ms":1244.1,"sm_graph_ms":744.3,"sm_hex_ms":444.2,"sm_index_ms":2.5,"sm_listing_cold_ms":269.8,"sm_listing_warm_ms":263.3,"sm_nav_ms":282.9,"sm_palette_ms":0.3,"sm_rename_ms":638.3,"sm_render_ms":255.6,"sm_search_ms":44.2,"sm_split_ms":1357.4,"fails":0}
* bench: cover a rename and the listing's recovery from itblasty3 days1-0/+66
|
* decomp_map: stop sweeping every column three times over. It allocated three ↵user3 days2-5/+29
| | | | | | ctree_item_t SWIG objects PER COLUMN, swept the tagged line length (124 columns for a 23-column line), and called dstr() — which formats a whole 'EA: description' string — for every column even though consecutive columns report the same ctree item. Now: one item, no head/tail, visible columns only, and dstr() only when the item's obj_id changes. Result: {"status":"keep","total_ms":19834.8,"lg_boot_ms":676.9,"lg_decomp_ms":2698.8,"lg_graph_ms":892.1,"lg_hex_ms":440.4,"lg_index_ms":93.6,"lg_listing_cold_ms":438.5,"lg_listing_warm_ms":420.8,"lg_nav_ms":6646.7,"lg_palette_ms":4.7,"lg_render_ms":215.1,"lg_search_ms":767,"lg_split_ms":1327.6,"pure_graph_ms":213.6,"sm_boot_ms":468,"sm_decomp_ms":1317.1,"sm_graph_ms":730.7,"sm_hex_ms":483.2,"sm_index_ms":2.6,"sm_listing_cold_ms":266.1,"sm_listing_warm_ms":268.1,"sm_nav_ms":288.6,"sm_palette_ms":0.3,"sm_render_ms":253,"sm_search_ms":47.1,"sm_split_ms":874.2,"fails":0}
* RE-BASELINE (v6 bench). The split view ('s') was not covered at all, and it ↵user3 days2-0/+51
| | | | | | turns out to be the most expensive thing in the app: 16.2s of a 33.5s session (lg_split 10189 + sm_split 6034), or 500ms per function on echo and 850ms on bash, just to open it. Result: {"status":"keep","total_ms":33502.3,"lg_boot_ms":772.2,"lg_decomp_ms":2546.9,"lg_graph_ms":903.4,"lg_hex_ms":434.1,"lg_index_ms":97.7,"lg_listing_cold_ms":420.2,"lg_listing_warm_ms":397.4,"lg_nav_ms":6501.1,"lg_palette_ms":4.9,"lg_render_ms":212.4,"lg_search_ms":760.4,"lg_split_ms":10189.2,"pure_graph_ms":212.3,"sm_boot_ms":443.8,"sm_decomp_ms":1275.3,"sm_graph_ms":720.5,"sm_hex_ms":451,"sm_index_ms":2.5,"sm_listing_cold_ms":263.3,"sm_listing_warm_ms":264.7,"sm_nav_ms":302.8,"sm_palette_ms":0.3,"sm_render_ms":248.4,"sm_search_ms":43.9,"sm_split_ms":6033.6,"fails":0}
* bench: cover the split view ('s'), whose decomp_map cost was entirely unmeasuredblasty3 days1-0/+48
|
* Confirmation re-run of #27, no code change: 17700 -> 17465, the best v5 ↵user3 days1-0/+1
| | | | | | reading. Confirms the wire-shape change holds and that the run-to-run spread is ~250ms even on the now-busier box. Result: {"status":"keep","total_ms":17464.6,"lg_boot_ms":744.8,"lg_decomp_ms":2600.3,"lg_graph_ms":889.3,"lg_hex_ms":436.1,"lg_index_ms":95.5,"lg_listing_cold_ms":436.4,"lg_listing_warm_ms":407.8,"lg_nav_ms":6747.4,"lg_palette_ms":4.6,"lg_render_ms":218.9,"lg_search_ms":760.9,"pure_graph_ms":215.7,"sm_boot_ms":429.7,"sm_decomp_ms":1263.5,"sm_graph_ms":650.2,"sm_hex_ms":435.3,"sm_index_ms":2.3,"sm_listing_cold_ms":262.8,"sm_listing_warm_ms":264.7,"sm_nav_ms":305.4,"sm_palette_ms":0.3,"sm_render_ms":251.4,"sm_search_ms":41.1,"fails":0}
* Keep a listing row's spans and operand extents exactly as they came off the ↵user3 days1-11/+16
| | | | | | wire instead of copying them into tuples. The copy re-proved types the worker's own tool guarantees, and it destroyed the object sharing the worker's line cache had created — 228k rows now reference 125k span lists, not 228k private tuples. Result: {"status":"keep","total_ms":17700.4,"lg_boot_ms":776,"lg_decomp_ms":2690.1,"lg_graph_ms":888.1,"lg_hex_ms":449.1,"lg_index_ms":97.4,"lg_listing_cold_ms":439.4,"lg_listing_warm_ms":404.6,"lg_nav_ms":6762.9,"lg_palette_ms":4.6,"lg_render_ms":230.8,"lg_search_ms":762.9,"pure_graph_ms":217.1,"sm_boot_ms":452.6,"sm_decomp_ms":1270,"sm_graph_ms":698,"sm_hex_ms":433.2,"sm_index_ms":2.4,"sm_listing_cold_ms":259.1,"sm_listing_warm_ms":260.5,"sm_nav_ms":300.4,"sm_palette_ms":0.3,"sm_render_ms":258.3,"sm_search_ms":42.7,"fails":0}
* autoresearch: final playbook update - budget, floors, dead endsblasty3 days2-8/+36
|
* Highlight ranges are computed per line on demand instead of for every match. ↵user3 days1-16/+79
| | | | | | Searching one character over bash matches 177k lines at 310k places, and all but the forty on screen were built and thrown away. _MatchRanges keeps the line SET eagerly and works out the offsets when a line is painted or the cursor lands on it; the blob scan now also skips to the next line after a hit. Result: {"status":"keep","total_ms":17829.9,"lg_boot_ms":724.4,"lg_decomp_ms":2353.5,"lg_graph_ms":1041,"lg_hex_ms":434.9,"lg_index_ms":95.5,"lg_listing_cold_ms":545.6,"lg_listing_warm_ms":407.9,"lg_nav_ms":6922,"lg_palette_ms":4.8,"lg_render_ms":218.1,"lg_search_ms":884.3,"pure_graph_ms":213.6,"sm_boot_ms":432,"sm_decomp_ms":1250.2,"sm_graph_ms":687.4,"sm_hex_ms":457.5,"sm_index_ms":2.6,"sm_listing_cold_ms":265.4,"sm_listing_warm_ms":288.8,"sm_nav_ms":303.2,"sm_palette_ms":0.3,"sm_render_ms":253.9,"sm_search_ms":43,"fails":0}
* autoresearch: note the machine-load caveat in the playbookblasty3 days2-0/+10
|
* CORRECTNESS REPAIR, kept on its merits. The full suite (which the gate was ↵user3 days3-57/+74
| | | | | | NOT running) revealed that the worker-connect poll change made test_project_ui flaky: 5ms polling on a background thread through a cold auto-analysis starved the UI thread enough that the loading overlay was still up when the test pressed Ctrl+O. Poll now backs off to a 25ms cap (keeps the boot win, no busy-wait), the racy boot wait is fixed, and checks.sh runs tests/run.py in full (830 checks) instead of just the scenario suite. Result: {"status":"keep","total_ms":18608,"lg_boot_ms":708.5,"lg_decomp_ms":2454.9,"lg_graph_ms":1034.5,"lg_hex_ms":431.9,"lg_index_ms":95.1,"lg_listing_cold_ms":530.2,"lg_listing_warm_ms":413.4,"lg_nav_ms":7057.9,"lg_palette_ms":5,"lg_render_ms":214.2,"lg_search_ms":1408.9,"pure_graph_ms":213.3,"sm_boot_ms":433.6,"sm_decomp_ms":1292,"sm_graph_ms":754,"sm_hex_ms":433.4,"sm_index_ms":2.6,"sm_listing_cold_ms":260.4,"sm_listing_warm_ms":280.5,"sm_nav_ms":286.1,"sm_palette_ms":0.3,"sm_render_ms":251,"sm_search_ms":46.5,"fails":0}
* autoresearch: record the bench history, the budget and the graph ↵blasty3 days6-461/+64
| | | | non-determinism finding
* Three targeted cuts: the graph's transposition pass counts keep and swap in ↵user3 days3-13/+77
| | | | | | one pass over the neighbour pairs (was four _pair_cross calls); the barycentre median answers degree 1 and 2 without sorting; and the search body is built from windowed model reads instead of one locked row lookup per line. Result: {"status":"keep","total_ms":17944.4,"lg_boot_ms":703.7,"lg_decomp_ms":2453.7,"lg_graph_ms":1000,"lg_hex_ms":478,"lg_index_ms":96,"lg_listing_cold_ms":453,"lg_listing_warm_ms":411.2,"lg_nav_ms":6555.9,"lg_palette_ms":4.9,"lg_render_ms":218.4,"lg_search_ms":1308.8,"pure_graph_ms":212.3,"sm_boot_ms":432.4,"sm_decomp_ms":1267.8,"sm_graph_ms":742.7,"sm_hex_ms":440,"sm_index_ms":2.5,"sm_listing_cold_ms":264.3,"sm_listing_warm_ms":289.5,"sm_nav_ms":305.4,"sm_palette_ms":0.3,"sm_render_ms":258.6,"sm_search_ms":45,"fails":0}
* Baseline for the v5 bench (landing polls every 2ms instead of 10ms; the poll ↵user3 days1-0/+1
| | | | | | interval was measurement overhead inside the timed regions). Final measurement shape — no further bench changes. Result: {"status":"keep","total_ms":18516,"lg_boot_ms":663.9,"lg_decomp_ms":2663.2,"lg_graph_ms":1012.3,"lg_hex_ms":436.1,"lg_index_ms":98.4,"lg_listing_cold_ms":437.7,"lg_listing_warm_ms":409.4,"lg_nav_ms":6841.9,"lg_palette_ms":4.7,"lg_render_ms":218.8,"lg_search_ms":1474.7,"pure_graph_ms":241.6,"sm_boot_ms":433.3,"sm_decomp_ms":1266.3,"sm_graph_ms":737.9,"sm_hex_ms":424.2,"sm_index_ms":2.3,"sm_listing_cold_ms":262.9,"sm_listing_warm_ms":283.2,"sm_nav_ms":301.2,"sm_palette_ms":0.3,"sm_render_ms":253.9,"sm_search_ms":47.9,"fails":0}
* bench: poll landings every 2ms, not 10 -- the wait interval was measurement ↵blasty3 days2-1/+9
| | | | overhead
* RE-BASELINE (v4 bench). Adding a second repetition on the big target exposed ↵user3 days1-0/+1
| | | | | | the same flaw the graph phase had: decompile, search and the function index all cache their answer, so a second rep reported a dict lookup under the name of the thing a user waits for. Cold-sensitive phases (listing_cold, decomp, search, index) now run ONCE; repeatable ones (render, hex, graph, listing_warm) run every rep and take the median. pure_graph is median-of-3. Result: {"status":"keep","total_ms":18497.9,"lg_boot_ms":694.8,"lg_decomp_ms":2674,"lg_graph_ms":992.7,"lg_hex_ms":429.1,"lg_index_ms":94.9,"lg_listing_cold_ms":545.3,"lg_listing_warm_ms":410.7,"lg_nav_ms":6489,"lg_palette_ms":4.7,"lg_render_ms":220.6,"lg_search_ms":1489.8,"pure_graph_ms":241,"sm_boot_ms":467.1,"sm_decomp_ms":1308.5,"sm_graph_ms":761,"sm_hex_ms":431.7,"sm_index_ms":2.5,"sm_listing_cold_ms":273.9,"sm_listing_warm_ms":289,"sm_nav_ms":373,"sm_palette_ms":0.3,"sm_render_ms":257.2,"sm_search_ms":47,"fails":0}
* bench: run the cold-sensitive phases once, the repeatable ones every repblasty3 days2-10/+25
|
* bench: cut two noise sources (pure_graph median-of-3, two reps on the big ↵blasty3 days2-3/+12
| | | | target)
* Re-run of #18 (_CellRow slice assignment, one-string box borders, memoised ↵user3 days3-14/+138
| | | | | | Style sum), confirming it: 17590 -> 17501. lg_graph averages 951 over the two runs against 1136 before. Result: {"status":"keep","total_ms":17501.4,"lg_boot_ms":711,"lg_decomp_ms":2484.8,"lg_graph_ms":1017.2,"lg_hex_ms":431.9,"lg_index_ms":98.7,"lg_listing_cold_ms":528,"lg_listing_warm_ms":409.2,"lg_nav_ms":6488.2,"lg_palette_ms":4.7,"lg_render_ms":214.8,"lg_search_ms":1473.5,"pure_graph_ms":244.3,"sm_boot_ms":451.4,"sm_decomp_ms":596,"sm_graph_ms":691.3,"sm_hex_ms":444.5,"sm_index_ms":0,"sm_listing_cold_ms":263,"sm_listing_warm_ms":284.2,"sm_nav_ms":374.7,"sm_palette_ms":0.3,"sm_render_ms":252.1,"sm_search_ms":37.5,"fails":0}
* Re-run of #16 (keep the joined search body across a cancelled search and ↵user3 days5-11/+265
| | | | | | across navigation inside the same segment), confirming it. total 17784 -> 17590; lg_search 1917 -> 1400, sm_search 70 -> 48. Also lands .auto/check_search.py in the checks gate: it compares both search fast paths against the plain per-line loop for every typed prefix. Result: {"status":"keep","total_ms":17589.8,"lg_boot_ms":710.6,"lg_decomp_ms":2404.9,"lg_graph_ms":1136.1,"lg_hex_ms":425.6,"lg_index_ms":103.4,"lg_listing_cold_ms":417.2,"lg_listing_warm_ms":508.1,"lg_nav_ms":6553.8,"lg_palette_ms":4.8,"lg_render_ms":215.7,"lg_search_ms":1400.3,"pure_graph_ms":239.3,"sm_boot_ms":443.2,"sm_decomp_ms":668.4,"sm_graph_ms":698.1,"sm_hex_ms":443.6,"sm_index_ms":0,"sm_listing_cold_ms":267.3,"sm_listing_warm_ms":286,"sm_nav_ms":356.6,"sm_palette_ms":0.3,"sm_render_ms":258.6,"sm_search_ms":47.9,"fails":0}
* HexView.render_line emits style RUNS instead of one Segment per byte cell ↵user3 days3-12/+40
| | | | | | (35 -> 7 segments per row), and Head is a NamedTuple rather than a frozen dataclass (tuple.__new__ 1.9us vs a dataclass __init__ 2.9us, and it is built once per listing row walked). Result: {"status":"keep","total_ms":17784.1,"lg_boot_ms":680.8,"lg_decomp_ms":2344,"lg_graph_ms":1106.5,"lg_hex_ms":576.4,"lg_index_ms":96.5,"lg_listing_cold_ms":552.9,"lg_listing_warm_ms":431.3,"lg_nav_ms":6256.4,"lg_palette_ms":4.8,"lg_render_ms":224.1,"lg_search_ms":1916.5,"pure_graph_ms":241.3,"sm_boot_ms":442.6,"sm_decomp_ms":591.3,"sm_graph_ms":663.9,"sm_hex_ms":419.2,"sm_index_ms":0,"sm_listing_cold_ms":258.2,"sm_listing_warm_ms":281.8,"sm_nav_ms":374.4,"sm_palette_ms":0.3,"sm_render_ms":251.1,"sm_search_ms":69.8,"fails":0}
* Search the whole segment as ONE joined string. Every line is concatenated ↵user3 days1-26/+101
| | | | | | once (with a start-offset table) so finding a term is a C-level str.find walk instead of a python loop that rebuilds and case-folds 224k lines per keystroke. Falls back to the per-line loop if case-folding changes the string's length. Result: {"status":"keep","total_ms":18618.9,"lg_boot_ms":689.6,"lg_decomp_ms":2560,"lg_graph_ms":1035.1,"lg_hex_ms":684.2,"lg_index_ms":100.4,"lg_listing_cold_ms":551.1,"lg_listing_warm_ms":397.8,"lg_nav_ms":6687.8,"lg_palette_ms":4.7,"lg_render_ms":211.9,"lg_search_ms":1858.2,"pure_graph_ms":238.5,"sm_boot_ms":431.4,"sm_decomp_ms":671.2,"sm_graph_ms":728.1,"sm_hex_ms":552,"sm_index_ms":0,"sm_listing_cold_ms":257.4,"sm_listing_warm_ms":281,"sm_nav_ms":365.9,"sm_palette_ms":0.3,"sm_render_ms":242.8,"sm_search_ms":69.6,"fails":0}
* autoresearch: refresh the playbook and idea backlogblasty3 days3-12/+55
|
* Two independent constants: memoise the pygments token -> Rich style lookup ↵user3 days3-5/+33
| | | | | | (a decompilation uses ~18 distinct token types but each token walked up to nine 'token in ttype' hierarchy checks), and hold the worker-connect poll at 5ms for the first 5s instead of backing off geometrically from the first probe. Result: {"status":"keep","total_ms":18856.8,"lg_boot_ms":689.7,"lg_decomp_ms":2484.2,"lg_graph_ms":1120,"lg_hex_ms":700.1,"lg_index_ms":96.5,"lg_listing_cold_ms":425.6,"lg_listing_warm_ms":511.5,"lg_nav_ms":6590.7,"lg_palette_ms":4.8,"lg_render_ms":215.1,"lg_search_ms":2195.5,"pure_graph_ms":238.1,"sm_boot_ms":431.5,"sm_decomp_ms":667.2,"sm_graph_ms":686.9,"sm_hex_ms":569.8,"sm_index_ms":0,"sm_listing_cold_ms":258.1,"sm_listing_warm_ms":283.2,"sm_nav_ms":365.2,"sm_palette_ms":0.3,"sm_render_ms":243.7,"sm_search_ms":79,"fails":0}
* Fetch a graph's listing rows from the blocks' MERGED EXTENTS, not their ↵user3 days2-3/+33
| | | | | | convex hull, and assign them per block by bisect. IDA puts a function's cold/tail chunks far from its entry, so the hull of a 1.4KB function could be 680KB wide: it fetched 128k rows, took 3s, and still came back EMPTY for the far blocks because the pager's 64-page bound ran out first. lg_graph 9561 -> 1022. Result: {"status":"keep","total_ms":19062.1,"lg_boot_ms":751.3,"lg_decomp_ms":2605.7,"lg_graph_ms":1022,"lg_hex_ms":554.5,"lg_index_ms":103.9,"lg_listing_cold_ms":545,"lg_listing_warm_ms":405.6,"lg_nav_ms":6636.8,"lg_palette_ms":4.7,"lg_render_ms":212.2,"lg_search_ms":2269,"pure_graph_ms":239.9,"sm_boot_ms":538.5,"sm_decomp_ms":688.4,"sm_graph_ms":712.7,"sm_hex_ms":557.5,"sm_index_ms":0,"sm_listing_cold_ms":259.9,"sm_listing_warm_ms":260.2,"sm_nav_ms":363.8,"sm_palette_ms":0.3,"sm_render_ms":252.7,"sm_search_ms":77.5,"fails":0}
* RE-BASELINE on a corrected benchmark. phase_graph was measuring a cache hit: ↵user3 days1-0/+1
| | | | | | the fixture picker called Program.flowchart (which caches per function), so the timed Space press only did a dict lookup. Fixtures now use the raw flowchart tool and the graph cache is cleared before the phase. Cold graph opens cost lg_graph 9561ms — 34% of the total, previously invisible. Result: {"status":"keep","total_ms":27912.9,"lg_boot_ms":742.8,"lg_decomp_ms":2595.7,"lg_graph_ms":9561.4,"lg_hex_ms":554.4,"lg_index_ms":77.6,"lg_listing_cold_ms":548.6,"lg_listing_warm_ms":405.5,"lg_nav_ms":6552.9,"lg_palette_ms":4.9,"lg_render_ms":216.1,"lg_search_ms":2218,"pure_graph_ms":507.6,"sm_boot_ms":535.6,"sm_decomp_ms":620.2,"sm_graph_ms":962,"sm_hex_ms":587.4,"sm_index_ms":0,"sm_listing_cold_ms":261.8,"sm_listing_warm_ms":261.7,"sm_nav_ms":368.8,"sm_palette_ms":0.3,"sm_render_ms":252.4,"sm_search_ms":77.1,"fails":0}
* bench: time a COLD graph open, and stop the fixture picker warming its cacheblasty3 days2-2/+15
|
* Only re-apply a scroll after the next refresh when it actually clamped. Both ↵user3 days2-5/+19
| | | | | | _apply_scroll implementations unconditionally scheduled a deferred scroll_to + refresh(layout=True) — a whole-screen re-arrange on every scroll — as a workaround for scrolling before the view's size is computed. Now the deferred pass runs only when scroll_offset didn't reach the target. Result: {"status":"keep","total_ms":19005.8,"lg_boot_ms":788.9,"lg_decomp_ms":2752.5,"lg_graph_ms":899,"lg_hex_ms":565.2,"lg_index_ms":71.9,"lg_listing_cold_ms":416.1,"lg_listing_warm_ms":402.3,"lg_nav_ms":6756.3,"lg_palette_ms":4.7,"lg_render_ms":217.1,"lg_search_ms":2335.8,"pure_graph_ms":235.3,"sm_boot_ms":537.4,"sm_decomp_ms":600.9,"sm_graph_ms":693.1,"sm_hex_ms":549.9,"sm_index_ms":0,"sm_listing_cold_ms":280.9,"sm_listing_warm_ms":261.2,"sm_nav_ms":309,"sm_palette_ms":0.3,"sm_render_ms":249.3,"sm_search_ms":78.7,"fails":0}
* Two hot-path fixes found by profiling the plain-line builder: the ↵user3 days3-5/+12
| | | | | | opcode-bytes column used a per-byte f-string generator where bytes.hex(' ').upper() does it in one C call (12x), and ListingModel._phys/_head_index_at re-imported bisect on every call. _line_plain 2.64 -> 1.57 us/row. Result: {"status":"keep","total_ms":19476.3,"lg_boot_ms":756.3,"lg_decomp_ms":2495.1,"lg_graph_ms":946,"lg_hex_ms":905.4,"lg_index_ms":72,"lg_listing_cold_ms":547.5,"lg_listing_warm_ms":410.4,"lg_nav_ms":6723.4,"lg_palette_ms":5,"lg_render_ms":218.8,"lg_search_ms":2312.4,"pure_graph_ms":239.5,"sm_boot_ms":535.3,"sm_decomp_ms":616.4,"sm_graph_ms":682.2,"sm_hex_ms":824.9,"sm_index_ms":0,"sm_listing_cold_ms":258.9,"sm_listing_warm_ms":261,"sm_nav_ms":339.5,"sm_palette_ms":0.3,"sm_render_ms":248.7,"sm_search_ms":77.3,"fails":0}
* Three micro-wins on the listing-row path: merge the colour-tag and ↵user3 days3-6/+27
| | | | | | operand-tag dicts into one lookup, skip both isspace() probes when a span needs no whitespace collapsing at all (the common case), and give Head slots=True. Spans 11.07 -> 10.18 us/line; Head construction 3.18 -> 2.53 us/row. Result: {"status":"keep","total_ms":20412.8,"lg_boot_ms":752.9,"lg_decomp_ms":2418.6,"lg_graph_ms":811.6,"lg_hex_ms":1020.1,"lg_index_ms":73.2,"lg_listing_cold_ms":573.7,"lg_listing_warm_ms":456.7,"lg_nav_ms":6640.1,"lg_palette_ms":4.8,"lg_render_ms":237.6,"lg_search_ms":3210.2,"pure_graph_ms":242.5,"sm_boot_ms":536.5,"sm_decomp_ms":643.1,"sm_graph_ms":689.8,"sm_hex_ms":866.6,"sm_index_ms":0,"sm_listing_cold_ms":262.9,"sm_listing_warm_ms":265.9,"sm_nav_ms":330.1,"sm_palette_ms":0.3,"sm_render_ms":265.2,"sm_search_ms":110.5,"fails":0}
* Incremental search narrows instead of rescanning. Typing a character onto ↵user3 days3-14/+84
| | | | | | the term can only remove lines (a line holding "mov" holds "mo"), so _compute_matches rescans the previous hit list when the term grew and nothing else moved. Keyed on (term, case-fold, row count, line-source id) so a listing still streaming rows in behind the search falls back to a full scan. Result: {"status":"keep","total_ms":20835.7,"lg_boot_ms":754.4,"lg_decomp_ms":2593.7,"lg_graph_ms":929.5,"lg_hex_ms":1080.2,"lg_index_ms":76.1,"lg_listing_cold_ms":526.7,"lg_listing_warm_ms":410.1,"lg_nav_ms":6674.8,"lg_palette_ms":5,"lg_render_ms":225.2,"lg_search_ms":3310.3,"pure_graph_ms":239.3,"sm_boot_ms":537.9,"sm_decomp_ms":630.1,"sm_graph_ms":734.3,"sm_hex_ms":869.9,"sm_index_ms":0,"sm_listing_cold_ms":260.8,"sm_listing_warm_ms":262.3,"sm_nav_ms":334.8,"sm_palette_ms":0.3,"sm_render_ms":268.4,"sm_search_ms":111.6,"fails":0}
generated by cgit v1.3.1-sl0p (git 2.54.0) at 2026-08-10 08:35:30 +0000 — sl0p fork