{"type":"config","name":"ida-tui performance: cut the latency of navigation, listing, decomp, graph and search","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
{"run":1,"commit":"2910c93","metric":46572.1,"metrics":{"lg_boot_ms":863.5,"lg_decomp_ms":2881.3,"lg_graph_ms":951.7,"lg_hex_ms":900.9,"lg_index_ms":219.3,"lg_listing_cold_ms":434.4,"lg_listing_warm_ms":552.3,"lg_nav_ms":29106.9,"lg_palette_ms":4.8,"lg_render_ms":231.2,"lg_search_ms":5590.1,"pure_graph_ms":236.2,"sm_boot_ms":539.4,"sm_decomp_ms":621.4,"sm_graph_ms":702,"sm_hex_ms":850.5,"sm_index_ms":0,"sm_listing_cold_ms":270.8,"sm_listing_warm_ms":270.6,"sm_nav_ms":875.1,"sm_palette_ms":0.3,"sm_render_ms":272.3,"sm_search_ms":197.1,"fails":0},"status":"checks_failed","description":"Baseline run of the new bench harness. Benchmark clean (fails=0) but the pilot scenario suite reported 300 passed / 1 failed with ZERO code changes -> flaky, and checks.sh printed the tail instead of the FAIL line so the name is unknown.","timestamp":1786058161497,"segment":0,"confidence":null,"asi":{"hypothesis":"establish a baseline for total_ms","bottleneck":"lg_nav_ms=29107 is 62% of total_ms; lg_nav_worst_ms=28274 is ONE cold jump to a high address in bash. ListingModel.ensure_ea walks the segment forward in 500-head pages from seg_start, so landing near the end of a 224k-row listing costs ~450 sequential worker round trips.","second_bottleneck":"lg_search_ms=5590 (91783 hits over the whole segment)","cheap_phases":"palette/index/render/pure_graph are all <600ms; not where the time is","rollback_reason":"checks.sh flagged 1 scenario failure with no code change (flake)","next_action_hint":"make checks.sh print the FAIL line name on non-zero exit, re-run baseline, then attack ListingModel address->row lookup (needs a backend primitive in server/patch_server.py: heads walking anchored at an address, or a segment head-index built in one call)"}}
{"run":2,"commit":"a3f3400","metric":46685.5,"metrics":{"lg_boot_ms":885.7,"lg_decomp_ms":2871.8,"lg_graph_ms":930.9,"lg_hex_ms":947.3,"lg_index_ms":207.8,"lg_listing_cold_ms":564.3,"lg_listing_warm_ms":454.7,"lg_nav_ms":29018.5,"lg_palette_ms":4.6,"lg_render_ms":223.6,"lg_search_ms":5448.7,"pure_graph_ms":535.4,"sm_boot_ms":537.9,"sm_decomp_ms":643.8,"sm_graph_ms":680.4,"sm_hex_ms":854.9,"sm_index_ms":0,"sm_listing_cold_ms":265.5,"sm_listing_warm_ms":263.7,"sm_nav_ms":887.2,"sm_palette_ms":0.3,"sm_render_ms":264.1,"sm_search_ms":194.5,"fails":0},"status":"discard","description":"Baseline re-run with the fixed checks gate. Checks pass; total_ms reproduces to within 0.24% of run #1 (46572 -> 46686), so the noise floor is ~115ms on a 46.6s metric.","timestamp":1786058370297,"segment":0,"confidence":null,"asi":{"hypothesis":"confirm the baseline is reproducible and the checks gate is green","noise_floor_ms":115,"reproducibility":"run1 46572 / run2 46686 -> 0.24% spread; pure_graph_ms is the jumpiest single phase (236 -> 535, it is CPU-only and gets descheduled)","checks":"flaky-scenario retry logic works; suite green on a clean tree","next_action_hint":"attack ListingModel.ensure_ea / the heads tool: 29s of 46.6s is one cold address->row walk over bash"}}
{"run":3,"commit":"93240e2","metric":26923.9,"metrics":{"lg_boot_ms":762.2,"lg_decomp_ms":2631.7,"lg_graph_ms":941.8,"lg_hex_ms":1052,"lg_index_ms":67.2,"lg_listing_cold_ms":440.6,"lg_listing_warm_ms":530.5,"lg_nav_ms":10598.3,"lg_palette_ms":4.6,"lg_render_ms":227.8,"lg_search_ms":5265.5,"pure_graph_ms":237.9,"sm_boot_ms":535.3,"sm_decomp_ms":631.8,"sm_graph_ms":702.1,"sm_hex_ms":841.1,"sm_index_ms":0,"sm_listing_cold_ms":268.2,"sm_listing_warm_ms":269.4,"sm_nav_ms":443.6,"sm_palette_ms":0.3,"sm_render_ms":277.8,"sm_search_ms":194.4,"fails":0},"status":"keep","description":"Stop ida-pro-mcp installing a sys.setprofile hook around every tool call. Its deadline mechanism profiles every python call/return so a pure-python tool loop can be interrupted; our tools are call-heavy, so it taxed the whole backend 3.3x. Worker now sets IDA_MCP_TOOL_TIMEOUT_SEC=0 and arms the deadline itself with one polling watchdog thread + ida_kernwin.set_cancelled() (the half that actually frees the IDA main thread). Also rewrote _idatui_spans to jump between colour tags instead of walking characters (byte-identical over 258k real lines).","timestamp":1786059117542,"segment":0,"confidence":173.264550264548,"asi":{"hypothesis":"the heads tool is not IDA-bound; the ida-pro-mcp sync wrapper's sys.setprofile deadline is the tax","evidence":"in-process A/B on targets/bash: heads(count=500,annotate) 92.2us/row with IDA_MCP_TOOL_TIMEOUT_SEC unset vs 27.5us/row with it 0; domain ListingModel._load_next_page 116 -> 37 us/row","where_it_lives":"site-packages/ida_pro_mcp/ida_mcp/sync.py sync_wrapper() installs profilefunc via sys.setprofile whenever timeout>0; default _DEFAULT_TOOL_TIMEOUT_SEC=60","what_we_kept":"the native half: threading watchdog + ida_kernwin.set_cancelled(), which is what decompile/auto_wait/find_bytes actually poll. Only the interruption of a pure-python loop inside a tool body is gone, and every idatui tool loop is bounded by its count argument (<=2000).","watchdog_design":"polling at 0.25s instead of signalling, because arm/disarm is on the path of EVERY worker call (a scroll is dozens); arming is two attribute writes + clr_cancelled()","spans_rewrite":"_idatui_spans now re.finditer's over the 3 control chars instead of walking characters. Verified byte-identical vs HEAD over 258k real disasm lines across echo/ls_ttl/bash/libcrypto (.auto/diff_spans.py). Standalone it measured 12.5 -> 13.6 us/line i.e. slightly SLOWER, so it is probably a small net loss that the setprofile win is hiding -- A/B it on its own next.","gains":"total 46572 -> 26924 (-42%); lg_nav 29107 -> 10598; lg_nav_worst 28274 -> 10170; sm_nav_worst 656 -> 224; lg_index 219 -> 67","work_unchanged":"nav_rows/decomp_ok/graph_ok/search_hits/render_cells all identical -> the speedup is not less work","next_action_hint":"1) A/B the _idatui_spans rewrite alone (suspect regression). 2) lg_nav is still 10.6s: ListingModel.ensure_ea is O(distance) at ~34us/row; a skeleton (ea,size,kind only) walk or a bigger PAGE would cut it further. 3) lg_search_ms 5265 is now the #2 term."}}
{"run":4,"commit":"b6a95ec","metric":26050.2,"metrics":{"lg_boot_ms":727.4,"lg_decomp_ms":2618.1,"lg_graph_ms":935.1,"lg_hex_ms":954.2,"lg_index_ms":67.4,"lg_listing_cold_ms":459.3,"lg_listing_warm_ms":545.7,"lg_nav_ms":9257.8,"lg_palette_ms":4.7,"lg_render_ms":238.8,"lg_search_ms":5472.5,"pure_graph_ms":530.7,"sm_boot_ms":534.6,"sm_decomp_ms":635.8,"sm_graph_ms":748.1,"sm_hex_ms":858.7,"sm_index_ms":0,"sm_listing_cold_ms":268.7,"sm_listing_warm_ms":269.2,"sm_nav_ms":441.6,"sm_palette_ms":0.3,"sm_render_ms":281.7,"sm_search_ms":200,"fails":0},"status":"keep","description":"_idatui_spans: one capturing re.split over the tag pairs instead of finditer+char-slicing, and collapse whitespace with ' '.join(txt.split()) instead of a regex sub. 13.15 -> 11.07 us/line (the previous finditer attempt was 14.4, i.e. SLOWER than the original char loop it replaced).","timestamp":1786059375617,"segment":0,"confidence":2.0769472107521656,"asi":{"hypothesis":"the span walker can beat the original char loop if the tokenisation is one C-level split and the whitespace collapse avoids re.sub","microbench_us_per_line":{"original_char_loop":13.15,"finditer_attempt":14.42,"re.split_version":11.07},"lesson":"re.finditer per tag is SLOWER than a plain character loop -- Match objects and .start() calls cost more than the ~54 trivial loop iterations they replace. A single capturing re.split that hands back [text, tag, text, ...] is what actually wins.","lesson2":"re.sub for whitespace collapse cost ~1us per call at ~6.5 calls/line; ' '.join(txt.split()) splits on exactly str.isspace() and is far cheaper. Leading/trailing space has to be re-attached by hand to keep cross-span runs collapsing the same way.","equivalence":"0 mismatches vs the pre-autoresearch implementation over 258k real disasm lines on echo/ls_ttl/bash/libcrypto (.auto/diff_spans.py --ref 2b0ae8d)","gains":"total 26924 -> 26050 (-3.2%); lg_nav 10598 -> 9258; lg_nav_worst 10170 -> 8869","work_unchanged":"every NOTES counter identical","next_action_hint":"lg_nav 9.3s and lg_search 5.5s are now the top two. For nav: ListingModel walks 500 heads/call at ~25us/row and the ROW TEXT is entirely wasted when the walk is only trying to reach an address -- a skeleton (ea,size,kind) mode on the heads tool would make ensure_ea nearly free. For search: _compute_matches/_line_plain over 224k rows."}}
{"run":5,"commit":"b6a95ec","metric":23259.6,"metrics":{"lg_boot_ms":752.7,"lg_decomp_ms":2475.3,"lg_graph_ms":963,"lg_hex_ms":950.4,"lg_index_ms":70.8,"lg_listing_cold_ms":533.2,"lg_listing_warm_ms":406.1,"lg_nav_ms":6862.6,"lg_palette_ms":4.7,"lg_render_ms":228.9,"lg_search_ms":5401.5,"pure_graph_ms":510.5,"sm_boot_ms":538.8,"sm_decomp_ms":666.7,"sm_graph_ms":715.7,"sm_hex_ms":856.8,"sm_index_ms":0,"sm_listing_cold_ms":260.3,"sm_listing_warm_ms":262.5,"sm_nav_ms":335.5,"sm_palette_ms":0.3,"sm_render_ms":270.8,"sm_search_ms":192.5,"fails":0},"status":"checks_failed","description":"Memoise per-line rendering in the worker (lru_cache on a new _idatui_line_parts) + build listing Heads with their opcode bytes already attached instead of dataclasses.replace-ing them in. total 26050 -> 23260, lg_nav 9258 -> 6863. Reverted: 3 graph_minimap checks fail -- but the cause is a RACE IN THE SCENARIO that the speedup wins, not a functional regression (proved below).","timestamp":1786060327739,"segment":0,"confidence":5.600496684223448,"asi":{"hypothesis":"cache the per-line render (tagged line -> text/spans/ops) in the worker, and stop double-constructing Heads client-side","change_A":"server/patch_server.py: new _idatui_line_parts(line) = (text, spans, ops), functools.lru_cache(16384). bash: 196618 listing lines are only 53363 distinct, so hit rate is ~70% and cost falls 10.4 -> 3.9 us/line. Bonus: pickle memoises the shared span lists so pages serialise smaller.","change_B":"idatui/domain.py: ListingModel._build_page reads the code extent FIRST and passes raw into Head.from_raw, replacing _attach_opcode_bytes' dataclasses.replace (which re-ran __init__ per code head). from_raw now uses tuple(map(tuple,...)) instead of a coercing genexpr.","measured":"cold ListingModel paging 35.7 -> 25.7 us/row; PAGE size (500/1000/2000) makes NO difference, do not bother tuning it","failure_root_cause":"tests/test_scenarios.py graph_minimap. _open_graph() leaves _graph_sticky=True; the scenario then does c.open(big,'listing') and presses space expecting to ENTER the graph. With sticky on, the navigation itself schedules _load_graph, and if that async load lands before the space press then space LEAVES graph mode instead -> the following 60s wait times out (scenario 1.9s -> 65.5s) and every minimap click lands on a hidden widget.","proof":"/tmp/mmrace.py drives the same steps and prints _active right before the space press: NEW code 'after open(big): active=graph', OLD code 'active=listing'. Bisected: stashing idatui/domain.py alone still fails, stashing server/patch_server.py alone passes -> it is purely the speedup winning the race, no behaviour changed.","equivalence_evidence":"diff_spans.py now compares _idatui_head_row too (whole row dict, not just spans): 0 mismatches over 118k lines on bash/echo/ls_ttl vs pre-autoresearch HEAD 2b0ae8d","work_preserved":".auto/wip-headcache.patch holds the reverted diff","next_action_hint":"re-apply the patch and make the graph_minimap SETUP deterministic (clear _graph_sticky before the second navigation). Assertions untouched; graph_sticky scenario already covers sticky behaviour. Record the amended tests/ rule in .auto/prompt.md."}}
{"run":6,"commit":"cf45e11","metric":22980.2,"metrics":{"lg_boot_ms":738.2,"lg_decomp_ms":2401.8,"lg_graph_ms":944.1,"lg_hex_ms":920.6,"lg_index_ms":75.2,"lg_listing_cold_ms":538.5,"lg_listing_warm_ms":411.1,"lg_nav_ms":6813.9,"lg_palette_ms":4.9,"lg_render_ms":221.8,"lg_search_ms":5630.1,"pure_graph_ms":240.7,"sm_boot_ms":537.5,"sm_decomp_ms":595.1,"sm_graph_ms":715.7,"sm_hex_ms":858.8,"sm_index_ms":0,"sm_listing_cold_ms":263.3,"sm_listing_warm_ms":265.3,"sm_nav_ms":335.2,"sm_palette_ms":0.3,"sm_render_ms":271.4,"sm_search_ms":196.5,"fails":0},"status":"keep","description":"Re-apply #5 (lru_cache on the per-line render + Heads built with their opcode bytes already attached) with the graph_minimap scenario's racy SETUP made deterministic: clear _graph_sticky before the second navigation so Space is known to be entering the graph, not leaving it. No assertion changed.","timestamp":1786060531752,"segment":0,"confidence":7.006489167396753,"asi":{"hypothesis":"the graph_minimap failure in #5 was a racy scenario setup, not lost functionality","proof_a_bisect":"stashing idatui/domain.py alone still failed; stashing server/patch_server.py alone passed -> the flip is caused purely by the backend getting faster","proof_b_race":"/tmp/mmrace.py replays the scenario's steps outside the suite and prints _active just before the Space press: NEW 'active=graph', OLD 'active=listing'. Same steps, two states. With sticky on, the navigation itself schedules _load_graph; whether it lands before the keypress decides whether Space enters or leaves graph mode.","proof_c_both_ways":"the repaired scenario passes on the fast code AND on the stashed slow code (graph_minimap + graph_sticky, 15 passed 0 failed)","test_edit_scope":"two setup lines (app._graph_sticky = False; wait for _active == listing). Every c.check is byte-identical. graph_sticky scenario still covers sticky navigation.","real_bug_noted":"there IS a genuine UX wart underneath: with sticky graph mode on, a keypress right after a navigation means something different depending on whether the async graph reload has landed. Out of scope for perf work -> .auto/ideas.md","gains":"total 26050 -> 22980 (-11.8%); lg_nav 9258 -> 6814; lg_nav_worst 8869 -> 6613; sm_nav_worst 213 -> 128; cumulative vs baseline -50.7%","work_unchanged":"every NOTES counter identical to baseline","next_action_hint":"lg_search_ms 5630 is now the largest single term after lg_nav 6814. Search runs _compute_matches + _line_plain over 224k rows client-side; profile SearchMixin._compute_matches / ListingView._line_plain next."}}
{"run":7,"commit":"5045ba1","metric":20835.7,"metrics":{"lg_boot_ms":754.4,"lg_decomp_ms":2593.7,"lg_graph_ms":929.5,"lg_hex_ms":1080.2,"lg_index_ms":76.1,"lg_listing_cold_ms":526.7,"lg_listing_warm_ms":410.1,"lg_nav_ms":6674.8,"lg_palette_ms":5,"lg_render_ms":225.2,"lg_search_ms":3310.3,"pure_graph_ms":239.3,"sm_boot_ms":537.9,"sm_decomp_ms":630.1,"sm_graph_ms":734.3,"sm_hex_ms":869.9,"sm_index_ms":0,"sm_listing_cold_ms":260.8,"sm_listing_warm_ms":262.3,"sm_nav_ms":334.8,"sm_palette_ms":0.3,"sm_render_ms":268.4,"sm_search_ms":111.6,"fails":0},"status":"keep","description":"Incremental search narrows instead of rescanning. Typing a character onto the term can only remove lines (a line holding \"mov\" holds \"mo\"), so _compute_matches rescans the previous hit list when the term grew and nothing else moved. Keyed on (term, case-fold, row count, line-source id) so a listing still streaming rows in behind the search falls back to a full scan.","timestamp":1786060806344,"segment":0,"confidence":8.383192182410422,"asi":{"hypothesis":"as-you-type search rescans every row per keystroke; the match set is monotonically shrinking so it need not","gains":"total 22980 -> 20836 (-9.3%); lg_search 5630 -> 3310 (-41%); sm_search 196 -> 112 (-43%)","results_identical":"search_hits 91783 (bash) / 2461 (echo) unchanged from baseline -- the same lines still match","equivalence_test":"/tmp/searcheq.py drives the real ListingView keystroke by keystroke over 9 terms (mov/call/rsp/Mov/1a/push/e/lea/sub_) and compares the narrowed _matches AND _ranges against a forced full rescan at every prefix: 0 mismatches","invalidation_traps":["case folding is per-term (_ci = term.islower()) and can FLIP as you type: '1' -> '1a' goes False -> True, which can ADD matches. The key stores _ci and only narrows when it is unchanged.","the listing streams rows in behind the search, so row count is in the key -- rows that arrived since the last pass have never been looked at.","action_toggle_opcodes changes the plain line's prefix, so it changes WHICH rows match (the opcode hex is searchable text), not just the highlight offsets. It now clears the key too.","every site that resets _matches/_ranges must reset _matched_key, or a cancelled search leaves a stale prefix and the next search narrows from an empty list. Six sites."],"next_action_hint":"lg_nav 6675 is again the biggest term, then lg_decomp 2594 and lg_search 3310. For search what is left is _line_plain: it rebuilds the whole formatted row (address gutter + opcode field + name prefix + text) per row on the first, unnarrowed pass. For nav see .auto/ideas.md (skeleton walk)."}}
{"run":8,"commit":"8b40fd2","metric":20412.8,"metrics":{"lg_boot_ms":752.9,"lg_decomp_ms":2418.6,"lg_graph_ms":811.6,"lg_hex_ms":1020.1,"lg_index_ms":73.2,"lg_listing_cold_ms":573.7,"lg_listing_warm_ms":456.7,"lg_nav_ms":6640.1,"lg_palette_ms":4.8,"lg_render_ms":237.6,"lg_search_ms":3210.2,"pure_graph_ms":242.5,"sm_boot_ms":536.5,"sm_decomp_ms":643.1,"sm_graph_ms":689.8,"sm_hex_ms":866.6,"sm_index_ms":0,"sm_listing_cold_ms":262.9,"sm_listing_warm_ms":265.9,"sm_nav_ms":330.1,"sm_palette_ms":0.3,"sm_render_ms":265.2,"sm_search_ms":110.5,"fails":0},"status":"keep","description":"Three micro-wins on the listing-row path: merge the colour-tag and operand-tag dicts into one lookup, skip both isspace() probes when a span needs no whitespace collapsing at all (the common case), and give Head slots=True. Spans 11.07 -> 10.18 us/line; Head construction 3.18 -> 2.53 us/row.","timestamp":1786061248935,"segment":0,"confidence":8.593443053776156,"asi":{"hypothesis":"shave the remaining per-row constants now that the structural wins are in","cost_model_measured":{"note":"cold walk of targets/bash .text, per LISTING ROW","worker_tool_compute_cold_us":18.1,"worker_tool_compute_warm_cache_us":10.3,"pickle_dumps_us":0.9,"pickle_loads_us":2.4,"client_build_page_us":4.5,"socket_round_trip_us_per_call":25},"worker_internals_us_per_head":{"generate_disasm_line":5.9,"head_row_warm_cache":8.4,"spans_on_a_cache_miss":10.2,"get_func":0.4,"get_ea_name":0.4,"struct_member_rows_per_DATA_row":3.8},"wire_shape_dead_end":"tried costing dict-with-hex-string-ea vs dict-with-int-ea vs plain tuples for the heads payload: 0.75/0.70/0.54 us dumps and 1.11/1.26/1.04 us loads per row. At most 0.3 us/row for a breaking change to the tool's wire format -- not worth it, do not revisit.","equivalence":"spans still byte-identical vs 2b0ae8d over 118k lines (bash/echo/ls_ttl); all NOTES counters unchanged","gains":"total 20836 -> 20413 (-2.0%); lg_graph 930 -> 812","next_action_hint":"lg_nav 6640 (33% of total) is now mostly irreducible per-row worker cost: generate_disasm_line is 5.9us of the ~18us and is IDA's. The only big lever left is NOT DOING IT -- a skeleton (ea,size,kind + row counts) walk so ensure_ea can find a row index without rendering text (.auto/ideas.md). Beware: the worker is a single serial process, so work moved to the background does not overlap; a skeleton only wins if the text is never needed."}}
{"run":9,"commit":"4148738","metric":19476.3,"metrics":{"lg_boot_ms":756.3,"lg_decomp_ms":2495.1,"lg_graph_ms":946,"lg_hex_ms":905.4,"lg_index_ms":72,"lg_listing_cold_ms":547.5,"lg_listing_warm_ms":410.4,"lg_nav_ms":6723.4,"lg_palette_ms":5,"lg_render_ms":218.8,"lg_search_ms":2312.4,"pure_graph_ms":239.5,"sm_boot_ms":535.3,"sm_decomp_ms":616.4,"sm_graph_ms":682.2,"sm_hex_ms":824.9,"sm_index_ms":0,"sm_listing_cold_ms":258.9,"sm_listing_warm_ms":261,"sm_nav_ms":339.5,"sm_palette_ms":0.3,"sm_render_ms":248.7,"sm_search_ms":77.3,"fails":0},"status":"keep","description":"Two hot-path fixes found by profiling the plain-line builder: the opcode-bytes column used a per-byte f-string generator where bytes.hex(' ').upper() does it in one C call (12x), and ListingModel._phys/_head_index_at re-imported bisect on every call. _line_plain 2.64 -> 1.57 us/row.","timestamp":1786061547254,"segment":0,"confidence":9.517985106084026,"asi":{"hypothesis":"search's remaining cost is _line_plain, and _line_plain is dominated by something silly","profile_us_per_row_before":{"model.get":0.67,"model._phys":0.44,"_op_field":1.29,"_line_plain":2.64},"profile_us_per_row_after":{"model.get":0.56,"model._phys":0.34,"_op_field":0.4,"_line_plain":1.57},"finding":"the opcode-bytes column (op_mode=1 by default, so it is ALWAYS built) was ' '.join(f'{b:02X}' for b in raw): 1.74us vs 0.14us for raw.hex(' ').upper(). Verified byte-identical for every length 0..19.","finding2":"ListingModel._phys and _head_index_at each did 'import bisect' inside the function body, on a path that runs once per rendered row and once per row a search reads. bisect is already imported at module scope.","gains":"total 20413 -> 19476 (-4.6%); lg_search 3210 -> 2312 (-28%); sm_search 111 -> 77; sm_render 265 -> 249","work_unchanged":"search_hits, render_cells, all counters identical","remaining_budget_ms":{"lg_nav":6723,"lg_decomp+sm_decomp":3112,"lg_search":2312,"hex(lg+sm)":1730,"graph(lg+sm)":1628,"listing cold+warm(lg+sm)":1478,"boot(lg+sm)":1292,"render":468,"pure_graph":240},"next_action_hint":"decomp: 12 bash functions cost 1370ms of Hex-Rays (irreducible) + 240ms of pygments highlight_c (70us/line -- a hand-rolled C lexer would be faster but risks colour changes). hex: 1730ms for 120 frames = 14ms/frame, unprofiled, look there next. ALSO NOTED: domain.decomp_map costs 280ms per function (more than decompile itself) and is on the split-view path, which the bench does not cover."}}
{"run":10,"commit":"f8fb9b7","metric":19005.8,"metrics":{"lg_boot_ms":788.9,"lg_decomp_ms":2752.5,"lg_graph_ms":899,"lg_hex_ms":565.2,"lg_index_ms":71.9,"lg_listing_cold_ms":416.1,"lg_listing_warm_ms":402.3,"lg_nav_ms":6756.3,"lg_palette_ms":4.7,"lg_render_ms":217.1,"lg_search_ms":2335.8,"pure_graph_ms":235.3,"sm_boot_ms":537.4,"sm_decomp_ms":600.9,"sm_graph_ms":693.1,"sm_hex_ms":549.9,"sm_index_ms":0,"sm_listing_cold_ms":280.9,"sm_listing_warm_ms":261.2,"sm_nav_ms":309,"sm_palette_ms":0.3,"sm_render_ms":249.3,"sm_search_ms":78.7,"fails":0},"status":"keep","description":"Only re-apply a scroll after the next refresh when it actually clamped. Both _apply_scroll implementations unconditionally scheduled a deferred scroll_to + refresh(layout=True) — a whole-screen re-arrange on every scroll — as a workaround for scrolling before the view's size is computed. Now the deferred pass runs only when scroll_offset didn't reach the target.","timestamp":1786061776694,"segment":0,"confidence":8.386589391381065,"asi":{"hypothesis":"the hex phase spends its time in the event loop, not in render_line -- something the app schedules per scroll is expensive","measurement":"profiling the hex sweep split it as ensure 0.1ms / scroll 6.7ms / pilot.pause 700ms / paint 106ms over 60 frames. The cost was in what the pause had to process.","isolation":"60 scroll frames on a live HexView: idle pause 73ms, scroll_to alone 497ms, scroll_to+refresh() 519ms, scroll_to+refresh(layout=True) 522ms, the shipped _apply_scroll 650ms. So the deferred call_after_refresh pass was ~25% on top of an already-costly scroll. After the fix: 453ms.","why_the_workaround_exists":"setting virtual_size then scrolling immediately clamps to 0 because max_scroll_y is not recomputed until layout (documented in the idatui skill). Keeping the deferred pass but only when scroll_offset actually missed the target preserves that and skips it otherwise. NOTE the pilot lays out synchronously, so under test the scroll always lands and the deferred pass is skipped -- the real-terminal path is the one that still schedules it.","gains":"total 19476 -> 19006 (-2.4%); hex(lg+sm) 1730 -> 1115 (-36%); lg_listing_cold 548 -> 416","noise_seen":"lg_decomp 2495 -> 2753 (+10%) with no change on that path -- Hex-Rays timing is the jumpiest phase; do not chase it","next_action_hint":"lg_nav 6756 is 36% of the total and is ~18us/row of worker time over 225k rows. Its parts: generate_disasm_line 5.9us (IDA's, irreducible), _idatui_spans ~10us on an lru miss, ~2us of dict building. Next: instrument _idatui_spans' four internal stages (split / token loop / whitespace collapse / operand extents) and see which is left."}}
{"type":"config","name":"ida-tui performance (v2 bench: graph opens are now measured cold)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
{"run":11,"commit":"26fa14c","metric":27912.9,"metrics":{"lg_boot_ms":742.8,"lg_decomp_ms":2595.7,"lg_graph_ms":9561.4,"lg_hex_ms":554.4,"lg_index_ms":77.6,"lg_listing_cold_ms":548.6,"lg_listing_warm_ms":405.5,"lg_nav_ms":6552.9,"lg_palette_ms":4.9,"lg_render_ms":216.1,"lg_search_ms":2218,"pure_graph_ms":507.6,"sm_boot_ms":535.6,"sm_decomp_ms":620.2,"sm_graph_ms":962,"sm_hex_ms":587.4,"sm_index_ms":0,"sm_listing_cold_ms":261.8,"sm_listing_warm_ms":261.7,"sm_nav_ms":368.8,"sm_palette_ms":0.3,"sm_render_ms":252.4,"sm_search_ms":77.1,"fails":0},"status":"keep","description":"RE-BASELINE on a corrected benchmark. phase_graph was measuring a cache hit: the fixture picker called Program.flowchart (which caches per function), so the timed Space press only did a dict lookup. Fixtures now use the raw flowchart tool and the graph cache is cleared before the phase. Cold graph opens cost lg_graph 9561ms — 34% of the total, previously invisible.","timestamp":1786062344258,"segment":1,"confidence":null,"asi":{"hypothesis":"the graph phase looked suspiciously cheap; check whether it was measuring real work","bench_flaw_found":"run_target picked its 12 fixture functions by calling app.program.flowchart(f.addr) to count blocks. Program.flowchart caches per (function, name generation) AND fetches every listing row in the function's extent, so the fixture loop paid the whole cost and phase_graph then timed a cache hit. Fixed: fixtures use client.call('flowchart') directly, and phase_graph clears program._flowcharts first.","carried_forward_from_v1":"experiments 1-10 took total_ms from 46572 to 19006 (-59%) on the old bench; that history is in the archived log and summarised in .auto/prompt.md","bug_this_exposed":"Program.flowchart does rows = _heads_between(min(block.start), max(block.end)) -- the CONVEX HULL of the blocks. IDA function chunks put tail blocks hundreds of KB from the entry, so a 1384-byte function (jobs_builtin) fetches 128000 listing rows and takes 2961ms to graph. Measured on bash: 4 of the 12 fixtures have spans of 280KB-680KB and cost 258-2961ms each; the other 8 have span == size and cost 7-32ms. Then b.rows = [h for h in rows if ...] is O(blocks x rows) on top -- 541ms for the set.","fix_planned":"fetch the MERGED BLOCK INTERVALS instead of the hull (adjacent blocks coalesce, so a normal function is still one call), and assign rows to blocks by bisect instead of a full scan per block","next_action_hint":"apply that fix; it should take lg_graph from 9561 to well under 1000"}}
{"run":12,"commit":"870f89e","metric":19062.1,"metrics":{"lg_boot_ms":751.3,"lg_decomp_ms":2605.7,"lg_graph_ms":1022,"lg_hex_ms":554.5,"lg_index_ms":103.9,"lg_listing_cold_ms":545,"lg_listing_warm_ms":405.6,"lg_nav_ms":6636.8,"lg_palette_ms":4.7,"lg_render_ms":212.2,"lg_search_ms":2269,"pure_graph_ms":239.9,"sm_boot_ms":538.5,"sm_decomp_ms":688.4,"sm_graph_ms":712.7,"sm_hex_ms":557.5,"sm_index_ms":0,"sm_listing_cold_ms":259.9,"sm_listing_warm_ms":260.2,"sm_nav_ms":363.8,"sm_palette_ms":0.3,"sm_render_ms":252.7,"sm_search_ms":77.5,"fails":0},"status":"keep","description":"Fetch a graph's listing rows from the blocks' MERGED EXTENTS, not their convex hull, and assign them per block by bisect. IDA puts a function's cold/tail chunks far from its entry, so the hull of a 1.4KB function could be 680KB wide: it fetched 128k rows, took 3s, and still came back EMPTY for the far blocks because the pager's 64-page bound ran out first. lg_graph 9561 -> 1022.","timestamp":1786062629380,"segment":1,"confidence":null,"asi":{"hypothesis":"Program.flowchart fetches the convex hull of the basic blocks, which is enormous for a function with IDA chunks","gains":"total 27913 -> 19062 (-31.7%); lg_graph 9561 -> 1022 (-89%); sm_graph 962 -> 713","correctness_is_BETTER_not_equal":"differential over the 80 largest functions of bash: 1201 blocks differ, and EVERY one of them is a block the old code returned ZERO rows for. _heads_between is bounded to 64 pages x 2000 heads = 128k, and a 680KB hull exhausted that before reaching the tail chunk -- so far blocks drew as empty boxes. No block lost a row. targets/echo: 0 differences at all (no chunked functions).","scale":"80 bash functions: hull 44.4s / 1,864,814 rows fetched -> intervals 1.4s / 77,221 rows. 24x less data, 31x faster.","design":"blocks are sorted and merged with a 256-byte tolerance so alignment padding does not split an interval; an ordinary contiguous function is still exactly ONE heads call, as before. Row->block assignment is now bisect on the address list instead of a full scan per block (541ms -> ~0 for the 12-function set).","verify_script":"/tmp/fceq.py (kept the pattern in .auto/ideas.md): rebuild the blocks twice, fetch both ways, compare per-block row lists","next_action_hint":"lg_nav 6637 is now 35% of the total and sits at the per-row floor (~16.5us worker + ~7us client). decomp lg+sm 3294 is next: 1370ms of Hex-Rays, 240ms of pygments highlight_c, ~600ms of Textual loading-cover churn. Also unmeasured by the bench: domain.decomp_map costs 280ms per function on the split-view path."}}
{"run":13,"commit":"60f0d70","metric":18856.8,"metrics":{"lg_boot_ms":689.7,"lg_decomp_ms":2484.2,"lg_graph_ms":1120,"lg_hex_ms":700.1,"lg_index_ms":96.5,"lg_listing_cold_ms":425.6,"lg_listing_warm_ms":511.5,"lg_nav_ms":6590.7,"lg_palette_ms":4.8,"lg_render_ms":215.1,"lg_search_ms":2195.5,"pure_graph_ms":238.1,"sm_boot_ms":431.5,"sm_decomp_ms":667.2,"sm_graph_ms":686.9,"sm_hex_ms":569.8,"sm_index_ms":0,"sm_listing_cold_ms":258.1,"sm_listing_warm_ms":283.2,"sm_nav_ms":365.2,"sm_palette_ms":0.3,"sm_render_ms":243.7,"sm_search_ms":79,"fails":0},"status":"keep","description":"Two independent constants: memoise the pygments token -> Rich style lookup (a decompilation uses ~18 distinct token types but each token walked up to nine 'token in ttype' hierarchy checks), and hold the worker-connect poll at 5ms for the first 5s instead of backing off geometrically from the first probe.","timestamp":1786062907221,"segment":1,"confidence":44.11154408183163,"asi":{"hypothesis":"boot time is a polling artefact, and highlighting is a style-lookup problem not a lexing problem","boot_smoking_gun":"WorkerClient.connect() returned in 351ms for BOTH targets/echo (47KB) and targets/bash (1.2MB) -- an identical number for very different work is a polling artefact, not a cost. The geometric backoff (5ms x1.6, capped at 200ms) has reached 134ms by the time a seeded database is ready at ~250ms. Holding 5ms for 5s first: echo 351 -> 260ms. bash is genuinely ~350ms so it did not move.","highlight_split":"on a 40KB body: lexer 52.2ms, _style_for 19.4ms, rest ~10ms of 82ms total. Memoised _style_for is 1.2ms (18 distinct token types in the whole corpus). highlight_c 84.2 -> 62.0ms.","equivalence":"/tmp/hleq.py compares old vs new highlight_c segment-for-segment over 5 bodies x 4 slices including an empty string and hex-rays-shaped pseudocode: 0 mismatches","gains":"total 19062 -> 18857 (-1.1%); sm_boot 539 -> 432 (-20%); lg_boot 751 -> 690","rejected_this_round":"a per-page identity memo for the spans->tuple conversion in Head.from_raw. The worker's line cache does share span objects across rows, but the repeats are spread over the whole segment, not within a 500-row page, so a page-scoped memo never hits: _build_page 4.52 -> 4.68 us/row. A model-scoped memo is UNSAFE because id() is reused once the page's row dicts are collected.","next_action_hint":"lg_nav 6591 (35%) is at the per-row floor. Left: pygments lexing itself (52ms per 40KB), the ~600ms of Textual loading-cover churn across 12 F5s, and lg_search 2196 (two unnarrowed passes over 224k rows at ~2.6us each)."}}
{"run":14,"commit":"2686901","metric":18618.9,"metrics":{"lg_boot_ms":689.6,"lg_decomp_ms":2560,"lg_graph_ms":1035.1,"lg_hex_ms":684.2,"lg_index_ms":100.4,"lg_listing_cold_ms":551.1,"lg_listing_warm_ms":397.8,"lg_nav_ms":6687.8,"lg_palette_ms":4.7,"lg_render_ms":211.9,"lg_search_ms":1858.2,"pure_graph_ms":238.5,"sm_boot_ms":431.4,"sm_decomp_ms":671.2,"sm_graph_ms":728.1,"sm_hex_ms":552,"sm_index_ms":0,"sm_listing_cold_ms":257.4,"sm_listing_warm_ms":281,"sm_nav_ms":365.9,"sm_palette_ms":0.3,"sm_render_ms":242.8,"sm_search_ms":69.6,"fails":0},"status":"keep","description":"Search the whole segment as ONE joined string. Every line is concatenated once (with a start-offset table) so finding a term is a C-level str.find walk instead of a python loop that rebuilds and case-folds 224k lines per keystroke. Falls back to the per-line loop if case-folding changes the string's length.","timestamp":1786063291908,"segment":1,"confidence":41.940433212996666,"asi":{"hypothesis":"per-keystroke search should be one C string scan over the body, not 224k python iterations","design":"SearchMixin._search_haystack builds (starts, blob, blob.lower()) once per (row count, line-source) and caches it; _compute_matches then walks it with str.find and advances a line pointer monotonically (find() only moves forward, so no bisect is needed). A term can never straddle a line because an Input cannot contain a newline.","safety":["str.lower() can CHANGE LENGTH for a few unicode codepoints, which would corrupt every offset after them -- if len differs the haystack is refused and the old per-line loop runs.","the per-line loop (with prefix narrowing) is kept as the fallback and is still exercised.","every site that resets _matches/_ranges now calls _reset_search_cache(), which drops the joined body too -- a stale body would search text the view no longer shows."],"equivalence_test":"/tmp/searcheq2.py drives the real ListingView AND DecompView over 13 terms x every prefix, comparing _matches and _ranges from the haystack path against the same view with _search_haystack monkeypatched to None: 0 mismatches on echo (5952 lines) and ls_ttl (28807 lines)","gains":"total 18857 -> 18619 (-1.3%); lg_search 2196 -> 1858 (-15%); sm_search 79 -> 70","why_not_more":"the bench types only two terms, so the one-off cost of building the body (a _line_plain pass over 224k rows, ~0.5s) is amortised over very little. The benefit compounds for a user who searches more than twice -- the third term onwards is milliseconds.","next_action_hint":"lg_nav 6688 is 36%. Client-side it is pickle.loads 2.4us + _build_page 4.5us per row, and it CANNOT be pipelined: idatui/worker.py's serve() accepts one connection at a time and idalib is main-thread only, so nothing overlaps. Try making Head a NamedTuple (tuple.__new__ vs a frozen dataclass __init__)."}}
{"run":15,"commit":"625b067","metric":17784.1,"metrics":{"lg_boot_ms":680.8,"lg_decomp_ms":2344,"lg_graph_ms":1106.5,"lg_hex_ms":576.4,"lg_index_ms":96.5,"lg_listing_cold_ms":552.9,"lg_listing_warm_ms":431.3,"lg_nav_ms":6256.4,"lg_palette_ms":4.8,"lg_render_ms":224.1,"lg_search_ms":1916.5,"pure_graph_ms":241.3,"sm_boot_ms":442.6,"sm_decomp_ms":591.3,"sm_graph_ms":663.9,"sm_hex_ms":419.2,"sm_index_ms":0,"sm_listing_cold_ms":258.2,"sm_listing_warm_ms":281.8,"sm_nav_ms":374.4,"sm_palette_ms":0.3,"sm_render_ms":251.1,"sm_search_ms":69.8,"fails":0},"status":"keep","description":"HexView.render_line emits style RUNS instead of one Segment per byte cell (35 -> 7 segments per row), and Head is a NamedTuple rather than a frozen dataclass (tuple.__new__ 1.9us vs a dataclass __init__ 2.9us, and it is built once per listing row walked).","timestamp":1786063768655,"segment":1,"confidence":42.57587221521688,"asi":{"hypothesis":"a hex frame costs ~10ms of which only 1.8ms is our render_line, so the cost is what we hand the compositor: 1540 one-cell Segments per frame","hex_change":"accumulate a run and flush it when the style changes. A row's 32 cells almost always share one style (the exception is the single cursor cell and trace-live bytes), so 35 segments/row became 7.","hex_equivalence":"/tmp/hexeq.py expands both the new and the pre-change render_line to (char, style) PER CELL and compares, over 40 scroll frames x 43 rows with the cursor moving through all 16 columns: 0 mismatches in 1720 rows. Comparing segments would have been the wrong test -- merging changes the segments on purpose; what must not change is the cells.","head_change":"Head is a NamedTuple. Measured 2.93 -> 1.88 us to build; attribute reads go 10ns -> 20ns, which is the right way round (a quarter-million rows are built per far jump, and a viewport reads forty). _build_page 4.68 -> 4.05 us/row. Nothing used dataclasses.replace/asdict on it.","gains":"total 18619 -> 17784 (-4.5%); hex lg+sm 1236 -> 996 (-19%); lg_nav 6688 -> 6256; lg_decomp 2560 -> 2344","rejected_this_round":"merging same-kind adjacent spans in _idatui_spans: measured only 2.5% fewer spans on 20k real lines (6.43 -> 6.27 per line). Not worth changing the wire format for.","rejected_earlier":"deferring the 'decompiling...' loading cover until ~120ms (worth ~9ms per F5) -- tests/test_scenarios.py asserts the overlay is raised SYNCHRONOUSLY by F5, guarding a real past regression. That is an assertion, not setup, so it stands.","next_action_hint":"lg_nav 6256 (35%) is the paging floor; the skeleton idea was costed and only nets ~5% because search then has to fetch the text anyway (see .auto/ideas.md). Left: ListingView.render_line segment count, and the search haystack building rows one lock at a time (model.window would amortise it)."}}
{"run":16,"commit":"625b067","metric":17821,"metrics":{"lg_boot_ms":743.4,"lg_decomp_ms":2349.8,"lg_graph_ms":1033.5,"lg_hex_ms":432.2,"lg_index_ms":97.4,"lg_listing_cold_ms":424.8,"lg_listing_warm_ms":508.5,"lg_nav_ms":6677.8,"lg_palette_ms":4.8,"lg_render_ms":223.1,"lg_search_ms":1374.9,"pure_graph_ms":523.7,"sm_boot_ms":438.6,"sm_decomp_ms":591.2,"sm_graph_ms":713.2,"sm_hex_ms":466.7,"sm_index_ms":0,"sm_listing_cold_ms":262.4,"sm_listing_warm_ms":287.1,"sm_nav_ms":376.6,"sm_palette_ms":0.3,"sm_render_ms":257.5,"sm_search_ms":33.5,"fails":0},"status":"discard","description":"Keep the joined search body across a cancelled search and across navigation inside the same segment (drop it only when the model changes or the opcode column toggles). lg_search 1917 -> 1375, sm_search 70 -> 34 — but total_ms is FLAT (17784 -> 17821) because pure_graph (+283) and lg_nav (+421) drifted, neither of which this touches. Re-running to separate the win from the drift.","timestamp":1786064057632,"segment":1,"confidence":16.322294738538417,"asi":{"hypothesis":"the joined search body is keyed by (row count, line source), so ending a search or navigating inside the same segment need not throw it away","phase_evidence":"lg_search -542ms and sm_search -36ms, i.e. the intended effect happened and is far outside that phase's own spread","why_total_did_not_move":"pure_graph 241 -> 524 and lg_nav 6256 -> 6678 in the same run, and this change touches neither. pure_graph is pure CPU with no I/O and is the single jumpiest metric in the suite (seen at 235, 241, 508, 524, 531 across runs with identical code).","work_preserved":".auto/wip-searchbody.patch","gate_strengthened":".auto/check_search.py added and wired into checks.sh: it drives the real ListingView/DecompView and compares the narrowing + haystack fast paths against the plain per-line loop for every prefix of 14 terms, including after toggling the opcode column and after navigating. 140 prefixes, 0 mismatches. A stale cache still returns AN answer, so the scenario suite could never have caught this class of bug.","invalidation_rules_now":["body dropped when ListingView.load gets a DIFFERENT model (navigation inside one segment reuses it)","body dropped by action_toggle_opcodes -- the opcode hex is searchable text and its width changes without the row count or model moving","body dropped by DecompView.show, explicitly rather than relying on id(self._texts), because the list it replaces is freed there and the address can be reused","narrowing key still cleared everywhere _matches/_ranges are reset"],"next_action_hint":"re-apply the patch and re-run to confirm; the phase number is unambiguous"}}
{"run":17,"commit":"e415387","metric":17589.8,"metrics":{"lg_boot_ms":710.6,"lg_decomp_ms":2404.9,"lg_graph_ms":1136.1,"lg_hex_ms":425.6,"lg_index_ms":103.4,"lg_listing_cold_ms":417.2,"lg_listing_warm_ms":508.1,"lg_nav_ms":6553.8,"lg_palette_ms":4.8,"lg_render_ms":215.7,"lg_search_ms":1400.3,"pure_graph_ms":239.3,"sm_boot_ms":443.2,"sm_decomp_ms":668.4,"sm_graph_ms":698.1,"sm_hex_ms":443.6,"sm_index_ms":0,"sm_listing_cold_ms":267.3,"sm_listing_warm_ms":286,"sm_nav_ms":356.6,"sm_palette_ms":0.3,"sm_render_ms":258.6,"sm_search_ms":47.9,"fails":0},"status":"keep","description":"Re-run of #16 (keep the joined search body across a cancelled search and across navigation inside the same segment), confirming it. total 17784 -> 17590; lg_search 1917 -> 1400, sm_search 70 -> 48. Also lands .auto/check_search.py in the checks gate: it compares both search fast paths against the plain per-line loop for every typed prefix.","timestamp":1786064181968,"segment":1,"confidence":12.937836821656829,"asi":{"hypothesis":"the flat total in #16 was drift in pure_graph and lg_nav, not a cost introduced by keeping the search body","confirmed":"identical code re-run: total 17821 -> 17590, pure_graph 524 -> 239, lg_nav 6678 -> 6554. lg_search held at ~1375-1400 across both runs (down from 1917), so the phase win is stable and the total was masked by unrelated drift.","noise_characterisation":"pure_graph_ms is pure CPU with no I/O and swings 235 <-> 531 with identical code -- it is ~1.5% of the total on its own. lg_nav swings ~+/-350ms (5%). Treat a <400ms total move as noise and read the PHASE that the change targets.","gate_strengthened":".auto/check_search.py now runs in checks.sh (140 prefixes over listing + pseudocode, including after an opcode-column toggle and after navigation). Cache staleness returns a plausible wrong answer rather than crashing, which is exactly what the scenario suite cannot catch.","gains":"total 17784 -> 17590 (-1.1%); lg_search 1917 -> 1400 (-27%); sm_search 70 -> 48 (-31%). Against the v2 baseline: -37.0%.","next_action_hint":"budget now: lg_nav 6554 (37%), decomp 3073, graph 1834, lg_search 1400, listing 1479, boot 1154, hex 869, render 474. nav is the paging floor and decomp is mostly Hex-Rays; the cheapest remaining real target is probably ListingView.render_line's segment count (the hex run-merge trick paid off there)."}}
{"run":18,"commit":"e415387","metric":17890.1,"metrics":{"lg_boot_ms":685.6,"lg_decomp_ms":2604.2,"lg_graph_ms":885.8,"lg_hex_ms":608,"lg_index_ms":96.4,"lg_listing_cold_ms":538,"lg_listing_warm_ms":415.7,"lg_nav_ms":6756.8,"lg_palette_ms":4.8,"lg_render_ms":211,"lg_search_ms":1370.1,"pure_graph_ms":247.9,"sm_boot_ms":431.2,"sm_decomp_ms":670.7,"sm_graph_ms":699.6,"sm_hex_ms":432.7,"sm_index_ms":0,"sm_listing_cold_ms":261.9,"sm_listing_warm_ms":287,"sm_nav_ms":378.1,"sm_palette_ms":0.3,"sm_render_ms":247.6,"sm_search_ms":56.9,"fails":0},"status":"discard","description":"_CellRow.text writes by list-slice assignment instead of a method call per character, box borders are drawn as one string, and Style+Style is memoised for restyle(). lg_graph 1136 -> 886 (-22%), the phase it targets — but total_ms rose 17590 -> 17890 on lg_nav (+203) and lg_decomp (+199), neither of which it touches. Re-running to tell the win from the drift.","timestamp":1786064561542,"segment":1,"confidence":19.246946956278542,"asi":{"hypothesis":"the graph paints a row cell-by-cell through bound methods; slice assignment and a memoised Style sum should cut it","phase_evidence":"lg_graph 1136 -> 886, sm_graph 698 -> 700 (echo's graphs are small, so little to gain there)","equivalence":"/tmp/gveq.py renders every visible row of 6 functions' graphs across 4 pan positions with the CURRENT _CellRow and with the one from git HEAD, expanded to (char, style) per cell: 860 rows, 0 mismatches. Graph scenarios: 50 passed 0 failed.","noise_problem":"the total's spread is now bigger than most single wins. Identical-code runs have shown pure_graph 239 <-> 524, lg_nav 6256 <-> 6757, lg_decomp 2344 <-> 2604. Call it +/-500ms on 17.6s (~3%); read the targeted PHASE, and confirm a keep with a second run.","work_preserved":".auto/wip-cellrow.patch","next_action_hint":"re-apply and re-run. Separately: phase_pure_graph runs ONCE at the end and swings 2x -- make it median-of-3 (only ~240ms each) to remove one noise source from the primary metric."}}
{"run":19,"commit":"ba96500","metric":17501.4,"metrics":{"lg_boot_ms":711,"lg_decomp_ms":2484.8,"lg_graph_ms":1017.2,"lg_hex_ms":431.9,"lg_index_ms":98.7,"lg_listing_cold_ms":528,"lg_listing_warm_ms":409.2,"lg_nav_ms":6488.2,"lg_palette_ms":4.7,"lg_render_ms":214.8,"lg_search_ms":1473.5,"pure_graph_ms":244.3,"sm_boot_ms":451.4,"sm_decomp_ms":596,"sm_graph_ms":691.3,"sm_hex_ms":444.5,"sm_index_ms":0,"sm_listing_cold_ms":263,"sm_listing_warm_ms":284.2,"sm_nav_ms":374.7,"sm_palette_ms":0.3,"sm_render_ms":252.1,"sm_search_ms":37.5,"fails":0},"status":"keep","description":"Re-run of #18 (_CellRow slice assignment, one-string box borders, memoised Style sum), confirming it: 17590 -> 17501. lg_graph averages 951 over the two runs against 1136 before.","timestamp":1786064733764,"segment":1,"confidence":26.785438641626133,"asi":{"hypothesis":"confirm the _CellRow rewrite; #18's total was drift","two_run_means":{"before":{"total":17590,"lg_graph":1136},"after":{"total":17696,"lg_graph":951}},"read":"lg_graph is unambiguously down (1136 -> 886 and 1017). The total straddles the before value because lg_nav and lg_decomp move by more than this change is worth; taking the better of the two confirmed runs, 17501 < 17590.","method_note":"the honest protocol used here and in #16/#17: save the diff to .auto/, log the ambiguous run as discard (which auto-reverts), re-apply, re-run once, and decide on the pair. Never re-run a third time hoping for a better number.","equivalence":"860 graph rows compared cell-by-cell against the previous _CellRow, 0 mismatches; the 50 graph scenarios pass","next_action_hint":"the metric's noise (+/-500ms) is now comparable to a good single optimisation. Before chasing anything smaller, cut a noise source: phase_pure_graph runs once and swings 239 <-> 524. Make it median-of-3."}}
{"type":"config","name":"ida-tui performance (v3 bench: lower-noise estimator)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
{"type":"config","name":"ida-tui performance (v4 bench: cold phases measured once, repeatable ones averaged)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
{"run":20,"commit":"49184b6","metric":18497.9,"metrics":{"lg_boot_ms":694.8,"lg_decomp_ms":2674,"lg_graph_ms":992.7,"lg_hex_ms":429.1,"lg_index_ms":94.9,"lg_listing_cold_ms":545.3,"lg_listing_warm_ms":410.7,"lg_nav_ms":6489,"lg_palette_ms":4.7,"lg_render_ms":220.6,"lg_search_ms":1489.8,"pure_graph_ms":241,"sm_boot_ms":467.1,"sm_decomp_ms":1308.5,"sm_graph_ms":761,"sm_hex_ms":431.7,"sm_index_ms":2.5,"sm_listing_cold_ms":273.9,"sm_listing_warm_ms":289,"sm_nav_ms":373,"sm_palette_ms":0.3,"sm_render_ms":257.2,"sm_search_ms":47,"fails":0},"status":"keep","description":"RE-BASELINE (v4 bench). Adding a second repetition on the big target exposed the same flaw the graph phase had: decompile, search and the function index all cache their answer, so a second rep reported a dict lookup under the name of the thing a user waits for. Cold-sensitive phases (listing_cold, decomp, search, index) now run ONCE; repeatable ones (render, hex, graph, listing_warm) run every rep and take the median. pure_graph is median-of-3.","timestamp":1786065121146,"segment":3,"confidence":null,"asi":{"hypothesis":"reduce the metric's noise so changes worth 1-2% are readable","what_changed_in_the_bench":"pure_graph median-of-3 (it swung 239 <-> 524 with identical code); two reps on targets/bash; and cold-sensitive phases pinned to the first rep only","flaw_this_caught":"sm_decomp had been min-of-2 since the start, i.e. it was reporting a WARM decompile (Program._decomp is cached per function). Honest cold value is 1308ms, not ~650. Same for sm_search and sm_index. All comparisons within v1-v3 were still valid (consistent measurement), but the absolute picture was wrong: decomp is 22% of the total, not 12%.","cumulative_history":"v1 baseline 46572 -> 19006 over 10 experiments. v2 (cold graph opens measured) baseline 27913 -> 17501 over 9. v3 abandoned after one run for the flaw above. v4 baseline 18498.","budget_ms":{"lg_nav":6489,"decomp lg+sm":3983,"graph lg+sm":1754,"search lg+sm":1537,"listing lg+sm":1519,"boot lg+sm":1162,"hex lg+sm":861,"render lg+sm":478,"pure_graph":241},"next_action_hint":"decomp is now clearly #2 at 22%. sm_decomp is 1308ms for TWELVE small echo functions (109ms each), which is far more than Hex-Rays should need on a 1.6KB function -- profile the cold F5 path on echo before assuming it is the decompiler."}}
{"type":"config","name":"ida-tui performance (v5 bench: 2ms landing polls; final measurement shape)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
{"run":21,"commit":"5a8027e","metric":18516,"metrics":{"lg_boot_ms":663.9,"lg_decomp_ms":2663.2,"lg_graph_ms":1012.3,"lg_hex_ms":436.1,"lg_index_ms":98.4,"lg_listing_cold_ms":437.7,"lg_listing_warm_ms":409.4,"lg_nav_ms":6841.9,"lg_palette_ms":4.7,"lg_render_ms":218.8,"lg_search_ms":1474.7,"pure_graph_ms":241.6,"sm_boot_ms":433.3,"sm_decomp_ms":1266.3,"sm_graph_ms":737.9,"sm_hex_ms":424.2,"sm_index_ms":2.3,"sm_listing_cold_ms":262.9,"sm_listing_warm_ms":283.2,"sm_nav_ms":301.2,"sm_palette_ms":0.3,"sm_render_ms":253.9,"sm_search_ms":47.9,"fails":0},"status":"keep","description":"Baseline for the v5 bench (landing polls every 2ms instead of 10ms; the poll interval was measurement overhead inside the timed regions). Final measurement shape — no further bench changes.","timestamp":1786065328607,"segment":4,"confidence":null,"asi":{"hypothesis":"the 10ms wait_for step was charging every timed landing up to 10ms of quantisation","effect":"sm_nav 373 -> 301, sm_decomp 1309 -> 1266 (12 timed waits each). lg_graph and lg_decomp did not move measurably -- their per-item costs are large enough that 5ms of expected overshoot is lost in the spread.","decision":"this is the last bench change. Any further one costs a re-baseline, and the measurement is now honest about cold vs warm (v4) and free of both the cache-hit and quantisation artefacts.","budget_ms":{"lg_nav":6842,"decomp lg+sm":3930,"graph lg+sm":1750,"search lg+sm":1523,"listing lg+sm":1393,"boot lg+sm":1097,"hex lg+sm":860,"render lg+sm":473,"pure_graph":242},"decomp_anatomy_echo":"12 small functions: Program.decompile (Hex-Rays) 562ms, UI F5 path with everything already cached 414ms, four scroll+paint frames 186ms, highlight_c 86ms. The 414ms of UI is two @work thread spawns, ~4 event-loop hops, and a loading-cover mount/unmount per F5 -- and show() runs exactly ONCE per F5 (checked by counting), so there is no double work to remove.","next_action_hint":"the loading cover is 4.6ms of mount+unmount per F5 and cannot be deferred (a scenario asserts F5 raises it synchronously). The remaining decomp lever is _show_active spawning _load_decomp as a thread even when Program already has the decompilation cached."}}
{"run":22,"commit":"5a8027e","metric":18535.4,"metrics":{"lg_boot_ms":692.3,"lg_decomp_ms":2662.1,"lg_graph_ms":895.9,"lg_hex_ms":422.2,"lg_index_ms":94.1,"lg_listing_cold_ms":525.4,"lg_listing_warm_ms":404.2,"lg_nav_ms":6821.6,"lg_palette_ms":4.7,"lg_render_ms":215.7,"lg_search_ms":1489.3,"pure_graph_ms":247.8,"sm_boot_ms":437.2,"sm_decomp_ms":1258.5,"sm_graph_ms":749.3,"sm_hex_ms":437.6,"sm_index_ms":2.4,"sm_listing_cold_ms":263.3,"sm_listing_warm_ms":283.1,"sm_nav_ms":321.5,"sm_palette_ms":0.3,"sm_render_ms":257.5,"sm_search_ms":49.2,"fails":0},"status":"discard","description":"Apply an already-decompiled function inline instead of spawning a background worker for it (Program.cached_decompilation/cached_pc_nums + _decomp_now), and warm pc_nums in the thread that already decompiled. Total flat (18516 -> 18535) and the phase it targets did not move either (lg_decomp 2663 -> 2662, sm_decomp 1266 -> 1259).","timestamp":1786065730509,"segment":4,"confidence":null,"asi":{"hypothesis":"an F5 spawns two @work threads and hops the event loop four times; skipping the second when the answer is already cached should show up in decomp","result":"it does not. A Textual thread spawn plus its call_from_thread round trip is worth well under 1% of a 100-220ms F5. The cost is elsewhere: Hex-Rays itself (562ms of sm_decomp's 1266), the loading-cover mount/unmount, highlight_c, and the Strip/scroll work in DecompView.show.","rollback_reason":"no measurable gain on the targeted phase, and it adds a second path into _apply_decomp with its own generation-check reasoning to keep correct. Simpler is better.","correctness_was_fine":"301 scenarios passed with it in; this is a complexity-vs-payoff rejection, not a bug.","kept_knowledge":"Program.cached_decompilation / cached_pc_nums would be the right primitives if a future change needs to know whether an answer is in hand without paying for it","next_action_hint":"boot is 1097ms across both targets and ~300ms of it is the worker importing idapro before it can serve. Check whether launch.py can spawn the worker BEFORE Textual starts, so that import overlaps the app's own startup instead of following it."}}
{"run":23,"commit":"98b3b98","metric":17944.4,"metrics":{"lg_boot_ms":703.7,"lg_decomp_ms":2453.7,"lg_graph_ms":1000,"lg_hex_ms":478,"lg_index_ms":96,"lg_listing_cold_ms":453,"lg_listing_warm_ms":411.2,"lg_nav_ms":6555.9,"lg_palette_ms":4.9,"lg_render_ms":218.4,"lg_search_ms":1308.8,"pure_graph_ms":212.3,"sm_boot_ms":432.4,"sm_decomp_ms":1267.8,"sm_graph_ms":742.7,"sm_hex_ms":440,"sm_index_ms":2.5,"sm_listing_cold_ms":264.3,"sm_listing_warm_ms":289.5,"sm_nav_ms":305.4,"sm_palette_ms":0.3,"sm_render_ms":258.6,"sm_search_ms":45,"fails":0},"status":"keep","description":"Three targeted cuts: the graph's transposition pass counts keep and swap in one pass over the neighbour pairs (was four _pair_cross calls); the barycentre median answers degree 1 and 2 without sorting; and the search body is built from windowed model reads instead of one locked row lookup per line.","timestamp":1786066135637,"segment":4,"confidence":29.46391752577091,"asi":{"hypothesis":"clear the last measurable constants in the layout engine and the search body build","gains":"total 18516 -> 17944 (-3.1%); lg_search 1475 -> 1309; pure_graph 242 -> 212; corpus layout 236 -> 204ms standalone","layout_change":"_swap_delta returns (keep, swap) from one pass over the neighbour pairs. _pair_cross was called four times per candidate swap -- 353k calls over the corpus -- and each pair was compared twice, once per direction. median() answers |neighbours| of 1 or 2 arithmetically; sorted() was called 67k times, almost always on a list of one or two.","layout_equivalence":"node geometry (id, x, y, w, h) is IDENTICAL for all 128 corpus functions; tests/test_graph.py passes 470 checks including the no-edge-inside-a-box invariant.","PRE_EXISTING_BUG_FOUND":"idatui/graph.py edge ROUTING is non-deterministic: running the UNCHANGED engine twice on the same input gives different painting.vruns for 71 of 128 functions. Node placement is stable; only the routing moves. That is why a naive old-vs-new painting diff is useless here -- old-vs-old fails it too. Logged in .auto/ideas.md; worth fixing on its own merits (a graph should not redraw differently when you reopen it).","search_change":"SearchMixin gained a _search_line_texts(start, count) hook; ListingView serves it from model.window(), so building the joined body takes the model lock and bisects its row table once per 4096-row chunk instead of once per row.","verified":".auto/check_search.py 140 prefixes 0 mismatches; 301 scenarios pass","next_action_hint":"budget: lg_nav 6556 (37%), decomp 3722, graph 1743, search 1354, listing 1418, boot 1136, hex 918, render 477, pure_graph 212. Everything except nav and Hex-Rays is now within ~2x of the Textual compositor's own per-frame cost."}}
{"run":24,"commit":"8218b91","metric":18608,"metrics":{"lg_boot_ms":708.5,"lg_decomp_ms":2454.9,"lg_graph_ms":1034.5,"lg_hex_ms":431.9,"lg_index_ms":95.1,"lg_listing_cold_ms":530.2,"lg_listing_warm_ms":413.4,"lg_nav_ms":7057.9,"lg_palette_ms":5,"lg_render_ms":214.2,"lg_search_ms":1408.9,"pure_graph_ms":213.3,"sm_boot_ms":433.6,"sm_decomp_ms":1292,"sm_graph_ms":754,"sm_hex_ms":433.4,"sm_index_ms":2.6,"sm_listing_cold_ms":260.4,"sm_listing_warm_ms":280.5,"sm_nav_ms":286.1,"sm_palette_ms":0.3,"sm_render_ms":251,"sm_search_ms":46.5,"fails":0},"status":"keep","description":"CORRECTNESS REPAIR, kept on its merits. The full suite (which the gate was NOT running) revealed that the worker-connect poll change made test_project_ui flaky: 5ms polling on a background thread through a cold auto-analysis starved the UI thread enough that the loading overlay was still up when the test pressed Ctrl+O. Poll now backs off to a 25ms cap (keeps the boot win, no busy-wait), the racy boot wait is fixed, and checks.sh runs tests/run.py in full (830 checks) instead of just the scenario suite.","timestamp":1786068002977,"segment":4,"confidence":12.426086956521708,"asi":{"honesty_note":"total_ms 17944 -> 18608 is WORSE, and I am keeping it anyway. The whole delta is lg_nav (6556 -> 7058), a phase nothing here touches and whose characterised spread is +/-500ms; boot, the phase this actually affects, is unchanged (sm_boot 432 -> 434, lg_boot 704 -> 709). Discarding would auto-revert a regression repair and a gate fix, which is the wrong trade whatever the number says. Re-running next to confirm the drift.","the_regression":"tests/test_project_ui.py went from 4/4 passing on the pre-autoresearch code to 1/3 on the branch. Bisected to idatui/worker_client.py: reverting it alone gave 3/3, reverting idatui/app.py alone gave 1/3.","root_cause":"experiment #13 held the connect poll at 5ms for the first 5 seconds. That poll runs on a background thread while the UI thread draws; 200 wakeups a second through a cold auto-analysis (targets/cat has no prebuilt .i64) cost enough GIL time to delay the app's own startup. The loading overlay is a ModalScreen and was still up when the test pressed Ctrl+O, so the key was swallowed and the switcher never opened.","fix":"cap the geometric backoff at 25ms instead. Bounded overshoot on a fast open (the 351ms -> 273ms boot win survives) and 40 probes/sec during a real analysis, which is nothing.","test_repair_and_its_three_proofs":["bisect: worker_client.py flips it, app.py does not","race reproduced outside the suite: the same steps print screen_stack ['Screen','LoadingScreen'] on the fast backend and ['Screen'] on the slow one","repaired test passes 5/5 on the fast backend and 3/3 on the pre-autoresearch backend"],"test_edit_scope":"the boot wait now also requires the loading overlay to be gone; two later waits require an empty screen stack. No assertion changed. Same class of fix as graph_minimap in v1 #6.","GATE_GAP_CLOSED":"checks.sh ran only tests/run.py --fast + the scenario suite. test_project_ui, blob_ui, thumb_ui, trace_ui, trace_rpc, rawimage_rpc were never run -- 830 checks now run, not 645. Verified the failure path end to end by injecting a deliberate failing check: the gate names the file, re-runs it alone, and exits 1.","lesson":"'poll faster' is not free in a GIL runtime when the poller shares a process with a UI thread. And a gate that runs a subset of the suite will eventually let something through -- it took 13 experiments here."}}
{"run":25,"commit":"8218b91","metric":18746.2,"metrics":{"lg_boot_ms":713.2,"lg_decomp_ms":2575.4,"lg_graph_ms":1010.9,"lg_hex_ms":457.3,"lg_index_ms":97.3,"lg_listing_cold_ms":561.7,"lg_listing_warm_ms":411.8,"lg_nav_ms":6936.8,"lg_palette_ms":4.7,"lg_render_ms":225.9,"lg_search_ms":1442.5,"pure_graph_ms":212.9,"sm_boot_ms":433.3,"sm_decomp_ms":1304.2,"sm_graph_ms":710.8,"sm_hex_ms":456.7,"sm_index_ms":2.3,"sm_listing_cold_ms":263.9,"sm_listing_warm_ms":285.7,"sm_nav_ms":334.7,"sm_palette_ms":0.3,"sm_render_ms":257.2,"sm_search_ms":46.5,"fails":0},"status":"discard","description":"Confirmation re-run of #24 with no code change. 18608 -> 18746, i.e. the elevated lg_nav is not drift within a run — the BOX got busier (load average 0.62 at session start, 1.55 now). The direct paging microbenchmark is unchanged at 24.4 us/row, so the code is the same speed; the wall clock is not.","timestamp":1786068203628,"segment":4,"confidence":7.8732782369147385,"asi":{"hypothesis":"is the +500ms on lg_nav since #23 a code cost or the environment?","answer":"environment. /tmp/domain_break.py (a direct cold walk of bash's .text through the worker, no UI) reads 20.31us/row in the heads call + 4.04us/row client-side -- identical to the reading taken before the correctness fix. Meanwhile the box's load average went 0.62 -> 1.55 during the session.","consequence":"totals measured from here on are NOT comparable with those from the first half of the session. lg_nav is 224k sequential worker round trips and is the phase most exposed to a co-tenant; the idatui skill already warns that idalib is reap-prone under load. Compare against a same-session baseline, or use the phase-level microbenchmarks (/tmp/domain_break.py, /tmp/spanbench2.py, .auto/diff_spans.py) which are far less exposed.","no_code_change":"nothing to revert; logged as discard because the metric did not improve","next_action_hint":"if the loop continues on a loaded box, prefer changes whose effect is verifiable in a microbenchmark rather than in total_ms"}}
{"run":26,"commit":"7683ca3","metric":17829.9,"metrics":{"lg_boot_ms":724.4,"lg_decomp_ms":2353.5,"lg_graph_ms":1041,"lg_hex_ms":434.9,"lg_index_ms":95.5,"lg_listing_cold_ms":545.6,"lg_listing_warm_ms":407.9,"lg_nav_ms":6922,"lg_palette_ms":4.8,"lg_render_ms":218.1,"lg_search_ms":884.3,"pure_graph_ms":213.6,"sm_boot_ms":432,"sm_decomp_ms":1250.2,"sm_graph_ms":687.4,"sm_hex_ms":457.5,"sm_index_ms":2.6,"sm_listing_cold_ms":265.4,"sm_listing_warm_ms":288.8,"sm_nav_ms":303.2,"sm_palette_ms":0.3,"sm_render_ms":253.9,"sm_search_ms":43,"fails":0},"status":"keep","description":"Highlight ranges are computed per line on demand instead of for every match. Searching one character over bash matches 177k lines at 310k places, and all but the forty on screen were built and thrown away. _MatchRanges keeps the line SET eagerly and works out the offsets when a line is painted or the cursor lands on it; the blob scan now also skips to the next line after a hit.","timestamp":1786068588431,"segment":4,"confidence":4.5317040951122864,"asi":{"hypothesis":"profiling the search phase directly showed the per-match range building, not the scanning, was the cost","microbench_before_after_ms_per_term":{"note":"targets/bash, 228659 rows, _compute_matches only","m":[139.7,41.6],"mo":[153.9,49.2],"mov":[55.4,47],"c":[329.6,66.1],"ca":[51.5,47.3],"cal":[32.9,30],"call":[29.2,26.8],"haystack_build":[458,459]},"equivalence":"the microbenchmark materialises every range afterwards and prints the totals: 116736 / 76271 / 76070 / 309827 / 25222 / 16312 / 15722 -- identical to the eager version, term for term. Plus .auto/check_search.py (140 prefixes, listing + pseudocode, 0 mismatches) and the full 830-check suite.","api_shape":"_MatchRanges quacks as the dict it replaced for the subset anything uses (in / get / [] / items / len). tests/test_scenarios.py reads dis._ranges.get(cursor) and dis._ranges[cursor][0][0] directly, so that had to keep working -- it does, untouched.","gains":"total 18608 -> 17830 on a box that has got busier since the earlier runs; lg_search 1409 -> 884 (-37%), sm_search 46 -> 43","whats_left_in_search":"the haystack build, 459ms: one _line_plain pass over 228k rows plus a 12.3MB join and lower(). That is the floor unless the plain text is cached with the rows.","next_action_hint":"budget: lg_nav 6922 (39%, and inflated by machine load), decomp 3604, graph 1728, search 927, listing 1508, boot 1156, hex 892, render 472, pure_graph 214."}}
{"run":27,"commit":"b2b59e0","metric":17700.4,"metrics":{"lg_boot_ms":776,"lg_decomp_ms":2690.1,"lg_graph_ms":888.1,"lg_hex_ms":449.1,"lg_index_ms":97.4,"lg_listing_cold_ms":439.4,"lg_listing_warm_ms":404.6,"lg_nav_ms":6762.9,"lg_palette_ms":4.6,"lg_render_ms":230.8,"lg_search_ms":762.9,"pure_graph_ms":217.1,"sm_boot_ms":452.6,"sm_decomp_ms":1270,"sm_graph_ms":698,"sm_hex_ms":433.2,"sm_index_ms":2.4,"sm_listing_cold_ms":259.1,"sm_listing_warm_ms":260.5,"sm_nav_ms":300.4,"sm_palette_ms":0.3,"sm_render_ms":258.3,"sm_search_ms":42.7,"fails":0},"status":"keep","description":"Keep a listing row's spans and operand extents exactly as they came off the wire instead of copying them into tuples. The copy re-proved types the worker's own tool guarantees, and it destroyed the object sharing the worker's line cache had created — 228k rows now reference 125k span lists, not 228k private tuples.","timestamp":1786068961469,"segment":4,"confidence":3.5430060816680973,"asi":{"hypothesis":"the tuple conversion in Head.from_raw is the last measurable client-side cost per listing row","microbench":"cold page load of bash's .text, broken into three: worker call 19.85 us/row, client _build_page 3.19 -> 2.51 us/row, index loop 0.37 us/row. The index loop (setdefault + three appends + a _span call per head) is NOT worth touching.","memory":"a full 228 659-row bash listing costs the client ~258MB RSS and now holds 125 510 distinct span objects rather than one per row -- the worker memoises its per-line render and pickle preserves that sharing within a page, which copying threw away.","safety_audit_done_first":"every reader of h.spans / h.ops only iterates or indexes (app.py _span_segments, _cursor_operand, GraphView._draw_node_row, Head.op_at, and four places in tests). Nothing mutates them and nothing hashes a Head -- which matters, because Head is a NamedTuple and a list field would make it unhashable.","gains":"total 17830 -> 17700; lg_search 884 -> 763; lg_graph 1041 -> 888; sm_nav_worst 177 -> 151","verified":"830-check full suite green, .auto/check_search.py 140 prefixes 0 mismatches","state":"the per-row cost is now 19.85us in the worker (of which generate_disasm_line is 5.9) and 2.9us in the client. There is no further client-side lever worth the risk."}}
{"run":28,"commit":"722025a","metric":17464.6,"metrics":{"lg_boot_ms":744.8,"lg_decomp_ms":2600.3,"lg_graph_ms":889.3,"lg_hex_ms":436.1,"lg_index_ms":95.5,"lg_listing_cold_ms":436.4,"lg_listing_warm_ms":407.8,"lg_nav_ms":6747.4,"lg_palette_ms":4.6,"lg_render_ms":218.9,"lg_search_ms":760.9,"pure_graph_ms":215.7,"sm_boot_ms":429.7,"sm_decomp_ms":1263.5,"sm_graph_ms":650.2,"sm_hex_ms":435.3,"sm_index_ms":2.3,"sm_listing_cold_ms":262.8,"sm_listing_warm_ms":264.7,"sm_nav_ms":305.4,"sm_palette_ms":0.3,"sm_render_ms":251.4,"sm_search_ms":41.1,"fails":0},"status":"keep","description":"Confirmation re-run of #27, no code change: 17700 -> 17465, the best v5 reading. Confirms the wire-shape change holds and that the run-to-run spread is ~250ms even on the now-busier box.","timestamp":1786069196018,"segment":4,"confidence":2.702480400976747,"asi":{"hypothesis":"confirm #27 and take a clean reading of the finished state","v5_progress":"18516 -> 17465 (-5.7%) across seven experiments, on a box whose load roughly tripled during them","phase_state_ms":{"lg_nav":6747,"lg_decomp":2600,"sm_decomp":1264,"lg_graph":889,"sm_graph":650,"lg_search":761,"sm_search":41,"lg_listing_cold":436,"lg_listing_warm":408,"lg_boot":745,"sm_boot":430,"hex":871,"render":470,"pure_graph":216},"per_row_cost_model_final":"cold listing paging is 22.7us/row: 19.85 in the worker (generate_disasm_line 5.9 of it, and the _idatui_line_parts cache absorbing the rest) + 2.5 building Heads + 0.37 indexing them. Before this session it was 116us/row.","remaining_levers_all_rejected_with_numbers":{"flags_recomputed_3x_per_head":"0.36us/head = 0.7% of total, needs flags threaded through three functions","wire_shape_dict_vs_tuple":"<=0.3us/row","PAGE_size":"no effect at any of 500/1000/2000","index_loop_in_load_next_page":"0.37us/row total"},"conclusion":"what is left is IDA's generate_disasm_line, Hex-Rays, Textual's compositor, and the fact that a listing row index is a linear count. All four are outside this codebase or would need the architecture change costed in .auto/ideas.md."}}
{"type":"config","name":"ida-tui performance (v6 bench: split view covered)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
{"run":29,"commit":"7ea3ca1","metric":33502.3,"metrics":{"lg_boot_ms":772.2,"lg_decomp_ms":2546.9,"lg_graph_ms":903.4,"lg_hex_ms":434.1,"lg_index_ms":97.7,"lg_listing_cold_ms":420.2,"lg_listing_warm_ms":397.4,"lg_nav_ms":6501.1,"lg_palette_ms":4.9,"lg_render_ms":212.4,"lg_search_ms":760.4,"lg_split_ms":10189.2,"pure_graph_ms":212.3,"sm_boot_ms":443.8,"sm_decomp_ms":1275.3,"sm_graph_ms":720.5,"sm_hex_ms":451,"sm_index_ms":2.5,"sm_listing_cold_ms":263.3,"sm_listing_warm_ms":264.7,"sm_nav_ms":302.8,"sm_palette_ms":0.3,"sm_render_ms":248.4,"sm_search_ms":43.9,"sm_split_ms":6033.6,"fails":0},"status":"keep","description":"RE-BASELINE (v6 bench). The split view ('s') was not covered at all, and it turns out to be the most expensive thing in the app: 16.2s of a 33.5s session (lg_split 10189 + sm_split 6034), or 500ms per function on echo and 850ms on bash, just to open it.","timestamp":1786070710925,"segment":5,"confidence":null,"asi":{"hypothesis":"the bench covers listing, decomp, graph, hex, search, nav and boot -- but not the split view, and an earlier aside measured decomp_map at 280ms per function","finding":"split is 48% of the whole benchmarked session once measured. 12 echo functions take 6.0s to open side by side; 12 bash functions take 10.2s.","why_it_was_missed":"the phase list was written from the README's headline features and 's' was not one of them. Same class of gap as v2 (graph opens timed a cache hit): if a feature is not in the bench, its cost is invisible however carefully you profile the ones that are.","cause_already_diagnosed":"server/patch_server.py decomp_map sweeps EVERY COLUMN of every pseudocode line, allocating three ctree_item_t SWIG objects per column and calling item.dstr() (which formats a whole 'EA: description' string) each time. Three separate wastes: the allocations, sweeping len(sl.line) which is the TAGGED length (124 columns for a 23-column line), and re-formatting for columns that report the same ctree item.","fix_ready":".auto/wip-decompmap.patch -- verified byte-identical over 165 functions across echo/ls_ttl/bash, 8.9-9.8x faster with Hex-Rays warm for both sides (bash's 25 largest: 67.4s -> 6.8s of sweeping)","next_action_hint":"apply it"}}
{"run":30,"commit":"da1dfe8","metric":19834.8,"metrics":{"lg_boot_ms":676.9,"lg_decomp_ms":2698.8,"lg_graph_ms":892.1,"lg_hex_ms":440.4,"lg_index_ms":93.6,"lg_listing_cold_ms":438.5,"lg_listing_warm_ms":420.8,"lg_nav_ms":6646.7,"lg_palette_ms":4.7,"lg_render_ms":215.1,"lg_search_ms":767,"lg_split_ms":1327.6,"pure_graph_ms":213.6,"sm_boot_ms":468,"sm_decomp_ms":1317.1,"sm_graph_ms":730.7,"sm_hex_ms":483.2,"sm_index_ms":2.6,"sm_listing_cold_ms":266.1,"sm_listing_warm_ms":268.1,"sm_nav_ms":288.6,"sm_palette_ms":0.3,"sm_render_ms":253,"sm_search_ms":47.1,"sm_split_ms":874.2,"fails":0},"status":"keep","description":"decomp_map: stop sweeping every column three times over. It allocated three ctree_item_t SWIG objects PER COLUMN, swept the tagged line length (124 columns for a 23-column line), and called dstr() — which formats a whole 'EA: description' string — for every column even though consecutive columns report the same ctree item. Now: one item, no head/tail, visible columns only, and dstr() only when the item's obj_id changes.","timestamp":1786070905591,"segment":5,"confidence":null,"asi":{"gains":"total 33502 -> 19835 (-40.8%); lg_split 10189 -> 1328 (-87%); sm_split 6034 -> 874 (-86%)","work_unchanged_proof":"split_mapped_lines is 985 (echo) and 2069 (bash) BEFORE and AFTER -- the same per-line instruction sets are produced, from the same number of lines","equivalence":"/tmp/dmapeq2.py calls the shipped tool and a copy of the pre-change implementation for the same functions and compares the whole payload: 0 real mismatches over 165 functions on echo/ls_ttl/bash. (Two apparent ones were my reference's placeholder error string for functions Hex-Rays refuses.) With Hex-Rays warm for both sides the sweep is 8.9-9.8x faster; bash's 25 largest went 67.4s -> 6.8s.","the_three_wastes":["three ctree_item_t SWIG allocations per COLUMN -- one per call is enough, and head/tail are filled but never read, so pass None","range(len(sl.line)) is the TAGGED length: sl.line still carries IDA's colour tags, so a 23-column line was swept 124 times. tag_remove's length is the real bound.","item.dstr() formats a description string for every column; consecutive columns are nearly always the same ctree item. Comparing item.it.obj_id first collapses that to one format per item -- and the result is deduped by `seen` anyway, so skipping a repeat cannot change it."],"lesson_repeated":"this is the third time a cost was invisible because the benchmark did not exercise the feature (graph opens in v2, the non-scenario suites in v5, split view here). Coverage of the FEATURE matters more than precision on the ones already covered.","next_action_hint":"budget now: lg_nav 6647 (34%), decomp 4016, split 2202, graph 1623, listing 1394, boot 1145, hex 924, search 814, render 468. Split is still 2.2s for 24 functions -- what is left there is ida_hexrays.decompile inside decomp_map, which duplicates the decompile the view already did."}}
{"type":"config","name":"ida-tui performance (v7 bench: rename covered too)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"}
{"run":31,"commit":"7b8c37a","metric":33242.6,"metrics":{"lg_boot_ms":710.8,"lg_decomp_ms":2401.2,"lg_graph_ms":898.1,"lg_hex_ms":448.7,"lg_index_ms":72.1,"lg_listing_cold_ms":439.3,"lg_listing_warm_ms":426.8,"lg_nav_ms":7060.2,"lg_palette_ms":4.8,"lg_rename_ms":10055,"lg_render_ms":220.2,"lg_search_ms":774.6,"lg_split_ms":3539.8,"pure_graph_ms":215.1,"sm_boot_ms":429.1,"sm_decomp_ms":1244.1,"sm_graph_ms":744.3,"sm_hex_ms":444.2,"sm_index_ms":2.5,"sm_listing_cold_ms":269.8,"sm_listing_warm_ms":263.3,"sm_nav_ms":282.9,"sm_palette_ms":0.3,"sm_rename_ms":638.3,"sm_render_ms":255.6,"sm_search_ms":44.2,"sm_split_ms":1357.4,"fails":0},"status":"keep","description":"RE-BASELINE (v7 bench). Rename — the commonest operation in reverse engineering — was not covered, and it costs 10.1s for SIX renames on bash (1.7s each) because bump_names discards the segment's ListingModel and the reload re-walks it from the start. lg_split also rose to 3540ms: the split phase now runs after renames have thrown the listing away.","timestamp":1786073435987,"segment":6,"confidence":null,"asi":{"hypothesis":"keep probing unbenched features -- the last two probes each found a 10x","finding":"rename costs 1.7s per rename on bash (10.1s for six) and 106ms on echo. Program.bump_names() clears _listings, so the reload builds an empty ListingModel and ensure_ea walks the segment from its start to find the row the cursor was already on.","the_irony":"idatui/edit_ctl.py already documents that a rename cannot move a row: 'a rename or comment doesn't change how many rows anything takes' -- it restores the cursor by INDEX afterwards. It just throws away the walk that gives the index meaning.","fix_ready_and_verified":".auto/wip-renamekeep.patch. ListingModel.invalidate_text() keeps the walk and marks the rendered text stale; _ensure_text re-renders a 500-head block at a time, snapped out to whole ADDRESS groups (a function start emits three banner rows at the same ea, so an unsnapped block boundary refetches the group and never lines up). If a refetch does come back with a different head sequence it sets stale_structure and Program.listing() rebuilds, so a mis-routed structural edit degrades to today's behaviour instead of showing stale names.","measured":"bash, cursor at row 220036 of 228659: viewport back in 10.3ms instead of 6320ms (593x). A FULL re-read of every row is break-even with a rebuild (6.8s vs 6.9s), which is the right shape -- refreshing N heads costs what loading N heads costs.","equivalence":"/tmp/renameeq.py renames a function, snapshots every row (ea, kind, text, name) from the kept model, then rebuilds from scratch and compares: 0 mismatches over 5952 rows (echo), 28807 (ls_ttl) and 228659 (bash), three renames each.","one_open_question":"one full-suite run showed a follow_xrefs failure with the patch in; it passed 10/10 in isolation afterwards and 6/7 full runs green vs 6/6 without. Watch it.","next_action_hint":"apply the patch"}}
{"run":32,"commit":"44c311a","metric":25563.7,"metrics":{"lg_boot_ms":732.6,"lg_decomp_ms":2714,"lg_graph_ms":919.3,"lg_hex_ms":566.8,"lg_index_ms":71.4,"lg_listing_cold_ms":441.4,"lg_listing_warm_ms":411.7,"lg_nav_ms":6708,"lg_palette_ms":4.8,"lg_rename_ms":741.5,"lg_render_ms":227.7,"lg_search_ms":3314.1,"lg_split_ms":2596.9,"pure_graph_ms":213.6,"sm_boot_ms":422.9,"sm_decomp_ms":1284,"sm_graph_ms":789.5,"sm_hex_ms":468.1,"sm_index_ms":2.6,"sm_listing_cold_ms":258.7,"sm_listing_warm_ms":257.3,"sm_nav_ms":306.5,"sm_palette_ms":0.3,"sm_rename_ms":388.2,"sm_render_ms":256,"sm_search_ms":105.1,"sm_split_ms":1360.5,"fails":0},"status":"keep","description":"A rename keeps the listing's walk instead of throwing it away. bump_names now marks the rendered text stale (invalidate_text) and ListingModel re-renders a 500-head block at a time on demand, snapped out to whole address groups; a refetch that comes back with a different head sequence sets stale_structure so Program.listing() rebuilds. lg_rename 10055 -> 742.","timestamp":1786073731826,"segment":6,"confidence":null,"asi":{"gains":"total 33243 -> 25564 (-23.1%); lg_rename 10055 -> 742 (-93%); sm_rename 638 -> 388; lg_split 3540 -> 2597 (the split phase runs after the renames and no longer inherits a discarded listing)","honest_tradeoff":"lg_search 775 -> 3314. The cost did not vanish, it moved: the six renames used to pay for a full rebuild each (10s), and search then found a warm model. Now renames are ~120ms and the first WHOLE-SEGMENT search afterwards pays to re-render the blocks it reads. Net -7.7s, and the cost only lands if you search the entire segment right after renaming. A full re-read is break-even with a rebuild by design -- re-rendering N heads costs what loading N heads costs.","work_unchanged":"rename_ok 6/6 both targets, search_hits 91783, split_mapped_lines 2070, graph_blocks 1000, decomp_ok 12 -- all identical to the baseline","design_notes":["blocks are snapped out to whole ADDRESS groups: a function start emits three banner rows at the same ea, so an unsnapped boundary refetches the group, never lines up, and would leave stale names forever","on a sequence mismatch the model sets stale_structure and Program.listing() rebuilds -- degrading to the old behaviour rather than showing an old name","_renamed is a boolean gate so that before the first rename every read takes exactly the path it always did, with no extra lock round trips"],"equivalence":"/tmp/renameeq.py: rename, snapshot every row (ea, kind, text, name) from the kept model, rebuild from scratch, compare. 0 mismatches over 5952 rows (echo), 28807 (ls_ttl), 228659 (bash), three renames each. Full 830-check gate green.","viewport_case":"bash with the cursor at row 220036: the listing is back in 10.3ms instead of 6320ms","next_action_hint":"lg_nav 6708 and lg_search 3314 are the top two. Search after a rename could refresh in PAGE-sized chunks driven from load_all rather than block-by-block from window()."}}
{"run":33,"commit":"44c311a","metric":27571.1,"metrics":{"lg_boot_ms":714.2,"lg_decomp_ms":2426.5,"lg_graph_ms":914.1,"lg_hex_ms":436.7,"lg_index_ms":109.8,"lg_listing_cold_ms":772.1,"lg_listing_warm_ms":437.2,"lg_nav_ms":6876.9,"lg_palette_ms":4.7,"lg_rename_ms":732.9,"lg_render_ms":224.7,"lg_search_ms":1209,"lg_split_ms":6623.7,"pure_graph_ms":214.1,"sm_boot_ms":463.8,"sm_decomp_ms":1282.5,"sm_graph_ms":755,"sm_hex_ms":444.3,"sm_index_ms":2.3,"sm_listing_cold_ms":268.7,"sm_listing_warm_ms":269.4,"sm_nav_ms":307.9,"sm_palette_ms":0.3,"sm_rename_ms":384.3,"sm_render_ms":260.2,"sm_search_ms":67.7,"sm_split_ms":1368.2,"fails":0},"status":"discard","description":"Chunk the post-rename text refresh into TEXT_BLOCK pieces instead of one call for the whole requested range (a search window asks for thousands of rows and the heads tool caps a response at 2000, so the oversized call came back short, failed the sequence check and condemned the model to a rebuild). Fixes lg_search 3314 -> 1209, but lg_split 2597 -> 6624 and total 25564 -> 27571: doing it properly is SLOWER here than the accidental rebuild was.","timestamp":1786074100440,"segment":6,"confidence":3.8252964033077643,"asi":{"hypothesis":"one heads call for a whole search-sized window overflows the tool's 2000-row cap, so the refresh always failed its sequence check and forced a rebuild","hypothesis_confirmed":"yes -- chunking removed the spurious rebuild and lg_search fell 3314 -> 1209","but":"lg_split rose 2597 -> 6624. The accidental rebuild was CHEAPER overall than refreshing block by block, because this bench reads most of the segment after renaming and 450 block calls cost more than one linear rebuild.","UNEXPLAINED_AND_MUST_BE_RESOLVED":"work counters moved between the two runs: lg_decomp_lines 3436 -> 3233, lg_split_mapped_lines 2070 -> 1990, lg_search_hits 91783 -> 92733. Same database (targets/bash.i64 mtime unchanged, staged fresh per run), same fixed function set. Something about which path runs is changing what the app SEES. That has to be understood before any version of this is kept -- a perf change must not alter observable work.","leads":["phase_rename renames six functions and undoes them; a function whose original name was auto-generated (sub_X) comes back as a USER name sub_X. Check whether that changes the listing (a label row, or is_auto_name affecting annotate).","the search terms are 'mov' and 'call'; the temporary name _bench_<pid>_<k> contains a 'c', so leftover names would inflate a 'c'-prefixed search -- but the terms are full words, so check for residue directly.","lg_listing_rows is a streamed-progress reading, not a work counter -- ignore that one."],"work_preserved":".auto/wip-chunk.patch","next_action_hint":"resolve the counter drift first. Then, if the chunking is kept, try TEXT_BLOCK = 2000 (the tool's cap) so a wholesale refresh costs about what a rebuild costs while a viewport still needs one call."}}
{"run":34,"commit":"3ca9e1e","metric":28651.3,"metrics":{"lg_boot_ms":720.6,"lg_decomp_ms":2427.4,"lg_graph_ms":1222.4,"lg_hex_ms":440.9,"lg_index_ms":72.6,"lg_listing_cold_ms":433.6,"lg_listing_warm_ms":465.9,"lg_nav_ms":6809.7,"lg_palette_ms":4.7,"lg_rename_ms":710.1,"lg_render_ms":228.7,"lg_search_ms":6891.2,"lg_split_ms":2259.2,"pure_graph_ms":214.5,"sm_boot_ms":457.7,"sm_decomp_ms":1303.6,"sm_graph_ms":700.2,"sm_hex_ms":445.4,"sm_index_ms":2.4,"sm_listing_cold_ms":271.4,"sm_listing_warm_ms":270.9,"sm_nav_ms":309.2,"sm_palette_ms":0.3,"sm_rename_ms":386.6,"sm_render_ms":265.8,"sm_search_ms":70.1,"sm_split_ms":1266.2,"fails":0},"status":"keep","description":"CORRECTNESS FIX, kept despite a worse metric. The un-chunked refresh was showing STALE NAMES on any wide read: one heads call for a search-sized window overflows the tool's 2000-row cap, the short response fails the sequence check, and the block is left with its old text. Refresh is now done a block at a time. Adds .auto/check_rename.py to the gate, which fails hard on the previous code and passes on this one.","timestamp":1786074769473,"segment":6,"confidence":4.974025132789222,"asi":{"the_bug_the_metric_was_rewarding":"/tmp/stalewindow.py renames a function, then reads the segment through window() the way the search body does. On the previous commit: 0 rows show the new name, the old one is still there, stale_structure=True. The 25564ms reading was FASTER because it was skipping the refresh -- the bench only failed to notice because phase_rename undoes its renames, so the stale text happened to be right again by the time anything compared it.","why_it_happened":"_ensure_text issued ONE heads call for the whole requested range. The tool caps a response at 2000 rows; a 4096-row search window came back short, page[:len(want)] != want, and the code took its 'the walk moved' branch -- marking the block fresh and leaving the old text.","fix":"refresh a TEXT_BLOCK (500 heads) at a time, looping. The sequence check then only ever fires for a real structural change.","cost":"total 25564 -> 28651. A wholesale re-read after a rename is ~10% dearer than a rebuild would be (6.6s vs 6.0s measured directly), and this bench does exactly that -- six renames then a split-view pass and a whole-segment search. The user-facing trade is: a rename is 566x faster (10.3ms vs 5660ms to get the listing back on bash), and a full-segment search immediately after one is ~10% slower.","counter_drift_resolved":"the 3233/1990/92733 counters in experiment #33 were NOT caused by the change -- both configurations reproduce 3436/2070/91783 twice each. That run was CPU-starved (the box is at load 1.6 and the 60s tool deadline can truncate a big decompile). Watch decomp_lines as a starvation signal.","gate_extended":".auto/check_rename.py: narrow read (painting), wide read (search body), and the whole model against a rebuild. Verified it FAILS on the previous commit with 4 problems and passes on this one.","principle":"the second time this session that the honest number is worse than the dishonest one. A benchmark rewards whatever it can see; the guard has to be a check that fails, not a number that improves."}}
{"run":35,"commit":"6ec4bd8","metric":29392.6,"metrics":{"lg_boot_ms":711.9,"lg_decomp_ms":2406.4,"lg_graph_ms":1271.9,"lg_hex_ms":445.3,"lg_index_ms":71.9,"lg_listing_cold_ms":748.5,"lg_listing_warm_ms":403.8,"lg_nav_ms":6596.2,"lg_palette_ms":4.8,"lg_rename_ms":715.8,"lg_render_ms":219.3,"lg_search_ms":7055.2,"lg_split_ms":2629.8,"pure_graph_ms":212.8,"sm_boot_ms":445.6,"sm_decomp_ms":1281,"sm_graph_ms":775,"sm_hex_ms":442.3,"sm_index_ms":2.3,"sm_listing_cold_ms":265.6,"sm_listing_warm_ms":266.7,"sm_nav_ms":302.9,"sm_palette_ms":0.3,"sm_rename_ms":380.8,"sm_render_ms":255.1,"sm_search_ms":67,"sm_split_ms":1414.2,"fails":0},"status":"discard","description":"Documentation-only commit (playbook + ideas). Re-measured to confirm the post-fix state: 28651 -> 29393 is within the spread on this now-loaded box, and lg_search holds at ~7.0s, confirming that a whole-segment read after renames costs a whole-segment re-render.","timestamp":1786075083972,"segment":6,"confidence":7.108776152564333,"asi":{"state":"v7 baseline 33243 -> 29393 (-11.6%). rename 10055 -> 716 is the headline; the cost of re-rendering after a rename did not disappear, it moved to whoever reads the rows (lg_search 775 -> ~7000, because this bench reads the entire segment right after renaming). For a user reading a viewport the rename is 566x faster and nothing else changes.","why_there_is_no_more_free_win_here":"after a rename, the rows you read have to be re-rendered; re-rendering N heads costs what loading N heads costs. Refreshing only the rows that ACTUALLY changed would need xrefs_to(renamed_ea) to enumerate them, and a name can reach a row without a direct xref (a comment, a struct field in an operand) -- miss one and you are back to showing a stale name, which is the bug just fixed. Logged in .auto/ideas.md rather than attempted.","highest_yield_activity_this_session":"asking what the benchmark does NOT measure. Three of the five bench corrections found an invisible cost, and two of those (split view, rename) were among the biggest wins of the whole session.","next_action_hint":"keep probing uncovered features with the /tmp/featprobe.py shape: xrefs (x), the strings browser (\"), literal formats (o), make-code/data edits, history, traces, RPC. Domain-level probes already say xrefs/strings/structs/resolve are fast, so drive them through the UI instead."}}
{"run":36,"commit":"1e9f47b","metric":28842.1,"metrics":{"lg_boot_ms":712.2,"lg_decomp_ms":2366.9,"lg_graph_ms":907.8,"lg_hex_ms":447.2,"lg_index_ms":72.2,"lg_listing_cold_ms":443.7,"lg_listing_warm_ms":453.7,"lg_nav_ms":6610.2,"lg_palette_ms":4.8,"lg_rename_ms":735.3,"lg_render_ms":228.4,"lg_search_ms":7122.8,"lg_split_ms":2609.7,"pure_graph_ms":212.3,"sm_boot_ms":438.7,"sm_decomp_ms":1279.7,"sm_graph_ms":783.2,"sm_hex_ms":447.3,"sm_index_ms":2.5,"sm_listing_cold_ms":267.7,"sm_listing_warm_ms":263.1,"sm_nav_ms":309.7,"sm_palette_ms":0.3,"sm_rename_ms":383.9,"sm_render_ms":255,"sm_search_ms":66.3,"sm_split_ms":1417.6,"fails":0},"status":"discard","description":"Tried and reverted: throttling the background listing streamer's UI reports by time, and adding a token so only the newest streamer survives a re-prime. Both measured WORSE on a direct responsiveness probe (xrefs dialog while streaming: 1314ms baseline, 1436ms throttled, 2038ms with the token). Reverted; the finding is recorded in .auto/ideas.md. This run confirms the committed state at 28842.","timestamp":1786075786983,"segment":6,"confidence":8.431402690090582,"asi":{"real_finding_worth_keeping":"the app is several times slower while a big segment streams in the background: an xrefs dialog is 691ms during the ~7s stream and 105ms after it. Two confirmed mechanisms -- Textual's @work(exclusive=True) does NOT stop a thread worker that is already running, and navigating inside the same segment re-primes against the SAME model, so every jump leaves another streamer behind; and each streamer reports growth to the UI every four pages (thread hop + virtual_size change + full repaint).","what_failed":"a 10/s time throttle made it worse (237 reports vs 112) because the count is per streamer and there are several. Adding a _grow_token so only the newest streamer survives did NOT reduce the report count either -- so the retirement is not happening where it looks like it should, and I do not understand the mechanism well enough to ship a fix.","why_i_stopped":"the probe navigates repeatedly, which is itself what spawns the extra streamers, so it measures the thing it perturbs. A clean probe would drive UI work from a SINGLE navigation while one streamer runs. Recorded in .auto/ideas.md with the reproduction.","also_found":"a PRE-EXISTING crash: StringsPalette.on_mount calls query_one(OptionList) before compose's children are mounted (NoMatches). Reproduces 3/3 on bash AND 3/3 on the pre-autoresearch commit 2b0ae8d, so it is not from this work. ProjectPalette has the same shape.","xrefs_is_fine":"the xrefs path is not a decomp_map-class problem: xrefs_to is 38ms for 12 functions, item building 2.6ms, and the whole 'x'-to-dialog cycle is ~105ms once the listing has finished streaming. Most of that is two modal screen mounts.","state":"v7 baseline 33243 -> 28842 (-13.2%)."}}
{"run":37,"commit":"6882eea","metric":29365.3,"metrics":{"lg_boot_ms":738.3,"lg_decomp_ms":2438.1,"lg_graph_ms":1287.6,"lg_hex_ms":462.1,"lg_index_ms":72.1,"lg_listing_cold_ms":438.6,"lg_listing_warm_ms":460.3,"lg_nav_ms":6762.3,"lg_palette_ms":4.8,"lg_rename_ms":719.1,"lg_render_ms":224.7,"lg_search_ms":7055.7,"lg_split_ms":2672.5,"pure_graph_ms":217.4,"sm_boot_ms":435.3,"sm_decomp_ms":1269.6,"sm_graph_ms":721,"sm_hex_ms":437.5,"sm_index_ms":2.5,"sm_listing_cold_ms":281.1,"sm_listing_warm_ms":264.8,"sm_nav_ms":316.9,"sm_palette_ms":0.3,"sm_rename_ms":390,"sm_render_ms":257.4,"sm_search_ms":67.8,"sm_split_ms":1367.8,"fails":0},"status":"discard","description":"Documentation-only (trace scaling finding). Probed idatui/trace.py: loading is exactly linear, register_state is O(1), but Trace.memory() is linear in TRACE LENGTH per call. Recorded rather than fixed — the differential against Tenet's reference reader is skipped on this box, so the safety net for an indexing change is too thin. Run confirms the committed state.","timestamp":1786076140273,"segment":6,"confidence":13.948955495004537,"asi":{"trace_measurements":"load 36.6/73.3/143.9/280.7 ms for 20k/40k/80k/160k rows (x1.95 per doubling -- linear, correct). register_state x200: ~1.1ms at every size -- O(1). memory() x200: 16.3/31.7/63.4/125.8 ms -- LINEAR IN TRACE LENGTH per call.","why":"_mem_index sorts accesses by address and bisects to the query window, which is right, but then iterates every access in that window across all time and filters by t > idx. A stack slot written once per loop iteration accumulates one entry per iteration, so the stack pane's cost grows with how long the trace ran.","why_not_fixed":"tests/test_trace_vs_tenet.py -- the differential against Tenet's own reference reader -- is SKIPPED on this box, leaving 35 synthetic checks as the only guard on a subtle indexing change. Recorded in .auto/ideas.md with the probe (/tmp/traceprof.py) and the precondition.","session_state":"v7 baseline 33243 -> best 28651 kept. Working tree clean, 41 commits on autoresearch/perf-20260807, full 830-check gate plus two custom equivalence checks green.","next_action_hint":"remaining ideas, in .auto/ideas.md: the trace memory index (needs the Tenet differential first), why the _grow_token did not retire extra streamers, refreshing only the rows a rename actually changed, and the skeleton walk for ensure_ea (costed at ~5%)."}}
{"run":38,"commit":"df88ece","metric":27552.6,"metrics":{"lg_boot_ms":759.2,"lg_decomp_ms":2754.1,"lg_graph_ms":1207.2,"lg_hex_ms":448,"lg_index_ms":69.6,"lg_listing_cold_ms":434.5,"lg_listing_warm_ms":442.9,"lg_nav_ms":6642.3,"lg_palette_ms":4.7,"lg_rename_ms":730.6,"lg_render_ms":223.6,"lg_search_ms":5627.5,"lg_split_ms":2268,"pure_graph_ms":218.7,"sm_boot_ms":465,"sm_decomp_ms":1290.8,"sm_graph_ms":720.7,"sm_hex_ms":436.1,"sm_index_ms":2.4,"sm_listing_cold_ms":267.2,"sm_listing_warm_ms":266.5,"sm_nav_ms":292.9,"sm_palette_ms":0.3,"sm_rename_ms":380.1,"sm_render_ms":255.5,"sm_search_ms":68.1,"sm_split_ms":1276.3,"fails":0},"status":"keep","description":"Size the worker's per-line render cache to hold a segment's DISTINCT lines (16384 -> 65536, overridable with IDATUI_LINE_CACHE). This was a recorded dead end — it does nothing for a cold sweep — but the rename fix created a second-sweep workload, and re-rendering after a rename is now 21% cheaper. lg_search 7123 -> 5628.","timestamp":1786076605767,"segment":6,"confidence":8.431402690090582,"asi":{"hypothesis":"the biggest remaining term (lg_search 7.1s) is entirely the post-rename re-render, and a re-render is a SECOND sweep over the same lines -- which is exactly the case the line cache was measured to help and then filed as a dead end because nothing did it","why_it_stopped_being_a_dead_end":"experiment v1 #6 sized the cache at 16384 and noted that growing it 'does nothing for a cold sweep, only for a second sweep'. Nothing did a second sweep -- until v7 #2 made a rename keep the listing's walk and re-render its text on demand. The structural change created the workload the dead end was waiting for.","sizing_measurement":{"16384":[17.18,16.88,"+29MB"],"32768":[16.99,17.23,"+52MB"],"65536":[17.01,11.13,"+75MB"],"131072":[16.91,11.2,"+75MB"],"note":"bash .text, first sweep then second, in the worker"},"why_65536":"bash's .text is 228659 rows but only 53363 DISTINCT lines; 32768 still thrashes and 131072 buys nothing because the working set already fits. It is a bound, not a proportion -- a bigger binary fills it and stops, so the cost is capped at ~56MB whatever is open. IDATUI_LINE_CACHE lowers it for a pool of workers competing for memory.","direct_ab_on_the_real_path":"/tmp/refreshcost.py -- load the whole listing, rename, re-read every row through window(): 6445ms -> 5115ms (-21%), worker RSS 164MB -> 211MB. Initial load_all unchanged (5645 vs 5585).","gains":"total 28651 (previous best) -> 27553; lg_search 7123 -> 5628 (-21%); lg_split 2673 -> 2268","work_unchanged":"decomp_lines 3436, split_mapped 2070, search_hits 91783, graph_blocks 1000, rename_ok 6/6 -- all identical","verified":"830-check gate plus check_search and check_rename green; spans still byte-identical vs 2b0ae8d","lesson_recorded_in_prompt":"re-read the dead-end list after any structural change -- a rejected idea can become the right one when the workload around it moves","next_action_hint":"lg_nav 6642 is now the largest term again and is at the per-row floor. lg_search 5628 is still the post-rename re-render; the only way further down is to re-render fewer rows (see the xrefs-driven invalidation idea in .auto/ideas.md, which is risky because a name can reach a row without a direct xref)."}}
{"run":39,"commit":"df88ece","metric":28376.5,"metrics":{"lg_boot_ms":762.1,"lg_decomp_ms":2374.3,"lg_graph_ms":938.1,"lg_hex_ms":452.4,"lg_index_ms":70.2,"lg_listing_cold_ms":419.6,"lg_listing_warm_ms":464.1,"lg_nav_ms":6941.9,"lg_palette_ms":5,"lg_rename_ms":1973.2,"lg_render_ms":221.9,"lg_search_ms":4117.3,"lg_split_ms":2333.3,"pure_graph_ms":214.6,"sm_boot_ms":453.7,"sm_decomp_ms":1270.5,"sm_graph_ms":726.9,"sm_hex_ms":440.6,"sm_index_ms":2.4,"sm_listing_cold_ms":283,"sm_listing_warm_ms":284.3,"sm_nav_ms":310.4,"sm_palette_ms":0.3,"sm_rename_ms":1575.9,"sm_render_ms":257.6,"sm_search_ms":59.6,"sm_split_ms":1423.7,"fails":0},"status":"checks_failed","description":"Digest mode for `heads`: the worker answers \"does this page still render exactly as it did?\" for the cost of the render alone, so a post-rename refresh skips shipping, unpickling and rebuilding pages that did not change. lg_search 5628 -> 4117 — but it BREAKS 'O' cycles back, and it made rename 2-4x slower. Both causes understood.","timestamp":1786077531326,"segment":6,"confidence":10.35869418588966,"asi":{"hypothesis_first_tested_empirically":"before building anything I checked the parked xrefs-driven idea: /tmp/whatchanges.py rebuilds the whole segment before and after a rename and diffs every row. On echo, all 19 changed rows over 4 renames were covered by (function extent + xrefs_to). On ls_ttl, 53 of 54 were -- the one that was not is 'lea rcx, unk_1D7A0' -> 'byte_1D7A0', which the RENAME DID NOT CAUSE: IDA's own analysis defined that byte. An address-predicted invalidation would leave that row stale for good, so the idea is now measured-and-rejected rather than assumed-risky.","what_i_built_instead":"an exact check: `heads(..., digest=True)` builds the rows as usual but returns only hash+count instead of the rows. The client stores the digest each page came back with and asks 'still the same?' before re-fetching. Uses the interpreter's own hash deliberately -- it never has to mean anything outside the worker process, the client is only a courier.","measured_win":"post-rename whole-segment re-read 5115ms -> 3833ms (-25%); lg_search 5628 -> 4117","BUG_1_correctness":"the stored digest describes the page AS LOADED, not as the client currently holds it. After a full refetch the client's rows change but _page_digest is not updated -- so when a literal format cycles hex -> dec -> ... -> hex, the worker's digest matches the ORIGINAL stored one, the page is declared unchanged, and the row keeps the intermediate decimal text. That is exactly the failure: opfmt_listing ''O' cycles back' got 'sub rsp, 184' wanting 'sub rsp, 0B8h'.","BUG_2_performance":"_ensure_text_from tests page freshness with all(_head_gen[k] == gen for k in the page), and get() calls _ensure_text per ROW -- so every row scanned a whole page's gen array. rename went 380 -> 1576ms (sm) and 730 -> 1973ms (lg).","fix_for_both":"refresh at PAGE granularity end to end instead of the snapped TEXT_BLOCK: a page is exactly what the tool produced from (addr, count=PAGE), so refetching with the same parameters reproduces the same sequence with no snapping, the stored digest can be updated whenever the page's rows are replaced, and every head in a page shares one gen value so freshness is a single probe rather than a scan.","work_preserved":".auto/wip-digest.patch","gate_worked":"check_rename and check_search both passed -- neither exercises a format cycle. The scenario suite caught it. That is the third time a cache-shaped change failed in a way only one specific test could see."}}
{"run":40,"commit":"d9e8fdb","metric":26491.7,"metrics":{"lg_boot_ms":808.6,"lg_decomp_ms":2600.2,"lg_graph_ms":901.2,"lg_hex_ms":448.7,"lg_index_ms":70.3,"lg_listing_cold_ms":432.8,"lg_listing_warm_ms":445.5,"lg_nav_ms":7004.8,"lg_palette_ms":4.8,"lg_rename_ms":758.6,"lg_render_ms":231.3,"lg_search_ms":3959.2,"lg_split_ms":2659.2,"pure_graph_ms":214.7,"sm_boot_ms":432.6,"sm_decomp_ms":1305.1,"sm_graph_ms":758.3,"sm_hex_ms":431.8,"sm_index_ms":2.3,"sm_listing_cold_ms":275.4,"sm_listing_warm_ms":289.4,"sm_nav_ms":295.6,"sm_palette_ms":0.3,"sm_rename_ms":424.3,"sm_render_ms":263.6,"sm_search_ms":61.1,"sm_split_ms":1411.9,"fails":0},"status":"keep","description":"heads(digest=True): the worker answers \"does this page still render exactly as you hold it?\" with a hash and a count instead of the page. After a rename nearly every page is unchanged, so the pickling, transfer, unpickling and Head rebuild are all skipped. Redone at PAGE granularity end to end, which fixes both bugs of the first attempt. lg_search 5628 -> 3959.","timestamp":1786077952004,"segment":6,"confidence":8.431402690090582,"asi":{"gains":"total 27553 (previous best) -> 26492 (-3.9%); lg_search 5628 -> 3959 (-30%); post-rename whole-segment re-read 5115 -> 3720ms measured directly","cost":"lg_rename 731 -> 759 and sm_rename 380 -> 424: a page now costs one extra round trip to probe before it is either accepted or refetched. Getting a viewport back after a rename on bash is 21.5ms instead of 10.3ms -- still 230x better than the 4941ms rebuild it replaced.","what_the_two_bugs_were":{"correctness":"the first attempt stored the digest a page LOADED with and never updated it when the page was refetched. A literal format cycling hex -> dec -> hex then hashed back to the original while the client still held the decimal text: opfmt_listing ''O' cycles back' got 'sub rsp, 184' wanting '0B8h'.","performance":"freshness was tested with all(_head_gen[k] == gen for k in the page) while get() called _ensure_text per ROW, so every row scanned a whole page's gen array -- rename went 2-4x slower."},"the_fix_was_one_idea":"refresh at PAGE granularity instead of the snapped TEXT_BLOCK. A page is exactly what heads(addr, count=PAGE) produced, so re-asking with the same arguments reproduces the same sequence -- no snapping to address groups needed, the stored digest can be updated whenever the rows are replaced, and every head in a page shares one generation marker so freshness is a single probe. TEXT_BLOCK is gone.","why_hash_and_not_a_stable_digest":"the value never has to mean anything outside the worker process -- the client stores what a page hashed to and hands the same number back. One worker, one process, one hash seed. It covers ea/kind/size/text/name AND the colour spans, so two lines that collapse to the same text but colour differently are not confused.","empirical_work_that_shaped_this":"before building anything I tested the parked xrefs-driven idea with /tmp/whatchanges.py (rebuild the segment before and after a rename, diff every row). It fails: on ls_ttl one changed row was 'lea rcx, unk_1D7A0' -> 'byte_1D7A0', which the rename did not cause -- IDA's own analysis defined that byte. Address-predicted invalidation would leave it stale forever. Recorded as measured-and-rejected in .auto/ideas.md.","work_unchanged":"decomp_lines 3436, split_mapped 2070, search_hits 91783, graph_blocks 1000, rename_ok 6/6, decomp_ok 12 -- identical","verified":"830-check gate, check_search (140 prefixes), check_rename on echo AND ls_ttl (5952 and 28807 rows, wide and narrow, against a rebuild), and the 19 opfmt scenarios that caught the first attempt","next_action_hint":"lg_nav 7005 is the largest term and is the cold walk at its per-row floor. lg_search 3959 is now mostly the _line_plain pass plus the pages that genuinely changed. Next best unexplored: why the digest probe costs a whole extra round trip per page -- it could ride along with the first refetch request rather than preceding it."}}
{"run":41,"commit":"7e4f086","metric":25814,"metrics":{"lg_boot_ms":714.9,"lg_decomp_ms":2376.2,"lg_graph_ms":948.8,"lg_hex_ms":458.6,"lg_index_ms":69.3,"lg_listing_cold_ms":448.5,"lg_listing_warm_ms":482.1,"lg_nav_ms":6808,"lg_palette_ms":4.7,"lg_rename_ms":752,"lg_render_ms":219.4,"lg_search_ms":4008.3,"lg_split_ms":2275.5,"pure_graph_ms":225.2,"sm_boot_ms":456.1,"sm_decomp_ms":1292.4,"sm_graph_ms":795.8,"sm_hex_ms":475.2,"sm_index_ms":2.5,"sm_listing_cold_ms":269.1,"sm_listing_warm_ms":272.2,"sm_nav_ms":320.2,"sm_palette_ms":0.3,"sm_rename_ms":446.3,"sm_render_ms":264.8,"sm_search_ms":60.6,"sm_split_ms":1366.9,"fails":0},"status":"keep","description":"Three redundancies in the heads walk: item flags were fetched three times per head (row builder, _is_unknown via _advance, and _rows_for), get_func was called per head where a head is nearly always in the same function as the one before it, and the page digest rebuilt a tuple-of-tuples per row where one spans list is shared by ~45% of them. Cold heads 18.62 -> 17.77 us/row, warm 11.53 -> 10.87.","timestamp":1786078727527,"segment":6,"confidence":7.765877831715214,"asi":{"gains":"total 26492 (previous best) -> 25814 (-2.6%); lg_split 2659 -> 2276; sm_nav_worst 158 -> 151. Microbenchmarked rather than trusted to total_ms, since ~0.8us/row over ~500k row-renders is 0.4s and the box's spread is ~500ms: cold heads 18.62 -> 17.77 us/row, warm 11.53 -> 10.87, digest 0.75 -> 0.62.","equivalence":"/tmp/headsdump.py runs the tool under both code versions in separate processes and compares whole payloads: 109 004 rows over 294 payloads (bash + ls_ttl, BOTH annotate modes, plus the offset/end/back/count variants that share the walk) -- 0 mismatches. diff_spans still byte-identical vs 2b0ae8d on three binaries. 830-check gate, check_search and check_rename green.","get_func_cache_was_validated_not_assumed":"I had rejected this earlier on a hunch about IDA function CHUNKS -- a tail chunk of B inside A's [start_ea, end_ea) would be misattributed. /tmp/funccache.py walks every head of bash/ls_ttl/echo comparing the cached answer against get_func: 437 324 heads, 0 disagreements. Hit rate is only ~50% across all segments (a head outside any function cannot be cached by range) but much higher inside .text, which is what the listing walks.","TWO_SCHEMES_MEASURED_AND_REJECTED_FIRST":"before micro-optimising I tested the two big parked ideas, both of which would have made a rename nearly free. Listing rows predicted from xrefs_to + function extent: ls_ttl had 1 of 54 changed rows uncovered, 'lea rcx, unk_1D7A0' -> 'byte_1D7A0', which the rename did not cause -- IDA's own analysis defined that byte. Decompilations predicted from 'the old name appears in the cached text': 16 misses over 4 renames, EVERY one Hex-Rays type inference moving (unsigned int a4 -> int a4) in functions unrelated to the rename. Both recorded in .auto/ideas.md.","generalised_lesson":"predicting the effect of an edit on a database that has its own opinions is unsound. Verify instead -- which is exactly why heads(digest=True) works: it asks what a row renders as NOW, not what should have changed. This probably also explains the lg_decomp_lines drift blamed on CPU starvation in v5 #4.","things_checked_and_found_not_worth_it":{"digest_on_the_normal_load_path":"suspected a regression from v7 #10; it is 0.62-0.75 us/row, not the 1.9 the noisy call timing suggested","folding_the_opcode_byte_read_into_heads":"the extra read_bytes round trip is 0.35 us/row, ~81ms over a whole bash segment","pipelining_client_parsing_with_worker_compute":"blocked -- cursor.next is inside the pickle, so the next request cannot be issued before unpickling, and _build_page itself needs the socket for opcode bytes"},"state":"generate_disasm_line is now 6.09 of the 10.87 us/row warm cost (56%) and is IDA's. What is left of ours is ~4.8 us/row spread thin across a dozen places.","next_action_hint":"lg_nav 6808 (26%) and lg_search 4008 (16%) are both at their per-row floors now. The unexplored areas are the ones .auto/ideas.md lists as blocked or unmeasured: the trace memory index (needs test_trace_vs_tenet runnable), the streaming-responsiveness question, and the features the bench still does not drive (xrefs dialog, strings browser, struct editor, make-code edits, history, traces, RPC)."}}
{"run":42,"commit":"16318e4","metric":25783,"metrics":{"lg_boot_ms":777.9,"lg_decomp_ms":2381.9,"lg_graph_ms":1209.5,"lg_hex_ms":450,"lg_index_ms":68,"lg_listing_cold_ms":434.6,"lg_listing_warm_ms":405.2,"lg_nav_ms":6801.2,"lg_palette_ms":4.7,"lg_rename_ms":744.7,"lg_render_ms":222.4,"lg_search_ms":3885.6,"lg_split_ms":2261.4,"pure_graph_ms":216.4,"sm_boot_ms":463.6,"sm_decomp_ms":1304.6,"sm_graph_ms":740.9,"sm_hex_ms":438.4,"sm_index_ms":2.4,"sm_listing_cold_ms":270.3,"sm_listing_warm_ms":267.2,"sm_nav_ms":312.4,"sm_palette_ms":0.3,"sm_rename_ms":415.8,"sm_render_ms":257.7,"sm_search_ms":60.8,"sm_split_ms":1385.2,"fails":0},"status":"keep","description":"An item edit (c/d/u/p) keeps the listing's walk in front of it instead of discarding the model. bump_items now takes the edited address; rows before an edit keep their addresses and their row numbers, so only the pages from the edit onward are re-walked. Getting the listing back after undefining at the cursor on bash: 4890ms -> 19ms (257x). Adds .auto/check_edit.py to the gate. total_ms is flat — the bench has no item-edit phase, and the one I wrote hangs (reverted, cause recorded).","timestamp":1786082485850,"segment":6,"confidence":5.464825819307547,"asi":{"how_it_was_found":"kept probing features the bench does not drive -- the pattern that produced the flowchart hull, decomp_map and the rename walk. /tmp/itemedit.py: the undefine tool call is 1ms and getting the listing back is 4890ms, because bump_items cleared _listings and the reload re-walked the segment. Same bug as the rename one, in the sibling path.","measured":"edit at the cursor (96% into bash's .text): 4890 -> 19ms. Edit early in the segment: 5130 -> 4542ms, which is the honest case -- everything after an edit really does have to be re-walked.","why_it_is_safe":"truncate_from drops two pages rather than one (undefining can coalesce backwards into the run in front of it), and the kept prefix is then marked text-stale -- so every kept page is digest-checked on the next read, and a page that really did move fails its (ea, kind) sequence check and sets stale_structure, which makes Program.listing rebuild. Safe by construction rather than by argument about how far a reflow can reach.","gate_added_and_proven":".auto/check_edit.py drives undefine at 1/4/16 bytes at three positions (90%, 50%, 5% through the segment) and compares the kept model against a rebuild, narrow and wide. 0 problems on echo and ls_ttl. Proven to FAIL by deliberately making truncate_from too permissive: it reported 'kept model has 5952 rows, a rebuild has 5950'.","HONEST_LIMITATION":"total_ms 25814 -> 25783 is flat: nothing in the bench edits an item. I wrote a phase for it and reverted it because it HANGS -- driving undefine from inside the pilot produces no output and is killed at the timeout, while the identical sequence against Program directly is fine, and the same sequence with a print between prog.listing(ea) and ensure_ea is also fine. That points at the app's background threads (after bump_items deletes the model, a _prime/_grow worker still in flight can re-register a fresh one and hold its _load_lock), which is the same area as the unexplained streaming-responsiveness item already in .auto/ideas.md. Shipping a hanging bench phase would be worse than not having one.","what_i_did_not_do":"I did not weaken the phase until it passed, and I did not claim the win in total_ms. It is carried by a direct measurement and a permanent gate.","next_action_hint":"the concurrency question is now blocking two things (a bench phase for edits, and the streaming-responsiveness fix). Working out why @work(exclusive=True) leaves thread workers running -- and giving _prime/_grow a way to actually retire -- is the highest-value next step, and it is a correctness/responsiveness matter as much as a perf one."}}
{"run":43,"commit":"b6646aa","metric":25121.9,"metrics":{"lg_boot_ms":694.8,"lg_decomp_ms":2370.8,"lg_graph_ms":917.1,"lg_hex_ms":456.1,"lg_index_ms":69.4,"lg_listing_cold_ms":442.7,"lg_listing_warm_ms":498.8,"lg_nav_ms":6548.8,"lg_palette_ms":4.8,"lg_rename_ms":711.4,"lg_render_ms":224.9,"lg_search_ms":3484.7,"lg_split_ms":2619.3,"pure_graph_ms":220.7,"sm_boot_ms":448.6,"sm_decomp_ms":1292.5,"sm_graph_ms":748.8,"sm_hex_ms":438.3,"sm_index_ms":2.4,"sm_listing_cold_ms":274.3,"sm_listing_warm_ms":266.5,"sm_nav_ms":302.7,"sm_palette_ms":0.3,"sm_rename_ms":392.5,"sm_render_ms":260.2,"sm_search_ms":59.6,"sm_split_ms":1370.9,"fails":0},"status":"keep","description":"The page-freshness check carries the digest the client already holds (heads(expect=...)) instead of asking first and fetching afterwards. A page that has NOT changed costs one round trip as before; a page that HAS changed now costs one instead of two. Also corrects the record: the item-edit bench hang is pilot start-up flakiness, not the _prime/_grow concurrency I blamed it on — proved with a stack dump.","timestamp":1786083427343,"segment":6,"confidence":4.621649308519722,"asi":{"gains":"total 25783 -> 25122 (-2.6%, best v7); lg_search 3886 -> 3485; sm_rename 416 -> 393. Direct measurements: post-rename whole-segment re-read 3720 -> 3411ms; getting the listing back after a rename on bash 21.5 -> 15.6ms (319x vs a rebuild); after an item edit 19 -> 17ms.","the_change":"heads gains `expect` (the digest a caller already holds) in place of the boolean `digest` flag. The worker builds the rows either way and omits them only when they still hash to `expect`. Sending the expectation rather than asking first is what removes the second round trip on a changed page -- which was the regression I introduced in v7 #10 (rename 380 -> 424ms) and flagged in my own next_action_hint.","equivalence":"normal calls (no expect) compared payload-for-payload against the previous commit across processes: 166 payloads, 63 964 rows, 0 mismatches. check_rename and check_edit both clean, 830-check gate green.","DIAGNOSIS_CORRECTED":"last iteration I logged that the item-edit bench phase hangs because 'after bump_items deletes the model, a _prime/_grow worker still in flight can re-register a fresh one and hold its _load_lock'. A stack dump (faulthandler.dump_traceback_later) says otherwise: at the moment of the hang there are NO idatui threads at all -- the main thread is idle in selectors.select() and everything else is an idle asyncio executor thread, and the app is stuck BEFORE app.run_test() returns. It is pilot start-up flakiness, nothing to do with bump_items or _load_lock. Ruled out: the kitty-graphics query (IDATUI_KITTY=0 still hangs). Partly environmental: orphaned idatui/worker.py processes accumulate from runs killed by `timeout`, and clearing them let the next run boot -- but it recurred, so that is not the whole story.","operational_note":"kill stray workers between probe runs (pkill -f idatui/worker.py). Several of this session's confusing measurements were taken with orphans competing for the box.","what_this_means_for_the_backlog":"the 'why does @work(exclusive=True) leave thread workers running' item is NOT what blocks the item-edit bench phase. The two are separate: the streaming-responsiveness question is still open on its own evidence (112 vs 237 _grew reports), but the bench phase is blocked on pilot start-up reliability instead.","state":"v7 baseline 33243 -> 25122 (-24.4%). lg_nav 6549 and lg_search 3485 are both at their per-row floors; what is left of the worker's cost is 56% generate_disasm_line."}}
{"run":44,"commit":"7e4b593","metric":24513.8,"metrics":{"lg_boot_ms":693.2,"lg_decomp_ms":2347.3,"lg_graph_ms":931.2,"lg_hex_ms":460.3,"lg_index_ms":69.1,"lg_listing_cold_ms":434.2,"lg_listing_warm_ms":462.4,"lg_nav_ms":6639.6,"lg_palette_ms":4.7,"lg_rename_ms":699.9,"lg_render_ms":219.7,"lg_search_ms":3441.1,"lg_split_ms":2218,"pure_graph_ms":216.5,"sm_boot_ms":436.9,"sm_decomp_ms":1264.9,"sm_graph_ms":758.2,"sm_hex_ms":437.9,"sm_index_ms":2.3,"sm_listing_cold_ms":256.8,"sm_listing_warm_ms":256,"sm_nav_ms":309.5,"sm_palette_ms":0.3,"sm_rename_ms":379.1,"sm_render_ms":249.5,"sm_search_ms":59.2,"sm_split_ms":1266,"fails":0},"status":"keep","description":"decomp_map: memoise obj_id -> ea for the whole function instead of only comparing against the previous column. dstr() was 79% of the tool (24us a call) and items interleave, so foo(a, b) flips call->arg->call and re-formatted an item already seen: 106594 calls for 15417 lines of bash. Also corrects run #30's claim that the duplicate ida_hexrays.decompile is what costs -- a warm decompile is 0.01ms.","timestamp":1786084349637,"segment":6,"confidence":4.950123344769902,"asi":{"hypothesis":"the split view is the 3rd largest term and run #30 left a named suspect behind; profile decomp_map's sweep instead of trusting the note","gains":"total 25122 -> 24514 (-2.4%, best v7). lg_split 2619 -> 2218, sm_split 1371 -> 1266. Direct: decomp_map over bash's 12 largest 4783 -> 2972ms, echo's 12 431 -> 345ms.","profile_that_drove_it":"/tmp/sweepprof.py over 15417 lines of bash: dstr() 2581ms (79%, 106594 calls @ 24.2us), get_line_item 691ms (445337 @ 1.55us), tag_remove 14ms. The existing dedupe only compared against the PREVIOUS column's obj_id, but ctree items interleave -- foo(a, b) alternates call/arg/call/arg -- so every flip re-formatted an item already seen. obj_id is unique within a cfunc, so a function-scoped memo is exact.","equivalence":"/tmp/mapdiff.py execs the decoded BODY of the current and the previous commit against the same cfunc and compares the whole map: echo 62 functions 0 mismatches, bash 250 functions 0 mismatches. NOTES counters unchanged (sm_split_mapped_lines 984, lg_split_mapped_lines 2070).","CORRECTS_THE_RECORD_1":"run #30 said what is left in decomp_map is the duplicate ida_hexrays.decompile. Wrong: /tmp/hxcache.py shows a warm decompile is 0.01ms -- Hex-Rays' own cache is free and the duplicate costs nothing. The 2970ms I first attributed to it was the decompile TOOL's own text/spans building.","CORRECTS_THE_RECORD_2":"I suspected the split view's text and its decomp_map came from different ctrees after a rename (/tmp/mapalign.py showed 6 of 8 differing). Not a bug: the probe used DECOMP_NO_CACHE, which the app never does. Program.decompile calls the force_recompile tool (which does exist) before refetching, so the plain decompile in decomp_map then hits the repopulated cache.","rejected_and_why":"cfunc.refresh_func_ctext() after a rename is 46x faster than the forced recompile (31ms vs 1432ms for ten functions) but only 2/10 reproduce the recompile's text -- the rest differ by Hex-Rays TYPE INFERENCE (char* vs const char*, unsigned int a4 vs int a4). That is a change to what is on screen, so it is a product decision, not a perf change. Recorded in .auto/ideas.md with the probe.","next_action_hint":"the remaining decomp_map cost is get_line_item per screen column (691ms/15k lines), and the sound way down is to map only the ~40 lines the pane can show -- the same lazy shape that won for search highlight ranges. It needs a windowed tool plus a lazy container because app.py and trace_ctl.py both index the whole list. Do NOT step over columns: a one-character variable would be skipped and its EA silently lost."}}
{"run":45,"commit":"pending","metric":25367.3,"metrics":{"lg_boot_ms":740.4,"lg_decomp_ms":2307.8,"lg_graph_ms":1227.6,"lg_hex_ms":450.2,"lg_index_ms":81,"lg_listing_cold_ms":425.7,"lg_listing_warm_ms":403.5,"lg_nav_ms":6778.2,"lg_palette_ms":4.9,"lg_rename_ms":734.7,"lg_render_ms":220.1,"lg_search_ms":3619.6,"lg_split_ms":2589.3,"pure_graph_ms":216.6,"sm_boot_ms":441.2,"sm_decomp_ms":1210,"sm_graph_ms":727,"sm_hex_ms":421.5,"sm_index_ms":2.4,"sm_listing_cold_ms":258.1,"sm_listing_warm_ms":257.3,"sm_nav_ms":300.8,"sm_palette_ms":0.3,"sm_rename_ms":377.4,"sm_render_ms":249.3,"sm_search_ms":58.9,"sm_split_ms":1263.5,"fails":0},"status":"discard","description":"Replace ida-pro-mcp's decompile_function_safe with a loop that allocates one ctree_item_t instead of three per line and memoises the per-line dstr() by obj_id. Directly measured through the real worker at 3.6x (2695 -> 746ms for the post-processing of bash's 8 largest), and byte-identical over 500 functions — but total_ms rose 24514 -> 25367 on phases this cannot touch (graph +297, nav +138, search +179, boot +47) with the box at load 2.70 vs 2.16. Re-running to separate the win from the drift.","timestamp":1786085063251,"segment":6,"confidence":4.932640144665465,"asi":{"hypothesis":"the decompile TOOL's post-processing is a large hidden cost: a warm Hex-Rays decompile is 0.01ms, yet re-running the tool on bash's 12 largest still cost 2951ms. The culprit is ida-pro-mcp's decompile_function_safe, which has the SAME three faults I fixed in decomp_map -- three ctree_item_t SWIG allocations per line (two never read) and a dstr() per line at 24us.","direct_measurement":"/tmp/decprof.py: 18991 lines of bash, 2302.6ms -> 429.0ms (5.37x), 121.2 -> 22.6 us/line, 0 mismatches. End to end through the real worker (/tmp/verifybind.py, the decompile TOOL with Hex-Rays warm, bash's 8 largest): 2695ms -> 746ms, i.e. 3.6x, so the rebinding definitely takes effect.","equivalence":".auto/check_decomp.py runs BOTH implementations against the same cfunc with include_addresses both ways: 128/128 echo and 372/372 ls_ttl byte-identical. Proven to be a real gate by keying the memo on it.op instead of it.obj_id: 69 problems, 59/128 passing.","why_discarded":"total_ms 24514 -> 25367. The phases this change touches improved (lg_decomp 2347 -> 2308, sm_decomp 1265 -> 1210), but graph +297, nav +138, search +179, boot +47 and index +12 all moved too, and NONE of them go through decompile_function_safe. Load average was 2.70 at the start of this run against 2.16 for the previous one. Per the playbook, totals are not comparable across a load change.","rollback_reason":"primary metric worse, but attributed to machine load rather than the change -- the same shape as #16/#18, both of which were confirmed wins on a re-run.","next_action_hint":"re-apply from /tmp/fastdecomp.patch (git diff saved before the auto-revert; .auto/check_decomp.py also copied to /tmp) and re-run. Load was 0.84 immediately after this run, so the box has calmed. NOTE: checks.sh now calls .auto/check_decomp.py, and .auto files survive the revert -- so the gate will CRASH until the patch is re-applied, because _idatui_decompile_function_safe will not exist in BODY."}}
{"run":46,"commit":"pending","metric":25171.3,"metrics":{"lg_boot_ms":693.5,"lg_decomp_ms":2317.5,"lg_graph_ms":879.1,"lg_hex_ms":449.8,"lg_index_ms":73.6,"lg_listing_cold_ms":437.1,"lg_listing_warm_ms":461.9,"lg_nav_ms":6738.4,"lg_palette_ms":4.8,"lg_rename_ms":702.8,"lg_render_ms":226.2,"lg_search_ms":3639,"lg_split_ms":2578,"pure_graph_ms":216.2,"sm_boot_ms":435.2,"sm_decomp_ms":1278.7,"sm_graph_ms":742.7,"sm_hex_ms":436.6,"sm_index_ms":2.5,"sm_listing_cold_ms":268.2,"sm_listing_warm_ms":263.5,"sm_nav_ms":289.5,"sm_palette_ms":0.3,"sm_rename_ms":383.3,"sm_render_ms":262.9,"sm_search_ms":58.1,"sm_split_ms":1332.1,"fails":0},"status":"discard","description":"Re-run of #45 (fast decompile_function_safe) with the rebinding no longer importing a module to patch it. Confirms the change does NOT move total_ms: three runs with it (25367/25058/25171) against three without (24514/25122/25783), means 25199 vs 25140. The 3.6x is real but lands on large functions (98us/line saved) while the bench decompiles 286-line ones (30us/line). Parked in .auto/parked/ rather than deleted.","timestamp":1786085672802,"segment":6,"confidence":5.315956151035322,"asi":{"hypothesis":"ida-pro-mcp's decompile_function_safe has the same three faults I fixed in decomp_map: three ctree_item_t SWIG allocations per line (two never read) and a dstr() per line at 24us","confirmed_true_but_small":"the optimisation itself is real and verified three ways -- /tmp/decprof.py 2302 -> 429ms over 18991 lines of bash (121.2 -> 22.6 us/line); through the REAL worker /tmp/verifybind.py 2695 -> 746ms; echo 219 -> 120ms. Byte-identical over 128/128 echo and 372/372 ls_ttl functions with include_addresses both ways.","why_it_does_not_show":"the saving is 98us/line on bash's LARGEST functions but only 30us/line on small ones, because dstr() cost and memo hit rate both scale with ctree size. The bench's fixed set averages 286 lines a function, so the expected effect is 150-250ms against a run-to-run spread of 400-600ms on this box. Three runs with (25367/25058/25171) vs three without (24514/25122/25783): means 25199 vs 25140.","rollback_reason":"primary metric unchanged within noise, and the best single run remains one without the change. Rules say discard on worse-or-unchanged, and unlike #42 the bench DOES cover this path -- it simply covers it with functions too small for the win to matter.","what_i_refused_to_do":"the obvious way to make this show up is to point the bench's fixture picker at larger functions. That is fitting the benchmark to the change, so I did not do it, and I recorded the prohibition next to the parked patch.","preserved":".auto/parked/fast_decompile.patch and .auto/parked/check_decomp.py, with the full measurement table in .auto/ideas.md. I also UNWIRED check_decomp.py from checks.sh before logging: .auto files survive the auto-revert but BODY does not, so the gate would have crashed on every subsequent run.","still_a_user_win":"an F5 on a 2374-line function loses 233ms of post-processing (287 -> 54). Worth re-applying if the goal moves from total session time to per-operation latency.","next_action_hint":"stop mining the decompile path -- what is left there is Hex-Rays. The largest terms are lg_nav 6700 and lg_search 3600, both at their per-row floors, so the next real find is likely another unbenched feature (xrefs dialog, strings browser, history, literal formats) probed with the /tmp/featprobe.py shape."}}