Re-run of #16 (keep the joined search body across a cancelled search and across navigation inside the same segment), confirming it. total 17784 -> 17590; lg_search 1917 -> 1400, sm_search 70 -> 48. Also lands .auto/check_search.py in the checks gate: it compares both search fast paths against the plain per-line loop for every typed prefix.

Result: {"status":"keep","total_ms":17589.8,"lg_boot_ms":710.6,"lg_decomp_ms":2404.9,"lg_graph_ms":1136.1,"lg_hex_ms":425.6,"lg_index_ms":103.4,"lg_listing_cold_ms":417.2,"lg_listing_warm_ms":508.1,"lg_nav_ms":6553.8,"lg_palette_ms":4.8,"lg_render_ms":215.7,"lg_search_ms":1400.3,"pure_graph_ms":239.3,"sm_boot_ms":443.2,"sm_decomp_ms":668.4,"sm_graph_ms":698.1,"sm_hex_ms":443.6,"sm_index_ms":0,"sm_listing_cold_ms":267.3,"sm_listing_warm_ms":286,"sm_nav_ms":356.6,"sm_palette_ms":0.3,"sm_render_ms":258.6,"sm_search_ms":47.9,"fails":0}

1 files changed, 2 insertions, 0 deletions

diff --git a/.auto/log.jsonl b/.auto/log.jsonl
index d52098a..3aa42c4 100644
--- a/.auto/log.jsonl
+++ b/.auto/log.jsonl
@@ -14,3 +14,5 @@
 {"run":12,"commit":"870f89e","metric":19062.1,"metrics":{"lg_boot_ms":751.3,"lg_decomp_ms":2605.7,"lg_graph_ms":1022,"lg_hex_ms":554.5,"lg_index_ms":103.9,"lg_listing_cold_ms":545,"lg_listing_warm_ms":405.6,"lg_nav_ms":6636.8,"lg_palette_ms":4.7,"lg_render_ms":212.2,"lg_search_ms":2269,"pure_graph_ms":239.9,"sm_boot_ms":538.5,"sm_decomp_ms":688.4,"sm_graph_ms":712.7,"sm_hex_ms":557.5,"sm_index_ms":0,"sm_listing_cold_ms":259.9,"sm_listing_warm_ms":260.2,"sm_nav_ms":363.8,"sm_palette_ms":0.3,"sm_render_ms":252.7,"sm_search_ms":77.5,"fails":0},"status":"keep","description":"Fetch a graph's listing rows from the blocks' MERGED EXTENTS, not their convex hull, and assign them per block by bisect. IDA puts a function's cold/tail chunks far from its entry, so the hull of a 1.4KB function could be 680KB wide: it fetched 128k rows, took 3s, and still came back EMPTY for the far blocks because the pager's 64-page bound ran out first. lg_graph 9561 -> 1022.","timestamp":1786062629380,"segment":1,"confidence":null,"asi":{"hypothesis":"Program.flowchart fetches the convex hull of the basic blocks, which is enormous for a function with IDA chunks","gains":"total 27913 -> 19062 (-31.7%); lg_graph 9561 -> 1022 (-89%); sm_graph 962 -> 713","correctness_is_BETTER_not_equal":"differential over the 80 largest functions of bash: 1201 blocks differ, and EVERY one of them is a block the old code returned ZERO rows for. _heads_between is bounded to 64 pages x 2000 heads = 128k, and a 680KB hull exhausted that before reaching the tail chunk -- so far blocks drew as empty boxes. No block lost a row. targets/echo: 0 differences at all (no chunked functions).","scale":"80 bash functions: hull 44.4s / 1,864,814 rows fetched -> intervals 1.4s / 77,221 rows. 24x less data, 31x faster.","design":"blocks are sorted and merged with a 256-byte tolerance so alignment padding does not split an interval; an ordinary contiguous function is still exactly ONE heads call, as before. Row->block assignment is now bisect on the address list instead of a full scan per block (541ms -> ~0 for the 12-function set).","verify_script":"/tmp/fceq.py (kept the pattern in .auto/ideas.md): rebuild the blocks twice, fetch both ways, compare per-block row lists","next_action_hint":"lg_nav 6637 is now 35% of the total and sits at the per-row floor (~16.5us worker + ~7us client). decomp lg+sm 3294 is next: 1370ms of Hex-Rays, 240ms of pygments highlight_c, ~600ms of Textual loading-cover churn. Also unmeasured by the bench: domain.decomp_map costs 280ms per function on the split-view path."}}
 {"run":13,"commit":"60f0d70","metric":18856.8,"metrics":{"lg_boot_ms":689.7,"lg_decomp_ms":2484.2,"lg_graph_ms":1120,"lg_hex_ms":700.1,"lg_index_ms":96.5,"lg_listing_cold_ms":425.6,"lg_listing_warm_ms":511.5,"lg_nav_ms":6590.7,"lg_palette_ms":4.8,"lg_render_ms":215.1,"lg_search_ms":2195.5,"pure_graph_ms":238.1,"sm_boot_ms":431.5,"sm_decomp_ms":667.2,"sm_graph_ms":686.9,"sm_hex_ms":569.8,"sm_index_ms":0,"sm_listing_cold_ms":258.1,"sm_listing_warm_ms":283.2,"sm_nav_ms":365.2,"sm_palette_ms":0.3,"sm_render_ms":243.7,"sm_search_ms":79,"fails":0},"status":"keep","description":"Two independent constants: memoise the pygments token -> Rich style lookup (a decompilation uses ~18 distinct token types but each token walked up to nine 'token in ttype' hierarchy checks), and hold the worker-connect poll at 5ms for the first 5s instead of backing off geometrically from the first probe.","timestamp":1786062907221,"segment":1,"confidence":44.11154408183163,"asi":{"hypothesis":"boot time is a polling artefact, and highlighting is a style-lookup problem not a lexing problem","boot_smoking_gun":"WorkerClient.connect() returned in 351ms for BOTH targets/echo (47KB) and targets/bash (1.2MB) -- an identical number for very different work is a polling artefact, not a cost. The geometric backoff (5ms x1.6, capped at 200ms) has reached 134ms by the time a seeded database is ready at ~250ms. Holding 5ms for 5s first: echo 351 -> 260ms. bash is genuinely ~350ms so it did not move.","highlight_split":"on a 40KB body: lexer 52.2ms, _style_for 19.4ms, rest ~10ms of 82ms total. Memoised _style_for is 1.2ms (18 distinct token types in the whole corpus). highlight_c 84.2 -> 62.0ms.","equivalence":"/tmp/hleq.py compares old vs new highlight_c segment-for-segment over 5 bodies x 4 slices including an empty string and hex-rays-shaped pseudocode: 0 mismatches","gains":"total 19062 -> 18857 (-1.1%); sm_boot 539 -> 432 (-20%); lg_boot 751 -> 690","rejected_this_round":"a per-page identity memo for the spans->tuple conversion in Head.from_raw. The worker's line cache does share span objects across rows, but the repeats are spread over the whole segment, not within a 500-row page, so a page-scoped memo never hits: _build_page 4.52 -> 4.68 us/row. A model-scoped memo is UNSAFE because id() is reused once the page's row dicts are collected.","next_action_hint":"lg_nav 6591 (35%) is at the per-row floor. Left: pygments lexing itself (52ms per 40KB), the ~600ms of Textual loading-cover churn across 12 F5s, and lg_search 2196 (two unnarrowed passes over 224k rows at ~2.6us each)."}}
 {"run":14,"commit":"2686901","metric":18618.9,"metrics":{"lg_boot_ms":689.6,"lg_decomp_ms":2560,"lg_graph_ms":1035.1,"lg_hex_ms":684.2,"lg_index_ms":100.4,"lg_listing_cold_ms":551.1,"lg_listing_warm_ms":397.8,"lg_nav_ms":6687.8,"lg_palette_ms":4.7,"lg_render_ms":211.9,"lg_search_ms":1858.2,"pure_graph_ms":238.5,"sm_boot_ms":431.4,"sm_decomp_ms":671.2,"sm_graph_ms":728.1,"sm_hex_ms":552,"sm_index_ms":0,"sm_listing_cold_ms":257.4,"sm_listing_warm_ms":281,"sm_nav_ms":365.9,"sm_palette_ms":0.3,"sm_render_ms":242.8,"sm_search_ms":69.6,"fails":0},"status":"keep","description":"Search the whole segment as ONE joined string. Every line is concatenated once (with a start-offset table) so finding a term is a C-level str.find walk instead of a python loop that rebuilds and case-folds 224k lines per keystroke. Falls back to the per-line loop if case-folding changes the string's length.","timestamp":1786063291908,"segment":1,"confidence":41.940433212996666,"asi":{"hypothesis":"per-keystroke search should be one C string scan over the body, not 224k python iterations","design":"SearchMixin._search_haystack builds (starts, blob, blob.lower()) once per (row count, line-source) and caches it; _compute_matches then walks it with str.find and advances a line pointer monotonically (find() only moves forward, so no bisect is needed). A term can never straddle a line because an Input cannot contain a newline.","safety":["str.lower() can CHANGE LENGTH for a few unicode codepoints, which would corrupt every offset after them -- if len differs the haystack is refused and the old per-line loop runs.","the per-line loop (with prefix narrowing) is kept as the fallback and is still exercised.","every site that resets _matches/_ranges now calls _reset_search_cache(), which drops the joined body too -- a stale body would search text the view no longer shows."],"equivalence_test":"/tmp/searcheq2.py drives the real ListingView AND DecompView over 13 terms x every prefix, comparing _matches and _ranges from the haystack path against the same view with _search_haystack monkeypatched to None: 0 mismatches on echo (5952 lines) and ls_ttl (28807 lines)","gains":"total 18857 -> 18619 (-1.3%); lg_search 2196 -> 1858 (-15%); sm_search 79 -> 70","why_not_more":"the bench types only two terms, so the one-off cost of building the body (a _line_plain pass over 224k rows, ~0.5s) is amortised over very little. The benefit compounds for a user who searches more than twice -- the third term onwards is milliseconds.","next_action_hint":"lg_nav 6688 is 36%. Client-side it is pickle.loads 2.4us + _build_page 4.5us per row, and it CANNOT be pipelined: idatui/worker.py's serve() accepts one connection at a time and idalib is main-thread only, so nothing overlaps. Try making Head a NamedTuple (tuple.__new__ vs a frozen dataclass __init__)."}}
+{"run":15,"commit":"625b067","metric":17784.1,"metrics":{"lg_boot_ms":680.8,"lg_decomp_ms":2344,"lg_graph_ms":1106.5,"lg_hex_ms":576.4,"lg_index_ms":96.5,"lg_listing_cold_ms":552.9,"lg_listing_warm_ms":431.3,"lg_nav_ms":6256.4,"lg_palette_ms":4.8,"lg_render_ms":224.1,"lg_search_ms":1916.5,"pure_graph_ms":241.3,"sm_boot_ms":442.6,"sm_decomp_ms":591.3,"sm_graph_ms":663.9,"sm_hex_ms":419.2,"sm_index_ms":0,"sm_listing_cold_ms":258.2,"sm_listing_warm_ms":281.8,"sm_nav_ms":374.4,"sm_palette_ms":0.3,"sm_render_ms":251.1,"sm_search_ms":69.8,"fails":0},"status":"keep","description":"HexView.render_line emits style RUNS instead of one Segment per byte cell (35 -> 7 segments per row), and Head is a NamedTuple rather than a frozen dataclass (tuple.__new__ 1.9us vs a dataclass __init__ 2.9us, and it is built once per listing row walked).","timestamp":1786063768655,"segment":1,"confidence":42.57587221521688,"asi":{"hypothesis":"a hex frame costs ~10ms of which only 1.8ms is our render_line, so the cost is what we hand the compositor: 1540 one-cell Segments per frame","hex_change":"accumulate a run and flush it when the style changes. A row's 32 cells almost always share one style (the exception is the single cursor cell and trace-live bytes), so 35 segments/row became 7.","hex_equivalence":"/tmp/hexeq.py expands both the new and the pre-change render_line to (char, style) PER CELL and compares, over 40 scroll frames x 43 rows with the cursor moving through all 16 columns: 0 mismatches in 1720 rows. Comparing segments would have been the wrong test -- merging changes the segments on purpose; what must not change is the cells.","head_change":"Head is a NamedTuple. Measured 2.93 -> 1.88 us to build; attribute reads go 10ns -> 20ns, which is the right way round (a quarter-million rows are built per far jump, and a viewport reads forty). _build_page 4.68 -> 4.05 us/row. Nothing used dataclasses.replace/asdict on it.","gains":"total 18619 -> 17784 (-4.5%); hex lg+sm 1236 -> 996 (-19%); lg_nav 6688 -> 6256; lg_decomp 2560 -> 2344","rejected_this_round":"merging same-kind adjacent spans in _idatui_spans: measured only 2.5% fewer spans on 20k real lines (6.43 -> 6.27 per line). Not worth changing the wire format for.","rejected_earlier":"deferring the 'decompiling...' loading cover until ~120ms (worth ~9ms per F5) -- tests/test_scenarios.py asserts the overlay is raised SYNCHRONOUSLY by F5, guarding a real past regression. That is an assertion, not setup, so it stands.","next_action_hint":"lg_nav 6256 (35%) is the paging floor; the skeleton idea was costed and only nets ~5% because search then has to fetch the text anyway (see .auto/ideas.md). Left: ListingView.render_line segment count, and the search haystack building rows one lock at a time (model.window would amortise it)."}}
+{"run":16,"commit":"625b067","metric":17821,"metrics":{"lg_boot_ms":743.4,"lg_decomp_ms":2349.8,"lg_graph_ms":1033.5,"lg_hex_ms":432.2,"lg_index_ms":97.4,"lg_listing_cold_ms":424.8,"lg_listing_warm_ms":508.5,"lg_nav_ms":6677.8,"lg_palette_ms":4.8,"lg_render_ms":223.1,"lg_search_ms":1374.9,"pure_graph_ms":523.7,"sm_boot_ms":438.6,"sm_decomp_ms":591.2,"sm_graph_ms":713.2,"sm_hex_ms":466.7,"sm_index_ms":0,"sm_listing_cold_ms":262.4,"sm_listing_warm_ms":287.1,"sm_nav_ms":376.6,"sm_palette_ms":0.3,"sm_render_ms":257.5,"sm_search_ms":33.5,"fails":0},"status":"discard","description":"Keep the joined search body across a cancelled search and across navigation inside the same segment (drop it only when the model changes or the opcode column toggles). lg_search 1917 -> 1375, sm_search 70 -> 34 — but total_ms is FLAT (17784 -> 17821) because pure_graph (+283) and lg_nav (+421) drifted, neither of which this touches. Re-running to separate the win from the drift.","timestamp":1786064057632,"segment":1,"confidence":16.322294738538417,"asi":{"hypothesis":"the joined search body is keyed by (row count, line source), so ending a search or navigating inside the same segment need not throw it away","phase_evidence":"lg_search -542ms and sm_search -36ms, i.e. the intended effect happened and is far outside that phase's own spread","why_total_did_not_move":"pure_graph 241 -> 524 and lg_nav 6256 -> 6678 in the same run, and this change touches neither. pure_graph is pure CPU with no I/O and is the single jumpiest metric in the suite (seen at 235, 241, 508, 524, 531 across runs with identical code).","work_preserved":".auto/wip-searchbody.patch","gate_strengthened":".auto/check_search.py added and wired into checks.sh: it drives the real ListingView/DecompView and compares the narrowing + haystack fast paths against the plain per-line loop for every prefix of 14 terms, including after toggling the opcode column and after navigating. 140 prefixes, 0 mismatches. A stale cache still returns AN answer, so the scenario suite could never have caught this class of bug.","invalidation_rules_now":["body dropped when ListingView.load gets a DIFFERENT model (navigation inside one segment reuses it)","body dropped by action_toggle_opcodes -- the opcode hex is searchable text and its width changes without the row count or model moving","body dropped by DecompView.show, explicitly rather than relying on id(self._texts), because the list it replaces is freed there and the address can be reused","narrowing key still cleared everywhere _matches/_ranges are reset"],"next_action_hint":"re-apply the patch and re-run to confirm; the phase number is unambiguous"}}