SPEED.md: record the post-fix backend numbers and the 2ms round-trip floor

1 files changed, 26 insertions, 17 deletions

diff --git a/.fastfeedback/SPEED.md b/.fastfeedback/SPEED.md
index 8dbe860..f5b1f84 100644
--- a/.fastfeedback/SPEED.md
+++ b/.fastfeedback/SPEED.md
@@ -135,33 +135,42 @@ may legitimately do nothing (e.g. carving random bytes).
 | `~/ida-venv/bin/python tests/test_rawimage_rpc.py` | ~60s | 21 checks, owns batch rename |
 | pilot boot (first scenario) | ~10-25s | opens the DB; seeded from `.pristine.i64` |
 | pilot `--only graph` | ~10s | 50 checks |
-| **pilot full (`test_scenarios.py`)** | **80.5s** | 56 scenarios, 301 checks (was 166s) |
-| `test_trace_ui.py` | 22.0s | 39 checks |
-| `test_rawimage_rpc.py` | 13.1s | 21 checks, owns batch rename |
+| **pilot full (`test_scenarios.py`)** | **62.2s** | 56 scenarios, 301 checks (was 166s) |
+| `test_trace_ui.py` | 19.4s | 39 checks |
+| `test_rawimage_rpc.py` | 7.5s | 21 checks, owns batch rename |
 | `test_project_ui.py` | 8.1s | 30 checks |
 | `test_thumb_ui.py` | **8.5s** | 20 checks — was 313s AND crashing |
 | `test_blob_ui.py` | **4.1s** | 30 checks — was 39.8s |
-| `tests/run.py` (everything) | ~2m20s | was ~9m20s. BACKGROUND ONLY |
+| `tests/run.py` (everything) | **115s** | 788 checks. was ~9m20s. BACKGROUND ONLY |
 
 IDA-suite total went from ~9m21s to ~2m16s (4.1x) by fixing the four wastes above —
 not by removing a single check.
 
-### Backend performance note (why the pilot is 2.8x slower on codemode)
-`heads` is the listing's paging call. Measured on `targets/echo`, 200 rows:
+### Backend performance (worker vs Code Mode), after the two transport fixes
 
-| | master (worker: pickle over unix socket) | codemode (HTTP + JSON + `to_jsonable`) |
-|---|---|---|
-| `heads count=200` | 2.6 ms | 92 ms |
-| `heads count=500` | 6.3 ms | 214 ms |
-| empty round trip | — | 2.0 ms |
+| op | worker | codemode | |
+|---|---|---|---|
+| `heads` 200 rows | 2.65ms | 5.85ms | 2.2x |
+| `heads` expect-hit | 2.16ms | 4.61ms | 2.1x |
+| `disasm` 200 | 9.03ms | 5.25ms | **0.6x** |
+| `decompile` cold | 162.6ms | 30.7ms | **0.2x** |
+| `decomp_map` | 45.8ms | 47.9ms | 1.0x |
+| empty round trip | ~0.07ms | **2.0ms** | the floor |
+
+Two things dominated and are fixed (see `idatui/codemode_client.py:_script`):
+`to_jsonable` walking every returned object (snippets now return one
+pre-serialised JSON string), and `sys.settrace` — the runtime installs a trace
+that returns itself, i.e. LINE tracing in every frame, which made
+`ida_bytes.get_flags` 52x slower than native. The snippet detaches it and
+restores it in a finally; `IDATUI_CODEMODE_TRACE=1` keeps the stock behaviour.
 
-So the transport floor is 2ms/call and the rest is per-row work + `to_jsonable`
-(~71ms of a 200-row page). This is architectural, not a bug in the port. The
-digest/`expect` path (a page that has not changed is answered with a hash and a count,
-no rows) is the main mitigation and is implemented.
+**What is left is the 2ms round-trip floor.** A trivial op (`data_type`,
+`force_recompile`, one xref query) is ~2.5ms wall clock and looks like 40x
+against an in-process worker. That is fixed by making FEWER calls, not faster
+ones — which is what the digest/`expect` path does for the listing.
 
-**Corollary for testing:** the codemode backend is slow enough that UI settle races
-appear that never appear on master. See KNOWN-FLAKY.
+Benchmark harness: `/tmp/cmport/bench.py` + `compare.py` (backend-agnostic; it
+picks whichever client the checked-out tree has, so it runs on master too).
 
 ---