diff options
| -rw-r--r-- | .fastfeedback/SPEED.md | 43 |
1 files changed, 26 insertions, 17 deletions
diff --git a/.fastfeedback/SPEED.md b/.fastfeedback/SPEED.md index 8dbe860..f5b1f84 100644 --- a/.fastfeedback/SPEED.md +++ b/.fastfeedback/SPEED.md @@ -135,33 +135,42 @@ may legitimately do nothing (e.g. carving random bytes). | `~/ida-venv/bin/python tests/test_rawimage_rpc.py` | ~60s | 21 checks, owns batch rename | | pilot boot (first scenario) | ~10-25s | opens the DB; seeded from `.pristine.i64` | | pilot `--only graph` | ~10s | 50 checks | -| **pilot full (`test_scenarios.py`)** | **80.5s** | 56 scenarios, 301 checks (was 166s) | -| `test_trace_ui.py` | 22.0s | 39 checks | -| `test_rawimage_rpc.py` | 13.1s | 21 checks, owns batch rename | +| **pilot full (`test_scenarios.py`)** | **62.2s** | 56 scenarios, 301 checks (was 166s) | +| `test_trace_ui.py` | 19.4s | 39 checks | +| `test_rawimage_rpc.py` | 7.5s | 21 checks, owns batch rename | | `test_project_ui.py` | 8.1s | 30 checks | | `test_thumb_ui.py` | **8.5s** | 20 checks — was 313s AND crashing | | `test_blob_ui.py` | **4.1s** | 30 checks — was 39.8s | -| `tests/run.py` (everything) | ~2m20s | was ~9m20s. BACKGROUND ONLY | +| `tests/run.py` (everything) | **115s** | 788 checks. was ~9m20s. BACKGROUND ONLY | IDA-suite total went from ~9m21s to ~2m16s (4.1x) by fixing the four wastes above — not by removing a single check. -### Backend performance note (why the pilot is 2.8x slower on codemode) -`heads` is the listing's paging call. Measured on `targets/echo`, 200 rows: +### Backend performance (worker vs Code Mode), after the two transport fixes -| | master (worker: pickle over unix socket) | codemode (HTTP + JSON + `to_jsonable`) | -|---|---|---| -| `heads count=200` | 2.6 ms | 92 ms | -| `heads count=500` | 6.3 ms | 214 ms | -| empty round trip | — | 2.0 ms | +| op | worker | codemode | | +|---|---|---|---| +| `heads` 200 rows | 2.65ms | 5.85ms | 2.2x | +| `heads` expect-hit | 2.16ms | 4.61ms | 2.1x | +| `disasm` 200 | 9.03ms | 5.25ms | **0.6x** | +| `decompile` cold | 162.6ms | 30.7ms | **0.2x** | +| `decomp_map` | 45.8ms | 47.9ms | 1.0x | +| empty round trip | ~0.07ms | **2.0ms** | the floor | + +Two things dominated and are fixed (see `idatui/codemode_client.py:_script`): +`to_jsonable` walking every returned object (snippets now return one +pre-serialised JSON string), and `sys.settrace` — the runtime installs a trace +that returns itself, i.e. LINE tracing in every frame, which made +`ida_bytes.get_flags` 52x slower than native. The snippet detaches it and +restores it in a finally; `IDATUI_CODEMODE_TRACE=1` keeps the stock behaviour. -So the transport floor is 2ms/call and the rest is per-row work + `to_jsonable` -(~71ms of a 200-row page). This is architectural, not a bug in the port. The -digest/`expect` path (a page that has not changed is answered with a hash and a count, -no rows) is the main mitigation and is implemented. +**What is left is the 2ms round-trip floor.** A trivial op (`data_type`, +`force_recompile`, one xref query) is ~2.5ms wall clock and looks like 40x +against an in-process worker. That is fixed by making FEWER calls, not faster +ones — which is what the digest/`expect` path does for the listing. -**Corollary for testing:** the codemode backend is slow enough that UI settle races -appear that never appear on master. See KNOWN-FLAKY. +Benchmark harness: `/tmp/cmport/bench.py` + `compare.py` (backend-agnostic; it +picks whichever client the checked-out tree has, so it runs on master too). --- |