diff options
| -rw-r--r-- | .auto/log.jsonl | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/.auto/log.jsonl b/.auto/log.jsonl index d7d8b7f..a1baf16 100644 --- a/.auto/log.jsonl +++ b/.auto/log.jsonl @@ -22,3 +22,4 @@ {"type":"config","name":"ida-tui performance (v3 bench: lower-noise estimator)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"} {"type":"config","name":"ida-tui performance (v4 bench: cold phases measured once, repeatable ones averaged)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"} {"run":20,"commit":"49184b6","metric":18497.9,"metrics":{"lg_boot_ms":694.8,"lg_decomp_ms":2674,"lg_graph_ms":992.7,"lg_hex_ms":429.1,"lg_index_ms":94.9,"lg_listing_cold_ms":545.3,"lg_listing_warm_ms":410.7,"lg_nav_ms":6489,"lg_palette_ms":4.7,"lg_render_ms":220.6,"lg_search_ms":1489.8,"pure_graph_ms":241,"sm_boot_ms":467.1,"sm_decomp_ms":1308.5,"sm_graph_ms":761,"sm_hex_ms":431.7,"sm_index_ms":2.5,"sm_listing_cold_ms":273.9,"sm_listing_warm_ms":289,"sm_nav_ms":373,"sm_palette_ms":0.3,"sm_render_ms":257.2,"sm_search_ms":47,"fails":0},"status":"keep","description":"RE-BASELINE (v4 bench). Adding a second repetition on the big target exposed the same flaw the graph phase had: decompile, search and the function index all cache their answer, so a second rep reported a dict lookup under the name of the thing a user waits for. Cold-sensitive phases (listing_cold, decomp, search, index) now run ONCE; repeatable ones (render, hex, graph, listing_warm) run every rep and take the median. pure_graph is median-of-3.","timestamp":1786065121146,"segment":3,"confidence":null,"asi":{"hypothesis":"reduce the metric's noise so changes worth 1-2% are readable","what_changed_in_the_bench":"pure_graph median-of-3 (it swung 239 <-> 524 with identical code); two reps on targets/bash; and cold-sensitive phases pinned to the first rep only","flaw_this_caught":"sm_decomp had been min-of-2 since the start, i.e. it was reporting a WARM decompile (Program._decomp is cached per function). Honest cold value is 1308ms, not ~650. Same for sm_search and sm_index. All comparisons within v1-v3 were still valid (consistent measurement), but the absolute picture was wrong: decomp is 22% of the total, not 12%.","cumulative_history":"v1 baseline 46572 -> 19006 over 10 experiments. v2 (cold graph opens measured) baseline 27913 -> 17501 over 9. v3 abandoned after one run for the flaw above. v4 baseline 18498.","budget_ms":{"lg_nav":6489,"decomp lg+sm":3983,"graph lg+sm":1754,"search lg+sm":1537,"listing lg+sm":1519,"boot lg+sm":1162,"hex lg+sm":861,"render lg+sm":478,"pure_graph":241},"next_action_hint":"decomp is now clearly #2 at 22%. sm_decomp is 1308ms for TWELVE small echo functions (109ms each), which is far more than Hex-Rays should need on a 1.6KB function -- profile the cold F5 path on echo before assuming it is the decompiler."}} +{"type":"config","name":"ida-tui performance (v5 bench: 2ms landing polls; final measurement shape)","metricName":"total_ms","metricUnit":"ms","bestDirection":"lower"} |