listing: syntax-highlight assembly from IDA's own token tags

The listing showed the mnemonic bright and every operand in one body colour.
IDA already classifies each token, for every processor it supports:
generate_disasm_line() emits \x01<tag>text\x02<tag> and the tag says what the
text IS. We were calling tag_remove() and throwing that away.

So: no lexer. A pygments asm lexer would be a worse guess and would need one
dialect per architecture — this is arch-correct for free, including the ARM/MIPS
blobs the loader work just made openable.

  lea rcx, function; "usage"
  insn  reg punct   name   cmt

_idatui_spans() parses the tags into [[kind, text], ...], heads rows carry
"spans", Head.spans holds them, and _span_segments() renders them with a
fallback to the old mnemonic/rest split for older workers.

Palette rule: NEUTRALS for the machine (mnemonic brightest — it's the column you
scan; registers at body weight because they're most of the text), HUES only where
they mean something (numbers, strings, symbols), structure recedes so commas and
brackets stop competing with operands.

Two things that fail SILENTLY and are now encoded:
* The constants are SCOLOR_DATNAME / SCOLOR_CODNAME. There is no SCOLOR_DNAME —
  a wrong guess leaves the tag unmapped, symbols render as plain body text, and
  nothing tells you why. Probed the live IDA to get the real names.
* Spans must be whitespace-collapsed exactly as `text` is, walking characters
  rather than per span, because a run of IDA's column padding straddles span
  boundaries. A row only gets spans when they reconstruct `text` exactly, so a
  mismatch degrades to the old rendering instead of corrupting the line.

The reason this was parked yesterday was NOT a bug in it. listing_view's
"undefining a data head yields an unknown run" waits for
`index_of_ea(dea) >= 0` — but dea is the head it just undefined, so it is in the
OLD model too and the predicate passes instantly, asserting against pre-edit
rows. It only ever passed because the model swap won the race; spans made pages
3x bigger, the swap lost, and the check accused working code. It now waits for
the model to be REPLACED.

Cost measured on libcrypto: 95KB per 500-row page, 50ms; model ensure(2000)
228ms. Acceptable for what it buys.

tests: new asm_highlight scenario (+7) — >90% of code rows carry spans, insn/reg/
punct present, every span kind has a style, spans reconstruct the row text
exactly, mnemonic is the first span. 202/0 scenarios, 26/0 blob, 30/0 project UI.

TODO: DisasmView appears to be dead code (never instantiated; Ctx.dis returns
ListingView), which is why this only needed doing once.

6 files changed, 233 insertions, 294 deletions

diff --git a/TODO b/TODO
index 78568ae..a97cb4f 100644
--- a/TODO
+++ b/TODO
@@ -91,42 +91,13 @@ before trusting a failure that appeared without a code change.
 Coverage for "an edit must not move the view" lives in tests/test_blob_ui.py,
 which builds its own throwaway binary and can mutate freely.
 
-## Assembly syntax highlighting (WIP, patch in docs/wip-asm-spans.patch)
+## DisasmView looks like dead code
 
-The listing shows the mnemonic bright and everything after it in one body
-colour. IDA already classifies every token, for every processor, so there is
-nothing to lex — generate_disasm_line() emits \x01<tag>text\x02<tag> and the tag
-says what the text IS. A pygments asm lexer would be a worse guess and would
-need a dialect per architecture.
+idatui/app.py defines DisasmView (Line-based, function-scoped) but the app never
+instantiates or queries it — the unified ListingView replaced it, and even the
+test harness's `Ctx.dis` returns ListingView. DisasmModel is still used, but only
+to compute indices in _do_edit_item, never to render.
 
-The patch: _idatui_spans() in patch_server parses those tags into
-[[kind, text], ...] (insn/reg/num/str/name/seg/cmt/punct), heads rows carry
-"spans", Head.spans holds them, and ListingView renders via _span_segments()
-with a fallback to today's mnemonic/rest split. Verified working on echo:
-
-  lea rcx, function; "usage"  ->  insn text reg punct text name cmt
-  coverage over 400 rows: insn 344, reg 402, punct 397, name 156, num 47, cmt 44
-
-Palette rule used: NEUTRALS for the machine (mnemonic brightest, registers at
-body weight since they are most of the text), HUES only where they carry meaning
-(numbers, strings, symbols), structure recedes so commas and brackets stop
-competing with operands.
-
-Two things learned that the patch encodes:
-* The constants are SCOLOR_DATNAME / SCOLOR_CODNAME — there is no SCOLOR_DNAME.
-  Guessing fails SILENTLY: an unmapped tag renders as body text, so symbols just
-  aren't blue and nothing says why.
-* Spans must be whitespace-collapsed exactly as `text` is, walking characters
-  rather than per-span, because a run of spaces straddles spans. The row only
-  gets spans when the two agree, so a mismatch degrades instead of corrupting.
-
-NOT MERGED because it breaks one check: listing_view "undefining a data head
-yields an unknown run" now reports kind=data. Reproducible, and bisected to this
-change (10/0 with it stashed, 9/1 with it applied) on a fresh .i64. Cause not yet
-found — the span code doesn't touch `kind`, which is computed from the flags
-before spans are attached, so the suspicion is that adding "spans" to the row
-dict perturbs something in the undefine path's re-read (caching keyed on the row
-shape?). Find that before merging.
-
-Also still plain: DisasmView (the function-scoped view) renders Line objects from
-the `disasm` tool, which doesn't emit spans. Same treatment needed there.
+Worth confirming and deleting: it's ~180 lines carrying its own rendering,
+search and cursor logic that no longer runs, and it misled me into thinking
+assembly highlighting needed doing twice.
diff --git a/docs/wip-asm-spans.patch b/docs/wip-asm-spans.patch
deleted file mode 100644
index 6b43da6..0000000
--- a/docs/wip-asm-spans.patch
+++ /dev/null
@@ -1,247 +0,0 @@
-diff --git a/idatui/app.py b/idatui/app.py
-index dd7c222..6506016 100644
---- a/idatui/app.py
-+++ b/idatui/app.py
-@@ -53,6 +53,23 @@ from .domain import DisasmModel, Func, Head, ListingModel, Program, Struct
- _S_ADDR = Style(color="#6b7684")
- _S_LABEL = Style(color="#7aa2f7", bold=True)
- _S_INSN = Style(color="#c3cad3")
-+#: IDA's token kinds -> the measured palette. The rule that keeps a dense
-+#: disassembly readable: NEUTRALS for the machine (mnemonic brightest because you
-+#: scan down that column, registers at body weight because they're most of the
-+#: text), HUES only where they mean something (numbers, strings, symbols),
-+#: structure recedes so brackets and commas stop competing with operands.
-+_S_SPAN = {
-+    "insn": Style(color="#e8ecf2"),      # 15.3:1  mnemonic / directive
-+    "reg": Style(color="#c3cad3"),       # 11.0:1  registers = body weight
-+    "num": Style(color="#d8a657"),       #  8.2:1  immediates, offsets
-+    "str": Style(color="#9ece6a"),       #  9.9:1  string literals
-+    "name": Style(color="#7aa2f7"),      #  7.2:1  symbols / xref targets
-+    "seg": Style(color="#93aee0"),       #  8.1:1  segment names
-+    "cmt": Style(color="#7c8b9e", italic=True),   # 5.2:1
-+    "punct": Style(color="#626c7a"),     #  3.4:1  brackets, commas, +/-
-+    "err": Style(color="#c9762f"),       #         IDA's own error marker
-+    "text": Style(color="#c3cad3"),      # 11.0:1  anything unclassified
-+}
- _S_MNEM = Style(color="#e8ecf2")
- _S_OPBYTES = Style(color="#5e6875")  # raw opcode bytes column
- _S_DATA = Style(color="#d8a657")
-@@ -1077,6 +1094,24 @@ class ListingView(SearchMixin, NavMixin, ColumnCursor, ScrollView, can_focus=Tru
-             return " ".join(f"{b:02X}" for b in raw[:_OP_LIMIT]) + "\u2026"
-         return " ".join(f"{b:02X}" for b in raw)
- 
-+    @staticmethod
-+    def _span_segments(h: Head, fallback: Style):
-+        """Segments for a row's disassembly text.
-+
-+        Uses IDA's own token classification when the worker supplied it; falls
-+        back to the old mnemonic/rest split so an older worker (or a row whose
-+        spans didn't match the text) still renders.
-+        """
-+        if h.spans:
-+            return [Segment(t, _S_SPAN.get(k, fallback)) for k, t in h.spans]
-+        if h.kind == "code":
-+            mnem, _, rest = h.text.partition(" ")
-+            segs = [Segment(mnem, _S_MNEM)]
-+            if rest:
-+                segs.append(Segment(" " + rest, fallback))
-+            return segs
-+        return [Segment(h.text, fallback)]
-+
-     def _op_field(self, h: Head) -> str:
-         """The padded opcode-bytes column (empty when hidden). Shared format so
-         cursor/search offsets line up."""
-@@ -1301,17 +1336,11 @@ class ListingView(SearchMixin, NavMixin, ColumnCursor, ScrollView, can_focus=Tru
-                 segs.append(Segment(_LST_INDENT, _S_MEMBER))
-             if h.name:
-                 segs.append(Segment(f"{h.name}  ", _S_LABEL))
--            if h.kind == "code":
--                mnem, _, rest = h.text.partition(" ")
--                segs.append(Segment(mnem, _S_MNEM))
--                if rest:
--                    segs.append(Segment(" " + rest, _S_INSN))
--            elif h.kind == "data":
--                segs.append(Segment(h.text, _S_DATA))
--            elif h.kind == "member":
-+            if h.kind == "member":
-                 segs.append(Segment(h.text, _S_MEMBER))
-             else:
--                segs.append(Segment(h.text, _S_UNK))
-+                base = {"code": _S_INSN, "data": _S_DATA}.get(h.kind, _S_UNK)
-+                segs.extend(self._span_segments(h, base))
-             strip = Strip(segs)
-         linked = idx in self._link_rows
-         if linked:
-diff --git a/idatui/domain.py b/idatui/domain.py
-index e4fbec9..606cf42 100644
---- a/idatui/domain.py
-+++ b/idatui/domain.py
-@@ -100,6 +100,10 @@ class Head:
-     text: str
-     name: str | None = None
-     raw: bytes | None = None  # opcode/item bytes (filled in for code by the model)
-+    #: [(kind, text)] from IDA's own colour tags — mnem/reg/num/name/str/punct/…
-+    #: None when the worker didn't provide them (older worker, or the spans
-+    #: disagreed with the plain text, in which case the text wins).
-+    spans: tuple[tuple[str, str], ...] | None = None
- 
-     @property
-     def label(self) -> str | None:  # Line-compatible alias
-@@ -107,12 +111,15 @@ class Head:
- 
-     @classmethod
-     def from_raw(cls, d: dict) -> "Head":
-+        sp = d.get("spans")
-         return cls(
-             ea=_as_int(d["ea"]),
-             kind=d.get("kind", "unknown"),
-             size=int(d.get("size", 0) or 0),
-             text=d.get("text", ""),
-             name=d.get("name"),
-+            spans=(tuple((str(k), str(t)) for k, t in sp)
-+                   if isinstance(sp, list) and sp else None),
-         )
- 
- 
-diff --git a/server/patch_server.py b/server/patch_server.py
-index 4c806a7..6601a0f 100644
---- a/server/patch_server.py
-+++ b/server/patch_server.py
-@@ -305,12 +305,137 @@ def _idatui_head_row(ea):
-         "size": int(ida_bytes.get_item_size(ea)),
-         "text": text,
-     }
-+    if line:
-+        # Keep IDA's own token classification for syntax highlighting. Built from
-+        # the SAME line as `text`, then whitespace-collapsed identically so the
-+        # two never disagree about what the row says.
-+        spans = _idatui_spans(line)
-+        joined = "".join(t for _k, t in spans)
-+        if " ".join(joined.split()) == text:
-+            row["spans"] = spans
-     nm = ida_name.get_ea_name(ea)
-     if nm:
-         row["name"] = nm
-     return row
- 
- 
-+#: IDA colour tag -> the semantic kind the TUI styles. IDA already classifies
-+#: every token in a disassembly line, for every processor it supports, so there
-+#: is nothing to lex: generate_disasm_line emits \x01<tag>text\x02<tag> and the
-+#: tag says what the text IS. A pygments assembly lexer would be a worse guess at
-+#: this and would need one dialect per architecture.
-+_IDATUI_SPAN_KINDS = {
-+    "insn": ("SCOLOR_INSN", "SCOLOR_KEYWORD", "SCOLOR_ASMDIR", "SCOLOR_MACRO"),
-+    "reg": ("SCOLOR_REG",),
-+    "num": ("SCOLOR_NUMBER", "SCOLOR_CHAR", "SCOLOR_BINPREF"),
-+    "str": ("SCOLOR_STRING",),
-+    # NB the real constant names: DATNAME/CODNAME, not "DNAME". Guessing here
-+    # fails silently — an unmapped tag renders as plain body text, so symbols
-+    # just quietly aren't blue and nothing tells you why.
-+    "name": ("SCOLOR_DATNAME", "SCOLOR_CODNAME", "SCOLOR_LOCNAME",
-+             "SCOLOR_IMPNAME", "SCOLOR_DEMNAME", "SCOLOR_LIBNAME",
-+             "SCOLOR_CNAME", "SCOLOR_DNAME",
-+             "SCOLOR_CREF", "SCOLOR_DREF", "SCOLOR_CREFTAIL", "SCOLOR_DREFTAIL"),
-+    "seg": ("SCOLOR_SEGNAME",),
-+    "cmt": ("SCOLOR_AUTOCMT", "SCOLOR_REGCMT", "SCOLOR_RPTCMT", "SCOLOR_VOIDOP"),
-+    "punct": ("SCOLOR_SYMBOL", "SCOLOR_ALTOP", "SCOLOR_HIDNAME"),
-+    "err": ("SCOLOR_ERROR",),
-+}
-+
-+
-+def _idatui_tag_map():
-+    """{tag character: kind}, built once from whatever this IDA actually has."""
-+    import ida_lines
-+    out = {}
-+    for kind, names in _IDATUI_SPAN_KINDS.items():
-+        for n in names:
-+            v = getattr(ida_lines, n, None)
-+            if isinstance(v, str) and v:
-+                out[v[0]] = kind
-+            elif isinstance(v, int):
-+                out[chr(v)] = kind
-+    return out
-+
-+
-+_IDATUI_TAGS = None
-+
-+
-+def _idatui_spans(line):
-+    """A tagged disasm line as [[kind, text], ...], colour tags resolved.
-+
-+    Unknown tags become 'text' rather than being dropped: a processor module can
-+    emit a colour we don't classify, and losing the characters would corrupt the
-+    line."""
-+    global _IDATUI_TAGS
-+    import ida_lines
-+    if _IDATUI_TAGS is None:
-+        _IDATUI_TAGS = _idatui_tag_map()
-+    on, off, esc = "\x01", "\x02", "\x03"
-+    addr_tag = chr(getattr(ida_lines, "COLOR_ADDR", 0x28))
-+    addr_len = int(getattr(ida_lines, "COLOR_ADDR_SIZE", 16))
-+    spans, stack, buf = [], [], []
-+    i, n = 0, len(line)
-+
-+    def flush():
-+        if buf:
-+            spans.append([stack[-1] if stack else "text", "".join(buf)])
-+            del buf[:]
-+
-+    while i < n:
-+        ch = line[i]
-+        if ch == on and i + 1 < n:
-+            tag = line[i + 1]
-+            if tag == addr_tag:
-+                # An embedded target address, not display text: 16 hex digits
-+                # that must not reach the screen.
-+                i += 2 + addr_len
-+                continue
-+            flush()
-+            stack.append(_IDATUI_TAGS.get(tag, "text"))
-+            i += 2
-+            continue
-+        if ch == off and i + 1 < n:
-+            flush()
-+            if stack:
-+                stack.pop()
-+            i += 2
-+            continue
-+        if ch == esc and i + 1 < n:      # escaped literal
-+            buf.append(line[i + 1])
-+            i += 2
-+            continue
-+        buf.append(ch)
-+        i += 1
-+    flush()
-+    # Collapse IDA's column padding EXACTLY as the plain text does. A run of
-+    # spaces can straddle two spans, so this walks characters rather than
-+    # collapsing each span on its own — otherwise the spans and `text` disagree
-+    # about the line and the row silently loses its highlighting.
-+    out, prev_space = [], False
-+    for kind, txt in spans:
-+        acc = []
-+        for ch in txt:
-+            if ch.isspace():
-+                if prev_space:
-+                    continue
-+                acc.append(" ")
-+                prev_space = True
-+            else:
-+                acc.append(ch)
-+                prev_space = False
-+        if acc:
-+            out.append([kind, "".join(acc)])
-+    while out and out[0][1] == " ":
-+        out.pop(0)
-+    while out and out[-1][1] == " ":
-+        out.pop()
-+    if out and out[0][1].startswith(" "):
-+        out[0][1] = out[0][1].lstrip()
-+    if out and out[-1][1].endswith(" "):
-+        out[-1][1] = out[-1][1].rstrip()
-+    return [[k, t] for k, t in out if t]
-+
-+
- def _idatui_unknown_row(ea, size):
-     """One collapsed row for a run of ``size`` undefined bytes starting at
-     ``ea``. A single byte is rendered normally (shows its value); a longer run
diff --git a/idatui/app.py b/idatui/app.py
index dd7c222..6506016 100644
--- a/idatui/app.py
+++ b/idatui/app.py
@@ -53,6 +53,23 @@ from .domain import DisasmModel, Func, Head, ListingModel, Program, Struct
 _S_ADDR = Style(color="#6b7684")
 _S_LABEL = Style(color="#7aa2f7", bold=True)
 _S_INSN = Style(color="#c3cad3")
+#: IDA's token kinds -> the measured palette. The rule that keeps a dense
+#: disassembly readable: NEUTRALS for the machine (mnemonic brightest because you
+#: scan down that column, registers at body weight because they're most of the
+#: text), HUES only where they mean something (numbers, strings, symbols),
+#: structure recedes so brackets and commas stop competing with operands.
+_S_SPAN = {
+    "insn": Style(color="#e8ecf2"),      # 15.3:1  mnemonic / directive
+    "reg": Style(color="#c3cad3"),       # 11.0:1  registers = body weight
+    "num": Style(color="#d8a657"),       #  8.2:1  immediates, offsets
+    "str": Style(color="#9ece6a"),       #  9.9:1  string literals
+    "name": Style(color="#7aa2f7"),      #  7.2:1  symbols / xref targets
+    "seg": Style(color="#93aee0"),       #  8.1:1  segment names
+    "cmt": Style(color="#7c8b9e", italic=True),   # 5.2:1
+    "punct": Style(color="#626c7a"),     #  3.4:1  brackets, commas, +/-
+    "err": Style(color="#c9762f"),       #         IDA's own error marker
+    "text": Style(color="#c3cad3"),      # 11.0:1  anything unclassified
+}
 _S_MNEM = Style(color="#e8ecf2")
 _S_OPBYTES = Style(color="#5e6875")  # raw opcode bytes column
 _S_DATA = Style(color="#d8a657")
@@ -1077,6 +1094,24 @@ class ListingView(SearchMixin, NavMixin, ColumnCursor, ScrollView, can_focus=Tru
             return " ".join(f"{b:02X}" for b in raw[:_OP_LIMIT]) + "\u2026"
         return " ".join(f"{b:02X}" for b in raw)
 
+    @staticmethod
+    def _span_segments(h: Head, fallback: Style):
+        """Segments for a row's disassembly text.
+
+        Uses IDA's own token classification when the worker supplied it; falls
+        back to the old mnemonic/rest split so an older worker (or a row whose
+        spans didn't match the text) still renders.
+        """
+        if h.spans:
+            return [Segment(t, _S_SPAN.get(k, fallback)) for k, t in h.spans]
+        if h.kind == "code":
+            mnem, _, rest = h.text.partition(" ")
+            segs = [Segment(mnem, _S_MNEM)]
+            if rest:
+                segs.append(Segment(" " + rest, fallback))
+            return segs
+        return [Segment(h.text, fallback)]
+
     def _op_field(self, h: Head) -> str:
         """The padded opcode-bytes column (empty when hidden). Shared format so
         cursor/search offsets line up."""
@@ -1301,17 +1336,11 @@ class ListingView(SearchMixin, NavMixin, ColumnCursor, ScrollView, can_focus=Tru
                 segs.append(Segment(_LST_INDENT, _S_MEMBER))
             if h.name:
                 segs.append(Segment(f"{h.name}  ", _S_LABEL))
-            if h.kind == "code":
-                mnem, _, rest = h.text.partition(" ")
-                segs.append(Segment(mnem, _S_MNEM))
-                if rest:
-                    segs.append(Segment(" " + rest, _S_INSN))
-            elif h.kind == "data":
-                segs.append(Segment(h.text, _S_DATA))
-            elif h.kind == "member":
+            if h.kind == "member":
                 segs.append(Segment(h.text, _S_MEMBER))
             else:
-                segs.append(Segment(h.text, _S_UNK))
+                base = {"code": _S_INSN, "data": _S_DATA}.get(h.kind, _S_UNK)
+                segs.extend(self._span_segments(h, base))
             strip = Strip(segs)
         linked = idx in self._link_rows
         if linked:
diff --git a/idatui/domain.py b/idatui/domain.py
index e4fbec9..606cf42 100644
--- a/idatui/domain.py
+++ b/idatui/domain.py
@@ -100,6 +100,10 @@ class Head:
     text: str
     name: str | None = None
     raw: bytes | None = None  # opcode/item bytes (filled in for code by the model)
+    #: [(kind, text)] from IDA's own colour tags — mnem/reg/num/name/str/punct/…
+    #: None when the worker didn't provide them (older worker, or the spans
+    #: disagreed with the plain text, in which case the text wins).
+    spans: tuple[tuple[str, str], ...] | None = None
 
     @property
     def label(self) -> str | None:  # Line-compatible alias
@@ -107,12 +111,15 @@ class Head:
 
     @classmethod
     def from_raw(cls, d: dict) -> "Head":
+        sp = d.get("spans")
         return cls(
             ea=_as_int(d["ea"]),
             kind=d.get("kind", "unknown"),
             size=int(d.get("size", 0) or 0),
             text=d.get("text", ""),
             name=d.get("name"),
+            spans=(tuple((str(k), str(t)) for k, t in sp)
+                   if isinstance(sp, list) and sp else None),
         )
 
 
diff --git a/server/patch_server.py b/server/patch_server.py
index 4c806a7..6601a0f 100644
--- a/server/patch_server.py
+++ b/server/patch_server.py
@@ -305,12 +305,137 @@ def _idatui_head_row(ea):
         "size": int(ida_bytes.get_item_size(ea)),
         "text": text,
     }
+    if line:
+        # Keep IDA's own token classification for syntax highlighting. Built from
+        # the SAME line as `text`, then whitespace-collapsed identically so the
+        # two never disagree about what the row says.
+        spans = _idatui_spans(line)
+        joined = "".join(t for _k, t in spans)
+        if " ".join(joined.split()) == text:
+            row["spans"] = spans
     nm = ida_name.get_ea_name(ea)
     if nm:
         row["name"] = nm
     return row
 
 
+#: IDA colour tag -> the semantic kind the TUI styles. IDA already classifies
+#: every token in a disassembly line, for every processor it supports, so there
+#: is nothing to lex: generate_disasm_line emits \x01<tag>text\x02<tag> and the
+#: tag says what the text IS. A pygments assembly lexer would be a worse guess at
+#: this and would need one dialect per architecture.
+_IDATUI_SPAN_KINDS = {
+    "insn": ("SCOLOR_INSN", "SCOLOR_KEYWORD", "SCOLOR_ASMDIR", "SCOLOR_MACRO"),
+    "reg": ("SCOLOR_REG",),
+    "num": ("SCOLOR_NUMBER", "SCOLOR_CHAR", "SCOLOR_BINPREF"),
+    "str": ("SCOLOR_STRING",),
+    # NB the real constant names: DATNAME/CODNAME, not "DNAME". Guessing here
+    # fails silently — an unmapped tag renders as plain body text, so symbols
+    # just quietly aren't blue and nothing tells you why.
+    "name": ("SCOLOR_DATNAME", "SCOLOR_CODNAME", "SCOLOR_LOCNAME",
+             "SCOLOR_IMPNAME", "SCOLOR_DEMNAME", "SCOLOR_LIBNAME",
+             "SCOLOR_CNAME", "SCOLOR_DNAME",
+             "SCOLOR_CREF", "SCOLOR_DREF", "SCOLOR_CREFTAIL", "SCOLOR_DREFTAIL"),
+    "seg": ("SCOLOR_SEGNAME",),
+    "cmt": ("SCOLOR_AUTOCMT", "SCOLOR_REGCMT", "SCOLOR_RPTCMT", "SCOLOR_VOIDOP"),
+    "punct": ("SCOLOR_SYMBOL", "SCOLOR_ALTOP", "SCOLOR_HIDNAME"),
+    "err": ("SCOLOR_ERROR",),
+}
+
+
+def _idatui_tag_map():
+    """{tag character: kind}, built once from whatever this IDA actually has."""
+    import ida_lines
+    out = {}
+    for kind, names in _IDATUI_SPAN_KINDS.items():
+        for n in names:
+            v = getattr(ida_lines, n, None)
+            if isinstance(v, str) and v:
+                out[v[0]] = kind
+            elif isinstance(v, int):
+                out[chr(v)] = kind
+    return out
+
+
+_IDATUI_TAGS = None
+
+
+def _idatui_spans(line):
+    """A tagged disasm line as [[kind, text], ...], colour tags resolved.
+
+    Unknown tags become 'text' rather than being dropped: a processor module can
+    emit a colour we don't classify, and losing the characters would corrupt the
+    line."""
+    global _IDATUI_TAGS
+    import ida_lines
+    if _IDATUI_TAGS is None:
+        _IDATUI_TAGS = _idatui_tag_map()
+    on, off, esc = "\x01", "\x02", "\x03"
+    addr_tag = chr(getattr(ida_lines, "COLOR_ADDR", 0x28))
+    addr_len = int(getattr(ida_lines, "COLOR_ADDR_SIZE", 16))
+    spans, stack, buf = [], [], []
+    i, n = 0, len(line)
+
+    def flush():
+        if buf:
+            spans.append([stack[-1] if stack else "text", "".join(buf)])
+            del buf[:]
+
+    while i < n:
+        ch = line[i]
+        if ch == on and i + 1 < n:
+            tag = line[i + 1]
+            if tag == addr_tag:
+                # An embedded target address, not display text: 16 hex digits
+                # that must not reach the screen.
+                i += 2 + addr_len
+                continue
+            flush()
+            stack.append(_IDATUI_TAGS.get(tag, "text"))
+            i += 2
+            continue
+        if ch == off and i + 1 < n:
+            flush()
+            if stack:
+                stack.pop()
+            i += 2
+            continue
+        if ch == esc and i + 1 < n:      # escaped literal
+            buf.append(line[i + 1])
+            i += 2
+            continue
+        buf.append(ch)
+        i += 1
+    flush()
+    # Collapse IDA's column padding EXACTLY as the plain text does. A run of
+    # spaces can straddle two spans, so this walks characters rather than
+    # collapsing each span on its own — otherwise the spans and `text` disagree
+    # about the line and the row silently loses its highlighting.
+    out, prev_space = [], False
+    for kind, txt in spans:
+        acc = []
+        for ch in txt:
+            if ch.isspace():
+                if prev_space:
+                    continue
+                acc.append(" ")
+                prev_space = True
+            else:
+                acc.append(ch)
+                prev_space = False
+        if acc:
+            out.append([kind, "".join(acc)])
+    while out and out[0][1] == " ":
+        out.pop(0)
+    while out and out[-1][1] == " ":
+        out.pop()
+    if out and out[0][1].startswith(" "):
+        out[0][1] = out[0][1].lstrip()
+    if out and out[-1][1].endswith(" "):
+        out[-1][1] = out[-1][1].rstrip()
+    return [[k, t] for k, t in out if t]
+
+
 def _idatui_unknown_row(ea, size):
     """One collapsed row for a run of ``size`` undefined bytes starting at
     ``ea``. A single byte is rendered normally (shows its value); a longer run
diff --git a/tests/test_scenarios.py b/tests/test_scenarios.py
index 6ccbf0d..abffe3c 100644
--- a/tests/test_scenarios.py
+++ b/tests/test_scenarios.py
@@ -395,6 +395,53 @@ async def s_load_options(c: Ctx):
     await c.wait(lambda: not isinstance(app.screen, LoadOptionsScreen), 10)
 
 
+@scenario("asm_highlight")
+async def s_asm_highlight(c: Ctx):
+    """Assembly is highlighted from IDA's OWN token tags.
+
+    generate_disasm_line() emits \x01<tag>text\x02<tag>, and the tag says what
+    the text is — for every processor IDA supports. Nothing is lexed here, so
+    what this pins is that the tags survive the trip: parsed server-side, agreed
+    with the plain text, carried on the Head, and mapped to a style.
+    """
+    from idatui.app import _S_SPAN
+    app = c.app
+    await c.open_biggest("listing")
+    lst = c.lst
+    ok = await c.wait(lambda: lst.model is not None and lst.total > 20, 25)
+    if not ok:
+        c.check("a listing to highlight", False, f"total={lst.total}")
+        return
+    lst.model.ensure(400)
+    rows = [lst.model.get(i) for i in range(min(lst.model.loaded(), 400))]
+    rows = [h for h in rows if h is not None]
+    code = [h for h in rows if h.kind == "code"]
+    c.check("code rows carry IDA's token spans",
+            code and sum(1 for h in code if h.spans) > len(code) * 0.9,
+            f"{sum(1 for h in code if h.spans)}/{len(code)} have spans")
+
+    kinds = {k for h in rows for k, _ in (h.spans or ())}
+    # If a tag isn't mapped it renders as body text and nothing says why, so the
+    # ones that carry real meaning are worth asserting explicitly.
+    for want in ("insn", "reg", "punct"):
+        c.check(f"the palette sees {want} tokens", want in kinds, f"{sorted(kinds)}")
+    c.check("every span kind has a style",
+            all(k in _S_SPAN for k in kinds), f"unstyled: {sorted(kinds - set(_S_SPAN))}")
+
+    # Spans must describe the SAME text the row shows, or the row renders
+    # different characters than search/width calculations think it has.
+    bad = [h for h in rows if h.spans
+           and "".join(t for _k, t in h.spans) != h.text]
+    c.check("spans reconstruct the row text exactly", not bad,
+            f"{[(hex(h.ea), h.text) for h in bad[:2]]}")
+
+    # And the mnemonic must be the loudest thing on the line (the column you
+    # scan), not just any styled token.
+    mn = next((h for h in code if h.spans and h.spans[0][0] == "insn"), None)
+    c.check("the mnemonic is the first span",
+            mn is not None, f"{code[0].spans if code else None}")
+
+
 @scenario("command_palette")
 async def s_command_palette(c: Ctx):
     app = c.app
@@ -1949,9 +1996,16 @@ async def s_listing_view(c: Ctx):
     try:
         c.prog.undefine(dea, size=4)
         c.prog.bump_items()
-        # reopen the listing so the model reflects the new undefined run
+        # Reopen the listing so the model reflects the new undefined run, and
+        # wait for the model to be REPLACED. Waiting on index_of_ea(dea) >= 0 is
+        # no test at all: dea is in the OLD model too (it's the head we just
+        # undefined), so the predicate passes instantly and we then assert
+        # against pre-edit rows. It only ever passed because the swap happened to
+        # win the race, and it started failing the moment page loads got bigger.
+        stale = c.lst.model
         await c.goto_ui(hex(dea))
         await c.wait(lambda: app._active == "listing" and c.lst.total > 0
+                     and c.lst.model is not stale
                      and c.lst.model.index_of_ea(dea) >= 0, 25)
         ui = c.lst.model.index_of_ea(dea)
         c.check("undefining a data head yields an unknown run in the listing",