sl0ppty: decode the PNGs programs actually send

Kitty graphics were green in the suite and a real program still drew
nothing. libghostty-vt ships no image codec: built as a library it sets
its `sys.decode_png` hook to null, and a null hook rejects every `f=100`
transmission outright -- no image stored, so no placement exists, so the
re-emission layer this project is mostly *about* is never reached. The
program gets no error and no picture.

Raw RGB worked throughout, which is why it looked done: every test here
sent raw pixels, because that is what is easy to write in a test. It is
the same lesson the last kitty graphics bug taught -- write the test the
way a program actually behaves -- and a program with a picture on disk
has a PNG. Found because ida-tui's startup logo never appeared, and its
detection probe answers `OK` first, so it fails silently rather than
falling back to its block art.

PNG is the whole of the gap and cannot grow: the protocol has exactly
three formats (`f=24`, `f=32`, `f=100`), the first two are raw pixels
lib-vt already handles, and `decode_png` is the only codec hook it has.
Decoding only, never encoding.

The decoder is the stb_image lib-vt already vendors and we already
commit -- upstream compiles that same header with the same
`STBI_ONLY_PNG` for this same purpose, it is just not in the
`-Demit-lib-vt` build. So nothing new is pulled in; a file we already
ship is finally compiled. src/png.c is the hook, a header pre-check and
a copy: dimensions are read before decoding because the buffer we return
is allocated through lib-vt's limited allocator but stb's own working
allocation is plain malloc and answers to nothing.

Tests cover a PNG placed at the size it asked for (RGB and RGBA sources,
both normalised to RGBA), the client being handed `f=32` rather than the
png it was given, and a malformed png placing nothing without taking the
pane with it. They fail on the previous binary.

8 files changed, 238 insertions, 2 deletions

diff --git a/DESIGN.md b/DESIGN.md
index d5eed95..ac3beaf 100644
--- a/DESIGN.md
+++ b/DESIGN.md
@@ -160,6 +160,7 @@ unreadable.
 | **D15** | **Shaders may be loaded as native shared libraries**, from a directory beside the config. This is the one hole in D9 and it is argued for rather than drifted into (and D16 has since removed most of the *need* for it). A shader is called per cell, per pass, per frame — tens of thousands of times at up to 120Hz — so the subprocess model D9 prescribes cannot reach that path at all, and the measured alternative, interpreting a bytecode per cell, is 56ns against 8ns for the compiled function. What is loaded is also the narrowest possible thing: a pure function from one cell and its position to that cell's colours, unable to touch text, width, neighbours, panes or the session, because `shader_abi.h` is the only header it sees and nothing else is declared in it. On trust: the config file can already say `shell` and a layout can already say `command=`, so a file in the config directory can already run anything as you — a `.so` in the same directory is not a new boundary. It is refused unless it declares our ABI version *and* matching struct sizes, so a stale plugin fails at load instead of garbling colours. Nothing is ever unloaded: a `shade_fn` may be sitting in a live config or on a pane, so `dlclose()` would be a jump into an unmapped page — a reload therefore picks up newly *added* plugins only, and replacing one needs a new session. |
 | **D16** | **A shader's `amount` may be an expression**, compiled to bytecode at config load. The scope is the argument: every built-in here is already `colour_op(amount)` with the amount a pure function of position, so the part worth handing to a config is the *number*, not the colour maths. That keeps mixing in C, gives an expression no way to produce an invalid cell (it produces one byte), and makes the result cacheable — a program that does not read the clock is evaluated once per cell into a map and reused until something it reads changes, so a config-written positional shader measures 5.2ns/cell against a compiled vignette's 6.1ns. Reading the clock costs 35ns/cell, per frame, which is the honest price of animation and is opt-in. The language is **total**: no loops, no recursion, no jumps — even `?:` evaluates both sides and selects, because nothing in it has an effect. So evaluation is O(program length), a config cannot spin, and there is nothing to sandbox; division by zero is 0 rather than a trap, because an expression that is wrong should look wrong rather than take the session with it. What a program reads is **derived by the compiler**, not declared, and that dependency set *is* the cache key — a stale map is unrepresentable rather than merely unlikely, which is the same rule the layout follows. An expression that does not compile drops that one shader with a warning: not the config, and not a half-strength version of what was asked for. |
 | **D17** | **Images need pixels, and we were not carrying any.** Kitty graphics worked in the test suite and not in real life, because three separate things all quietly assumed a cell has no size. `pty_spawn`/`pty_resize` left `ws_xpixel`/`ws_ypixel` zero, so a program asking the tty how big a cell is got nothing and guessed. `ghostty_terminal_resize` was called with `0, 0` for the cell size, so lib-vt could not work out how many cells an image covers when the placement did not say — and a placement that covers zero cells is dropped, which is a picture that silently does not appear. And the vendored lib freed image *data* on `ED 2`, so a full-screen program lost everything it had uploaded. The fix runs the whole way through: the client reads its terminal's pixel size (`TIOCGWINSZ`, re-read on every resize because a window can move to another monitor), `MSG_HELLO`/`MSG_RESIZE` carry it, the app hands it to every pane in every tab, and each pane gives it to both its pty and its terminal. Where nobody knows, the default is **8x16 rather than 0**: a plausible cell means an image appears at roughly the right size, and zero means it does not appear at all. Tested end to end, including that a program can read real pixel dimensions out of its own pty. **Re-emission has to preserve what the program meant, not what it currently covers.** Two things fall out of that. A placement carries a sub-cell offset (`X=`/`Y=`, where the image starts *inside* its first cell); dropping it is invisible in a still picture and makes anything moving jump a whole cell at a time. And `c=`/`r=` mean *scale into this many cells*, so they are passed on only when the program asked for them — sending the count a natural-size image happens to cover makes it change size as it slides across a cell boundary, because that count goes up by one. Cropping follows the same split: a natural image is cropped by the screen pixels the pane has left (measured from the offset, not the cell edge), a scaled one proportionally, because a lost cell is worth `source/cells` pixels there. |
+| **D18** | **We decode PNG ourselves, with the stb_image we already vendor.** lib-vt ships no image codec: built as a library its `sys.decode_png` hook is null, and a null hook rejects every `f=100` transmission outright — no image stored, no placement, nothing to re-emit. A program that uploads a PNG (which is what a program with a picture on disk does) drew nothing at all, silently, while raw RGB worked. Found because ida-tui's startup logo never appeared. PNG is the only format this can ever be about: the protocol has exactly three (`f=24`, `f=32`, `f=100`) and the first two are raw pixels lib-vt already handles, and `decode_png` is the only codec hook it has — so there is no scope to creep. Decoding only, never encoding; the client is handed RGBA. The decoder is `vendor/libghostty-vt/src/stb/stb_image.h`, which is **already in the tree and already pinned** by the vendor commit — upstream compiles that same header with the same `STBI_ONLY_PNG` for the same reason (`src/stb/stb.c` says so), it is simply not part of the `-Demit-lib-vt` build. So this is not a new dependency, it is a file we already ship finally being compiled, and `src/png.c` is a hook and a copy around it. Hand-rolling a subset decoder was considered and rejected: the bulk of a PNG decoder is a DEFLATE inflate, and "just the colour type ida-tui uses" breaks on the first palette PNG, which is what every png optimiser emits. That is a lot of new bounds-checked parsing, fed by whatever a pane decides to send, to replace a file already in the repo. Two consequences to know: we re-emit **decoded** pixels, so a 819KB logo goes to the client as 3.3MB of base64 (once per image, again after a reattach clears `sent`) — passing the original PNG through would be cheaper but lib-vt hands back only the decoded form; and lib-vt caps image storage at **10MB** in library builds, so images are bigger than they look. |
 
 ### D6 — responsive layout as a pure function
 
diff --git a/HANDOVER.md b/HANDOVER.md
index fba59d8..76daa6f 100644
--- a/HANDOVER.md
+++ b/HANDOVER.md
@@ -227,6 +227,32 @@ it answers with the escape sequences the client is actually sent, so a test
 can assert on the bytes rather than on the model that produced them. If you
 touch graphics, assert on the bytes.
 
+### And then a PNG arrived
+
+Same feature, same shape of bug, a year of confidence later: kitty graphics
+were green and a real program still drew nothing. lib-vt ships **no image
+codec** — built as a library it sets `sys.decode_png` to null, and a null hook
+rejects every `f=100` transmission outright, so no image is stored, no
+placement exists, and the re-emission layer is never even reached. Raw RGB
+worked, which is why it looked fine here for as long as it did: every test in
+this file sent raw pixels, because that is what is easy to write in a test.
+That is the *same* lesson as the section above, learned again — write the test
+the way a program actually behaves. A program with a picture has a PNG.
+
+The fix is `src/png.c`: `ghostty_sys_set(GHOSTTY_SYS_OPT_DECODE_PNG, ...)`
+over the `stb_image.h` that libghostty-vt already vendors and we already
+commit. See D18 for why that rather than a hand-rolled one.
+
+**A false lead worth not repeating.** The first bisect built its escape
+sequences with `sh printf`, and `printf` mangles adjacent backslash escapes:
+`\033\\` immediately followed by `\033` reaches the pty as a literal `\033`.
+So the pane never received the second escape, and the evidence pointed
+convincingly at "two APC sequences back to back are dropped by the parser" —
+a bug in the vendored lib that did not exist. What killed it was dumping the
+bytes at `pane_pump`, where the pty read shows what *actually* arrived. If a
+repro's conclusion implicates the parser, check the pty bytes before you
+believe it, and build test streams in python where a byte is a byte.
+
 **Anything written after the frame must leave the cursor where the frame put
 it.** Images go out after the cell diff on purpose (a repainted cell must not
 land on top of a placement), and placing an image parks the cursor on its
diff --git a/Makefile b/Makefile
index 7174c5c..76df199 100644
--- a/Makefile
+++ b/Makefile
@@ -53,6 +53,12 @@ $(BIN): $(OBJ) $(VT_LIB)
 build/%.o: src/%.c build/version.h $(VT_LIB) | build
 	$(CC) $(CFLAGS) -c $< -o $@
 
+# stb_image comes with the vendored libghostty-vt, which is where the include
+# path points. It is third-party and does not build clean under our warnings,
+# so it gets its own rule rather than every file paying for it.
+build/png.o: src/png.c build/version.h $(VT_LIB) | build
+	$(CC) $(CFLAGS) -I$(VT)/src/stb -Wno-unused-function -c $< -o $@
+
 # Rewritten only when the value actually changes, so `make` after a commit
 # rebuilds, and `make` twice in a row does not.
 .PHONY: FORCE
diff --git a/README.md b/README.md
index 624c77b..1c3436b 100644
--- a/README.md
+++ b/README.md
@@ -121,8 +121,10 @@ mistyped command in a fresh session no longer closes the session.
   thing other multiplexers drop — including the useful half of the protocol,
   where a program uploads an image once and then places it every frame. Your
   terminal's cell size is carried through to each pane, so programs can size
-  images and place them without saying how many cells to cover. (Sixel is not
-  supported: the terminal core has no sixel decoder.)
+  images and place them without saying how many cells to cover. PNG uploads
+  are decoded here and handed to your terminal as pixels, because the terminal
+  core ships no image decoder of its own. (Sixel is not supported: there is no
+  sixel decoder either, and unlike PNG nobody has asked.)
 - **Small terminals degrade gracefully.** When panes no longer fit, the tab
   becomes a list of one-line headers with the focused pane open below them —
   and returns to exactly the layout you had when there is room again.
diff --git a/src/main.c b/src/main.c
index 5a52a18..68170f1 100644
--- a/src/main.c
+++ b/src/main.c
@@ -21,6 +21,7 @@
 #include "proto.h"
 #include "server.h"
 #include "config.h"
+#include "png.h"
 #include "version.h"
 
 int run_headless(const char *const argv[], uint16_t cols, uint16_t rows,
@@ -45,6 +46,12 @@ static void usage(void) {
 }
 
 int main(int argc, char **argv) {
+  /* Process-global and required before any terminal exists, so it happens
+   * here rather than in whichever mode we turn out to be: server, headless
+   * and the script driver all create panes, and a pane created without this
+   * silently drops every PNG a program sends it. */
+  png_init();
+
   bool headless = false, script = false, server = false;
   uint16_t cols = 80, rows = 24;
   int idle_ms = 300;
diff --git a/src/png.c b/src/png.c
new file mode 100644
index 0000000..daf0f2e
--- /dev/null
+++ b/src/png.c
@@ -0,0 +1,87 @@
+#define _GNU_SOURCE
+#include "png.h"
+
+#include <stdbool.h>
+#include <stddef.h>
+#include <stdint.h>
+#include <string.h>
+
+/* stb_image is already in the tree: libghostty-vt vendors it, and upstream
+ * compiles that same header with exactly these defines (see
+ * vendor/libghostty-vt/src/stb/stb.c) to answer exactly this question —
+ * "the only use case we have right now is the Kitty Graphics protocol which
+ * only supports PNG as a format besides raw RGB/RGBA buffers". So this is not
+ * a new dependency; it is a file we already ship and pin, finally compiled.
+ *
+ * ONLY_PNG keeps the other decoders out of the binary, NO_STDIO the file
+ * paths: the bytes always arrive in memory, straight off a pane's pty, so a
+ * decoder that can open files is surface we have no use for. */
+#define STBI_ONLY_PNG
+#define STBI_NO_STDIO
+#define STB_IMAGE_IMPLEMENTATION
+#include "stb_image.h"
+
+#include <ghostty/vt.h>
+
+/* The library refuses anything past this (`max_size` in
+ * kitty/graphics_image.zig), so decoding past it can only ever be work done
+ * to have the result thrown away. Matched rather than tightened: being
+ * stricter here would silently reject images the library would have taken. */
+#define MAX_DECODED (400u * 1024u * 1024u)
+
+static bool decode(void *ud, const GhosttyAllocator *alloc, const uint8_t *data,
+                   size_t data_len, GhosttySysImage *out) {
+  if (!alloc || !data || !data_len || !out) return false;
+  if (data_len > INT32_MAX) return false; /* stb counts in int */
+
+  /* Read the header before decoding it. The buffer we return is allocated
+   * through the library's allocator, which is wrapped in a limit — but
+   * stb_image's own working allocation is plain malloc and answers to nothing,
+   * so a header claiming 60000x60000 would have us try for 14GB before anyone
+   * got to refuse it. This costs an IHDR parse instead. */
+  int w = 0, h = 0, channels = 0;
+  if (!stbi_info_from_memory(data, (int)data_len, &w, &h, &channels))
+    return false;
+  if (w <= 0 || h <= 0) return false;
+  if ((uint64_t)w * (uint64_t)h * 4u > MAX_DECODED) return false;
+
+  /* 4 = "give me RGBA whatever the file is", so palette, greyscale and
+   * 16-bit all arrive in the one layout the protocol wants. */
+  int got_w = 0, got_h = 0;
+  stbi_uc *px =
+      stbi_load_from_memory(data, (int)data_len, &got_w, &got_h, &channels, 4);
+  if (!px) return false;
+  if (got_w <= 0 || got_h <= 0) {
+    stbi_image_free(px);
+    return false;
+  }
+
+  /* Sized from what came back, not from what the header promised. */
+  uint64_t len = (uint64_t)got_w * (uint64_t)got_h * 4u;
+  if (len > MAX_DECODED) {
+    stbi_image_free(px);
+    return false;
+  }
+
+  /* Copied into the library's allocator because that is what it frees the
+   * buffer with. Routing stb's own allocation there instead would mean
+   * smuggling a per-call allocator into a global, for one memcpy that is
+   * already far cheaper than the decode that produced it. */
+  uint8_t *buf = ghostty_alloc(alloc, (size_t)len);
+  if (!buf) {
+    stbi_image_free(px);
+    return false;
+  }
+  memcpy(buf, px, (size_t)len);
+  stbi_image_free(px);
+
+  *out = (GhosttySysImage){.width = (uint32_t)got_w,
+                           .height = (uint32_t)got_h,
+                           .data = buf,
+                           .data_len = (size_t)len};
+  return true;
+}
+
+void png_init(void) {
+  ghostty_sys_set(GHOSTTY_SYS_OPT_DECODE_PNG, (const void *)(uintptr_t)decode);
+}
diff --git a/src/png.h b/src/png.h
new file mode 100644
index 0000000..757edeb
--- /dev/null
+++ b/src/png.h
@@ -0,0 +1,22 @@
+/* PNG decoding for the kitty graphics protocol.
+ *
+ * libghostty-vt ships no image codec: built as a library it sets its
+ * `decode_png` hook to null, and a null hook means every `f=100` transmission
+ * is rejected outright — no image stored, no placement, nothing for the
+ * graphics layer to re-emit. A program that uploads a PNG (which is what a
+ * program with a picture on disk naturally does) therefore drew nothing at
+ * all, silently, while raw RGB worked. Installing this hook is the whole fix.
+ *
+ * PNG is the only format this can be about: the protocol has exactly three
+ * (`f=24` RGB, `f=32` RGBA, `f=100` PNG) and the first two are raw pixels the
+ * library already handles. Decoding only, never encoding — we hand the client
+ * RGBA.
+ */
+#ifndef SL0PPTY_PNG_H
+#define SL0PPTY_PNG_H
+
+/* Install the process-global PNG decoder. Must be called before any terminal
+ * exists, and is idempotent. */
+void png_init(void);
+
+#endif /* SL0PPTY_PNG_H */
diff --git a/tests/test_graphics.py b/tests/test_graphics.py
index 241498e..3ead630 100644
--- a/tests/test_graphics.py
+++ b/tests/test_graphics.py
@@ -7,7 +7,9 @@ cannot collide between panes. That collision is the whole reason tmux and
 zellij drop images instead, and it is the first thing tested here.
 """
 import base64
+import struct
 import sys
+import zlib
 
 from harness import Session, check, report
 
@@ -15,6 +17,24 @@ from harness import Session, check, report
 PX = base64.b64encode(bytes([255, 0, 0, 0, 255, 0, 0, 0, 255, 255, 255, 0] * 2)).decode()
 
 
+def png(w=4, h=2, color_type=2):
+    """A minimal PNG, built here so the suite needs no binary fixtures.
+
+    `color_type` 2 is RGB and 6 is RGBA; both are worth sending, because the
+    decoder is asked to normalise whatever the file is into RGBA.
+    """
+    def chunk(tag, body):
+        c = tag + body
+        return struct.pack(">I", len(body)) + c + struct.pack(">I", zlib.crc32(c))
+
+    px = [255, 0, 0] if color_type == 2 else [255, 0, 0, 255]
+    raw = b"".join(b"\x00" + bytes(px * w) for _ in range(h))
+    return (b"\x89PNG\r\n\x1a\n" +
+            chunk(b"IHDR", struct.pack(">IIBBBBB", w, h, 8, color_type, 0, 0, 0)) +
+            chunk(b"IDAT", zlib.compress(raw)) +
+            chunk(b"IEND", b""))
+
+
 def sends_image(image_id=7, cols=6, rows=2, after="sleep 5"):
     seq = f"\\033_Ga=T,f=24,s=4,v=2,i={image_id},q=2,c={cols},r={rows};{PX}\\033\\\\"
     return ["/bin/sh", "-c", f'stty raw -echo; printf "{seq}"; {after}']
@@ -287,6 +307,68 @@ def test_partial_visibility_crops_rather_than_squashes():
               f"{pl[0]} vs {p['content_h']}")
 
 
+def sends_png(data, place="c=6,r=2", image_id=7):
+    """Upload with `a=t,f=100` and place it: what a program with a picture on
+    disk does, and what ida-tui's splash does.
+
+    The child is python rather than sh because `printf` mangles adjacent
+    backslash escapes -- `\\033\\\\` followed by `\\033` arrives as a literal
+    `\\033` -- so a shell-built sequence does not reliably reach the pty as the
+    bytes you wrote. That cost an afternoon and a wrong diagnosis.
+    """
+    b64 = base64.b64encode(data).decode()
+    prog = ("import sys, time\n"
+            f"sys.stdout.write('\\033_Ga=t,f=100,t=d,i={image_id},q=2;{b64}\\033\\\\')\n"
+            f"sys.stdout.write('\\033_Ga=p,i={image_id},p=1,{place},q=2\\033\\\\')\n"
+            "sys.stdout.write('DONE')\n"
+            "sys.stdout.flush()\n"
+            "time.sleep(5)\n")
+    return ["python3", "-c", prog]
+
+
+def test_a_png_is_decoded_and_placed():
+    """libghostty-vt ships no image codec: built as a library its `decode_png`
+    hook is null, and a null hook rejects every `f=100` transmission outright.
+    No image was stored, so no placement existed and there was nothing to
+    re-emit -- a program that uploaded a PNG drew nothing at all, silently,
+    while raw RGB worked. Found because ida-tui's startup logo never appeared."""
+    for color_type, what in ((2, "RGB"), (6, "RGBA")):
+        with Session(sends_png(png(color_type=color_type)), cols=44, rows=10) as s:
+            s.until_text("DONE")
+            s.settle(200)
+            pl = places(s)
+            check(f"a {what} png is placed", len(pl) == 1, str(pl))
+            if pl:
+                check("at the size it asked for",
+                      (pl[0]["cols"], pl[0]["rows"]) == (6, 2), str(pl[0]))
+
+
+def test_a_decoded_png_reaches_the_client_as_raw_pixels():
+    """The client is told `f=32`: decoding is the point, and the terminal we
+    are re-emitting to should not have to decode it a second time."""
+    with Session(sends_png(png()), cols=44, rows=10) as s:
+        s.until_text("DONE")
+        s.settle(200)
+        raw = s.api("graphics", format="bytes")["bytes"]
+        tx = [c for c in raw.split("\x1b") if c.startswith("_Ga=t")]
+        check("the image is transmitted to the client", tx != [], repr(raw[:120]))
+        if tx:
+            check("as RGBA, not as the png we were given",
+                  "f=32" in tx[0] and "f=100" not in tx[0], tx[0][:80])
+
+
+def test_a_png_that_is_not_one_places_nothing():
+    """The bytes come off a pane's pty, so the decoder is fed whatever a
+    program feels like sending. Refusing it is all that is asked; the pane
+    must still be there afterwards."""
+    junk = b"\x89PNG\r\n\x1a\n" + b"\xde\xad\xbe\xef" * 32
+    with Session(sends_png(junk), cols=44, rows=10) as s:
+        s.until_text("DONE")
+        s.settle(200)
+        check("nothing is placed", places(s) == [], str(places(s)))
+        check("and the session survived it", s.pane() is not None)
+
+
 def test_scrolled_away_placements_are_dropped():
     prog = ("stty raw -echo; "
             f'printf "\\033_Ga=T,f=24,s=4,v=2,i=7,q=2,c=6,r=2;{PX}\\033\\\\"; '
@@ -299,6 +381,9 @@ def test_scrolled_away_placements_are_dropped():
 
 if __name__ == "__main__":
     test_a_pane_image_reaches_the_screen()
+    test_a_png_is_decoded_and_placed()
+    test_a_decoded_png_reaches_the_client_as_raw_pixels()
+    test_a_png_that_is_not_one_places_nothing()
     test_a_placement_that_does_not_say_how_big_it_is()
     test_the_cell_size_a_client_reports_is_what_sizes_an_image()
     test_a_program_can_read_the_pixel_size_from_its_pty()