slosh.git/src/graphics.c, branch v0.1.7

graphics: a retransmit invalidates every placement of the image

2026-09-07T13:54:34Z

Replacing an image's pixels deletes all of its placements on the client -- kitty drops them when the data is replaced, which is the very fact the place-once logic leans on. But the invalidation rode a local in one gfx_place() call, so only the placement said in the same breath as the transmit was owed again. The image's other placements -- a second one elsewhere, the other pieces of one a float has cut -- were judged unchanged against a client that no longer holds them, and stayed unsaid: a picture the client deleted and we believe it still has, gone until something moves it. Masked before place-once by the every-frame re-emission, so this is the regression that fix bought. The transmit now un-marks every placement of its image, and each is said again as its gfx_place() comes around. The live check drives one image placed twice, then replaces its pixels and places both again: both must reach the client after the second transmit, where the old code re-sent one.

graphics: place what a float leaves of an image, not nothing

2026-09-06T17:59:13Z

Occlusion for placements is clipping: the cell compositor gets it free from paint order, but images are sent after the diff and drawn over it, so what a float covers has to be taken out of the placement itself. A clean edge came out as a smaller rectangle; anything else -- a float across the middle, a float on a corner, a float wholly inside -- was refused, on the grounds that the remainder is an L and a placement is a rectangle. The image vanished, all of it, the moment a floating pane crossed it, on the terminal and in the mirror alike, and came back when the float moved off. The remainder is not one rectangle, but it is always a few: the strips above and below what is covered, and what is left beside it between them. So rect_subtract() cuts the visible remainder into rectangles and each is placed in its own right, with the source rectangle the same arithmetic the pane edge already used -- generalised to a cut on any side, and to a first piece that keeps the sub-cell offset while the others start on a cell boundary. Eight pieces is the ceiling (two floats' worth); past it the old refusal stands. The pieces of one placement need an id each, and the program's own placement id is one number that cannot be several, so the model now hands out the placement ids the client is told and keeps the program's as the identity it was. `graphics` reports the piece alongside it. Checked against a float parked over an image: the picture is cut around the float, the strips tile exactly the image minus what the float covers, and their source rectangles land where their cells do (a float inside a 40x10-cell natural-size image gives 320x32 above, 320x32 below, 56x96 left and 72x96 right). Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CRz3zwZJvBvzsR9k4cj6dd

graphics: state z=0 on a placement, so a mirror paints it over the text

2026-09-06T17:41:56Z

kitty's default z is 0, which draws an image above the text, and that is the value slosh has always relied on -- it places after the cell diff precisely so the picture ends up on top. Saying nothing works because the terminal supplies the default. Anything mirroring the stream has no such default to fall back on. It cannot tell "the emitter wants this over the text" from "the emitter said nothing", and shellglass's viewer paints an unstated image under the cells. That is fine until a cell over the picture carries an explicit background -- and dim_unfocused gives one to every cell of a pane at once, so the moment focus moved elsewhere the whole image disappeared from the mirrored copy while the terminal, which took the default, still showed it. So the placement states z=0. Nothing changes on a terminal, since that is the value it was already assuming; the mirror now paints the picture over the cells and the dim pass runs underneath it, where it belongs. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CRz3zwZJvBvzsR9k4cj6dd

graphics: place an image once, not once per frame

2026-09-06T17:41:18Z

Every frame re-emitted `a=p` for every visible placement. On a terminal that is invisible: a kitty placement is a standing instruction, not a per-frame one ("the other commands to erase text must have no effect on graphics"), so a repaint of the cells under a picture does not disturb it and saying it again says nothing new. It is not free anywhere else. A mirror cannot know that the identical placement is identical: shellglass re-decodes and re-encodes the image on every one, holds the placement back while that runs, and -- because an image change rides only in a full frame -- republishes the whole screen each time. Measured on one image and a pane printing text for twenty seconds, the mirror published 430 full frames and re-sent the image list 215 times; 215 of those fulls carried no image at all, because the placement was still mid-decode. That alternation is what was visible in the browser: a picture that flickered while anything on screen moved and vanished when it stopped. The same twenty seconds now costs one full frame, 412 diffs and one image list. So a placement is emitted only when it is new or has actually changed, and the client's state records what it is holding. Three things have to be right for that to be safe: a dropped frame un-marks the placements it carried (they are owed again, exactly as transmissions and deletions already were); a retransmit of the pixels invalidates the placement, because kitty deletes an image's placements when its data is replaced; and a full repaint clears the screen, which takes the images with it, so every placement is owed again -- gfx_repaint(), not gfx_reset(), because the pixels the terminal was sent survive a clear and a deletion still owed from a dropped frame is still owed. The live check drives a picture that does not move while the screen around it repaints: one placement across 121 frames, where the old behaviour emitted 120. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01CRz3zwZJvBvzsR9k4cj6dd

graphics: a frame the client never got must be said again

2026-08-29T19:31:41Z

A pane image is re-emitted to the client as decoded RGBA, which for one screenshot is ~4MB of base64 in a single indivisible message -- and conn_send silently dropped anything that would overflow the 4MB outbox, while push_graphics ignored the refusal and the model advanced anyway. Everything stateful in that dropped frame was lost for good: the image was marked sent (so its placements drew nothing, forever), and the deletions rode out with it (so a placement the client had was never revoked). Reported as screenshots in a recovered pi transcript parked at a fixed spot on the screen, immune to scrolling and pane swaps -- those were placements whose deletion left with a dropped frame. The 1000x800 one could never be delivered at all: its transmission alone was bigger than the cap. So the frame is now transactional. Deletions queue and are repeated every flush until gfx_commit() confirms the client heard them; transmissions recorded in a frame are unwound by an undelivered commit so they are sent again. A flush nobody commits counts as undelivered at the next gfx_begin(), which is what the `graphics` control command is: bytes rendered for a CLI, not the client. The server reports the verdict and keeps the paint pending until it lands. The outbox cap now bounds the backlog rather than backlog-plus-message, so one big transmission on a drained queue always fits -- and that surfaced a second way to lose: a queued image legitimately holds the backlog over the cap until the client swallows it, and push_frame read that as a dead client and closed the connection. The session auto-detached every time a screenshot crossed the wire. A full outbox is a client that is *behind*, not gone: push_frame now marks the screen for a full repaint (the dropped diff means the client's cells are unknown) and retries as POLLOUT drains the queue; a client that truly left still shows up as a write error there. The headless suite pins the owed-until-committed contract; the new tests/live_graphics.py drives the real socket with a bigger-than-the-cap image and a reader that stalls, and would have caught both bugs: the flood must not detach, the transmission must arrive whole, and a placement scrolled away must be deleted -- then come back on scroll without a second copy of the pixels.

sl0ppty: clang-format the tree

2026-08-17T12:22:38Z

The bumpy one. `make fmt` over every `.c` and `.h` outside `vendor/`: 32 files, 1,843 lines rewritten, no behaviour changed. Nothing here is worth reading -- the next commit adds it to `.git-blame-ignore-revs` so `git blame` walks past it. Two thirds of it is alignment the formatter now owns: trailing comments in a struct, continuation lines under a signature. The rest is the hand-packed tables opening up to one entry per line -- `ACTIONS`, `NAMED_KEYS`, `ACTION_HELP`. That is a real loss of density and a real gain in diffs: adding an action was a reflow of the block and is now one line. Anything worth keeping packed can be fenced with `// clang-format off`; nothing was, on the grounds that a table is the thing a formatter is for. **One change is not formatting.** `src/config.h` had three trailing comments glued onto one field: uint16_t hover_delay_ms; /* how long the pointer must rest to arm a guide */ /* how long an announcement stays up */ /* rows per wheel notch */ -- `toast_ms`'s and `scroll_lines`' comments, landed on their neighbour by an earlier edit, documenting the wrong things ever since. It is in this commit because it is what caught it: clang-format could not settle on that line, wrapping it one way and then re-indenting the wrap on a second pass, so `make fmt` was not idempotent. The comments are back where they belong and it is. Verified after the reformat, not before: `make clean && make all` is warning-free, `make retest` is 1801 checks green, and `make fmt` twice in a row changes nothing the second time. A formatter that alters behaviour would be worse than none, so the gate was the whole suite rather than a reading of the diff.

sl0ppty: answer the terminal when it is asked how big it is

2026-08-15T17:38:38Z

XTWINOPS (CSI 14/16/18 t) went unanswered. A program that draws images needs the cell size to keep an aspect ratio; the pty's pixel fields are the first way to get it (D17) and this query is the second, which is what a program reaches for when those come back zeroed -- as they do on the far side of an ssh hop. Unanswered, the query times out and the program falls back to a guess: ida-tui guesses 10x20, and against a real 9x22 cell that is every image stretched by a fifth, which looks like a bug in the program rather than in us. Found alongside the png bug and from the same program, which asks `CSI 16 t` in the same round trip as its graphics query -- so once the png decoded, the picture would have appeared at the wrong shape. lib-vt encodes and writes the reply once the callback says what the numbers are; they come from the pane's own geometry, so a program is told about the pane it is in rather than the window. Mode 2048 (the in-band report on resize) already worked, because that one is answered from the size passed to ghostty_terminal_resize. Also notes in graphics.c what re-emission actually costs: we send decoded pixels because that is the only form lib-vt keeps, so an 819KB png reaches the client as 3.3MB of base64, once per image and again after a reattach clears `sent`. Fine for a splash, and written down so the next person to wonder does not have to measure it again.

sl0ppty: the graphics stream gives the cursor back

2026-08-14T21:32:44Z

Reported as "the cursor is not rendered in zsh" in a newly spawned pane. It was not about zsh, or about that pane: it was about the pane next to it, which was animating an image. Placing an image means parking the cursor on its target cell first, and the graphics bytes go out *after* the cell diff -- deliberately, so a repainted cell cannot land on top of a placement. That is also after the frame has put the real cursor where the focused pane wants it. So with any image on screen, the last instruction the terminal received every frame was "go to wherever that image is", and the cursor sat there: in another pane, moving about with the picture. With a bouncing DVD logo next door it is quite a sight. gfx_begin() now opens the stream with DECSC and gfx_flush() closes it with DECRC, so the whole thing is cursor-neutral no matter what it contains. The save has to be written at the start of the frame rather than at flush time, because transmissions are emitted as images are recorded and are already in the buffer by then. A frame with nothing to say drops both and still emits not one byte, which the test asserts, because a save/restore pair on every idle frame is exactly the sort of cost that creeps in unnoticed. Two tests: that the stream is bracketed (and that it really does move the cursor, or the assertion proves nothing), and that an idle frame is empty. One C trap worth the comment it now has: "\x1b7" is a *single* hex escape, 0x1b7, not ESC followed by 7. The compiler catches that one because it is out of range -- and would not catch "\x1b1", which compiles quietly into one wrong byte. Escape then digit is always two strings.

sl0ppty: re-emit what the program meant, not what it covers

2026-08-14T21:25:46Z

Images rendered after the last commit, and were still wrong twice. Both mistakes are invisible in a still picture, which is what every test was looking at, and obvious the moment anything moves. **The sub-cell offset was dropped.** A placement says where the image starts *inside* its first cell (`X=`/`Y=`); that is what makes motion smooth rather than a slideshow. We tracked the placement and re-emitted it without those, so a bouncing logo moved in steps of a whole cell -- eight pixels across, sixteen down. **`c=`/`r=` were passed on unconditionally.** They mean *scale the image into this many cells*, and the count a natural-size image happens to cover goes up by one whenever it straddles one more boundary -- so the picture changed size as it slid along. They are now sent only when the program asked for them, which lib-vt reports as 0 when it did not. Cropping splits the same way, and getting it wrong took a test failure to notice: at natural size a lost cell costs the screen pixels the pane has left (measured from the offset, not the cell edge, or an offset image hangs up to a cell over the border); when the program asked to scale, a lost cell is worth source/cells pixels and the crop is proportional. Cropping a 4-pixel image scaled across 40 cells by screen pixels removes all of it. gfx_place() took twenty positional arguments, half of them uint32_t pixel quantities where a transposed pair compiles perfectly and lands the picture slightly wrong -- the exact bug class this file exists for. It takes a struct now. `{"cmd":"graphics","format":"bytes"}` answers with the escape sequences the client is actually sent. The model being right is not the same as the bytes being right, and this whole episode was a lesson in asserting on the first while the second was wrong. The new tests assert on the bytes.

sl0ptty: kitty graphics passthrough

2026-08-13T23:01:04Z

A program in a pane transmits an image and it appears. libghostty-vt parses the protocol and tracks images and placements; what a multiplexer has to add is re-emitting them to the client, which is exactly what tmux and zellij do not do — and the reason tools fall back to sixel under them. Per frame: walk the visible tab's panes, transmit each image once (a=t), place every visible placement (a=p,C=1) after the cell diff so a repainted cell cannot land on top of one, and delete placements that vanished. Image ids are remapped into a high range, so two panes both using image 7 stay two images and neither collides with what the client's terminal already had. A reattaching client is told everything again, because it has seen nothing. The cropping rule is the part worth remembering: a placement is N pixels drawn across M cells, so the placement itself tells you the pixels per cell — the client's font metrics never come into it. Clipping at a pane edge, or at the top of the viewport when a tall image is partly scrolled off, moves the source rectangle; asking for fewer columns or rows alone scales the image into them instead of cropping it. The first version did exactly that, and dropped partially-visible placements entirely rather than cropping them. Tests cover the multiplexer-specific case first: two panes, both using image id 7, must get different client ids. Then placement position in screen cells, following the layout, cropping at edges and when taller than the pane, nothing from collapsed panes, and removal when a pane closes or the image scrolls away. A live check confirms the wire: transmitted once, re-placed per frame, not re-transmitted. 416 checks green (389 headless + 27 live).