diff options
| author | user <user@clank> | 2026-08-19 20:11:10 +0200 |
|---|---|---|
| committer | user <user@clank> | 2026-08-19 20:11:10 +0200 |
| commit | 76f3868cd7b775cb59b85a32c2b223732f6afc30 (patch) | |
| tree | e38858cabf624732ca48ae7a8b4c49e9af65728c | |
| parent | slosh: layout files are *.layout, not *.layout.kdl (diff) | |
| download | slosh-76f3868cd7b775cb59b85a32c2b223732f6afc30.tar.gz slosh-76f3868cd7b775cb59b85a32c2b223732f6afc30.tar.xz slosh-76f3868cd7b775cb59b85a32c2b223732f6afc30.zip | |
slosh: the website lives in web/ -- one directory, one build, make www
Consolidate everything the website is made of:
- web_new/ -> web/: the front page, install.sh, and build-site
- contrib/webdemo/ -> web/demo/: the in-browser machine, committed for
the first time (it had never been checked in)
- make www at the top level assembles the whole site into build/www
(NO_DEMO=1 skips the guest cross-compile)
- web_plan.md deleted: the plan is fulfilled and web/README.md is now
the record
Path fallout: the overlay theme symlink gains one ../ (web/demo is a
level shallower relative to contrib/themes), PYSRC in the Makefile
follows the demo scripts, contrib/README.md drops its webdemo section.
ROOT := ../.. in the demo Makefile survives unchanged -- web/demo sits
at the same depth contrib/webdemo did.
Verified: make www builds front page + docs + demo end to end,
web/demo make check boots the image to slosh, tests/test_docs.py and
make fmt-check green.
32 files changed, 2339 insertions, 54 deletions
@@ -53,7 +53,7 @@ else endif .PHONY: all clean vendor run test retest test-live test-all smoke help coverage \ - docs fmt fmt-check hooks tools + docs www fmt fmt-check hooks tools .DEFAULT_GOAL := help help: ## show this @@ -192,6 +192,9 @@ smoke: $(BIN) ## compose a screen headlessly and print it docs: ## render docs/ into build/docs (no dependencies; open build/docs/index.html) $(Q)python3 contrib/gen-docs build/docs +www: ## assemble the website into build/www (NO_DEMO=1 to skip the browser demo) + $(Q)web/build-site + run: $(BIN) ## build and run ./$(BIN) @@ -259,7 +262,7 @@ coverage: $(COV_DIR)/slosh $(COV_UNITS) ## how much of the code the tests run CSRC := $(shell git ls-files '*.c' '*.h' 2>/dev/null | grep -v '^vendor/') PYSRC := $(shell git ls-files '*.py' 2>/dev/null | grep -v '^vendor/') \ contrib/coverage contrib/gen-docs contrib/shader-repl \ - contrib/webdemo/boot-check contrib/webdemo/build-rootfs contrib/webdemo/serve + web/demo/boot-check web/demo/build-rootfs web/demo/serve # clang-format is a system package. ruff usually is not, and `pip install --user` # is refused outright on a distro python (PEP 668), so `make tools` puts it in a diff --git a/contrib/README.md b/contrib/README.md index 5809694..d781636 100644 --- a/contrib/README.md +++ b/contrib/README.md @@ -136,21 +136,6 @@ that computes zero everywhere, because an example that does nothing reads as a broken shader system rather than a bad example. It found its first bug on its first run. -## webdemo/ - -slosh in a browser tab, for people who want to try it without installing it: -a riscv64 Linux booted by Bellard's TinyEMU compiled to WebAssembly, an Alpine -userland we build into a disk image, and slosh as the thing you land in. - -```sh -cd contrib/webdemo && make && make serve -``` - -The output is static files, so it can be served from anywhere that can serve a -directory. `make check` boots the same image in a native emulator and asserts -the session comes up, which is how a change to the guest is tested without -opening a browser at all. - ## coverage ```sh diff --git a/web_new/README.md b/web/README.md index b272b28..deaab69 100644 --- a/web_new/README.md +++ b/web/README.md @@ -1,21 +1,22 @@ -# web_new — the slosh website +# web — the slosh website -One front page, plus the two things the repository already builds, assembled -into a single static site: +Everything the website is made of lives here (plus the docs generator, which +`make docs` shares). One front page, the documentation, and the in-browser +demo, assembled into a single static site: | path | what | comes from | |---|---|---| | `/` | the front page: what it is, the demo video, install | `index.html` + `site.css`, hand-written, no build step | | `/docs/` | the documentation | `contrib/gen-docs`, the same pages `make docs` renders | -| `/demo/` | the in-browser machine | `contrib/webdemo`, on its own page so the front page stays light | +| `/demo/` | the in-browser machine | `demo/`, on its own page so the front page stays light | | `/install.sh` | what `curl \| sudo sh` fetches | `install.sh` here | ## Build ```sh -web_new/build-site # everything -> build/www -NO_DEMO=1 web_new/build-site # without the demo (it cross-compiles a guest) -contrib/webdemo/serve --dir build/www +make www # everything -> build/www (same as web/build-site) +NO_DEMO=1 web/build-site # without the demo (it cross-compiles a guest) +web/demo/serve --dir build/www ``` The output is only static files. Any host that serves a directory works, with diff --git a/web_new/build-site b/web/build-site index df1a20d..9066456 100755 --- a/web_new/build-site +++ b/web/build-site @@ -1,17 +1,17 @@ #!/bin/sh # Assemble the whole website into build/www: # -# / web_new/ the front page (this directory) +# / web/ the front page (this directory) # /docs/ contrib/gen-docs the rendered documentation -# /demo/ contrib/webdemo the in-browser machine, on its own page +# /demo/ web/demo/ the in-browser machine, on its own page # -# web_new/build-site everything -# NO_DEMO=1 web_new/build-site skip the demo (it cross-compiles a -# guest and needs zig + e2fsprogs) +# web/build-site everything (or: make www) +# NO_DEMO=1 web/build-site skip the demo (it cross-compiles a +# guest and needs zig + e2fsprogs) # # The output is only static files. Serve it with anything that serves a # directory; the one condition is .wasm as application/wasm, which -# `contrib/webdemo/serve --dir build/www` handles locally. +# `web/demo/serve --dir build/www` handles locally. set -eu @@ -32,10 +32,10 @@ python3 "$root/contrib/gen-docs" "$out/docs" if [ "${NO_DEMO:-0}" = 1 ]; then echo "build-site: skipping demo (NO_DEMO=1)" else - make -C "$root/contrib/webdemo" site + make -C "$here/demo" site mkdir -p "$out/demo" - cp -R "$root/contrib/webdemo/build/site/." "$out/demo/" + cp -R "$here/demo/build/site/." "$out/demo/" fi echo "build-site: $out" -echo "build-site: serve with: contrib/webdemo/serve --dir build/www" +echo "build-site: serve with: web/demo/serve --dir build/www" diff --git a/web/demo/.gitignore b/web/demo/.gitignore new file mode 100644 index 0000000..0287b08 --- /dev/null +++ b/web/demo/.gitignore @@ -0,0 +1,6 @@ +# Downloaded upstream artifacts (fetch-vendor puts them here, by sha256) and +# everything the build produces. The emulator is MIT, but the kernel and the +# busybox userland are GPL binaries: we link to Bellard's tarball rather than +# redistribute them without their source. +vendor/ +build/ diff --git a/web/demo/Makefile b/web/demo/Makefile new file mode 100644 index 0000000..2272c46 --- /dev/null +++ b/web/demo/Makefile @@ -0,0 +1,166 @@ +# slosh in a browser tab. +# +# Bellard's TinyEMU compiled to wasm boots a riscv64 Linux; we hand it a root +# filesystem with a cross-compiled slosh in it. `make` builds the whole thing +# into build/site, `make serve` puts it on a port. +# +# Run this from web/demo (or `make -C web/demo`), not from the +# repository root: it is a separate build with a separate target triple and it +# deliberately shares nothing with the top-level one but the source files. + +ROOT := ../.. +ZIG ?= $(HOME)/zig-0.16.0/zig +CC := $(ZIG) cc +TARGET := riscv64-linux-musl + +VT := $(ROOT)/vendor/libghostty-vt +VT_OUT := build/vt +VT_LIB := $(VT_OUT)/lib/libghostty-vt.a +VT_INC := $(VT_OUT)/include + +SRC := $(wildcard $(ROOT)/src/*.c) +OBJ := $(patsubst $(ROOT)/src/%.c,build/obj/%.o,$(SRC)) +DEPS := $(OBJ:.o=.d) +GUEST := build/slosh-riscv64 + +# -Os, no -g: every byte of this binary is pulled through a virtio block device +# by somebody's browser, 256 KiB at a time. +CFLAGS := -target $(TARGET) -std=c23 -Os -Wall -Wextra -Wno-unused-parameter \ + -MMD -MP -I$(VT_INC) -I$(ROOT)/include -Ibuild +VERSION := 0.1.0 +GITDESC = $(shell cd $(ROOT) && git describe --always --dirty 2>/dev/null || echo unknown) + +SITE := build/site +IMAGE := build/rootfs.bin +IMAGE_MB ?= 48 +PORT ?= 8000 + +# What the browser downloads, beyond our own page. The emulator comes from +# emulator/dist -- ours, built from Bellard's source -- not from vendor/: +# his prebuilt blob wedges on disks larger than 16 MiB (see emulator/). +VENDOR_SITE := bbl64.bin kernel-riscv64.bin xterm.js xterm.css +EMU_SITE := emulator/dist/riscvemu64-wasm.js emulator/dist/riscvemu64-wasm.wasm +WEB_SITE := index.html demo.js demo.css slosh.cfg + +V ?= 0 +ifeq ($(V),0) + Q := @ + say = @printf ' %-5s %s\n' '$(1)' '$(2)' +else + Q := + say = @: +endif + +.PHONY: all vendor guest rootfs site serve check clean distclean help +.DEFAULT_GOAL := help + +help: ## show this + @grep -hE '^[a-z-]+:.*##' $(MAKEFILE_LIST) | sed 's/:.*##/\t/' | expand -t14 + @printf '\n%-13s %s\n' 'PORT=8080' 'which port `make serve` uses' + @printf '%-13s %s\n' 'IMAGE_MB=64' 'how big the guest filesystem is' + +all: site ## build everything into build/site + +# ── the third-party half ──────────────────────────────────────────────────── + +vendor: vendor/.stamp ## download the emulator, kernel, base image and xterm.js + +vendor/.stamp: fetch-vendor + $(Q)./fetch-vendor + @touch $@ + +# ── slosh, for a machine nobody here owns ───────────────────────────────── +# +# -Dsimd=false is not a size choice: Google Highway, which libghostty-vt uses +# for SIMD, does not compile for rv64gc -- it emits RVV intrinsics for a vector +# extension this target does not have. The scalar paths are the same code +# without the hand-vectorised fast path, which for a demo VM is free. +$(VT_LIB): + $(call say,[ZIG],libghostty-vt $(TARGET)) + $(Q)cd $(VT) && PATH="$(dir $(ZIG)):$$PATH" zig build \ + -Demit-lib-vt -Dsimd=false -Doptimize=ReleaseFast -Dtarget=$(TARGET) \ + --prefix $(abspath $(VT_OUT)) --cache-dir $(abspath build/vt-cache) + +# FORCE, as at the top level: rewritten only when the value actually changes, +# so a new commit rebuilds and `make` twice in a row does not. +.PHONY: FORCE +build/version.h: FORCE | build + @printf '/* generated by web/demo/Makefile */\n#define SLOSH_VERSION "%s+%s"\n' \ + '$(VERSION)' '$(GITDESC)' > $@.tmp + @cmp -s $@.tmp $@ || mv $@.tmp $@ + @rm -f $@.tmp + +build build/obj: + @mkdir -p build/obj + +# The logo, exactly as the top-level Makefile bakes it: logo.txt is the +# source of truth and the splash includes this. +build/logo.h: $(ROOT)/logo.txt | build + $(call say,[GEN],$@) + $(Q){ printf '/* generated from logo.txt; edit that, not this */\n'; \ + printf 'static const char *const LOGO[] = {\n'; \ + sed -e 's/\\\\/\\\\\\\\/g' -e 's/"/\\\\"/g' -e 's/^/ "/' -e 's/$$/",/' $(ROOT)/logo.txt; \ + printf '};\n'; } > $@ + +build/obj/%.o: $(ROOT)/src/%.c build/version.h build/logo.h $(VT_LIB) | build/obj + $(call say,[CC],$(notdir $<)) + $(Q)$(CC) $(CFLAGS) -c $< -o $@ + +# stb_image is third-party and does not build clean under our warnings. +build/obj/png.o: $(ROOT)/src/png.c build/version.h $(VT_LIB) | build/obj + $(call say,[CC],$(notdir $<)) + $(Q)$(CC) $(CFLAGS) -I$(VT)/src/stb -Wno-unused-function -c $< -o $@ + +# Static, because the guest has glibc 2.24 and we built against musl. It also +# means the binary is the only thing that has to land in the image. +$(GUEST): $(OBJ) $(VT_LIB) + $(call say,[LD],$@) + $(Q)$(CC) -target $(TARGET) -Os $(OBJ) $(VT_LIB) -o $@ -s + +guest: $(GUEST) ## cross-compile slosh for riscv64 + +# ── the guest filesystem ──────────────────────────────────────────────────── +# +# The kernel we use has no initramfs support (CONFIG_BLK_DEV_INITRD is off: it +# ignores an initrd and panics for want of a root), so the root filesystem is an +# ext2 image on a virtio disk, split into 256 KiB blocks that the emulator +# fetches over HTTP as the guest touches them. A visitor downloads the parts of +# the image they use, not all of it. +rootfs: $(IMAGE) ## build and split the guest filesystem + +$(IMAGE): $(GUEST) build-rootfs vendor/.stamp $(shell find overlay -type f 2>/dev/null) + $(Q)./build-rootfs --base vendor/alpine-minirootfs-riscv64.tar.gz --slosh $(GUEST) \ + --apks vendor/apks --overlay overlay --out $@ --split $(SITE)/rootfs --size $(IMAGE_MB)M + +# Everything except the page can be checked without a browser: same emulator, +# same kernel, same image, driven over a pty. `make check` before you open +# devtools, always. +check: $(IMAGE) ## boot the image in a native emulator and check slosh runs + $(Q)./boot-check + +# ── the page ──────────────────────────────────────────────────────────────── + +site: $(IMAGE) ## assemble build/site + @mkdir -p $(SITE) + $(Q)cp $(addprefix web/,$(WEB_SITE)) $(SITE)/ + $(Q)cp $(addprefix vendor/,$(VENDOR_SITE)) $(SITE)/ + $(Q)cp $(EMU_SITE) $(SITE)/ + $(Q)cp vendor/TINYEMU-MIT-LICENSE.txt $(SITE)/ + $(Q)for f in kernel-riscv64.bin bbl64.bin riscvemu64-wasm.js riscvemu64-wasm.wasm \ + xterm.js xterm.css demo.js demo.css; do \ + gzip -9 -n -c $(SITE)/$$f > $(SITE)/$$f.gz; \ + done + @printf ' site %s (%s; %s gzipped)\n' "$(SITE)" \ + "$$(du -sh $(SITE) | cut -f1)" \ + "$$(du -shc $(SITE)/*.gz $(SITE)/rootfs/*.gz 2>/dev/null | tail -1 | cut -f1)" + +serve: site ## serve build/site on http://localhost:$(PORT) + ./serve --dir $(SITE) --port $(PORT) + +clean: ## remove build/ (keeps the downloads) + rm -rf build + +distclean: clean ## also remove vendor/ + rm -rf vendor + +-include $(DEPS) diff --git a/web/demo/README.md b/web/demo/README.md new file mode 100644 index 0000000..8e3502f --- /dev/null +++ b/web/demo/README.md @@ -0,0 +1,176 @@ +# webdemo + +slosh in a browser tab: a riscv64 Linux, booted by Fabrice Bellard's TinyEMU +compiled to WebAssembly, with a root filesystem we build and slosh running as +the thing you land in. Nothing is installed, nothing is uploaded, and reloading +the page throws the machine away. + +```sh +cd web/demo +make all # download, cross-compile, build the filesystem, assemble the site +make serve # http://localhost:8000 +``` + +A bare `make` prints the target list, as it does at the top level. + +`make serve` is a static file server. The output in `build/site` is *only* +static files, so anything that serves a directory will do -- including GitHub +Pages, S3, or `python3 -m http.server` -- with one condition: `.wasm` must be +served as `application/wasm` or the browser will not instantiate it. That is +the single thing our `serve` does that a default configuration might not. + +It cannot be opened from `file://`: the emulator fetches the VM config and the +disk blocks with XMLHttpRequest. + +## What you need + +| | | +|---|---| +| zig 0.16 | `$HOME/zig-0.16.0/zig`, or `make ZIG=/path/to/zig` -- the same one the top-level build wants | +| e2fsprogs | `mke2fs`, `debugfs`, `dumpe2fs` -- all used unprivileged | +| curl | for `fetch-vendor` | +| gcc | only for `make check`, which builds the native emulator | + +No npm, no bundler, no emscripten, and no root at any point. + +## The targets + +| | | +|---|---| +| `make vendor` | download the emulator, kernel, Alpine rootfs and xterm.js into `vendor/`, each pinned by sha256 | +| `make guest` | cross-compile slosh to a static riscv64 binary | +| `make rootfs` | build the guest filesystem and cut it into HTTP blocks | +| `make site` | assemble `build/site` | +| `make check` | boot the image in a native TinyEMU and check slosh comes up -- no browser | +| `make serve` | serve it (`PORT=8080` to move it) | +| `make clean` | remove `build/`; `distclean` also removes the downloads | + +`./boot-check --interactive` boots the same image in your terminal and hands +you the keyboard, which is the fastest way to try a change to the overlay. +`C-a x` quits the emulator -- note that this is also slosh's leader key, so +type it twice if the session swallows the first. + +## How it fits together + +```mermaid +flowchart LR + A[src/*.c + libghostty-vt] -->|zig cc riscv64-linux-musl| B[static slosh] + C[Alpine minirootfs riscv64] --> D[build-rootfs] + E[overlay/] --> D + B --> D + D -->|mke2fs -d| F[rootfs.bin] + F -->|256 KiB blocks| G[build/site/rootfs/] + H[riscvemu64-wasm] --> I[browser] + G -->|fetched lazily over HTTP| I + J[web/ + xterm.js] --> I +``` + +Four decisions are worth knowing about, because each one was a dead end +first. + +**riscv64, not x86.** The x86 emulator in the JSLinux tarball is a prebuilt +blob with no source in TinyEMU's release -- `x86_cpu.c` there is a 96-line KVM +shim, not a software core -- and both its kernel and its userland are 32-bit +i386. riscv64 is the only target with published source, a 64-bit userland, and +a toolchain we already have: zig cross-compiles to `riscv64-linux-musl` out of +the box. + +**A disk, not an initramfs.** The guest kernel is Bellard's prebuilt Linux +4.15, and it is built without `CONFIG_BLK_DEV_INITRD`: an `initrd:` line in the +VM config is accepted, ignored, and followed by a panic for want of a root. +The root filesystem is therefore an ext2 image on a virtio disk, split into +256 KiB files fetched over plain HTTP. `blk.txt` lists every block in its +`prefetch:` array, so the whole disk streams in at startup while boot +proceeds -- boot-critical blocks sit at the front, a demand read of a block +already in flight joins it, and once the stream ends the session never +touches the network again. A cold block fetched mid-keystroke reads as the +demo hanging, which is worse than the ~11 MiB the stream costs. An +initramfs would buy the same bytes-up-front without the overlap: nothing +boots until all of it has arrived. + +Compression happens on the wire: the build writes a `.gz` twin beside every +block and every big asset, `serve` answers `Accept-Encoding: gzip` with the +twin and `Content-Encoding: gzip`, and XHR decodes before the emulator looks. +The emulator neither knows nor cares. Any competent static host or CDN does +the equivalent on its own. + +**xterm.js, not the term.js in the tarball.** Bellard's terminal has no +truecolor and no wide-character handling. slosh paints 24-bit colour shaders +and lays out CJK, so that terminal would misrepresent both. The emulator +expects a global `term` with `write` and `getSize`; `web/demo.js` provides those +two methods over xterm.js and nothing else. + +**Our emulator build, not Bellard's blob.** The prebuilt +`riscvemu64-wasm.js` in the JSLinux tarball does not match the source +released beside it: past 16 MiB into the disk it stops issuing block +fetches, so the first command to stat an inode in a later block group -- +`ls -la`, typically -- wedges or segfaults in the guest. His demo images are +4 MiB, so upstream never crossed the line; our 32 MiB image did. The same +source compiled natively serves the same blocks flawlessly, so +`emulator/build-emulator` compiles the emulator from the tarball's own +source (ported to a current emscripten; +`emulator/patches/0001-modern-emscripten.patch` is the whole port) and the +result is committed in `emulator/dist/` -- because it needs the emscripten +SDK to rebuild, and nothing else here does. It is also half the size of the +blob it replaces. + +## The guest + +Alpine's riscv64 minirootfs -- musl, busybox, `apk`, `less`, `top` -- plus a +real vim with syntax highlighting (the `vim` package from pinned `.apk`s, its +26 MiB runtime pruned to the ~10 that highlighting uses: `syntax/`, filetype +detection, colour schemes; no docs, no spell, no netrw), plus `overlay/`, +which is the whole of what makes it ours: + +| | | +|---|---| +| `etc/inittab` | four lines: run `rcS`, then respawn slosh on the console | +| `etc/init.d/rcS` | mounts, and `devpts` in particular: no devpts, no panes | +| `etc/profile` | the environment every pane inherits, `COLORTERM` included | +| `etc/motd` | five lines, shown in a pane rather than on the console | +| `root/.config/slosh/config.kdl` | the demo's config: the `sl0p` theme | +| `root/.vimrc` | syntax on, truecolor, and nothing that wants the pruned parts | +| `root/demo.layout` | the three panes you land in | + +The theme is a symlink to `contrib/themes/sl0p.kdl`, so the demo cannot drift +from the palette the repository ships; `build-rootfs` copies it by content, +because a symlink in the image would dangle. + +Editing anything in `overlay/` and running `make` rebuilds the image. There is +no state to clear: the emulator's writes live in the browser's memory and die +with the tab. + +## Known limits + +- **The terminal size is fixed at boot.** TinyEMU sends the console size once, + from `console_init()`, and exports no way to send another. The page therefore + measures the viewport once, tells the guest, and afterwards scales the fixed + grid with a CSS transform instead of reflowing it. `?cols=`/`?rows=` override + the choice. +- **No network.** There is no `eth0` in the VM config, so `apk add` cannot + reach a repository. Everything in the demo has to be in the image. +- **Terminfo is an alias.** The image's only terminfo entries are ncurses' + base set, with `xterm-ghostty` (the `TERM` slosh sets in panes, src/pty.c) + symlinked to `xterm-256color` -- the honest subset. Truecolor still happens: + `COLORTERM=truecolor` is exported and vim's `termguicolors` is on. +- **It is an emulator.** A keystroke costs an interpreted instruction stream; + the session is responsive but the machine is not fast, and a `make` inside it + would take a while. + +## Provenance + +Nothing in `vendor/` is committed. `fetch-vendor` downloads each piece and +checks it against a sha256 recorded in the script, tarball and extracted member +both: + +- TinyEMU / JSLinux -- BIOS, kernel, and the emulator *source*. MIT, Fabrice + Bellard, <https://bellard.org/tinyemu/>. The licence text is copied into + `build/site` next to what it covers. +- Alpine Linux riscv64 minirootfs, <https://alpinelinux.org>. +- xterm.js 5.5.0, MIT. + +The one binary this directory commits is `emulator/dist/riscvemu64-wasm.{js,wasm}` +-- MIT, built from the pinned TinyEMU tarball by `emulator/build-emulator`, +reproducible from the patch beside it. The kernel and the Alpine userland are +GPL binaries; a repository that shipped those would owe their source, so we +point at their publishers instead. diff --git a/web/demo/boot-check b/web/demo/boot-check new file mode 100755 index 0000000..94db9c2 --- /dev/null +++ b/web/demo/boot-check @@ -0,0 +1,333 @@ +#!/usr/bin/env python3 +"""Boot the demo filesystem in a native TinyEMU and check slosh runs. + +The browser is the target, but it is a terrible place to find out that the +guest binary does not start: the emulator, the kernel and the image are the +same in both, so everything except the page can be checked here in about +twenty seconds, without a browser and without a web server. + +The same tarball fetch-vendor already downloaded for its licence text is the +native emulator's source, so this costs one gcc build the first time and +nothing after that. +""" + +import argparse +import os +import pty +import re +import select +import shutil +import signal +import subprocess +import sys +import tarfile +import time + +HERE = os.path.dirname(os.path.abspath(__file__)) +BUILD = os.path.join(HERE, "build") +TEMU_SRC = os.path.join(BUILD, "tinyemu") +TEMU = os.path.join(TEMU_SRC, "temu") +TARBALL = os.path.join(HERE, "vendor", ".cache", "tinyemu.tar.gz") + + +def build_temu(): + """Build the native emulator, without the parts that need libraries. + + CONFIG_FS_NET wants libcurl and openssl and exists to fetch a filesystem + over HTTP -- which is exactly what the browser does and exactly what we do + not, because here the image is a file. CONFIG_SDL wants SDL for a graphical + display we never open. Turning both off leaves a build that needs a C + compiler and nothing else. + """ + if os.path.exists(TEMU): + return + if not os.path.exists(TARBALL): + sys.exit(f"{TARBALL} is missing -- run ./fetch-vendor first") + print(" building the native emulator (once)") + if os.path.exists(TEMU_SRC): + shutil.rmtree(TEMU_SRC) + os.makedirs(TEMU_SRC) + with tarfile.open(TARBALL) as tf: + for m in tf.getmembers(): + parts = m.name.split("/", 1) + if len(parts) == 2 and parts[1]: + m.name = parts[1] + tf.extract(m, TEMU_SRC, filter="data") + mk = os.path.join(TEMU_SRC, "Makefile") + with open(mk) as f: + text = f.read() + text = text.replace("\nCONFIG_FS_NET=y", "\n#CONFIG_FS_NET=y") + text = text.replace("\nCONFIG_SDL=y", "\n#CONFIG_SDL=y") + with open(mk, "w") as f: + f.write(text) + r = subprocess.run( + ["make", "-s", f"-j{os.cpu_count() or 4}", "temu"], + cwd=TEMU_SRC, + capture_output=True, + text=True, + ) + if not os.path.exists(TEMU): + sys.exit(f"temu did not build:\n{r.stdout[-2000:]}\n{r.stderr[-2000:]}") + + +def write_cfg(path, image, kernel, bios, cmdline, mem): + with open(path, "w") as f: + f.write( + "{\n" + " version: 1,\n" + ' machine: "riscv64",\n' + f" memory_size: {mem},\n" + f' bios: "{bios}",\n' + f' kernel: "{kernel}",\n' + f' cmdline: "{cmdline}",\n' + f' drive0: {{ file: "{image}" }},\n' + "}\n" + ) + + +class VM: + """A booted guest on a pty, driven a line at a time.""" + + def __init__(self, cfg, cols=100, rows=30): + env = dict( + os.environ, TERM="xterm-256color", COLUMNS=str(cols), LINES=str(rows) + ) + self.pid, self.fd = pty.fork() + if self.pid == 0: + os.execve(TEMU, [TEMU, cfg], env) + os._exit(127) + self.buf = "" + + def read(self, deadline): + while True: + timeout = deadline - time.time() + if timeout <= 0: + return False + r, _, _ = select.select([self.fd], [], [], min(timeout, 0.5)) + if not r: + continue + try: + data = os.read(self.fd, 65536) + except OSError: + return False + if not data: + return False + self.buf += data.decode("utf-8", "replace") + return True + + def expect(self, pattern, timeout=60): + rx = re.compile(pattern, re.S) + deadline = time.time() + timeout + while True: + m = rx.search(self.buf) + if m: + return m + if not self.read(deadline): + return None + + def send(self, s): + os.write(self.fd, s.encode()) + + def close(self): + try: + os.kill(self.pid, signal.SIGKILL) + os.waitpid(self.pid, 0) + except OSError: + pass + try: + os.close(self.fd) + except OSError: + pass + + def tail(self, n=40): + return "\n".join(self.buf.replace("\r", "").splitlines()[-n:]) + + +def phase_shell(cfg_dir, image, kernel, bios, mem, timeout): + """Boot straight to a shell and put slosh through its paces. + + init=/bin/sh deliberately skips our inittab: this phase is about the + binary, not about the boot scripts, and a failure here means the + cross-compile is wrong rather than the overlay. + """ + cfg = os.path.join(cfg_dir, "boot-check-shell.cfg") + write_cfg( + cfg, image, kernel, bios, "console=hvc0 root=/dev/vda rw init=/bin/sh", mem + ) + vm = VM(cfg) + try: + if not vm.expect(r"[#$] $|/ #|# ", timeout=timeout): + return "no shell prompt after boot", vm.tail() + # devpts is what /etc/init.d/rcS does for the real boot; slosh cannot + # open a pane without it, so the failure is worth reproducing exactly. + vm.send( + "mkdir -p /dev/pts && mount -t devpts -o ptmxmode=0666 devpts /dev/pts\n" + ) + vm.send("echo MOUNT_RC=$?\n") + if not vm.expect(r"MOUNT_RC=0", timeout=20): + return "devpts would not mount", vm.tail() + + vm.send("/usr/bin/slosh --version; echo VERSION_RC=$?\n") + if not vm.expect(r"VERSION_RC=0", timeout=30): + return "slosh --version failed", vm.tail() + + # The same composition the top-level `make smoke` does, in the guest: + # a real pty, a real child process, a screen composed and printed. The + # child sleeps because a process that exits before the first compose + # races it: the pane is then drawn already dead, showing "[process + # exited]" and none of its output. That race exists on the host too -- + # it is not something the emulator introduced. + vm.send( + "/usr/bin/slosh --headless --cols 40 --rows 8 -- " + "/bin/sh -c 'printf \"HEADLESS-OK\\n\"; sleep 2'; echo SMOKE_RC=$?\n" + ) + if not vm.expect(r"HEADLESS-OK", timeout=60): + return "headless compose produced no output", vm.tail() + if not vm.expect(r"SMOKE_RC=0", timeout=30): + return "headless compose exited non-zero", vm.tail() + + # The config and layout we ship are read by the guest's own binary + # rather than by the one on this machine, which is the only reading + # that counts -- `--check` is what it does to them at startup anyway. + for path in ("/root/.config/slosh/config.kdl", "/root/demo.layout"): + vm.send(f"/usr/bin/slosh --check {path}; echo CHECK_RC=$?\n") + if not vm.expect(r"CHECK_RC=0", timeout=30): + return f"{path} did not pass --check", vm.tail() + + # vim, with the runtime pruned to syntax highlighting: prove the prune + # kept enough. Batch ex mode loads synload.vim and filetype detection + # exactly the way `:syntax on` does, and writes what it found -- if + # the autoload/dist detection functions or syntax/ went missing, this + # is what breaks, not `vim --version`. + vm.send( + "HOME=/root TERM=xterm-ghostty vim -es" + " -c 'syntax on' -c 'e /etc/init.d/rcS'" + ' -c \'call writefile([exists("g:syntax_on")."-".&filetype], "/tmp/vimprobe")\'' + " -c 'qa!'; cat /tmp/vimprobe; echo VIM_RC=$?\n" + ) + if not vm.expect(r"1-sh.*VIM_RC=0", timeout=60): + return "vim did not come up highlighting (want '1-sh')", vm.tail() + + # And once more full-screen in a pane, the way a visitor runs it, + # because -es skips plugin loading and cannot see a plugin startup + # trips over (the pruned netrw's loader did exactly that, with E919, + # while the batch probe stayed green). --idle-ms is load-bearing: + # headless composes at the first quiet moment and exits, and both a + # half-started vim and a vim blocked at its hit-enter prompt are + # quiet. Four seconds lets startup finish, so the one frame shows + # either the file's text or the error that stopped it. + mark = len(vm.buf) + vm.send( + ". /etc/profile; /usr/bin/slosh --headless --idle-ms 4000 " + "--cols 70 --rows 16 -- vim /etc/init.d/rcS; echo FS''VIM_END\n" + ) + if not vm.expect(r"FSVIM_END", timeout=120): + return "full-screen vim run did not finish", vm.tail(30) + frame = vm.buf[mark:] + if re.search(r"E\d{2,4}:", frame): + return "vim startup raised an error in a pane", vm.tail(30) + if "#!/bin/sh" not in frame: + return "vim did not paint the file in a pane", vm.tail(30) + return None, vm.tail(12) + finally: + vm.close() + + +def phase_boot(cfg_dir, image, kernel, bios, mem, expect, timeout): + """Boot the way a visitor will, and wait for what they should see.""" + cfg = os.path.join(cfg_dir, "boot-check-boot.cfg") + write_cfg(cfg, image, kernel, bios, "console=hvc0 root=/dev/vda rw", mem) + vm = VM(cfg) + try: + # Either outcome ends the wait: a panic is a result, not a timeout, and + # sitting out the full two minutes to report one is a waste of a loop. + m = vm.expect(rf"(?P<panic>Kernel panic)|(?P<ok>{expect})", timeout=timeout) + if m is None: + return f"never saw {expect!r} on the console", vm.tail() + if m.group("panic"): + return "kernel panic", vm.tail() + return None, vm.tail(20) + finally: + vm.close() + + +def main(): + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--image", default=os.path.join(BUILD, "rootfs.bin")) + ap.add_argument( + "--kernel", default=os.path.join(HERE, "vendor", "kernel-riscv64.bin") + ) + ap.add_argument("--bios", default=os.path.join(HERE, "vendor", "bbl64.bin")) + ap.add_argument("--mem", type=int, default=128, help="guest memory in MB") + ap.add_argument("--timeout", type=int, default=120) + ap.add_argument( + "--expect", + default=r"pane \d+/\d+", + # The status line, which only exists once a session has composed a + # screen. Matching the word "slosh" would also match the error a + # failing one prints. + help="regex the normal boot must print on the console", + ) + ap.add_argument("--phase", choices=("shell", "boot", "both"), default="both") + ap.add_argument( + "--interactive", + action="store_true", + help="boot it and hand over the keyboard (C-a x quits temu)", + ) + args = ap.parse_args() + + # Relative paths in a VM config are resolved against the config file, which + # lives in build/ -- so `--image build/rootfs.bin` would be looked for in + # build/build/. Absolute paths sidestep the question entirely. + args.image = os.path.abspath(args.image) + args.kernel = os.path.abspath(args.kernel) + args.bios = os.path.abspath(args.bios) + for p in (args.image, args.kernel, args.bios): + if not os.path.exists(p): + sys.exit(f"{p} is missing -- run `make` first") + build_temu() + + if args.interactive: + cfg = os.path.join(BUILD, "boot-check-interactive.cfg") + write_cfg( + cfg, + args.image, + args.kernel, + args.bios, + "console=hvc0 root=/dev/vda rw", + args.mem, + ) + os.execv(TEMU, [TEMU, cfg]) + + failures = 0 + if args.phase in ("shell", "both"): + err, tail = phase_shell( + BUILD, args.image, args.kernel, args.bios, args.mem, args.timeout + ) + print( + f" {'FAIL' if err else 'ok '} guest binary {err or 'runs, opens a pty, composes a screen'}" + ) + if err: + failures += 1 + print(tail) + if args.phase in ("boot", "both"): + err, tail = phase_boot( + BUILD, + args.image, + args.kernel, + args.bios, + args.mem, + args.expect, + args.timeout, + ) + print( + f" {'FAIL' if err else 'ok '} normal boot {err or 'init reaches slosh'}" + ) + if err: + failures += 1 + print(tail) + return 1 if failures else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/web/demo/build-rootfs b/web/demo/build-rootfs new file mode 100755 index 0000000..bec1e98 --- /dev/null +++ b/web/demo/build-rootfs @@ -0,0 +1,377 @@ +#!/usr/bin/env python3 +"""Assemble the guest filesystem: Alpine + slosh + overlay, then split it. + +Unprivileged from end to end -- no loop mount, no chroot, no sudo. The tarball +is unpacked into a staging directory, our files are laid on top, and `mke2fs +-d` turns the whole tree into an ext2 image in one go. The one thing that +needs root, and does not get it, is ownership: everything mke2fs writes is +owned by whoever ran it, so a debugfs pass sets every inode back to root. + +The output is two things: build/rootfs.bin, which `boot-check` hands to a +native emulator, and build/site/rootfs/, the same image cut into 256 KiB +pieces with an index -- which is how TinyEMU's HTTP block device reads a disk +off a plain static web server, fetching only the blocks the guest touches. +""" + +import argparse +import os +import re +import shutil +import stat +import subprocess +import sys +import tarfile + +# ext2, and no feature the 2018-vintage guest kernel might not know. Pinned +# rather than left to mke2fs's defaults so that a newer e2fsprogs on somebody +# else's machine cannot quietly produce an image this kernel will not mount. +FS_FEATURES = "^metadata_csum,^64bit,^huge_file,^dir_nlink,^extra_isize,^resize_inode" +FS_BLOCK = 1024 + + +def need(*tools): + missing = [t for t in tools if shutil.which(t) is None] + if missing: + sys.exit( + f"missing: {', '.join(missing)}\n" + "these come from e2fsprogs (apt install e2fsprogs)" + ) + + +def run(cmd, **kw): + return subprocess.run(cmd, capture_output=True, text=True, **kw) + + +def parse_size(s): + m = re.fullmatch(r"(\d+)([KMG]?)", s.strip(), re.I) + if not m: + sys.exit(f"cannot read a size from {s!r} (try 32M)") + return ( + int(m.group(1)) + * {"": 1, "K": 1024, "M": 1024**2, "G": 1024**3}[m.group(2).upper()] + ) + + +# What of vim's 26 MiB runtime actually serves syntax highlighting: the syntax +# definitions themselves, the filetype detection they key off (filetype.vim +# calls into autoload/dist/), the colour schemes, and the handful of top-level +# scripts `:syntax on` sources. Docs, spell, translations, GUI menus, indent +# and per-filetype plugins are none of that. +VIM_PRUNE = ( + "usr/share/vim/vim91/doc", + "usr/share/vim/vim91/spell", + "usr/share/vim/vim91/lang", + "usr/share/vim/vim91/tutor", + "usr/share/vim/vim91/pack", + "usr/share/vim/vim91/indent", + "usr/share/vim/vim91/ftplugin", + "usr/share/vim/vim91/compiler", + "usr/share/vim/vim91/keymap", + "usr/share/vim/vim91/print", + "usr/share/vim/vim91/macros", + "usr/share/vim/vim91/tools", + "usr/share/vim/vim91/menu.vim", + "usr/share/vim/vim91/synmenu.vim", + "usr/share/vim/vim91/optwin.vim", + # vim 9.1 moved netrw itself into pack/, which is pruned above; the loader + # left in plugin/ then greets every interactive start with E919. A + # directory browser is not syntax highlighting. + "usr/share/vim/vim91/plugin/netrwPlugin.vim", +) + + +def unpack_apks(apk_dir, root): + """Unpack pinned .apk files into the staging tree. + + An .apk is three concatenated gzip streams -- signature, control, data -- + and Python's gzip reads straight across the joins, so the whole thing + parses as one tar with some `.SIGN`/`.PKGINFO` entries to skip. No + apk-tools, no package database in the guest: on a machine with no network + and a lifetime of one browser tab, `apk list` has nobody to answer to. + """ + import gzip + import io + + for name in sorted(os.listdir(apk_dir)): + if not name.endswith(".apk"): + continue + with gzip.open(os.path.join(apk_dir, name)) as gz: + data = gz.read() + with tarfile.open(fileobj=io.BytesIO(data), mode="r:", ignore_zeros=True) as tf: + for m in tf.getmembers(): + if m.name.startswith(".") or m.isdev(): + continue + # The minirootfs seeds busybox applets as absolute symlinks + # (usr/bin/xxd -> /bin/busybox); extracting the real binary + # through one both fails tarfile's containment check and would + # be wrong. Replacing the applet is the point of the package. + dest = os.path.join(root, m.name) + if not m.isdir() and os.path.lexists(dest) and not os.path.isdir(dest): + os.remove(dest) + tf.extract(m, root, filter="tar") + + for rel in VIM_PRUNE: + p = os.path.join(root, rel) + if os.path.isdir(p): + shutil.rmtree(p) + elif os.path.exists(p): + os.remove(p) + + +def stage(base, apks, overlay, slosh, root): + """Unpack Alpine, lay our files on top, install the binary. + + Nothing here needs privilege: the minirootfs tarball contains no device + nodes (the kernel mounts devtmpfs over /dev anyway) and no setuid bits. + """ + if os.path.exists(root): + shutil.rmtree(root) + os.makedirs(root) + + with tarfile.open(base) as tf: + for m in tf.getmembers(): + if m.isdev(): + continue + tf.extract(m, root, filter="tar") + + if apks: + unpack_apks(apks, root) + # Panes get TERM=xterm-ghostty (src/pty.c), and no packaged terminfo + # knows that name. An alias to xterm-256color is the honest subset: + # slosh's VT speaks everything that entry claims, and vim gets its + # colours from COLORTERM=truecolor rather than from the palette here. + # Alpine's ncurses keeps its base entries in /etc/terminfo. + ti = os.path.join(root, "etc/terminfo/x") + if not os.path.isdir(ti): + sys.exit("ncurses-terminfo-base did not provide etc/terminfo/x") + if not os.path.lexists(os.path.join(ti, "xterm-ghostty")): + os.symlink("xterm-256color", os.path.join(ti, "xterm-ghostty")) + + installed = [] + overlay = os.path.abspath(overlay) + for dirpath, _dirs, files in os.walk(overlay): + for name in sorted(files): + host = os.path.join(dirpath, name) + rel = os.path.relpath(host, overlay) + dest = os.path.join(root, rel) + real = os.path.realpath(host) + if not os.path.exists(real): + sys.exit(f"{host} points at {real}, which is not there") + os.makedirs(os.path.dirname(dest), exist_ok=True) + shutil.copyfile(real, dest) + # A checkout on a filesystem with no permission bits, or a zip + # download, would otherwise produce an init script that cannot run. + executable = os.access(real, os.X_OK) or rel.split(os.sep)[0] in ( + "sbin", + "bin", + ) + os.chmod(dest, 0o755 if executable else 0o644) + installed.append(("/" + rel.replace(os.sep, "/"), os.path.getsize(real))) + + dest = os.path.join(root, "usr/bin/slosh") + os.makedirs(os.path.dirname(dest), exist_ok=True) + shutil.copyfile(slosh, dest) + os.chmod(dest, 0o755) + installed.append(("/usr/bin/slosh", os.path.getsize(slosh))) + if apks: + vim = os.path.join(root, "usr/bin/vim") + if not os.path.exists(vim): + sys.exit("apks were given but usr/bin/vim did not appear") + installed.append(("/usr/bin/vim", os.path.getsize(vim))) + + # Mountpoints the guest needs to exist before anything can mount on them. + for d in ("dev/pts", "proc", "sys", "tmp", "var/log", "root"): + os.makedirs(os.path.join(root, d), exist_ok=True) + os.chmod(os.path.join(root, "tmp"), 0o1777) + return installed + + +def mkfs(root, image, size): + if os.path.exists(image): + os.remove(image) + r = run( + [ + "mke2fs", + "-q", + "-F", + "-t", + "ext2", + "-b", + str(FS_BLOCK), + "-O", + FS_FEATURES, + "-d", + root, + "-E", + "root_owner=0:0", + "-L", + "slosh-demo", + image, + str(size // FS_BLOCK), + ] + ) + if r.returncode != 0 or not os.path.exists(image): + sys.exit(f"mke2fs failed:\n{r.stdout}{r.stderr}") + + +def chown_root(root, image): + """Give every inode back to root. + + mke2fs -d copies the staging tree's ownership, and the staging tree + belongs to whoever ran the build. Left alone, the guest boots with a + userland owned by uid 1000: root can still read and execute all of it, so + it works, but `ls -l` tells a lie about the machine and anything that + checks ownership is wrong. One debugfs run fixes the lot. + """ + paths = ["/"] + for dirpath, dirs, files in os.walk(root): + for name in sorted(dirs) + sorted(files): + p = os.path.join(dirpath, name) + paths.append("/" + os.path.relpath(p, root).replace(os.sep, "/")) + script = "".join(f"sif {p} uid 0\nsif {p} gid 0\n" for p in paths) + r = run(["debugfs", "-w", "-f", "/dev/stdin", image], input=script) + if "Bad magic" in (r.stdout + r.stderr): + sys.exit(f"debugfs would not open the image:\n{r.stdout}{r.stderr}") + return len(paths) + + +STAT_SIZE = re.compile(r"\bSize: (\d+)") +STAT_MODE = re.compile(r"\bMode:\s+(\d+)") +STAT_USER = re.compile(r"\bUser:\s+(\d+)") + + +def verify(image, installed): + """Read our own files back out of the finished image. + + Presence is not the check that matters. A build that ran out of space + leaves a file with the right name, the right mode and no contents, which + boots into something that looks installed and is not -- so this compares + sizes, and insists the binary is executable and owned by root. + """ + bad = [] + for guest, want in installed: + out = run(["debugfs", "-R", f"stat {guest}", image]).stdout + size, mode, user = ( + STAT_SIZE.search(out), + STAT_MODE.search(out), + STAT_USER.search(out), + ) + if not (size and mode and user): + bad.append(f"{guest}: not in the image") + continue + if int(size.group(1)) != want: + bad.append(f"{guest}: {size.group(1)} bytes in the image, {want} staged") + if user.group(1) != "0": + bad.append(f"{guest}: owned by uid {user.group(1)}") + if guest.endswith("/slosh") and not int(mode.group(1), 8) & stat.S_IXUSR: + bad.append(f"{guest}: mode {mode.group(1)}, not executable") + if bad: + sys.exit("the image is not what it should be:\n " + "\n ".join(bad)) + + +def split(image, outdir, block_kib): + """Cut the image up the way TinyEMU's splitimg does, index and all. + + Each block also gets a `.gz` twin. The emulator cannot be taught about + compression -- but it never has to be: `serve` (or any host that can map + Accept-Encoding onto a precompressed file) sends the .gz with + Content-Encoding: gzip, XHR decodes it before the emulator looks, and the + wire carries about half the bytes. mtime=0 so a rebuild of identical + content produces identical files. + """ + import gzip + + block = block_kib * 1024 + os.makedirs(outdir, exist_ok=True) + # A smaller image than last time would otherwise leave its tail behind and + # the guest would read a stale block as though it were live disk. + for stale in os.listdir(outdir): + if re.fullmatch(r"blk\d+\.bin(\.gz)?", stale): + os.remove(os.path.join(outdir, stale)) + n = 0 + with open(image, "rb") as f: + while True: + buf = f.read(block) + if not buf: + break + with open(os.path.join(outdir, f"blk{n:09d}.bin"), "wb") as out: + out.write(buf) + with open(os.path.join(outdir, f"blk{n:09d}.bin.gz"), "wb") as out: + with gzip.GzipFile( + fileobj=out, mode="wb", compresslevel=9, mtime=0 + ) as gz: + gz.write(buf) + n += 1 + # Every block, ascending, in the prefetch list: the emulator streams the + # whole disk in at device init while boot proceeds -- a demand read of a + # block already in flight joins it rather than re-requesting. Ascending + # order front-loads what boot touches (superblock, early inode tables, + # /bin and /lib), so time-to-prompt stays demand-paced and the tail of + # the disk arrives behind it. Once the stream ends, the session never + # touches the network again -- which is the point: a mid-session pause + # for a cold block reads as the demo hanging. + with open(os.path.join(outdir, "blk.txt"), "w") as f: + blocks = ",".join(str(i) for i in range(n)) + f.write( + "{\n block_size: %d,\n n_block: %d,\n prefetch: [ %s ],\n}\n" + % (block_kib, n, blocks) + ) + return n + + +def free_space(image): + r = run(["dumpe2fs", "-h", image]) + out = r.stdout + r.stderr + free = re.search(r"Free blocks:\s+(\d+)", out) + return "%.1f MiB" % (int(free.group(1)) * FS_BLOCK / 1024 / 1024) if free else "?" + + +def main(): + ap = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + ap.add_argument("--base", required=True, help="Alpine minirootfs tarball") + ap.add_argument("--slosh", required=True, help="the riscv64 binary to install") + ap.add_argument("--overlay", required=True, help="tree to copy in at /") + ap.add_argument( + "--apks", default=None, help="directory of pinned .apk files to unpack" + ) + ap.add_argument("--out", required=True, help="image to write") + ap.add_argument("--split", required=True, help="directory for the HTTP blocks") + ap.add_argument( + "--stage", default=None, help="staging directory (default: beside --out)" + ) + ap.add_argument("--size", default="32M", help="image size (default 32M)") + ap.add_argument( + "--block-size", type=int, default=256, help="block KiB (default 256)" + ) + args = ap.parse_args() + + need("mke2fs", "debugfs", "dumpe2fs") + for p in (args.base, args.slosh, args.overlay): + if not os.path.exists(p): + sys.exit(f"{p} is missing") + + out = os.path.abspath(args.out) + os.makedirs(os.path.dirname(out), exist_ok=True) + root = args.stage or os.path.join(os.path.dirname(out), "root") + + installed = stage(args.base, args.apks, args.overlay, args.slosh, root) + mkfs(root, out, parse_size(args.size)) + n_inodes = chown_root(root, out) + verify(out, installed) + n = split(out, args.split, args.block_size) + + print( + " fs %s: %d ours of %d files, %s free, %d x %d KiB blocks" + % ( + os.path.relpath(out), + len(installed), + n_inodes, + free_space(out), + n, + args.block_size, + ) + ) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/web/demo/emulator/build-emulator b/web/demo/emulator/build-emulator new file mode 100755 index 0000000..a47df6c --- /dev/null +++ b/web/demo/emulator/build-emulator @@ -0,0 +1,47 @@ +#!/bin/sh +# Rebuild dist/riscvemu64-wasm.{js,wasm} from the TinyEMU source tarball. +# +# The demo does NOT use Bellard's prebuilt emulator, and this is why: his +# 2019 wasm blob does not match the 2019 source release. Compiled natively, +# that source serves our 32 MiB image over HTTP flawlessly; his blob stops +# issuing block fetches at the 16 MiB boundary, so the first read past it -- +# `ls -la` stat-ing an inode table in a later block group -- wedges in the +# guest or, depending on timing, segfaults it. His own images are 4 MiB, so +# the public demo never crossed the line. Building from the released source +# gives us the code we actually audited, and it comes out half the size. +# +# The output is committed in dist/, because rebuilding it needs the +# emscripten SDK and nothing else in this demo does. Run this only to change +# or verify the emulator: +# +# . /path/to/emsdk/emsdk_env.sh && ./build-emulator +# +# patches/0001-modern-emscripten.patch is the whole port: emcc 6.x lost +# Pointer_stringify and Runtime.dynCall, dead-strips JS library functions +# that lack __deps, and no longer coerces JS strings passed as wasm i32 +# parameters. The C side is untouched except renaming our XHR helper so it +# cannot collide with emscripten's own library symbol of the same name. +set -eu + +here=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +tarball="$here/../vendor/.cache/tinyemu.tar.gz" +work="$here/../build/emulator-src" + +command -v emcc >/dev/null 2>&1 || { + echo "emcc not found -- source emsdk_env.sh first (https://emscripten.org)" >&2 + exit 1 +} +[ -f "$tarball" ] || { + echo "$tarball is missing -- run ../fetch-vendor first" >&2 + exit 1 +} + +rm -rf "$work" +mkdir -p "$work" +tar -xzf "$tarball" -C "$work" --strip-components=1 +patch -d "$work" -p1 -s <"$here/patches/0001-modern-emscripten.patch" + +make -C "$work" -f Makefile.emcc -j"$(nproc 2>/dev/null || echo 4)" >/dev/null + +cp "$work/riscvemu64-wasm.js" "$work/riscvemu64-wasm.wasm" "$here/dist/" +sha256sum "$here/dist/riscvemu64-wasm.js" "$here/dist/riscvemu64-wasm.wasm" diff --git a/web/demo/emulator/dist/riscvemu64-wasm.js b/web/demo/emulator/dist/riscvemu64-wasm.js new file mode 100644 index 0000000..4998b67 --- /dev/null +++ b/web/demo/emulator/dist/riscvemu64-wasm.js @@ -0,0 +1 @@ +var Module=typeof Module!="undefined"?Module:{};var ENVIRONMENT_IS_WEB=true;var ENVIRONMENT_IS_WORKER=false;var programArgs=[];var thisProgram="./this.program";var quit_=(status,toThrow)=>{throw toThrow};var _scriptName=globalThis.document?.currentScript?.src;var scriptDirectory="";function locateFile(path){if(Module["locateFile"]){return Module["locateFile"](path,scriptDirectory)}return scriptDirectory+path}var readAsync,readBinary;if(ENVIRONMENT_IS_WEB||ENVIRONMENT_IS_WORKER){try{scriptDirectory=new URL(".",_scriptName).href}catch{}{readAsync=async url=>{var response=await fetch(url,{credentials:"same-origin"});if(response.ok){return response.arrayBuffer()}throw new Error(response.status+" : "+response.url)}}}else{}var out=console.log.bind(console);var err=console.error.bind(console);var wasmBinary;var ABORT=false;var EXITSTATUS;class EmscriptenEH{}class EmscriptenSjLj extends EmscriptenEH{}var runtimeInitialized=false;function getMemoryBuffer(){return wasmMemory.buffer}function updateMemoryViews(){if(HEAP8?.buffer?.resizable)return;var b=getMemoryBuffer();HEAP8=new Int8Array(b);HEAPU8=new Uint8Array(b);HEAP32=new Int32Array(b);HEAPU32=new Uint32Array(b);HEAP64=new BigInt64Array(b)}function preRun(){var preRun=Module["preRun"];if(preRun){if(typeof preRun=="function")preRun=[preRun];onPreRuns.push(...preRun)}callRuntimeCallbacks(onPreRuns)}function initRuntime(){runtimeInitialized=true;wasmExports["B"]()}function postRun(){var postRun=Module["postRun"];if(postRun){if(typeof postRun=="function")postRun=[postRun];onPostRuns.push(...postRun)}callRuntimeCallbacks(onPostRuns)}function abort(what){Module["onAbort"]?.(what);what=`Aborted(${what})`;err(what);ABORT=true;what+=". Build with -sASSERTIONS for more info.";var e=new WebAssembly.RuntimeError(what);throw e}var wasmBinaryFile;function findWasmBinary(){return locateFile("riscvemu64-wasm.wasm")}function getBinarySync(file){if(readBinary){return readBinary(file)}throw"both async and sync fetching of the wasm failed"}async function getWasmBinary(binaryFile){if(!wasmBinary){try{var response=await readAsync(binaryFile);return new Uint8Array(response)}catch{}}return getBinarySync(binaryFile)}async function instantiateArrayBuffer(binaryFile,imports){try{var binary=await getWasmBinary(binaryFile);var instance=await WebAssembly.instantiate(binary,imports);return instance}catch(reason){err(`failed to asynchronously prepare wasm: ${reason}`);abort(reason)}}async function instantiateAsync(binary,binaryFile,imports){if(!binary){try{var response=fetch(binaryFile,{credentials:"same-origin"});var instantiationResult=await WebAssembly.instantiateStreaming(response,imports);return instantiationResult}catch(reason){err(`wasm streaming compile failed: ${reason}`);err("falling back to ArrayBuffer instantiation")}}return instantiateArrayBuffer(binaryFile,imports)}function getWasmImports(){var imports={a:wasmImports};return imports}async function createWasm(){function receiveInstance(instance){wasmExports=instance.exports;assignWasmExports(wasmExports);updateMemoryViews();return wasmExports}function receiveInstantiationResult(result){return receiveInstance(result["instance"])}var info=getWasmImports();var instantiateWasm=Module["instantiateWasm"];if(instantiateWasm){return new Promise(resolve=>{instantiateWasm(info,inst=>resolve(receiveInstance(inst)))})}wasmBinaryFile??=findWasmBinary();var result=await instantiateAsync(wasmBinary,wasmBinaryFile,info);var exports=receiveInstantiationResult(result);return exports}class ExitStatus{name="ExitStatus";constructor(status){this.message=`Program terminated with exit(${status})`;this.status=status}}var HEAP8;var callRuntimeCallbacks=callbacks=>{while(callbacks.length>0){callbacks.shift()(Module)}};var onPostRuns=[];var onPreRuns=[];var noExitRuntime=true;var stackRestore=val=>__emscripten_stack_restore(val);var stackSave=()=>_emscripten_stack_get_current();var UTF8Decoder=globalThis.TextDecoder&&new TextDecoder;var findStringEnd=(heapOrArray,idx,maxBytesToRead,ignoreNul)=>{var maxIdx=idx+maxBytesToRead;if(ignoreNul)return maxIdx;while(heapOrArray[idx]&&!(idx>=maxIdx))++idx;return idx};var UTF8ArrayToString=(heapOrArray,idx=0,maxBytesToRead,ignoreNul)=>{var endPtr=findStringEnd(heapOrArray,idx,maxBytesToRead,ignoreNul);if(endPtr-idx>16&&heapOrArray.buffer&&UTF8Decoder){return UTF8Decoder.decode(heapOrArray.subarray(idx,endPtr))}var str="";while(idx<endPtr){var u0=heapOrArray[idx++];if(!(u0&128)){str+=String.fromCharCode(u0);continue}var u1=heapOrArray[idx++]&63;if((u0&224)==192){str+=String.fromCharCode((u0&31)<<6|u1);continue}var u2=heapOrArray[idx++]&63;if((u0&240)==224){u0=(u0&15)<<12|u1<<6|u2}else{u0=(u0&7)<<18|u1<<12|u2<<6|heapOrArray[idx++]&63}if(u0<65536){str+=String.fromCharCode(u0)}else{var ch=u0-65536;str+=String.fromCharCode(55296|ch>>10,56320|ch&1023)}}return str};var HEAPU8;var UTF8ToString=(ptr,maxBytesToRead,ignoreNul)=>ptr?UTF8ArrayToString(HEAPU8,ptr,maxBytesToRead,ignoreNul):"";var ___assert_fail=(condition,filename,line,func)=>abort(`Assertion failed: ${UTF8ToString(condition)}, at: `+[filename?UTF8ToString(filename):"unknown filename",line,func?UTF8ToString(func):"unknown function"]);var __abort_js=()=>abort("");var isLeapYear=year=>year%4===0&&(year%100!==0||year%400===0);var MONTH_DAYS_LEAP_CUMULATIVE=[0,31,60,91,121,152,182,213,244,274,305,335];var MONTH_DAYS_REGULAR_CUMULATIVE=[0,31,59,90,120,151,181,212,243,273,304,334];var ydayFromDate=date=>{var leap=isLeapYear(date.getFullYear());var monthDaysCumulative=leap?MONTH_DAYS_LEAP_CUMULATIVE:MONTH_DAYS_REGULAR_CUMULATIVE;var yday=monthDaysCumulative[date.getMonth()]+date.getDate()-1;return yday};var INT53_MAX=9007199254740992;var INT53_MIN=-9007199254740992;var bigintToI53Checked=num=>num<INT53_MIN||num>INT53_MAX?NaN:Number(num);var HEAP32;function __localtime_js(time,tmPtr){time=bigintToI53Checked(time);var date=new Date(time*1e3);if(isNaN(date.getTime())){return 1}HEAP32[tmPtr>>2]=date.getSeconds();HEAP32[tmPtr+4>>2]=date.getMinutes();HEAP32[tmPtr+8>>2]=date.getHours();HEAP32[tmPtr+12>>2]=date.getDate();HEAP32[tmPtr+16>>2]=date.getMonth();HEAP32[tmPtr+20>>2]=date.getFullYear()-1900;HEAP32[tmPtr+24>>2]=date.getDay();var yday=ydayFromDate(date)|0;HEAP32[tmPtr+28>>2]=yday;HEAP32[tmPtr+36>>2]=-(date.getTimezoneOffset()*60);var start=new Date(date.getFullYear(),0,1);var summerOffset=new Date(date.getFullYear(),6,1).getTimezoneOffset();var winterOffset=start.getTimezoneOffset();var dst=(summerOffset!=winterOffset&&date.getTimezoneOffset()==Math.min(winterOffset,summerOffset))|0;HEAP32[tmPtr+32>>2]=dst;return 0}var stringToUTF8Array=(str,heap,outIdx,maxBytesToWrite)=>{if(!(maxBytesToWrite>0))return 0;var startIdx=outIdx;var endIdx=outIdx+maxBytesToWrite-1;for(var i=0;i<str.length;++i){var u=str.codePointAt(i);if(u<=127){if(outIdx>=endIdx)break;heap[outIdx++]=u}else if(u<=2047){if(outIdx+1>=endIdx)break;heap[outIdx++]=192|u>>6;heap[outIdx++]=128|u&63}else if(u<=65535){if(outIdx+2>=endIdx)break;heap[outIdx++]=224|u>>12;heap[outIdx++]=128|u>>6&63;heap[outIdx++]=128|u&63}else{if(outIdx+3>=endIdx)break;heap[outIdx++]=240|u>>18;heap[outIdx++]=128|u>>12&63;heap[outIdx++]=128|u>>6&63;heap[outIdx++]=128|u&63;i++}}heap[outIdx]=0;return outIdx-startIdx};var stringToUTF8=(str,outPtr,maxBytesToWrite)=>stringToUTF8Array(str,HEAPU8,outPtr,maxBytesToWrite);var HEAPU32;var __tzset_js=(timezone,daylight,std_name,dst_name)=>{var currentYear=(new Date).getFullYear();var winter=new Date(currentYear,0,1);var summer=new Date(currentYear,6,1);var winterOffset=winter.getTimezoneOffset();var summerOffset=summer.getTimezoneOffset();var stdTimezoneOffset=Math.max(winterOffset,summerOffset);HEAPU32[timezone>>2]=stdTimezoneOffset*60;HEAP32[daylight>>2]=Number(winterOffset!=summerOffset);var extractZone=timezoneOffset=>{var sign=timezoneOffset>=0?"-":"+";var absOffset=Math.abs(timezoneOffset);var hours=String(Math.floor(absOffset/60)).padStart(2,"0");var minutes=String(absOffset%60).padStart(2,"0");return`UTC${sign}${hours}${minutes}`};var winterName=extractZone(winterOffset);var summerName=extractZone(summerOffset);if(summerOffset<winterOffset){stringToUTF8(winterName,std_name,17);stringToUTF8(summerName,dst_name,17)}else{stringToUTF8(winterName,dst_name,17);stringToUTF8(summerName,std_name,17)}};var _emscripten_get_now=()=>performance.now();var _emscripten_date_now=()=>Date.now();var nowIsMonotonic=1;var checkWasiClock=clock_id=>clock_id>=0&&clock_id<=3;var HEAP64;function _clock_time_get(clk_id,ignored_precision,ptime){ignored_precision=bigintToI53Checked(ignored_precision);if(!checkWasiClock(clk_id)){return 28}var now;if(clk_id===0){now=_emscripten_date_now()}else if(nowIsMonotonic){now=_emscripten_get_now()}else{return 52}var nsec=Math.round(now*1e3*1e3);HEAP64[ptime>>3]=BigInt(nsec);return 0}function _console_get_size(pw,ph){var r;r=term.getSize();HEAPU32[pw>>2]=r[0];HEAPU32[ph>>2]=r[1]}function _console_write(opaque,buf,len){var str;str=String.fromCharCode.apply(String,HEAPU8.subarray(buf,buf+len));term.write(str)}var handleException=e=>{if(e instanceof ExitStatus||e=="unwind"){return EXITSTATUS}quit_(1,e)};var runtimeKeepaliveCounter=0;var keepRuntimeAlive=()=>noExitRuntime||runtimeKeepaliveCounter>0;var _proc_exit=code=>{EXITSTATUS=code;if(!keepRuntimeAlive()){Module["onExit"]?.(code);ABORT=true}quit_(code,new ExitStatus(code))};var exitJS=(status,implicit)=>{EXITSTATUS=status;_proc_exit(status)};var _exit=exitJS;var maybeExit=()=>{if(!keepRuntimeAlive()){try{_exit(EXITSTATUS)}catch(e){handleException(e)}}};var callUserCallback=func=>{if(ABORT){return}try{return func()}catch(e){handleException(e)}finally{maybeExit()}};var safeSetTimeout=(func,timeout)=>setTimeout(()=>{callUserCallback(func)},timeout);var _emscripten_set_main_loop_timing=(mode,value)=>{MainLoop.timingMode=mode;MainLoop.timingValue=value;if(!MainLoop.func){return 1}if(!MainLoop.running){MainLoop.running=true}if(mode==0){MainLoop.scheduler=function MainLoop_scheduler_setTimeout(){var timeUntilNextTick=Math.max(0,MainLoop.tickStartTime+value-_emscripten_get_now())|0;setTimeout(MainLoop.runner,timeUntilNextTick)}}else if(mode==1){MainLoop.scheduler=function MainLoop_scheduler_rAF(){MainLoop.requestAnimationFrame(MainLoop.runner)}}else{if(!MainLoop.setImmediate){if(globalThis.scheduler){MainLoop.setImmediate=scheduler.postTask.bind(scheduler)}else{var setImmediates=[];var emscriptenMainLoopMessageId="setimmediate";var MainLoop_setImmediate_messageHandler=event=>{if(event.data===emscriptenMainLoopMessageId){event.stopPropagation();setImmediates.shift()()}};addEventListener("message",MainLoop_setImmediate_messageHandler,true);MainLoop.setImmediate=func=>{setImmediates.push(func);if(ENVIRONMENT_IS_WORKER){postMessage(emscriptenMainLoopMessageId)}else{postMessage(emscriptenMainLoopMessageId,"*")}}}}MainLoop.scheduler=function MainLoop_scheduler_setImmediate(){MainLoop.setImmediate(MainLoop.runner)}}return 0};var setMainLoop=(iterFunc,fps,simulateInfiniteLoop,arg,noSetTiming)=>{MainLoop.func=iterFunc;MainLoop.arg=arg;var thisMainLoopId=MainLoop.currentlyRunningMainloop;function checkIsRunning(){if(thisMainLoopId<MainLoop.currentlyRunningMainloop){maybeExit();return false}return true}MainLoop.running=false;MainLoop.runner=function MainLoop_runner(){if(ABORT)return;if(MainLoop.queue.length>0){var start=Date.now();var blocker=MainLoop.queue.shift();blocker.func(blocker.arg);if(MainLoop.remainingBlockers){var remaining=MainLoop.remainingBlockers;var next=remaining%1==0?remaining-1:Math.floor(remaining);if(blocker.counted){MainLoop.remainingBlockers=next}else{next=next+.5;MainLoop.remainingBlockers=(8*remaining+next)/9}}MainLoop.updateStatus();if(!checkIsRunning())return;setTimeout(MainLoop.runner,0);return}if(!checkIsRunning())return;MainLoop.currentFrameNumber=MainLoop.currentFrameNumber+1|0;if(MainLoop.timingMode==1&&MainLoop.timingValue>1&&MainLoop.currentFrameNumber%MainLoop.timingValue!=0){MainLoop.scheduler();return}else if(MainLoop.timingMode==0){MainLoop.tickStartTime=_emscripten_get_now()}MainLoop.runIter(iterFunc);if(!checkIsRunning())return;MainLoop.scheduler()};if(!noSetTiming){if(fps>0){_emscripten_set_main_loop_timing(0,1e3/fps)}else{_emscripten_set_main_loop_timing(1,1)}MainLoop.scheduler()}if(simulateInfiniteLoop){throw"unwind"}};var MainLoop={running:false,scheduler:null,currentlyRunningMainloop:0,func:null,arg:0,timingMode:0,timingValue:0,currentFrameNumber:0,queue:[],preMainLoop:[],postMainLoop:[],pause(){MainLoop.scheduler=null;MainLoop.currentlyRunningMainloop++},resume(){MainLoop.currentlyRunningMainloop++;var timingMode=MainLoop.timingMode;var timingValue=MainLoop.timingValue;var func=MainLoop.func;MainLoop.func=null;setMainLoop(func,0,false,MainLoop.arg,true);_emscripten_set_main_loop_timing(timingMode,timingValue);MainLoop.scheduler()},updateStatus(){if(Module["setStatus"]){var message=Module["statusMessage"]||"Please wait...";var remaining=MainLoop.remainingBlockers??0;var expected=MainLoop.expectedBlockers??0;if(remaining){if(remaining<expected){Module["setStatus"](`{message} ({expected - remaining}/{expected})`)}else{Module["setStatus"](message)}}else{Module["setStatus"]("")}}},init(){},runIter(func){if(ABORT)return;for(var pre of MainLoop.preMainLoop){if(pre()===false){return}}callUserCallback(func);for(var post of MainLoop.postMainLoop){post()}},nextRAF:0,fakeRequestAnimationFrame(func){var now=Date.now();if(!MainLoop.nextRAF){MainLoop.nextRAF=now+1e3/60}else{while(now+2>=MainLoop.nextRAF){MainLoop.nextRAF+=1e3/60}}var delay=Math.max(MainLoop.nextRAF-now,0);setTimeout(func,delay)},requestAnimationFrame(func){if(globalThis.requestAnimationFrame){requestAnimationFrame(func)}else{MainLoop.fakeRequestAnimationFrame(func)}}};var safeRequestAnimationFrame=func=>MainLoop.requestAnimationFrame(()=>{callUserCallback(func)});var wasmTableMirror=[];var getWasmTableEntry=funcPtr=>{var func=wasmTableMirror[funcPtr];if(!func){wasmTableMirror[funcPtr]=func=wasmTable.get(funcPtr)}return func};var _emscripten_async_call=(func,arg,millis)=>{var wrapper=()=>getWasmTableEntry(func)(arg);if(millis>=0){safeSetTimeout(wrapper,millis)}else{safeRequestAnimationFrame(wrapper)}};var _emscripten_random=()=>Math.random();var getHeapMax=()=>2147483648;var alignMemory=(size,alignment)=>Math.ceil(size/alignment)*alignment;var growMemory=size=>{var oldHeapSize=wasmMemory.buffer.byteLength;var pages=(size-oldHeapSize+65535)/65536|0;try{wasmMemory.grow(pages);updateMemoryViews();return 1}catch(e){}};var _emscripten_resize_heap=requestedSize=>{var oldSize=HEAPU8.length;requestedSize>>>=0;var maxHeapSize=getHeapMax();if(requestedSize>maxHeapSize){return false}for(var cutDown=1;cutDown<=4;cutDown*=2){var overGrownHeapSize=oldSize*(1+.2/cutDown);overGrownHeapSize=Math.min(overGrownHeapSize,requestedSize+100663296);var newSize=Math.min(maxHeapSize,alignMemory(Math.max(requestedSize,overGrownHeapSize),65536));var replacement=growMemory(newSize);if(replacement){return true}}return false};function _fb_refresh(opaque,data,x,y,w,h,stride){var i,j,v,src,image_data,dst_pos,display,dst_pos1,image_stride;display=graphic_display;image_data=display.image.data;image_stride=display.width*4;dst_pos1=(y*display.width+x)*4;for(i=0;i<h;i=i+1|0){src=data;dst_pos=dst_pos1;for(j=0;j<w;j=j+1|0){v=HEAPU32[src>>2];image_data[dst_pos]=v>>16&255;image_data[dst_pos+1]=v>>8&255;image_data[dst_pos+2]=v&255;image_data[dst_pos+3]=255;src=src+4|0;dst_pos=dst_pos+4|0}data=data+stride|0;dst_pos1=dst_pos1+image_stride|0}display.ctx.putImageData(display.image,0,0,x,y,w,h)}var _fd_close=fd=>52;function _fd_seek(fd,offset,whence,newOffset){offset=bigintToI53Checked(offset);return 70}var printCharBuffers=[null,[],[]];var printChar=(stream,curr)=>{var buffer=printCharBuffers[stream];if(!curr||curr===10){(stream===1?out:err)(UTF8ArrayToString(buffer));buffer.length=0}else{buffer.push(curr)}};var _fd_write=(fd,iov,iovcnt,pnum)=>{var num=0;for(var i=0;i<iovcnt;i++){var ptr=HEAPU32[iov>>2];var len=HEAPU32[iov+4>>2];iov+=8;for(var j=0;j<len;j++){printChar(fd,HEAPU8[ptr+j])}num+=len}HEAPU32[pnum>>2]=num;return 0};function _file_buffer_init(bs){HEAPU32[bs>>2]=0;HEAPU32[bs+4>>2]=0}function _file_buffer_read(bs,offset,buf,size){var h,data,i;h=HEAPU32[bs>>2];if(h){data=Module.temuFbuf[h];for(i=0;i<size;i=i+1|0){HEAPU8[buf+i]=data[offset+i]}}}function _file_buffer_get_new_handle(){var h;if(typeof Module.temuFbuf=="undefined"){Module.temuFbuf=new Object;Module.temuFbufNext=1}for(;;){h=Module.temuFbufNext;Module.temuFbufNext++;if(Module.temuFbufNext==2147483648)Module.temuFbufNext=1;if(typeof Module.temuFbuf[h]=="undefined"){return h}}}function _file_buffer_resize(bs,new_size){var h,size,new_data,i,data;h=HEAPU32[bs>>2];size=HEAPU32[bs+4>>2];if(new_size==0){if(h!=0){delete Module.temuFbuf[h];h=0}}else if(size==0){h=_file_buffer_get_new_handle();new_data=new Uint8Array(new_size);Module.temuFbuf[h]=new_data}else if(size!=new_size){data=Module.temuFbuf[h];new_data=new Uint8Array(new_size);if(new_size>size){new_data.set(data,0)}else{for(i=0;i<new_size;i=i+1|0)new_data[i]=data[i]}Module.temuFbuf[h]=new_data}HEAPU32[bs>>2]=h;HEAPU32[bs+4>>2]=new_size;return 0}function _file_buffer_reset(bs){_file_buffer_resize(bs,0);_file_buffer_init(bs)}function _file_buffer_set(bs,offset,val,size){var h,data,i;h=HEAPU32[bs>>2];if(h){data=Module.temuFbuf[h];for(i=0;i<size;i=i+1|0){data[offset+i]=val}}}function _file_buffer_write(bs,offset,buf,size){var h,data,i;h=HEAPU32[bs>>2];if(h){data=Module.temuFbuf[h];for(i=0;i<size;i=i+1|0){data[offset+i]=HEAPU8[buf+i]}}}function _fs_export_file(filename,buf,buf_len){var _filename=UTF8ToString(filename);var data=HEAPU8.subarray(buf,buf+buf_len);var file=new Blob([data],{type:"application/octet-stream"});var url=URL.createObjectURL(file);var a=document.createElement("a");a.href=url;a.setAttribute("download",_filename);a.innerHTML="downloading";document.body.appendChild(a);setTimeout(function(){a.click();document.body.removeChild(a)},50)}function _fs_wget_update_downloading(flag){update_downloading(Boolean(flag))}function _net_recv_packet(bs,buf,buf_len){if(net_state){net_state.recv_packet(HEAPU8.subarray(buf,buf+buf_len))}}function _temu_wget_data(url,request,user,password,post_data,post_data_len,arg,free,onload,onerror,onprogress){var _url=UTF8ToString(url);var _request=UTF8ToString(request);var _user;var _password;var http=new XMLHttpRequest;if(user)_user=UTF8ToString(user);else _user=null;if(password)_password=UTF8ToString(password);else _password=null;http.open(_request,_url,true);http.responseType="arraybuffer";if(_user){http.setRequestHeader("Authorization","Basic "+btoa(_user+":"+_password))}var handle=Module.temuWgetHandle=(Module.temuWgetHandle||0)+1;http.onload=function http_onload(e){if(http.status==200||_url.substr(0,4).toLowerCase()!="http"){var byteArray=new Uint8Array(http.response);var buffer=_malloc(byteArray.length);HEAPU8.set(byteArray,buffer);if(onload)getWasmTableEntry(onload)(handle,arg,buffer,byteArray.length);if(free)_free(buffer)}else{if(onerror)getWasmTableEntry(onerror)(handle,arg,http.status,0)}delete(Module.temuWgetRequests=Module.temuWgetRequests||{})[handle]};http.onerror=function http_onerror(e){if(onerror){getWasmTableEntry(onerror)(handle,arg,http.status,0)}delete(Module.temuWgetRequests=Module.temuWgetRequests||{})[handle]};http.onprogress=function http_onprogress(e){if(onprogress)getWasmTableEntry(onprogress)(handle,arg,e.loaded,e.lengthComputable||e.lengthComputable===undefined?e.total:0)};http.onabort=function http_onabort(e){delete(Module.temuWgetRequests=Module.temuWgetRequests||{})[handle]};if(_request=="POST"){var _post_data=HEAPU8.subarray(post_data,post_data+post_data_len);http.setRequestHeader("Content-type","application/octet-stream");http.setRequestHeader("Content-length",post_data_len);http.setRequestHeader("Connection","close");http.send(_post_data)}else{http.send(null)}(Module.temuWgetRequests=Module.temuWgetRequests||{})[handle]=http;return handle}var getCFunc=ident=>{var func=Module["_"+ident];return func};var writeArrayToMemory=(array,buffer)=>{HEAP8.set(array,buffer)};var lengthBytesUTF8=str=>{var len=0;for(var i=0;i<str.length;++i){var c=str.charCodeAt(i);if(c<=127){len++}else if(c<=2047){len+=2}else if(c>=55296&&c<=57343){len+=4;++i}else{len+=3}}return len};var stackAlloc=sz=>__emscripten_stack_alloc(sz);var stringToUTF8OnStack=str=>{var size=lengthBytesUTF8(str)+1;var ret=stackAlloc(size);stringToUTF8(str,ret,size);return ret};var ccall=(ident,returnType,argTypes,args,opts)=>{var toC={string:str=>{var ret=0;if(str!==null&&str!==undefined&&str!==0){ret=stringToUTF8OnStack(str)}return ret},array:arr=>{var ret=stackAlloc(arr.length);writeArrayToMemory(arr,ret);return ret}};function convertReturnValue(ret){if(returnType==="string"){return UTF8ToString(ret)}if(returnType==="boolean")return Boolean(ret);return ret}var func=getCFunc(ident);var cArgs=[];var stack=0;if(args){for(var i=0;i<args.length;i++){var converter=toC[argTypes[i]];if(converter){if(!stack)stack=stackSave();cArgs[i]=converter(args[i])}else{cArgs[i]=args[i]}}}var ret=func(...cArgs);function onDone(ret){if(stack)stackRestore(stack);return convertReturnValue(ret)}ret=onDone(ret);return ret};var cwrap=(ident,returnType,argTypes,opts)=>{var numericArgs=!argTypes||argTypes.every(type=>type==="number"||type==="boolean");var numericRet=returnType!=="string";if(numericRet&&numericArgs&&!opts){return getCFunc(ident)}return(...args)=>ccall(ident,returnType,argTypes,args,opts)};Module["requestAnimationFrame"]=MainLoop.requestAnimationFrame;Module["pauseMainLoop"]=MainLoop.pause;Module["resumeMainLoop"]=MainLoop.resume;MainLoop.init();{if(Module["noExitRuntime"])noExitRuntime=Module["noExitRuntime"];if(Module["print"])out=Module["print"];if(Module["printErr"])err=Module["printErr"];if(Module["arguments"])programArgs=Module["arguments"];if(Module["thisProgram"])thisProgram=Module["thisProgram"];var preInit=Module["preInit"];if(preInit){if(typeof preInit=="function")Module["preInit"]=preInit=[preInit];while(preInit.length>0){preInit.shift()()}}}Module["ccall"]=ccall;Module["cwrap"]=cwrap;var _console_queue_char,_display_key_event,_display_mouse_event,_display_wheel_event,_net_write_packet,_net_set_carrier,_vm_start,_free,_malloc,_fs_import_file,__emscripten_stack_restore,__emscripten_stack_alloc,_emscripten_stack_get_current,memory,__indirect_function_table,wasmMemory,wasmTable;function assignWasmExports(wasmExports){_console_queue_char=Module["_console_queue_char"]=wasmExports["C"];_display_key_event=Module["_display_key_event"]=wasmExports["D"];_display_mouse_event=Module["_display_mouse_event"]=wasmExports["F"];_display_wheel_event=Module["_display_wheel_event"]=wasmExports["G"];_net_write_packet=Module["_net_write_packet"]=wasmExports["H"];_net_set_carrier=Module["_net_set_carrier"]=wasmExports["I"];_vm_start=Module["_vm_start"]=wasmExports["J"];_free=Module["_free"]=wasmExports["K"];_malloc=Module["_malloc"]=wasmExports["L"];_fs_import_file=Module["_fs_import_file"]=wasmExports["M"];__emscripten_stack_restore=wasmExports["N"];__emscripten_stack_alloc=wasmExports["O"];_emscripten_stack_get_current=wasmExports["P"];memory=wasmMemory=wasmExports["A"];__indirect_function_table=wasmTable=wasmExports["E"]}var wasmImports={a:___assert_fail,x:__abort_js,u:__localtime_js,v:__tzset_js,w:_clock_time_get,z:_console_get_size,p:_console_write,l:_emscripten_async_call,m:_emscripten_date_now,e:_emscripten_random,r:_emscripten_resize_heap,b:_exit,q:_fb_refresh,t:_fd_close,s:_fd_seek,k:_fd_write,h:_file_buffer_init,d:_file_buffer_read,i:_file_buffer_reset,g:_file_buffer_resize,y:_file_buffer_set,c:_file_buffer_write,n:_fs_export_file,f:_fs_wget_update_downloading,o:_net_recv_packet,j:_temu_wget_data};async function run(){preRun();var setStatus=Module["setStatus"];if(setStatus){setStatus("Running...");await new Promise(resolve=>setTimeout(resolve,1));setTimeout(setStatus,1,"")}if(ABORT)return;initRuntime();Module["onRuntimeInitialized"]?.();postRun()}var wasmExports;createWasm().then(()=>run()); diff --git a/web/demo/emulator/dist/riscvemu64-wasm.wasm b/web/demo/emulator/dist/riscvemu64-wasm.wasm Binary files differnew file mode 100755 index 0000000..4822f49 --- /dev/null +++ b/web/demo/emulator/dist/riscvemu64-wasm.wasm diff --git a/web/demo/emulator/patches/0001-modern-emscripten.patch b/web/demo/emulator/patches/0001-modern-emscripten.patch new file mode 100644 index 0000000..030dbe0 --- /dev/null +++ b/web/demo/emulator/patches/0001-modern-emscripten.patch @@ -0,0 +1,233 @@ +--- a/js/lib.js ++++ b/js/lib.js +@@ -41,7 +41,7 @@ + + fs_export_file: function(filename, buf, buf_len) + { +- var _filename = Pointer_stringify(filename); ++ var _filename = UTF8ToString(filename); + // console.log("exporting " + _filename); + var data = HEAPU8.subarray(buf, buf + buf_len); + var file = new Blob([data], { type: "application/octet-stream" }); +@@ -58,20 +58,20 @@ + }, 50); + }, + +- emscripten_async_wget3_data: function(url, request, user, password, post_data, post_data_len, arg, free, onload, onerror, onprogress) { +- var _url = Pointer_stringify(url); +- var _request = Pointer_stringify(request); ++ temu_wget_data: function(url, request, user, password, post_data, post_data_len, arg, free, onload, onerror, onprogress) { ++ var _url = UTF8ToString(url); ++ var _request = UTF8ToString(request); + var _user; + var _password; + + var http = new XMLHttpRequest(); + + if (user) +- _user = Pointer_stringify(user); ++ _user = UTF8ToString(user); + else + _user = null; + if (password) +- _password = Pointer_stringify(password); ++ _password = UTF8ToString(password); + else + _password = null; + +@@ -81,7 +81,7 @@ + http.setRequestHeader("Authorization", "Basic " + btoa(_user + ':' + _password)); + } + +- var handle = Browser.getNextWgetRequestHandle(); ++ var handle = (Module.temuWgetHandle = (Module.temuWgetHandle || 0) + 1); + + // LOAD + http.onload = function http_onload(e) { +@@ -89,37 +89,32 @@ + var byteArray = new Uint8Array(http.response); + var buffer = _malloc(byteArray.length); + HEAPU8.set(byteArray, buffer); +- if (onload) Runtime.dynCall('viiii', onload, [handle, arg, buffer, byteArray.length]); ++ if (onload) {{{ makeDynCall('viiii', 'onload') }}}(handle, arg, buffer, byteArray.length); + if (free) _free(buffer); + } else { +- if (onerror) Runtime.dynCall('viiii', onerror, [handle, arg, http.status, http.statusText]); ++ if (onerror) {{{ makeDynCall('viiii', 'onerror') }}}(handle, arg, http.status, 0); + } +- delete Browser.wgetRequests[handle]; ++ delete (Module.temuWgetRequests = Module.temuWgetRequests || {})[handle]; + }; + + // ERROR + http.onerror = function http_onerror(e) { + if (onerror) { +- Runtime.dynCall('viiii', onerror, [handle, arg, http.status, http.statusText]); ++ {{{ makeDynCall('viiii', 'onerror') }}}(handle, arg, http.status, 0); + } +- delete Browser.wgetRequests[handle]; ++ delete (Module.temuWgetRequests = Module.temuWgetRequests || {})[handle]; + }; + + // PROGRESS + http.onprogress = function http_onprogress(e) { +- if (onprogress) Runtime.dynCall('viiii', onprogress, [handle, arg, e.loaded, e.lengthComputable || e.lengthComputable === undefined ? e.total : 0]); ++ if (onprogress) {{{ makeDynCall('viiii', 'onprogress') }}}(handle, arg, e.loaded, e.lengthComputable || e.lengthComputable === undefined ? e.total : 0); + }; + + // ABORT + http.onabort = function http_onabort(e) { +- delete Browser.wgetRequests[handle]; ++ delete (Module.temuWgetRequests = Module.temuWgetRequests || {})[handle]; + }; + +- // Useful because the browser can limit the number of redirection +- try { +- if (http.channel instanceof Ci.nsIHttpChannel) +- http.channel.redirectionLimit = 0; +- } catch (ex) { /* whatever */ } + + if (_request == "POST") { + var _post_data = HEAPU8.subarray(post_data, post_data + post_data_len); +@@ -132,7 +127,7 @@ + http.send(null); + } + +- Browser.wgetRequests[handle] = http; ++ (Module.temuWgetRequests = Module.temuWgetRequests || {})[handle] = http; + + return handle; + }, +@@ -181,16 +176,16 @@ + file_buffer_get_new_handle: function() + { + var h; +- if (typeof Browser.fbuf_table == "undefined") { +- Browser.fbuf_table = new Object(); +- Browser.fbuf_next_handle = 1; ++ if (typeof Module.temuFbuf == "undefined") { ++ Module.temuFbuf = new Object(); ++ Module.temuFbufNext = 1; + } + for(;;) { +- h = Browser.fbuf_next_handle; +- Browser.fbuf_next_handle++; +- if (Browser.fbuf_next_handle == 0x80000000) +- Browser.fbuf_next_handle = 1; +- if (typeof Browser.fbuf_table[h] == "undefined") { ++ h = Module.temuFbufNext; ++ Module.temuFbufNext++; ++ if (Module.temuFbufNext == 0x80000000) ++ Module.temuFbufNext = 1; ++ if (typeof Module.temuFbuf[h] == "undefined") { + // console.log("new handle=" + h); + return h; + } +@@ -212,15 +207,15 @@ + size = HEAPU32[(bs + 4) >> 2]; + if (new_size == 0) { + if (h != 0) { +- delete Browser.fbuf_table[h]; ++ delete Module.temuFbuf[h]; + h = 0; + } + } else if (size == 0) { + h = _file_buffer_get_new_handle(); + new_data = new Uint8Array(new_size); +- Browser.fbuf_table[h] = new_data; ++ Module.temuFbuf[h] = new_data; + } else if (size != new_size) { +- data = Browser.fbuf_table[h]; ++ data = Module.temuFbuf[h]; + new_data = new Uint8Array(new_size); + if (new_size > size) { + new_data.set(data, 0); +@@ -228,13 +223,14 @@ + for(i = 0; i < new_size; i = (i + 1) | 0) + new_data[i] = data[i]; + } +- Browser.fbuf_table[h] = new_data; ++ Module.temuFbuf[h] = new_data; + } + HEAPU32[bs >> 2] = h; + HEAPU32[(bs + 4) >> 2] = new_size; + return 0; + }, + ++ file_buffer_reset__deps: ['file_buffer_resize', 'file_buffer_init'], + file_buffer_reset: function(bs) + { + _file_buffer_resize(bs, 0); +@@ -246,7 +242,7 @@ + var h, data, i; + h = HEAPU32[bs >> 2]; + if (h) { +- data = Browser.fbuf_table[h]; ++ data = Module.temuFbuf[h]; + for(i = 0; i < size; i = (i + 1) | 0) { + data[offset + i] = HEAPU8[buf + i]; + } +@@ -258,7 +254,7 @@ + var h, data, i; + h = HEAPU32[bs >> 2]; + if (h) { +- data = Browser.fbuf_table[h]; ++ data = Module.temuFbuf[h]; + for(i = 0; i < size; i = (i + 1) | 0) { + HEAPU8[buf + i] = data[offset + i]; + } +@@ -270,7 +266,7 @@ + var h, data, i; + h = HEAPU32[bs >> 2]; + if (h) { +- data = Browser.fbuf_table[h]; ++ data = Module.temuFbuf[h]; + for(i = 0; i < size; i = (i + 1) | 0) { + data[offset + i] = val; + } +--- a/fs_wget.c ++++ b/fs_wget.c +@@ -92,7 +92,7 @@ + s->cb(s->opaque, 0, data, size); + } + +-extern int emscripten_async_wget3_data(const char* url, const char* requesttype, const char *user, const char *password, const uint8_t *post_data, int post_data_len, void *arg, int free, em_async_wget2_data_onload_func onload, em_async_wget2_data_onerror_func onerror, em_async_wget2_data_onprogress_func onprogress); ++extern int temu_wget_data(const char* url, const char* requesttype, const char *user, const char *password, const uint8_t *post_data, int post_data_len, void *arg, int free, em_async_wget2_data_onload_func onload, em_async_wget2_data_onerror_func onerror, em_async_wget2_data_onprogress_func onprogress); + + XHRState *fs_wget2(const char *url, const char *user, const char *password, + WGetReadCallback *read_cb, uint64_t post_data_len, +@@ -116,7 +116,7 @@ + } + fs_wget_update_downloading_count(1); + +- emscripten_async_wget3_data(url, request, user, password, ++ temu_wget_data(url, request, user, password, + post_data, post_data_len, s, 1, fs_wget_onload, + fs_wget_onerror, NULL); + if (post_data_len != 0) +--- a/b/Makefile.emcc ++++ b/Makefile.emcc +@@ -0,0 +1,23 @@ ++# riscvemu64-wasm with a modern emscripten (6.x), from the 2019 TinyEMU source. ++EMCC=emcc ++EMCFLAGS=-O2 -w -DEMSCRIPTEN -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -MMD \ ++ -fno-strict-aliasing -DCONFIG_FS_NET -DMAX_XLEN=64 -DCONFIG_RISCV_MAX_XLEN=64 ++EMLDFLAGS=-O3 -s EXPORTED_FUNCTIONS=_console_queue_char,_vm_start,_fs_import_file,_display_key_event,_display_mouse_event,_display_wheel_event,_net_write_packet,_net_set_carrier,_malloc,_free \ ++ -s EXPORTED_RUNTIME_METHODS=ccall,cwrap \ ++ -s INITIAL_MEMORY=67108864 -s ALLOW_MEMORY_GROWTH=1 \ ++ -s ENVIRONMENT=web -s FILESYSTEM=0 --no-entry \ ++ --js-library js/lib.js ++ ++OBJS=jsemu.o softfp.o virtio.o fs.o fs_net.o fs_wget.o fs_utils.o simplefb.o \ ++ pci.o json.o block_net.o iomem.o cutils.o aes.o sha256.o \ ++ riscv_cpu.o riscv_machine.o machine.o ++ ++all: riscvemu64-wasm.js ++ ++%.o: %.c ++ $(EMCC) $(EMCFLAGS) -c -o $@ $< ++ ++riscvemu64-wasm.js: $(OBJS) js/lib.js ++ $(EMCC) $(EMLDFLAGS) -o $@ $(OBJS) ++ ++-include $(wildcard *.d) diff --git a/web/demo/emulator/patches/0002-sfence-vma-invalidates-one-page.patch b/web/demo/emulator/patches/0002-sfence-vma-invalidates-one-page.patch new file mode 100644 index 0000000..e156f30 --- /dev/null +++ b/web/demo/emulator/patches/0002-sfence-vma-invalidates-one-page.patch @@ -0,0 +1,21 @@ +--- a/riscv_cpu.c ++++ b/riscv_cpu.c +@@ -600,9 +600,17 @@ + tlb_init(s); + } + ++/* `sfence.vma rs1` names one page, and the three TLBs are direct-mapped on the ++ * page number, so that page's translation can only ever live at one index in ++ * each of them: clearing those three entries is complete. The original flushed ++ * all 768 entries, which Linux pays for on every unmap - and it unmaps ++ * constantly, on every fork, exec and munmap. */ + static void tlb_flush_vaddr(RISCVCPUState *s, target_ulong vaddr) + { +- tlb_flush_all(s); ++ int i = (vaddr >> PG_SHIFT) & (TLB_SIZE - 1); ++ s->tlb_read[i].vaddr = -1; ++ s->tlb_write[i].vaddr = -1; ++ s->tlb_code[i].vaddr = -1; + } + + /* XXX: inefficient but not critical as long as it is seldom used */ diff --git a/web/demo/fetch-vendor b/web/demo/fetch-vendor new file mode 100755 index 0000000..576cbce --- /dev/null +++ b/web/demo/fetch-vendor @@ -0,0 +1,138 @@ +#!/bin/sh +# Fetch the third-party pieces of the web demo into vendor/. +# +# None of this is committed. The emulator is MIT and could be, but the kernel +# and the busybox userland are GPL binaries, and a repository that ships those +# owes their source -- so we point at the upstream tarball instead and let it +# stay Bellard's to distribute. Everything is pinned by sha256, tarball and +# extracted member both: a mirror that quietly changes underneath us should +# fail the build rather than boot something else in the visitor's browser. +set -eu + +here=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd) +vendor="$here/vendor" +cache="$vendor/.cache" +mkdir -p "$cache" + +JSLINUX_URL=https://bellard.org/tinyemu/jslinux-2019-12-21.tar.gz +JSLINUX_SHA=c57e46f8eab6656cd9ce5c7e2825230177919b147c8c2befd1f327bbdf254088 +JSLINUX_DIR=jslinux-2019-12-21 + +# The guest userland: Alpine's riscv64 minirootfs. A musl busybox userland with +# apk, vi, less and top in it, which is a machine worth being dropped into -- +# and, unlike a rootfs lifted out of the jslinux tarball, one whose provenance +# and licence we can point at. +ALPINE_VER=3.22.5 +ALPINE_URL=https://dl-cdn.alpinelinux.org/alpine/v3.22/releases/riscv64/alpine-minirootfs-$ALPINE_VER-riscv64.tar.gz +ALPINE_SHA=24cd94b1be923179737cd50cb357aa7e72ed086ac08c2360f7e866ae2eb0cba5 + +# Only for the licence text: the jslinux tarball ships none, and we redistribute +# Bellard's MIT-licensed emulator in build/site. +TINYEMU_URL=https://bellard.org/tinyemu/tinyemu-2019-12-21.tar.gz +TINYEMU_SHA=be8351f2121819b3172fcedce5cb1826fa12c87da1b7ed98f269d3e802a05555 +TINYEMU_DIR=tinyemu-2019-12-21 + +XTERM_VER=5.5.0 +XTERM_JS_URL=https://unpkg.com/@xterm/xterm@$XTERM_VER/lib/xterm.js +XTERM_JS_SHA=1f991ac3b4b283ebf96e60ae23a00a52765dd3a2e46fa6fdda9f1aab032f7495 +XTERM_CSS_URL=https://unpkg.com/@xterm/xterm@$XTERM_VER/css/xterm.css +XTERM_CSS_SHA=ba8e6985669488981ccf40c0cefe3aba80722cb6c92de7ad628b0bd717faf2b6 + +sha() { sha256sum "$1" | cut -d' ' -f1; } + +# Re-downloading a 4 MB tarball on every make is rude to bellard.org; the cache +# is checked by hash, so a truncated download is refetched and a good one is not. +fetch() { # url sha dest + if [ -f "$3" ] && [ "$(sha "$3")" = "$2" ]; then + printf ' have %s\n' "$(basename "$3")" + return 0 + fi + printf ' get %s\n' "$(basename "$3")" + curl -fsSL --retry 3 -o "$3.part" "$1" + got=$(sha "$3.part") + if [ "$got" != "$2" ]; then + rm -f "$3.part" + printf 'sha256 mismatch for %s\n expected %s\n got %s\n' "$1" "$2" "$got" >&2 + exit 1 + fi + mv "$3.part" "$3" +} + +# member sha dest: extract one file from a tarball and check it on its own, so a +# right tarball unpacked wrong is still an error. +take() { # tarball member sha dest + tar -xzf "$1" -O "$2" >"$4.part" + got=$(sha "$4.part") + if [ "$got" != "$3" ]; then + rm -f "$4.part" + printf 'sha256 mismatch for %s\n expected %s\n got %s\n' "$2" "$3" "$got" >&2 + exit 1 + fi + mv "$4.part" "$4" + printf ' ok %s\n' "$(basename "$4")" +} + +fetch "$JSLINUX_URL" "$JSLINUX_SHA" "$cache/jslinux.tar.gz" +fetch "$TINYEMU_URL" "$TINYEMU_SHA" "$cache/tinyemu.tar.gz" + +# Deliberately NOT taken from the tarball: riscvemu64-wasm.{js,wasm}. Bellard's +# prebuilt blob does not match his source release -- it stops issuing block +# fetches at 16 MiB into the disk, which his 4 MiB demo images never noticed +# and our 32 MiB one hit as a hang or a segfault in the guest. The emulator we +# ship is built from this same tarball's source by emulator/build-emulator and +# committed in emulator/dist/. + +# bbl is the BIOS, and the kernel is Bellard's 4.15 riscv64 build. We keep his +# kernel deliberately: it is what these emulator binaries were tested against, +# and building our own means building a riscv toolchain to get one feature +# (CONFIG_BLK_DEV_INITRD) that a virtio root device does not need. +take "$cache/jslinux.tar.gz" "$JSLINUX_DIR/bbl64.bin" \ + 293610cea7af6c75e4a8337e16c0d62834becbf31ffd9cca35e0c211602349db \ + "$vendor/bbl64.bin" +take "$cache/jslinux.tar.gz" "$JSLINUX_DIR/kernel-riscv64.bin" \ + 65ca70a6560a730396ab61488d1cf46089b32da5af7e535e03c180cd5bcc8e75 \ + "$vendor/kernel-riscv64.bin" + +# The guest userland. build-rootfs unpacks this into a staging tree, lays the +# overlay and slosh on top, and makes an image out of the result -- the +# tarball itself is never modified. Bellard's own root-riscv64.bin is the +# obvious alternative and is deliberately not used: it is a 2017 glibc busybox +# image with 1.4 MiB free, no package manager, and /sbin/init as a shell script +# that loops a bare shell. +fetch "$ALPINE_URL" "$ALPINE_SHA" "$vendor/alpine-minirootfs-riscv64.tar.gz" + +# Guest packages beyond the minirootfs, as pinned .apk files. The machine has +# no network, so anything it is to have must be baked into the image; +# build-rootfs unpacks these into the staging tree (and prunes them -- vim +# alone unpacks to 26 MiB of which the syntax machinery is under half). +# An .apk is tar.gz segments, so nothing here needs apk-tools. +APK_BASE=https://dl-cdn.alpinelinux.org/alpine/v3.22/main/riscv64 +mkdir -p "$vendor/apks" +fetch "$APK_BASE/vim-9.1.1566-r0.apk" \ + a6c60d2343b3432c7b1cf0953132e67b83e666da969a05196a23fb6a41f6dbfa \ + "$vendor/apks/vim.apk" +fetch "$APK_BASE/vim-common-9.1.1566-r0.apk" \ + cd0b9d6990cb1d89c9559c8683b071689eef79b6b78c10e7906551ef8d429937 \ + "$vendor/apks/vim-common.apk" +fetch "$APK_BASE/xxd-9.1.1566-r0.apk" \ + 9e8e77c39c286d5f515bd377be0c2c55e706d9821910ecaa51e4880b0c0706b1 \ + "$vendor/apks/xxd.apk" +fetch "$APK_BASE/libncursesw-6.5_p20250503-r0.apk" \ + 7d88384bd34c276bafcaae52289e62b895bc5265f74369dc0ad523ef204d9607 \ + "$vendor/apks/libncursesw.apk" +fetch "$APK_BASE/ncurses-terminfo-base-6.5_p20250503-r0.apk" \ + 8e523158d782d13d311fffde76d891812be9e4e0d5b4e30dd14ad1c21848243a \ + "$vendor/apks/ncurses-terminfo-base.apk" + +take "$cache/tinyemu.tar.gz" "$TINYEMU_DIR/MIT-LICENSE.txt" \ + a75690160a50d8085bcd25acf34faa5b1c484e15ffd0bd662e5f4d7a289da080 \ + "$vendor/TINYEMU-MIT-LICENSE.txt" + +# xterm.js rather than the term.js in the jslinux tarball: that one has no +# truecolor and no wide characters, and a demo of a multiplexer that renders +# 24-bit shaders and CJK would be lying about both. UMD bundle, so a plain +# <script> puts Terminal on the global object -- no npm, no bundler. +fetch "$XTERM_JS_URL" "$XTERM_JS_SHA" "$vendor/xterm.js" +fetch "$XTERM_CSS_URL" "$XTERM_CSS_SHA" "$vendor/xterm.css" + +printf 'vendor/ ready (%s)\n' "$vendor" diff --git a/web/demo/overlay/etc/init.d/rcS b/web/demo/overlay/etc/init.d/rcS new file mode 100755 index 0000000..48bf717 --- /dev/null +++ b/web/demo/overlay/etc/init.d/rcS @@ -0,0 +1,44 @@ +#!/bin/sh +# Everything this machine needs before the session starts, and nothing else. +# +# Quiet on purpose: this writes to the console the visitor is about to watch +# slosh draw on, and a wall of boot chatter under a terminal multiplexer +# reads as something having gone wrong. +# +# Deliberately not `set -e`: a mount that is already there must not stop the +# ones after it, and one of those is devpts -- without which the demo has no +# panes at all. + +mountpoint -q /proc || mount -t proc proc /proc +mountpoint -q /sys || mount -t sysfs sysfs /sys + +# The guest kernel is built with CONFIG_DEVTMPFS_MOUNT, so /dev is already +# populated by the time init runs. Kept as a guard rather than dropped: it is +# one line, and it is what makes the rest of this script true on a kernel +# without that option. +mountpoint -q /dev || mount -t devtmpfs devtmpfs /dev + +# Load-bearing, and the first thing to look at if the demo comes up with no +# panes in it: slosh gives every pane a pty of its own through +# posix_openpt(3), which is devpts or nothing. gid 5 is `tty` in Alpine's +# /etc/group, and ptmxmode is what makes /dev/ptmx openable at all. +mkdir -p /dev/pts +mountpoint -q /dev/pts || + mount -t devpts -o ptmxmode=0666,gid=5,mode=620 devpts /dev/pts +[ -e /dev/ptmx ] || ln -s pts/ptmx /dev/ptmx + +# Somewhere to write. The session's control socket lands under /tmp +# (/tmp/slosh-0/<name>.sock, since $XDG_RUNTIME_DIR is unset here), so a +# read-only /tmp is a session that cannot start. /dev/shm is not used by +# anything we ship, but musl's shm_open expects it and the cost is nothing. +mountpoint -q /tmp || mount -t tmpfs -o size=16m,mode=1777 tmpfs /tmp +mountpoint -q /run || mount -t tmpfs -o size=4m,mode=0755 tmpfs /run +mkdir -p /dev/shm +mountpoint -q /dev/shm || mount -t tmpfs -o size=8m,mode=1777 tmpfs /dev/shm + +hostname slosh-demo + +# The console is a terminal a full-screen program is about to own, and a kernel +# message arriving mid-session would be painted straight over it. Only panics +# get through from here on. +dmesg -n 1 2>/dev/null diff --git a/web/demo/overlay/etc/inittab b/web/demo/overlay/etc/inittab new file mode 100644 index 0000000..6523b0d --- /dev/null +++ b/web/demo/overlay/etc/inittab @@ -0,0 +1,25 @@ +# busybox init, on a machine whose entire job is to hold one slosh session. +# +# This replaces Alpine's own inittab, which hands everything to openrc -- a +# service manager that is not in the minirootfs and that this machine has no +# services for. Four lines is the whole boot. +# +# Fields are <tty>:<runlevels>:<action>:<command>. The tty field is empty so +# every entry inherits init's console, which is the one the kernel was told +# about on the command line (console=hvc0) rather than a device this file has +# to name. +::sysinit:/etc/init.d/rcS + +# The visitor lands *in* slosh. There is nothing else on this machine to do, +# and a shell prompt in front of a demo of a terminal multiplexer is a wasted +# screen. `respawn` because quitting the session -- C-a q, or closing the last +# pane -- has to bring it straight back: an empty console reads as a crash. +# +# It goes through a shell for two reasons: /etc/profile is the environment +# every pane will inherit, and busybox init would otherwise split this line on +# spaces itself and exec it directly. +::respawn:/bin/sh -c '. /etc/profile; exec /usr/bin/slosh --layout /root/demo.layout' + +# There is no reset button on a browser tab. +::ctrlaltdel:/sbin/reboot +::shutdown:/bin/umount -a -r diff --git a/web/demo/overlay/etc/motd b/web/demo/overlay/etc/motd new file mode 100644 index 0000000..c676471 --- /dev/null +++ b/web/demo/overlay/etc/motd @@ -0,0 +1,5 @@ +slosh: panes, tabs and sessions, on riscv64 Linux inside your browser. +The machine is yours and it is throwaway -- reload the page and it is gone. +The leader is C-a: C-a Enter splits, C-a z zooms, C-a ? lists every key. +These three panes are a file, /root/demo.layout, and so is the theme. +The real thing, and the source of all of this, is linked beside the terminal. diff --git a/web/demo/overlay/etc/profile b/web/demo/overlay/etc/profile new file mode 100644 index 0000000..057730e --- /dev/null +++ b/web/demo/overlay/etc/profile @@ -0,0 +1,24 @@ +# The environment every pane inherits. +# +# Sourced by the line in /etc/inittab that starts the session, so slosh gets +# it and hands it on. A pane's shell is not a login shell and reads nothing at +# startup, which is why PS1 is exported here rather than left in a file no pane +# will ever open. +# +# TERM is what *slosh itself* is looking at -- xterm.js in the browser. It is +# not what a program in a pane is told: slosh sets TERM=xterm-ghostty on the +# far side of every pty it opens (src/pty.c:67), because that is the terminal +# it implements. That name resolves: build-rootfs aliases xterm-ghostty to +# xterm-256color in the image's terminfo, which is the subset both speak. +# +# COLORTERM because it is true -- the VT does 24-bit colour -- and because it +# is what lets vim pick truecolor over the 256 the terminfo alias admits to. +TERM=xterm-256color +COLORTERM=truecolor +HOME=/root +PATH=/usr/bin:/usr/sbin:/bin:/sbin +PS1='slosh-demo:\w \$ ' +export TERM COLORTERM HOME PATH PS1 + +# The motd belongs in a pane, not on the console underneath the session: the +# layout gives it a pane of its own, where it can be read. diff --git a/web/demo/overlay/root/.config/slosh/config.kdl b/web/demo/overlay/root/.config/slosh/config.kdl new file mode 100644 index 0000000..3252da7 --- /dev/null +++ b/web/demo/overlay/root/.config/slosh/config.kdl @@ -0,0 +1,29 @@ +// The demo's config. Deliberately about twenty lines: the reference config with +// every setting and the reasoning attached is config/config.kdl in the source +// tree, and 48KB of commentary is the wrong thing to ship to a browser. +// +// Saving a config applies to a running session, so this file is also a thing to +// edit from inside the demo: `C-a e` opens it in a pane. + +// The default palette, from the file the repo ships rather than a second copy +// of it -- overlay/root/.config/slosh/themes/sl0p.kdl is a symlink to +// contrib/themes/sl0p.kdl, so this is the same sl0p as everywhere else and it +// cannot fall behind. An include is resolved against *this* file's directory. +include "themes/sl0p.kdl" + +// Busybox ash is the only shell on this machine and $SHELL is not set on it. +shell "/bin/sh" + +// A tenth of the default history. The whole machine is 32MB of disk and +// whatever RAM the page asked for, and nothing here will produce 1000 lines. +scrollback 1000 + +// Enough gap to see the frames, without spending a column this terminal does +// not have. `gap` is in rows and comes out square (gap_aspect). +gap 1 +padding 0 1 + +// No shader by default: a cursor-line wash reads as a rendering artifact to +// anyone who did not ask for it. The shader system is one paste away -- +// contrib/shaders/ has thirty-two of them, and `C-a e` opens this file in a +// pane; it is re-read the moment you save. diff --git a/web/demo/overlay/root/.config/slosh/themes/sl0p.kdl b/web/demo/overlay/root/.config/slosh/themes/sl0p.kdl new file mode 120000 index 0000000..9306464 --- /dev/null +++ b/web/demo/overlay/root/.config/slosh/themes/sl0p.kdl @@ -0,0 +1 @@ +../../../../../../../contrib/themes/sl0p.kdl
\ No newline at end of file diff --git a/web/demo/overlay/root/.vimrc b/web/demo/overlay/root/.vimrc new file mode 100644 index 0000000..6974506 --- /dev/null +++ b/web/demo/overlay/root/.vimrc @@ -0,0 +1,23 @@ +" The demo's vim: syntax highlighting on a pruned runtime -- syntax/, +" filetype detection, colours, and nothing else. :help is not installed. +set nocompatible +filetype on +syntax on + +" slosh's terminal does 24-bit colour (it is most of what the shaders are), +" and the pane's TERM is an alias of xterm-256color, whose terminfo does not +" say so. COLORTERM comes from /etc/profile; the t_8f/t_8b overrides are the +" standard incantation for truecolor on a TERM that is not *-direct. +if $COLORTERM ==# 'truecolor' + let &t_8f = "\<Esc>[38;2;%lu;%lu;%lum" + let &t_8b = "\<Esc>[48;2;%lu;%lu;%lum" + set termguicolors +endif +colorscheme habamax + +" A pane in an emulated machine: keep vim from wanting things that are not +" there. No swapfile chatter on a throwaway disk, no mouse fights with the +" multiplexer's own mouse handling. +set noswapfile +set mouse= +set laststatus=1 diff --git a/web/demo/overlay/root/demo.layout b/web/demo/overlay/root/demo.layout new file mode 100644 index 0000000..a5b6bd2 --- /dev/null +++ b/web/demo/overlay/root/demo.layout @@ -0,0 +1,27 @@ +// What the visitor lands in: one tab, three panes, one screen. +// +// slosh --layout /root/demo.layout +// +// which is the line /etc/inittab respawns. No `cwd=` anywhere in it: a relative +// path in a layout file means the directory the file is in, and a pane with no +// cwd at all starts there too -- so every pane here starts in /root. +layout { + tab name="demo" purpose="demo" { + + // The greeting, and a shell under it. This pane was *given* a command, + // so typing `exit` leaves the corpse and an offer to re-run it (`C-a r`) + // rather than closing the pane -- which is a pane state worth seeing. + pane focus=true purpose="demo:welcome" command="cat /etc/motd; exec /bin/sh" + + pane split="rows" { + + // Laid out, running nothing, started by your first keystroke: the + // pane shows what it *would* run. Here it is also the proof that + // the machine underneath is a real riscv64 one. + pane purpose="demo:machine" command="cat /proc/cpuinfo" suspended=true + + // No command: the session's shell, in /root. + pane purpose="demo:shell" + } + } +} diff --git a/web/demo/serve b/web/demo/serve new file mode 100755 index 0000000..07df7c3 --- /dev/null +++ b/web/demo/serve @@ -0,0 +1,92 @@ +#!/usr/bin/env python3 +"""Serve the built demo out of web/demo/build/site. + + web/demo/serve http://localhost:8000/ + web/demo/serve --port 9000 + web/demo/serve --dir build/site + +Not `python3 -m http.server`, for two reasons that both look like the emulator +being broken. Older mimetypes databases do not know .wasm, so the module +arrives as application/octet-stream and instantiation refuses it; and the +default handler caches, so a rebuilt root filesystem serves the old blocks +until someone thinks to hard-refresh. +""" + +import argparse +import functools +import http.server +import os +import sys + +TYPES = { + ".wasm": "application/wasm", + ".bin": "application/octet-stream", # bios, kernel, rootfs blocks + ".cfg": "text/plain; charset=utf-8", # the VM config, read by the emulator + ".txt": "text/plain; charset=utf-8", # the rootfs block index + ".html": "text/html; charset=utf-8", + ".js": "text/javascript; charset=utf-8", + ".css": "text/css; charset=utf-8", +} + + +class Handler(http.server.SimpleHTTPRequestHandler): + def guess_type(self, path): + return TYPES.get(os.path.splitext(path)[1].lower()) or super().guess_type(path) + + def send_head(self): + """Serve <path>.gz with Content-Encoding: gzip when the client allows. + + The build writes a .gz twin beside every block and every big asset; + XHR and the browser decode transparently, so the emulator sees the + same bytes off a wire carrying about half as many. This is what any + competent static host does on its own -- this server is just not one. + """ + path = self.translate_path(self.path) + gz = path + ".gz" + if ( + not path.endswith(".gz") + and os.path.isfile(gz) + and "gzip" in self.headers.get("Accept-Encoding", "") + ): + try: + f = open(gz, "rb") + except OSError: + return super().send_head() + self.send_response(200) + self.send_header("Content-Type", self.guess_type(path)) + self.send_header("Content-Encoding", "gzip") + self.send_header("Content-Length", str(os.fstat(f.fileno()).st_size)) + self.end_headers() + return f + return super().send_head() + + def end_headers(self): + self.send_header("Cache-Control", "no-store") + super().end_headers() + + def log_message(self, fmt, *args): + sys.stderr.write(" %s\n" % (fmt % args)) + + +def main(): + here = os.path.dirname(os.path.abspath(__file__)) + ap = argparse.ArgumentParser(description="serve the slosh web demo") + ap.add_argument("--port", type=int, default=8000) + ap.add_argument("--dir", default=os.path.join(here, "build", "site")) + args = ap.parse_args() + + if not os.path.isdir(args.dir): + sys.exit("%s does not exist -- run `make` in %s first" % (args.dir, here)) + + http.server.ThreadingHTTPServer.allow_reuse_address = True + handler = functools.partial(Handler, directory=args.dir) + with http.server.ThreadingHTTPServer(("", args.port), handler) as httpd: + print("serving %s at http://localhost:%d/" % (args.dir, args.port)) + try: + httpd.serve_forever() + except KeyboardInterrupt: + print() + + +if __name__ == "__main__": + main() diff --git a/web/demo/web/demo.css b/web/demo/web/demo.css new file mode 100644 index 0000000..3b7f608 --- /dev/null +++ b/web/demo/web/demo.css @@ -0,0 +1,184 @@ +/* Dark, quiet, one column. The terminal is the only thing on the page that + should be worth looking at, so everything else is grey and small. Colours are + contrib/themes/sl0p.kdl -- hot pink on near-black -- so the chrome and the + thing running inside it agree. */ + +:root { + --bg: #0b0b0e; + --panel: #141418; + --fg: #e8e8ea; + --dim: #8a8a95; + --faint: #45454a; + --pink: #ff5fd7; +} + +* { box-sizing: border-box; } + +body { + margin: 0 auto; + padding: 2rem 1.5rem 3rem; + max-width: 102rem; + background: var(--bg); + color: var(--fg); + font: 15px/1.55 -apple-system, "Segoe UI", Roboto, "Helvetica Neue", sans-serif; +} + +a { color: var(--pink); } +a:hover { color: #fff; } + +code, kbd { + font-family: "JetBrains Mono", Menlo, Consolas, "DejaVu Sans Mono", monospace; + font-size: 0.9em; +} + +header { margin-bottom: 1.25rem; } + +h1 { + margin: 0; + color: var(--pink); + font-family: "JetBrains Mono", Menlo, Consolas, monospace; + font-size: 1.6rem; + letter-spacing: 0.02em; +} + +/* The title is the way back to the front page, without growing a nav. */ +h1 a { color: inherit; text-decoration: none; } +h1 a:hover { color: #fff; } + +header p { + margin: 0.4rem 0 0; + max-width: 46rem; + color: var(--dim); +} + +/* The stage is a fixed box; the terminal inside it is a fixed grid scaled to + fit by demo.js. Absolute positioning keeps the scaled element from dragging + the layout around as it grows -- a transform does not reflow, but an + in-flow element that changed size would. */ +#stage { + position: relative; + overflow: hidden; + height: 78vh; + min-height: 360px; + padding: 10px; + border: 1px solid var(--faint); + border-radius: 6px; + background: var(--bg); +} + +#stage:focus-within { border-color: var(--pink); } + +#term_wrap { + position: absolute; + top: 10px; + left: 10px; + transform-origin: top left; +} + +#bar { + display: flex; + flex-wrap: wrap; + gap: 0.75rem; + align-items: baseline; + margin-top: 0.6rem; + color: var(--faint); + font-size: 13px; +} + +#bar code { color: var(--dim); } + +#status { + color: var(--pink); + font-family: "JetBrains Mono", Menlo, Consolas, monospace; +} + +#fetching { + visibility: hidden; + padding: 0 0.4rem; + border-radius: 3px; + background: var(--panel); + color: var(--pink); + font-family: "JetBrains Mono", Menlo, Consolas, monospace; +} + +#fetching.on { visibility: visible; } + +.spacer { flex: 1; } + +#geometry { font-family: "JetBrains Mono", Menlo, Consolas, monospace; } + +#keys_btn { + padding: 0.1rem 0.7rem; + border: 1px solid var(--faint); + border-radius: 3px; + background: var(--panel); + color: var(--dim); + font: inherit; + cursor: pointer; +} + +#keys_btn:hover { color: var(--pink); border-color: var(--pink); } + +/* The cheat-sheet, out of the way until asked for. A native <dialog>, so Esc + and the backdrop come from the browser rather than from more script. */ +#keys { + max-width: 44rem; + padding: 1.5rem 2rem; + border: 1px solid var(--faint); + border-radius: 6px; + background: var(--panel); + color: var(--fg); +} + +#keys::backdrop { background: rgba(0, 0, 0, 0.6); } + +#keys h2 { + margin: 0 0 0.3rem; + color: var(--fg); + font-size: 1rem; +} + +#keys p { + margin: 0 0 1rem; + color: var(--dim); +} + +#keys dl { + display: grid; + grid-template-columns: max-content 1fr; + gap: 0.35rem 1.25rem; + margin: 0; + font-size: 14px; +} + +#keys dt { text-align: right; white-space: nowrap; } +#keys dd { margin: 0; color: var(--dim); } + +#keys .dismiss { + margin: 1rem 0 0; + color: var(--faint); + font-size: 12px; +} + +kbd { + padding: 0.05em 0.35em; + border: 1px solid var(--faint); + border-radius: 3px; + background: var(--panel); + color: var(--pink); +} + +footer { + margin-top: 2.5rem; + padding-top: 1rem; + border-top: 1px solid var(--panel); + color: var(--faint); + font-size: 13px; +} + +footer p { margin: 0; max-width: 52rem; } + +@media (max-width: 40rem) { + #keys dl { grid-template-columns: 1fr; gap: 0 0; } + #keys dt { text-align: left; margin-top: 0.5rem; } +} diff --git a/web/demo/web/demo.js b/web/demo/web/demo.js new file mode 100644 index 0000000..e013e57 --- /dev/null +++ b/web/demo/web/demo.js @@ -0,0 +1,273 @@ +// The page half of the demo: build a terminal, hand its size to a riscv64 +// TinyEMU compiled to wasm, and shovel bytes between the two. +// +// Everything here talks to a *prebuilt* emulator -- Fabrice Bellard's +// riscvemu64-wasm.js, MIT, https://bellard.org/tinyemu/. That blob was +// compiled against TinyEMU's js/lib.js, whose emscripten glue reaches straight +// for globals: `term`, `update_downloading`, `graphic_display`, `net_state`, +// `Module`. So those are plain top-level `var`s and functions here, +// deliberately -- no module scope, no bundler, nothing that would hide them +// from a blob we do not compile. Rename one and the VM dies at boot with a +// bare ReferenceError from inside minified code. + +/* `term` is a two-method shim, not the xterm object: lib.js only ever calls + write() and getSize(), and keeping them apart means the emulator cannot + accidentally depend on the rest of xterm's API. */ +var term = null; +var graphic_display = null; /* text-only, but _fb_refresh reads it unguarded */ +var net_state = null; /* no networking: no socket to relay packets to */ +var Module = {}; /* emscripten adopts an existing global Module */ +var console_write1 = null; /* set in preRun: one byte -> the guest console */ + +/* Root filesystem blocks arrive over HTTP, lazily, 256 KiB at a time, so this + fires during ordinary use and not just at boot. Worth showing: a pause that + comes with a visible "fetching" reads as a download, and the same pause + without one reads as a hang. Hidden on a delay because bursts of block + fetches would otherwise strobe. */ +var downloading_timer = null; + +function update_downloading(flag) { + var el = document.getElementById("fetching"); + if (!el) + return; + if (flag) { + if (downloading_timer !== null) { + clearTimeout(downloading_timer); + downloading_timer = null; + } + el.classList.add("on"); + } else if (downloading_timer === null) { + downloading_timer = setTimeout(function () { + downloading_timer = null; + el.classList.remove("on"); + }, 500); + } +} + +(function () { + "use strict"; + + /* Parsed the way jslinux.js does it, so a URL that works there works here: + ?mem= (MB), ?cols=, ?rows=, ?cfg= (another VM config), ?cmdline= (extra + kernel arguments, appended to the config's own). */ + function get_params() { + var href = window.location.href; + var p = href.indexOf("?"); + var params = {}; + if (p < 0) + return params; + href.substr(p + 1).split("&").forEach(function (pair) { + var kv = pair.split("="); + params[decodeURIComponent(kv[0])] = decodeURIComponent(kv[1] || ""); + }); + return params; + } + + /* The emulator fetches the config with an XHR of its own and resolves + drive0's blocks relative to it, so it has to be absolute. */ + function get_absolute_url(fname) { + if (fname.indexOf(":") >= 0) + return fname; + var path = window.location.pathname; + var p = path.lastIndexOf("/"); + if (p < 0) + return fname; + return window.location.origin + path.slice(0, p + 1) + fname; + } + + function clamp(v, lo, hi) { + return v < lo ? lo : (v > hi ? hi : v); + } + + var params = get_params(); + var cfg_url = get_absolute_url(params["cfg"] || "slosh.cfg"); + var mem_size = (params["mem"] | 0) || 128; + var cmdline = params["cmdline"] || ""; + + var stage = document.getElementById("stage"); + var wrap = document.getElementById("term_wrap"); + var status_el = document.getElementById("status"); + + /* xterm.js rather than Bellard's term.js, which is the whole point of the + exercise: term.js has no truecolor (38;2) and no wide-character + handling, and slosh paints 24-bit colour shaders and lays out CJK. */ + var xt = new Terminal({ + cols: 80, + rows: 24, + cursorBlink: true, + convertEol: false, /* the guest sends proper CRLF; do not "help" */ + scrollback: 5000, + fontSize: 14, + fontFamily: '"JetBrains Mono", "Fira Code", Menlo, Consolas, "DejaVu Sans Mono", monospace', + theme: { + /* contrib/themes/sl0p.kdl: hot pink on near-black */ + background: "#0b0b0e", + foreground: "#e8e8ea", + cursor: "#ff5fd7", + cursorAccent: "#0b0b0e", + selectionBackground: "rgba(255, 95, 215, 0.30)", + black: "#141418", + red: "#ff5f5f", + green: "#5fd7a7", + yellow: "#ffd75f", + blue: "#5fafff", + magenta: "#ff5fd7", + cyan: "#5fd7d7", + white: "#e8e8ea", + brightBlack: "#45454a", + brightRed: "#ff8787", + brightGreen: "#87ffd7", + brightYellow: "#ffe787", + brightBlue: "#87cfff", + brightMagenta: "#ff87e7", + brightCyan: "#87e7e7", + brightWhite: "#ffffff" + } + }); + xt.open(wrap); + + /* Cell metrics come off the rendered grid rather than out of a table: the + font that actually resolved is the only one whose cell size is true. */ + function measure_cell() { + var screen = wrap.querySelector(".xterm-screen"); + if (screen) { + var r = screen.getBoundingClientRect(); + if (r.width > 0 && r.height > 0) + return { w: r.width / xt.cols, h: r.height / xt.rows }; + } + return { w: 8.5, h: 17 }; /* headless or display:none; keep going */ + } + + /* The room a scaled terminal has: the stage's content box, padding taken + off, since the wrapper sits inside that padding. */ + function room() { + var cs = window.getComputedStyle(stage); + var w = stage.clientWidth - parseFloat(cs.paddingLeft) - parseFloat(cs.paddingRight); + var h = stage.clientHeight - parseFloat(cs.paddingTop) - parseFloat(cs.paddingBottom); + return { w: w > 0 ? w : 1024, h: h > 0 ? h : 640 }; + } + + /* Geometry is chosen once, before vm_start, and then never again: jsemu.c + sets console_resize_pending only in console_init(), and exports no + resize entry point, so the guest hears the size exactly once at boot. + Nothing this page can do afterwards will reach it -- hence the scaling + below instead of a reflow. The caps exist only to keep a huge monitor + from asking the emulated CPU to repaint a stadium screen. */ + var cell = measure_cell(); + var box = room(); + var cols = clamp(Math.floor(box.w / cell.w), 80, 220); + var rows = clamp(Math.floor(box.h / cell.h), 24, 60); + if (params["cols"]) + cols = clamp(params["cols"] | 0, 20, 400); + if (params["rows"]) + rows = clamp(params["rows"] | 0, 8, 200); + xt.resize(cols, rows); + + var geom_el = document.getElementById("geometry"); + if (geom_el) + geom_el.textContent = cols + "x" + rows; + + /* Stretch the fixed grid to fill whatever room there is. transform-origin + is top left, so the translate re-centres what the scale left hanging off + one side. The grid stays the size the guest was told; only the pixels + move. */ + var natural = { w: 0, h: 0 }; + + function remeasure() { + wrap.style.transform = "none"; + var r = wrap.getBoundingClientRect(); + natural = { w: r.width, h: r.height }; + rescale(); + } + + function rescale() { + if (!(natural.w > 0) || !(natural.h > 0)) + return; + var b = room(); + var s = Math.min(b.w / natural.w, b.h / natural.h); + if (!(s > 0)) + return; + s = Math.min(s, 4); + var dx = Math.max(0, (b.w - natural.w * s) / 2); + wrap.style.transform = "translate(" + dx.toFixed(2) + "px, 0px) scale(" + s.toFixed(4) + ")"; + } + + remeasure(); + window.addEventListener("resize", rescale); + + /* Output: lib.js builds the string with String.fromCharCode over the + guest's bytes, so each char code is one byte and the string is latin1, + not decoded text. Widen it back to bytes and let xterm do the UTF-8 -- + that is what makes a wide glyph occupy two cells instead of arriving as + two mojibake ones. */ + var booted = false; + + term = { + write: function (str) { + if (!booted) { + booted = true; + if (status_el) + status_el.hidden = true; + } + xt.write(Uint8Array.from(str, function (c) { + return c.charCodeAt(0) & 0xff; + })); + }, + getSize: function () { + return [cols, rows]; + } + }; + + /* Input: the reverse, and the place jslinux.js has a bug worth not + copying. It sends charCodeAt(i) per JS char, which truncates anything + past U+00FF and splits everything else wrong; encode to UTF-8 and send + every byte. */ + var encoder = new TextEncoder(); + + xt.onData(function (s) { + if (!console_write1) + return; + var bytes = encoder.encode(s); + for (var i = 0; i < bytes.length; i++) + console_write1(bytes[i]); + }); + + stage.addEventListener("mousedown", function () { xt.focus(); }); + xt.focus(); + + /* The cheat-sheet lives in a native <dialog>: showModal() gives Esc and a + backdrop for free, and while it is open the terminal is not focused, so + nothing typed at the sheet leaks into the guest. Clicking the backdrop + is a click whose target is the dialog element itself. */ + var keys_dlg = document.getElementById("keys"); + var keys_btn = document.getElementById("keys_btn"); + if (keys_dlg && keys_btn && keys_dlg.showModal) { + keys_btn.addEventListener("click", function () { keys_dlg.showModal(); }); + keys_dlg.addEventListener("click", function (ev) { + if (ev.target === keys_dlg) + keys_dlg.close(); + }); + keys_dlg.addEventListener("close", function () { xt.focus(); }); + } + + /* preRun runs after the wasm instance exists and before main(), which is + the only window where cwrap works and the VM has not started. */ + function start() { + console_write1 = Module.cwrap("console_queue_char", null, ["number"]); + /* width/height 0 says text console; the 0 after them is "no network". */ + Module.ccall("vm_start", null, + ["string", "number", "string", "string", "number", "number", "number", "string"], + [cfg_url, mem_size, cmdline, "", 0, 0, 0, ""]); + } + + Module.preRun = start; + + /* Loaded last and by hand, so every global above is in place first. */ + var script = document.createElement("script"); + script.src = "riscvemu64-wasm.js"; + script.onerror = function () { + if (status_el) + status_el.textContent = "could not load riscvemu64-wasm.js -- run `make all`"; + }; + document.getElementsByTagName("head")[0].appendChild(script); +})(); diff --git a/web/demo/web/index.html b/web/demo/web/index.html new file mode 100644 index 0000000..f5041c6 --- /dev/null +++ b/web/demo/web/index.html @@ -0,0 +1,71 @@ +<!DOCTYPE html> +<html lang="en"> +<head> +<meta charset="utf-8"> +<meta name="viewport" content="width=device-width, initial-scale=1"> +<title>slosh — demo</title> +<link rel="stylesheet" href="xterm.css"> +<link rel="stylesheet" href="demo.css"> +</head> +<body> + +<header> + <h1><a href="../">slosh</a></h1> + <p>A demo machine with the terminal multiplexer running in it, booted inside + this tab. It is yours and it is throwaway — reload the page for a fresh + one.</p> +</header> + +<main> + <div id="stage"><div id="term_wrap"></div></div> + <div id="bar"> + <span id="status">booting…</span> + <span id="fetching">fetching…</span> + <span class="spacer"></span> + <span id="geometry"></span> + <button id="keys_btn" type="button">keys</button> + </div> +</main> + +<dialog id="keys"> + <h2>Keys</h2> + <p>Everything is the leader, <kbd>C-a</kbd>, and then a key. Press it twice to + send a literal <kbd>C-a</kbd> to whatever is in the pane. <kbd>C-a</kbd> + <kbd>?</kbd> shows the full sheet inside the session, built from the bindings + the config actually has.</p> + <dl> + <dt><kbd>Enter</kbd></dt><dd>split whichever way there is more room</dd> + <dt><kbd>\</kbd> <kbd>-</kbd></dt><dd>split into columns / rows, when you mean one</dd> + <dt><kbd>h</kbd> <kbd>j</kbd> <kbd>k</kbd> <kbd>l</kbd></dt><dd>move focus (arrows work too)</dd> + <dt><kbd>o</kbd></dt><dd>the next pane</dd> + <dt><kbd>H</kbd> <kbd>J</kbd> <kbd>K</kbd> <kbd>L</kbd></dt><dd>move the boundary between panes</dd> + <dt><kbd>=</kbd></dt><dd>give every visible pane an even share</dd> + <dt><kbd>Space</kbd></dt><dd>turn the layout a quarter turn</dd> + <dt><kbd>z</kbd></dt><dd>zoom this pane to fill the tab, and back</dd> + <dt><kbd>m</kbd></dt><dd>minimise it into the strip along the bottom</dd> + <dt><kbd>x</kbd> <kbd>X</kbd></dt><dd>close this pane / this whole tab</dd> + <dt><kbd>c</kbd></dt><dd>new tab</dd> + <dt><kbd>Tab</kbd> <kbd>1</kbd>…<kbd>9</kbd></dt><dd>next tab / that tab</dd> + <dt><kbd>f</kbd></dt><dd>find a pane by name</dd> + <dt><kbd>p</kbd></dt><dd>the command palette</dd> + <dt><kbd>e</kbd></dt><dd>edit the config, in a pane — it is re-read the moment you save</dd> + <dt><kbd>?</kbd></dt><dd>the cheatsheet</dd> + <dt><kbd>d</kbd> <kbd>q</kbd></dt><dd>detach, leaving it running / quit the session</dd> + </dl> + <p class="dismiss">Esc, or click outside, to close.</p> +</dialog> + +<footer> + <p>The emulator is <a href="https://bellard.org/tinyemu/">TinyEMU / JSLinux</a> + by Fabrice Bellard, MIT licence; the kernel and BIOS are his, unmodified. The + terminal is <a href="https://xtermjs.org/">xterm.js</a>. The machine's disk + streams in during startup — that is <em>fetching</em> blinking — + and after that the session is entirely local. The terminal size is fixed at + boot, so resizing this window scales the picture; <code>?cols=</code> and + <code>?rows=</code> pick another grid.</p> +</footer> + +<script src="xterm.js"></script> +<script src="demo.js"></script> +</body> +</html> diff --git a/web/demo/web/slosh.cfg b/web/demo/web/slosh.cfg new file mode 100644 index 0000000..e371365 --- /dev/null +++ b/web/demo/web/slosh.cfg @@ -0,0 +1,21 @@ +/* The VM the page boots. TinyEMU's own config format -- JSON with unquoted + keys, trailing commas and C comments -- read by the emulator over HTTP. + Paths are relative to this file. + + drive0 is a virtio block device rather than an initrd on purpose: Bellard's + prebuilt riscv64 kernel is built without CONFIG_BLK_DEV_INITRD, so an + `initrd:` line here is accepted, ignored, and then the kernel panics with + "Unable to mount root fs". blk.txt is the block index; the .bin blocks + beside it are fetched lazily, 256 KiB at a time, as the guest reads them. + + No eth0: nothing in this demo wants the network, and leaving the driver out + means no websocket relay is contacted. */ +{ + version: 1, + machine: "riscv64", + memory_size: 128, + bios: "bbl64.bin", + kernel: "kernel-riscv64.bin", + cmdline: "console=hvc0 root=/dev/vda rw", + drive0: { file: "rootfs/blk.txt" }, +} diff --git a/web_new/index.html b/web/index.html index e75b8ba..e75b8ba 100644 --- a/web_new/index.html +++ b/web/index.html diff --git a/web_new/install.sh b/web/install.sh index f2b8c97..f2b8c97 100755 --- a/web_new/install.sh +++ b/web/install.sh diff --git a/web_new/site.css b/web/site.css index a46e45f..a46e45f 100644 --- a/web_new/site.css +++ b/web/site.css diff --git a/web_plan.md b/web_plan.md deleted file mode 100644 index 677c266..0000000 --- a/web_plan.md +++ /dev/null @@ -1,21 +0,0 @@ -ok listen up dude, - -we're going to need a good webpage for this slosh thing - -we have some semi good (generated) docs/ website thing already - -also we have very cool web-based linux demo system in ./contrib/webdemo/ - -we should integrate all of these to a good single page website (minus the live demo thing maybe can live on its own page to reduce innitial clutter0 - -page will be simple in terms of getting the point accross what slosh is/does - -we'll render a good demo video (TBD, put placeholder) - -some `curl http://slosh.foo/install.sh | sudo sh` instructions - -make sure it doesn't look like 1 of those thousand a dozen claude-opus webpages - -super tired of those - -lets go, u put artifacts in ./web_new (mkdir, not exist yet) |
