Developers: quickstart and API

Sunstone North Labs LLC · burton@sunstonenorth.com

Quickstart

In the browser

Open blackwindow.xyz. The catalog grades models against this machine's WebGPU budget; pick one under runs in this browser and Load. Models up to about 8B run here; the memory encoder (bge-small) runs on WebGPU too.
Drop files on the page: text, code, PDF, Word, spreadsheets, EPUB, ZIP. Each is cut into passages and embedded on arrival; every question recalls the nearest passages and puts them in front of the model as numbered notes. Sound and video are transcribed in the tab beside the model (whisper-tiny on desktops; Moonshine-tiny on phones, English, 172 MB of wasm); an mp3 or WAV of any length is read in slices of a few minutes, each transcribed and woven before the next is decoded, so a two-hour recording never holds more than one slice in memory. A text, log or code file over 4 MB is read the same way, slice by slice into the weave, with its opening in front of the model. A picture is read by a tower: MobileCLIP-S0's image side (43 MB, beside the model on a phone) and two linear heads fitted on 118K COCO pairs give the objects it holds (80 kinds, mAP 0.75) and the three COCO captions it most resembles, in the memory's own space (val t2i R@1 0.32, R@10 0.71 against 0.42/0.80 for the 27B tower in the app); those words are what the model reads. No model looks at the pixels in the browser.
read it all on a long file's line (or a question about the whole of it) reads the file whole into notes, one per part, and a question about the whole document gets every note together. On a small window the part notes can outgrow the window; then a second and, if needed, a third pass reads the notes into fewer, each pass named and counted on the file line, and a question gets the finest level that fits. Every level stays searchable.
The window is a working set, not the transcript: the last turn or two sit beside the notes, every finished turn is woven into memory, and an earlier turn comes back by recall when a question reaches for it. The footer shows the weave against the window (weave 187 passages ≈ 50K tokens, 49× the window).
save writes one session file (transcript, passages, vectors, attachments, spider web) to your disk; the next visit offers it back with one click, and opening it is a read, not a re-embedding.

On a phone

The same page. Qwen3-0.6B runs in a Safari tab beside the memory and the media encoders (320 MB of weights, 1K window, 128-token prefill chunk from our own WebLLM build); Qwen3-1.7B (923 MB) runs from the Home Screen app, which gets more memory than a tab. Add the page with share → Add to Home Screen; close other tabs and keep the page in front while it reads or answers, since iOS reclaims a background tab first. The catalog marks each model against this phone, and a model that dies here is taken out of the picker until set > weave > forget crashes. set > weave > open the chat without a model opens the chat with nothing loaded, for reading files into the weave on their own.

On WebKit the media worker loads onnxruntime's plain wasm build rather than the WebGPU-capable one transformers.js ships: on Safari that build took the page from 243 MB to 3.8 GB over one recording; the plain build peaks at 1.1 GB for the same read. The model stays loaded through a read (bw.set("mediaAside", true) steps it aside instead). Sound is read by Moonshine-tiny in 30-second pieces (bw.set("speechModel", "whisper") for languages other than English).

A model this machine cannot hold

In the catalog, under needs a hosted box, pick the model and click rent a box for it. Step 1 links our vast.ai template for that model: image, start command, disk and memory filters preset. Pick a machine and rent, on your own vast.ai account.
When the model has loaded, click Open on the instance card. You land back here with the box connected; its model is a tile, and the catalog marks every other model the box could hold. The same link is in the instance's logs as a BLACK WINDOW: open … line.
Boxes are listed under set > hosted: stop, rename, hardware, key, remove. A box is an https URL plus llama-server's api key; both stay in this browser.

The Mac app

The same page inside a native shell: models run on Metal through llama.cpp, the Weave is a store on disk that outlives the session, images are looked at by vision models, audio and video are transcribed on the device, and boxes are reached over SSH with a run on <box> button per model.

What a box is

A box is any machine running llama.cpp's llama-server with an api key, reachable over https. Our templates make one from the image ghcr.io/ggml-org/llama.cpp:server-cuda with this on-start:

K=$(head -c 16 /dev/urandom | od -An -tx1 | tr -d " \n"); echo "$K" > /root/.bw_key
# a Cloudflare quick tunnel to the server, and a pairing line once its URL is known
cloudflared tunnel --url http://127.0.0.1:8080 | … echo "BLACK WINDOW: open https://blackwindow.xyz/#box=$(printf '{"url":"%s","key":"%s"}' "$u" "$K" | base64 -w0)"
/app/llama-server -hf <repo>:<quant> --host 127.0.0.1 --port 8080 --api-key-file /root/.bw_key -c 32768 -fa on --jinja

The pairing link is <page>#box=base64({"url", "key", "name"?}). Opening it, or pasting the whole line anywhere the page takes a box, adds the box. The fragment never reaches a server. The Open button on the vast card reaches a small responder on the box's port 1111 that redirects to that link once /health answers (403 without vast's per-instance token).

Anything else that speaks the OpenAI chat API (vLLM, LM Studio, Ollama, your own llama-server) works as your own server on the catalog page: an https URL, or http://localhost:PORT on the same machine. The page reads /v1/models and /props, streams /v1/chat/completions with stream_options.include_usage and timings_per_token, and sends Authorization: Bearer <key> when the box has one.

A box with many models

llama-server's router mode (--models-preset models.ini --models-max 1) serves every model on the box's disk and loads one on demand. The page reads /models (status unloaded, loading, loaded), shows one tile per model under on your server (the loaded one first), and Load posts /models/load and waits on the status with the wait in the header: a 9B in seconds, a 27B in a minute or two, a 544 GB Kimi in about thirty minutes cold. Tile ids are remote:<url>|<model>. A hosted pick in the recommended set that the box holds shows as the box's tile instead of a rent chip. The preset generator, start script and probes are in research/bench/ (box_preset.py, box_start.sh, box_caps.py); the design note is research/notes/box_router.md.

What a server model gets from the page: reasoning streamed as thinking (reasoning_content deltas), the server's own decode rate on the speed line, enable_thinking/thinking/reasoning_effort set together so Qwen, Kimi and gpt-oss all obey the same switch, and a prompt whose head is constant from turn to turn (the per-turn notes ride in the user message) so the server's prompt cache keeps the transcript.

The weave across models

A session is passages, turns and vectors; the model is a label on it. A weave written beside a large model opens beside a small one. Load a 7B (or a box), drop the documents, ask what matters, read the long ones whole, save; drop the file on the phone with +, with any model loaded or none. The text is re-indexed for that device's reader (a phone's lexical reader drops the desktop's dense vectors) and the small model answers from the large model's replies and notes at its own speed. What travels is knowledge as passages, not weights: a transformer touches every matrix for every token, so there is nothing in a model to recall selectively.

Calling the page

Three doors, all in this tab; nothing leaves it.

Console: `window.bw`

`await bw.ask("…")`	a turn, as if typed; resolves with the reply and its notes
`bw.search("…")`	the retrieval alone
`await bw.notes("…", k)`	the k nearest passages with scores
`bw.attach(name, text)`	index text under a source name
`await bw.files([File, …])`	read files as a drop would (sound, pictures, documents, a session file)
`bw.digest("file.pdf")`	read an attached file whole into notes
`bw.export()` · `bw.import(obj)` · `bw.save()`	the session as an object; open one; write the file
`bw.weave()` · `bw.store(name)` · `bw.open(source)` · `bw.forget(source)`	the store (app): size, named stores, documents read whole, removal
`bw.settings` · `bw.set(key, value)` · `bw.ready`	settings as an object, one setting, whether a turn can start

Settings worth knowing from the console: rowBits (8, 4 or 1: how dense memory rows are stored, default 4, centred by the pool mean so int4 ranks as f32 does), speechModel ("whisper" on a phone), mediaAside (step the model aside for a media read), speechGpu (whisper on WebGPU where there is one), digestW (chars per part when reading whole), digestLanes (parts in flight on a server with slots), ctx (window; "auto" sizes it to the device).

postMessage

From a page that embeds or opened this one: postMessage({bw:"ask", id, text}); the reply comes back as {bw:"reply", id, reply, …}. Requires set > api > serve this tab.

Local relay

For anything else on the same machine: run bw_relay.py (python3, no dependencies), turn on set > api > serve this tab, then

curl -s localhost:8770/v1/chat/completions -d '{"messages":[{"role":"user","content":"…"}]}'

The tab polls the relay and answers each request as an ordinary turn, so what the API answered is what you can see in the transcript.

Session file

{ format: "blackwindow-session", version: 2, turns[], passages[{key, source, text}], vectors{reader, dim, n, keys[], f16}, attachments[], spider{} }. Vectors are bge-small (384-d) as base64 float16; a file without them is re-embedded on open, and a phone re-indexes the text lexically. Passage keys name their source: audio:<file>#n (timed transcript segments), imagetext:<file>, digest:<source>#n (one note per part of a file read whole; ~2 and ~3 for the passes over the notes), memory:turn N# (a finished turn), wikipedia:<title>#n.

With more than one chat on the page, save writes them all: { format: "blackwindow-session", version: 3, current, chats[{id, name, created, updated, named, data}] }, where each data is a version 2 session. Each chat is its own weave; opening a version 3 file brings every chat back and returns to the current one. Version 2 files still open as one chat.

Measured

Numbers behind the claims in the catalog, from the engineering notes: bge-small on WebGPU indexes 229 to 896 passages/s (M2 Ultra); the store searches 2.8M passages in 36 ms in the app; the browser's lexical index answers over 100K passages in 10 ms after a 0.3 s build; 40K prose passages hold 17 MB of heap (text deflated in blocks, varint postings) and 9K dense rows 3 MB at int4; a 1T-parameter MoE (Kimi-K2.6) on a 1 TB RAM box answers at 7.5 to 9 tok/s through a tunnel; a box rented from a template was serving and paired in under 3 minutes for $0.002. On an iPhone: a 66-minute mp3 read into 173 timed passages beside the loaded 0.6B, then read whole into notes and answered; the 0.6B decodes at 25 to 30 tok/s there. Ask for the notes: burton@sunstonenorth.com.