Black Window

not loaded
hover a setting for the long version
offline

Offline: the page notices when the connection drops. Live lookup and link reading pause on their own and resume when it returns; your settings are not changed. A model that has been loaded once runs without the network.

read wholea question about the whole of a long file: read it into notes first (the slow read, minutes), or answer from its opening and nearest passages (the fast read, seconds)
no modelfiles drop into the weave and are read (sound, pictures, documents) with nothing else loaded; no replies. Tells a read's own cost apart from the model's.
windowtokens the model sees at once; auto sizes it to this device
the window and its cost

tokens the model can see at once. Auto sizes it to this device's memory; each 8K of window costs 0.3 to 1.2 GB of GPU memory depending on the model. Replies and the attachment budget grow with it. Only a turn that fills the window is slow: with 16K tokens in front of it a model reads for tens of seconds and answers several times slower than usual.

the weave

The Weave is the memory behind the window. Everything the tab reads is embedded as it arrives (on this device's GPU when it has one: hundreds of passages a second, a million tokens in under a minute); each question brings its nearest passages into the window as notes; a long file offers read it all on its line, which reads it once, part by part, into notes, so a question about the whole document has every part to draw on. In the console: bw.digest("file"), bw.notes("question").

read time grows with the model

Read time grows with the model. Embedding is done by a 33M-parameter encoder, which is why a file is indexed in seconds. Reading it into notes is done by the chat model, and every token goes through every weight once, so a bigger model reads slower: measured here on one device with a long prompt, 425 tokens/s for the 0.6B, 319 for the 1.7B, 134 for the 8B. Writing slows the same way, and more so with a full window, since each new token re-reads everything before it. What comes out grows with size too: a 0.6B writes short answers and thin notes (names and figures, little of the argument); a 1.7B to 3B sound answers and plain notes; a 4B good ones; a 7B or 8B the best notes and reasoning, at the slowest read. The model tiles show the read time for a 100-page file on this device once a rate has been measured, and what goes in front of the model whole is bounded by what it reads in about 30 seconds here, unless you set the window yourself above.

what a session file holds

A session file holds the transcript with each turn's evidence and timing, everything the tab indexed (memory, files, lookups), the attachments and the spider web. It is written to your device; nothing is uploaded. Opening one adds it to this tab and re-indexes its passages, so memory and the spider continue from it. In the console: bw.export(), bw.import(obj).

relay
how to call it

In this tab's console: await bw.ask("..."), bw.search("..."), bw.settings. From a page that embeds or opened this one: postMessage({bw:"ask", id, text}), replies as {bw:"reply", id, reply, ...}. From anything on this machine: run bw_relay.py (python3, no dependencies) and curl -s localhost:8770/v1/chat/completions -d '{"messages":[{"role":"user","content":"..."}]}'. The tab polls the relay; nothing else leaves it.

Places a model can run: this Mac, and every box below, reached over SSH or at an https URL. Each SSH box's llama-server is forwarded to a local port; the catalog marks the models each place holds and loads them there. Prompts go to the box a model runs on; files and the Weave stay here.

add box
vast key

how boxes work

For each box the app keeps an SSH tunnel (reconnecting when it drops, with keepalives) from a local port to the box's llama-server, probes the box (GPUs, RAM, CPU, where llama-server is), grades every catalog model against it, and shows what the box serves now as a tile. run on <box> on a tile stops what the box serves and starts its llama-server on that model; the Hub pull happens on the box. Speed, prompt-cache hits and window come from the server's own timings. Rent boxes on vast.ai through our link (the catalog's rent on vast); any machine with SSH and llama-server works the same. See HOSTED.md.

Black Window is a chat model on your own machine with a memory that does not forget: the Weave. Every file, picture, recording, link and reply is indexed the moment it arrives and recalled per question from millions of passages in milliseconds. A context window holds one conversation and forgets it when it fills; the Weave holds everything you have ever given it, at any size, and puts the parts that matter in front of the model each turn. The window itself is a working set, the last turn or two beside the notes, so a conversation never hits its end: earlier turns come back by recall. A long document reads once into notes so the whole of it answers. Save writes the session and the Weave to a file on your disk; the next visit offers it back with one click. Reads twenty kinds of file. Larger than this machine? Rent a box from the catalog, connected in three clicks; prompts go there, your files and memory stay here. Nothing leaves the device unless you turn on live lookup or a box, and the chat says so each time. Pick a model below and Load.

checking this device…
a box on vast.aifor a model this Mac cannot hold. You rent the box on your own vast.ai account; the app reaches it over SSH and the model appears in the catalog. Nothing to sign up for here, no payments through us.
1
Rent the box. Open vast.ai (our referral link) and rent a box with enough memory for . To see boxes free now, at the host's price:
2
Give vast this Mac's key so the box lets the app in: account > SSH keys, add …
3
Start the server on the box. In the instance's template use the image ghcr.io/ggml-org/llama.cpp:server-cuda and this on-start command 

      
4
Connect the box. From the instance card, the SSH line (ssh -p PORT root@sshN.vast.ai):    

      

      
A box connected here is listed under set > hosted, where it can be stopped, renamed, given its hardware, or removed; a box you already have (any llama-server behind an https URL, or an SSH line in the app) can be added there directly.

    

    
    

      
      
      
      
    

    
    

    

      
      
    

    
    
Works offline once a model has loaded. Turn the network off and the page says so in the header; lookup and link reading pause and resume on their own.