The live video-out: ritmolux --stream

Every other instrument on this page writes a file. This one writes a live video stream into another program, with no window on our side, no codec anywhere, and a latency of a frame or two (ADR-0125).

There are two sinks, and --sink picks between them.

# Spout, the default. Open a Syphon Spout In TOP in TouchDesigner and set its
# Sender Name to `ritmolux`.
ritmolux --stream --size 1280x720 --fps 60

# A pipe. Raw RGBA8 frames on stdout, for a parent process that spawned this.
ritmolux --stream --sink stdout --events | your-program

--sink spout: another application on the same machine

TouchDesigner, Resolume, OBS — anything that receives Spout. It exists only in a build with the spout feature. The shipped release ritmolux.exe has it; a plain cargo build does not, and --sink spout there fails with a named error rather than starting and publishing nowhere. To build it yourself you need the SDK staged first — it is third-party, pinned by hash and never committed:

powershell -File packaging/spout/fetch-sdk.ps1
cargo run -p standalone --bin ritmolux --features spout --release -- --stream

--sink stdout: raw frames on a pipe

Every platform, no feature, nothing installed. The frames go to standard output as tight width x height x 4 bytes of RGBA8, in order, with nothing between them — no header, no length prefix, no padding. A reader cuts the stream into frames by multiplying the geometry, which it learns from the stream event on standard error before the first byte arrives:

{"v":1,"ev":"stream","width":640,"height":360,"fps":30,"format":"rgba8"}

That event needs --events, which is what turns the structured report on at all (Configuration). Without it the frames still flow and the reader has to know the geometry some other way — which is why the studio always passes both.

format is rgba8 here because a headless run renders into an offscreen this engine chooses, and it is read off that texture rather than asserted. The windowed mirror below is the path where it can be the other value, so a reader that hard-codes one order works on this sink and paints red and blue swapped on that one.

The default is 640x360 at 30 fps, not the Spout path’s 1280x720 at 60: this sink exists to feed a preview canvas, and asking the engine for four times the pixels to shrink them into a panel costs the readback and the pipe for a picture nobody sees at that size. --size and --fps override it in either direction.

The writer blocks; it never drops. A reader that stops reading fills the pipe and stalls this process, and the deadline pacing absorbs that: frame n stays due at n * period from the start of the run, so a stalled second costs the frames that fell inside it and the run resumes at the frame index the wall clock has reached rather than drifting behind it. That is the right policy for a loop with no present deadline; a windowed preview has one and drops instead.

A headless run is a whole show, and it is drivable. It resolves and seeds the same per-user preset directory a window does, watches it, and hot-reloads a file you save into it; and it reports the same events — roster on every reload, preset when the picture changes, preset_error and preset_warning with the file and the line, health once a second. --control binds a listener here exactly as it does for a window, and hello carries the port it actually got, so a parent can move a parameter on the next frame. The window and the sink are what differ between the two modes; nothing in this paragraph does (ADR-0183).

On a machine with no per-user data directory at all — a bare CI runner — the run says so in one line and carries on with the presets built into the binary, which is the same degrade every other reader of that directory performs.

Standard output carries nothing else while this sink is open — every human-readable line goes to standard error, and a test holds the whole crate to that.

--preview stdout: the windowed show, mirrored

The sink above is a headless run: no window, and the pipe is the only picture there is. --preview stdout is the other half — a windowed run keeps its show on the projector and sends a copy of it up the same kind of pipe, announced by the same stream event. It is how a studio watches the picture the audience is actually seeing.

The two pipes are not the same thing, and the difference is the point.

	--stream --sink stdout	--preview stdout
the show	headless, no window	a window, unchanged
the frames	exact, at --size	a scaled, letterboxed copy
the geometry	the size you asked for	fixed at 640x360, or @WIDTHxHEIGHT
format	always rgba8	rgba8 or bgra8 — read it
a slow reader	blocks the run	drops frames, the show carries on

Read format. The mirror carries whatever the swapchain negotiated, which on a DX12 backend is commonly BGRA, and the stream event names the order the bytes are actually in. A reader that assumes RGBA paints the show’s oranges blue and its blues orange — a picture that looks like a colour-grading mistake rather than a decoding one, which is why the player refuses to open the pipe at all rather than guess a name for a format it cannot describe.

--sink stdout is an exact feed for ffmpeg, so its frames are the size named on the command line and nothing scales them. --preview is a mirror for a canvas, so it is fixed-size on purpose: the show’s window may be dragged, maximized or thrown fullscreen mid-run, and the mirror’s geometry does not move with it. A reader is told the size once, before the first byte, and never again — which is the only arrangement a headerless byte stream can survive, since bytes already in the pipe carry no mark saying where a new size began.

The show’s aspect is kept: a window of a shape the mirror is not gets bars rather than a squash. What that costs is pixel-scale judgement — a one-texel seam or the exact width of a contour is not a thing to read off a scaled copy. Look at the show window for those.

ritmolux --events --preview stdout            # 640x360
ritmolux --events --preview stdout@960x540    # something else

The TouchDesigner side

The operator is called Syphon Spout In, not “Spout In” — Derivative ships the two app-to-app transports as one TOP, Spout on Windows and Syphon on macOS, so that is the name in the OP Create Dialog. Set its Sender Name to whatever the mode prints, which is not always what you asked for:

publishing 1280x720 at 60 fps as Spout sender 'Ritmolux'

SetSenderName increments on collision — a run that was force-killed leaves its registration behind, and the next one comes up as rlx_1, then rlx_2. The mode prints the name it actually got for exactly this reason. A TOP pointed at a name nobody is publishing reports No Active Sender Found.

No colour setting is needed on either side. The engine reads back display-referred sRGB bytes and Spout publishes them untouched (SetSenderFormat(DXGI_FORMAT_R8G8B8A8_UNORM) matches the readback, so nothing swaps red and blue and nothing re-encodes). A receiving TOP put beside a Movie File In TOP of the same frame as a PNG is indistinguishable from it. If your picture looks washed out or crushed, something in your network is re-interpreting it, not the sender.

Which GPU, and why it is not a preference

On a machine with one GPU, skip this. On a hybrid laptop it is the difference between a picture and nothing at all.

A Spout sender shares a D3D11 texture by handle, and the receiver opens that handle on its own device — which succeeds only when both devices are the same physical GPU. Windows hands a plain console process the integrated GPU to save power while TouchDesigner runs on the discrete one, and the receiver then reports only Unable to open shared Spout Texture, naming neither adapter nor the mismatch.

ritmolux --list-adapters              # both rosters, with their own indices
ritmolux --stream --gpu "RTX 3080"    # one name moves the renderer AND the sender

--list-adapters prints two lists because there are two enumerations, and they are not assumed to agree on order — the renderer selects through wgpu, the sender through the Spout SDK, and on a machine with a software rasterizer installed the two lists are not even the same length. --gpu takes a name (a substring is enough, case-insensitively) or an index, and each side resolves it against its own roster; an ambiguous name is an error listing what it matched, never an arbitrary pick.

Unset, the renderer asks for the high-performance adapter and the sender follows it by name. Both resolved choices are printed at startup, always:

renderer : NVIDIA GeForce RTX 3080 Laptop GPU (Dx12, DiscreteGpu), driver 32.0.15.8142
sender   : adapter [1] NVIDIA GeForce RTX 3080 Laptop GPU

Both are fixed for the life of the run. The window’s settings menu can move a running show onto another adapter (Running the app), but a --stream run has no window and no menu, and its renderer keeps the adapter it was built on; the sender, whose adapter is a correctness constraint rather than a frame-rate one, is never re-opened underneath a connected receiver. To publish from the other GPU, stop the run and start it again with --gpu — the receiver re-acquires the sender by name as it does after any restart. [output] gpu in config.toml is the window’s stored choice and is not read here: this mode takes only the flag, so what a receiver connects to is always what the command line says. On Linux, --sink stdout writes pixels to a pipe and has no adapter tie at all.

Presets, and stopping

Presets rotate on the operator config’s [rotate] dwell timer exactly as they do in the window — and rotation is on here even when auto is off, because a headless source has nobody to press Space and a four-hour set on one scene is not what this mode is for. --preset <name> holds one scene and turns rotation off. Rotations are announced:

rotate   : frame 5400, AutoTimer -> 'Thomas Gallery'

Ctrl-C stops the run through its own exit path, which is what makes it print the three numbers a measurement needs — frames emitted, wall clock, scene clock. --frames N bounds a run so it terminates and reports on its own.

What it costs, and what those numbers mean

Every 30 s and at exit, the mode reports its per-stage cost and its resident set:

stream: draw+submit 7.79 ms, spout send 0.79 ms, mean over 1800 frames
render: resident set 277 MB, growth +0.2 MB across 5400 frames ...
stream: 36000 frames, 600.00 s wall, 599.99 s scene clock, on NVIDIA GeForce RTX 3080 ...

Two stages, not three. draw+submit is the cost of producing a frame: taking the previous frame’s readback, encoding the passes and submitting them. The tap keeps one frame in flight, so a frame is published while the next one is being drawn, and it waits for the GPU only when the GPU is behind. On an adapter that keeps up, the figure is CPU time alone. On one that does not, it carries the wait as well, and the run is held to the GPU’s rate rather than queueing frames it would never publish. What the GPU itself spent is the per-pass table below, which needs timestamp queries rather than a clock on this side. The second stage is the sink’s own, and it is named for the sink — spout send is the upload into the sender’s device, pipe write is the blocking write to standard output — so a figure copied out of a log says which one produced it. The split answers the question that matters, whether the sink is what limits the rate, and on the development machine it is not, by an order of magnitude.

One frame of latency, and it is the whole price. The first call publishes nothing, so a run emits one fewer frame than it draws, and every published frame is one frame behind the scene clock. --frames N still puts exactly N frames on the sink.

A figure taken before this is not comparable to one taken after it. The readback used to block on every frame, so the first stage always carried the GPU’s execution time as well as the CPU’s; now it carries a wait only on an adapter that is behind. The reading in the next paragraph is one of those, kept as the record of what the old figure was rather than as something to compare against a new run.

Measured, on one machine, once, against the blocking readback (RTX 3080 Laptop, 1280x720 at 60 fps, one preset held, nothing else on the GPU): a 30-minute run emitted 108,000 frames in 1800.00 s wall against 1799.99 s scene, at 3.67-7.82 ms of the first stage and 0.27-0.58 ms of Spout send per frame, with the resident set at a 280 MB peak growing 2.0 MB across the whole run. That is a reading from one box and one driver, not a specification — a machine that cannot hold the rate reports it the way the next paragraph describes.

A third reading sits under those two: what each GPU pass cost. Where the adapter offers timestamp queries, the same 30 s report and the exit print one row per labelled render or compute pass, in mean GPU milliseconds per frame, costliest first:

stream: pass costs, mean per frame over 1800 frames, grid scale 1.00 (post grid 1920x1024, trail grid 1920x1024)
  attractor-draw-pass    19.840 ms
  bloom-blur-h            4.220 ms
  trails-pass             4.125 ms
  kaleido-pass            2.980 ms
  tonemap-pass            0.576 ms
  background-pass         0.293 ms

Every labelled pass gets a row, however cheap: the table never folds a tail.

The label is the pass’s own, so a row names something you can find in the engine, and a pass encoded several times a frame — the bloom pyramid’s blur runs once per level — is summed into one row rather than split across them. The rows follow the preset: turn trails, kaleido_order or bloom_amount off and those rows leave the table, because an inactive stage encodes nothing.

These are GPU times taken at each pass’s own boundaries, so they do not add up to draw+submit above — that one is a CPU-side wall clock around the encode and the submit, and the GPU runs the passes it was handed concurrently with it. On a run that is holding its rate the pass table is the larger of the two, and it is the one to read when a preset will not fit the frame.

An adapter without timestamp queries — the software rasterizers — prints one line instead, at startup, and no table afterwards:

skipped  : per-pass GPU timings - this adapter offers no timestamp queries, so no pass cost table is reported

Nothing else changes on that adapter: the query set is the only thing this adds, it is built by the frame tap and by nothing else, and a window has no tap.

Wall clock against scene clock is the honest frame-rate reading. They track each other because dt is measured per frame rather than assumed, so a run that cannot hold the requested rate renders in correct real time and simply delivers fewer frames: the animation is never slow, the frame count is low. Compare the frames emitted against fps x wall to see whether the rate was held.

What it does not do

• --sink spout is Windows only. Spout has no macOS or Linux form; the macOS analogue is Syphon, a different SDK against a Metal/IOSurface seam, and the Linux build carries no sender at all. --sink stdout runs wherever the player does.
• No audio, on either sink. Both are video transports. A receiver takes audio from its own source.
• Spout is same-machine only. It shares GPU memory between processes on one box; there is nothing to send over a network. The pipe reaches whatever spawned the player and no further — a remote sink would be --render’s ffmpeg pipe pointed at SRT or RTSP, which is not built.
• No golden covers the picture. The mode is wall-clock paced, so its output is not reproducible and no baseline can assert on it. What is asserted, from outside the process, is the pipe’s shape: a bounded run puts exactly one frame’s bytes on stdout per frame, the geometry is announced before them, and a reader stalled for a second loses none. Whether the picture is right is still a byte-identity claim against a deterministic capture, or a human looking at a receiver.