SoundfishSoundfish tests its piano roll with a program that uses the editor the way a restless person does, at random, for a minute at a time, and checks a short list of rules after every step. It looks for bugs that no single button owns. The draw tool works, the arrow keys work, and shrinking the window works. The bug appears when you drag a note, switch tools halfway through a selection, resize the window to phone width, and then nudge with the arrow key, in an order nobody thought to try.
Bugs that live between the clicks
A lot of software now arrives finished-looking and fragile. A model writes a feature quickly, the demo works, and the second session breaks it. That is vibe-coded slop, and careful code can end up the same way when it sits on a fragile foundation: each part was tested alone, and nobody checked what happens when the parts meet in an order no one planned.
A music editor is full of those meetings. A piano roll has selections, tools, focus, scrolling, zoom, and a transport that keeps time while you edit. Ordinary tests walk a path someone wrote down. People click the toolbar twice, drag while playback runs, and change window size mid-edit. The failure report that comes back is "it broke after I'd been editing for a while," with no steps to reproduce.
What changes when a machine keeps clicking
Suppose the editor always starts from the same state, one you can open again by name. A program takes it from there and does a random mix of what a person does: clicks tools, drags across the grid, presses arrow keys, changes the window size. After each step it asks a few plain questions. Is the number of notes the same as before? Did playback come to rest? Is the page piling up elements it never removes?
When an answer is no, the run stops and keeps the full list of steps it took. That list turns "it broke after a while" into a sequence someone can read, replay, and turn into an ordinary test. The randomness finds orders nobody would write, the fixed start and saved steps make each find something a person can act on, and the rules decide what counts as broken. This is fuzzing an interface.
Twelve starting states, pinned before the first click
Soundfish uses Direct, a Hraness library that gives each app state a name and opens it by URL with local fixture data. The editor's own components render the score. Only the data behind them is fixed, and none of it touches the live site, an account, or the network.
The fuzzer is Bombadil, an open-source browser fuzzer from Antithesis. It drives Chrome, picks actions at random with weights the specification sets, and records every step.
Soundfish explores 12 of these named states. They include pointer editing, keyboard editing, the production piano roll at eight and sixteen bars, a dense score, a composite keyboard view, a phone-sized touch layout, and a playback scenario that starts and stops the transport. Each one has an exact starting picture written down in advance. The keyboard-editing score starts with one track and six notes. The pointer-editing score starts with two tracks and twelve notes. Nothing is selected and no grid has focus. The playback scenario has no score at all and starts with the transport idle.
The first rule checks that picture once, at the first moment the editor is fully on screen, and keeps the answer. In sketch form:
let firstReady = null;
function recordFirstReady(onScreen, matchesExpected) {
if (firstReady === null && onScreen) {
firstReady = { matchesExpected };
}
return firstReady;
}
The startup rule then requires that this first snapshot matched, within ten seconds of load. If the editor opened with the wrong tool, the wrong window size, or a note already selected, the run fails even if a later random action happens to fix it.
Six other named states are left out on purpose: two full studio views, the studio's phone layout, and three task benchmarks. The campaign file says why: they have no guarded action menu yet, so a fuzzer there would mostly wait.
The menu of things it may do
A fuzzer that can click anything will eventually click something that ends the experiment: a link off the page, a reload, or a button that deletes the score. Soundfish gives it a weighted menu instead:
- click tool and transport buttons, and the octave up and down controls;
- click a piano-roll grid to focus it, and drag across it;
- press the arrow keys, offered only while a grid has focus;
- switch between the window sizes that scenario allows;
- at a lower weight, Direct's shared conservative clicks, text inputs, scrolls and waits.
Grid clicks and drags are left off the menu while the draw tool is active.
Desktop scenarios alternate between 1024 by 768 and 1280 by 800. The phone scenario alternates between 360 by 800 and 390 by 844. A window size outside that list fails the run.
Every click passes one more filter. Anything that is a link, or whose label contains close, delete, erase, remove, or reset, is dropped. That keeps each walk inside the state it started from. It also means those actions are never fuzzed here, so they need their own tests.
Rules checked after every step
Soundfish holds the editor to the rules below. The sketches are simplified; the real versions read the same values from attributes the editor exposes on the page.
Safe actions never change how many notes exist. Dragging and nudging can move notes. They cannot create or lose them. Outside the playback scenario, the note count on the next step must equal the count on this one.
Every edit the editor tries to commit ends one of two ways.
const editsAccountedFor = (s) =>
s.attempted === s.accepted + s.rejected &&
s.deleteRequests === 0 &&
s.blockedNetworkRequests === 0;
An edit that is neither accepted nor rejected fails the run, as does any delete request or blocked network request. The editor harness's own failure counters, for activity, performance thresholds, and renderer memory, must also stay at zero.
The picture stays possible. The track count, grid count, and total note count must match the scenario's starting picture. The number of visible notes can never exceed the total.
Playback keeps one clock, and comes to rest. At every step, the transport may hold at most one scheduled tick and one subscription to the timer. Within ten seconds of any action, it must reach a resting state that matches what the user asked for:
const playbackAtRest = (p) =>
p.status === (p.wantsToPlay ? "playing" : "idle") &&
p.activeTicks === (p.wantsToPlay ? 1 : 0) &&
(p.audioRunning || !p.wantsToPlay);
Playing means exactly one tick in flight. Stopped means none. A double-click on Play that starts a second clock, or a Stop that leaves one running, breaks this rule. In the playback scenario, the fixture's delay from Play to the first loop point must also stay within 150 milliseconds while playing.
The page does not quietly grow. Within any 30-second window, Chrome's count of DOM nodes may grow by at most 2,000, and within any 5-second window its count of event listeners may grow by at most 500. A slow leak that no single test notices can trip these during a random walk.
Direct adds the rules every scenario shares: the named state must load within ten seconds, it must not change identity during the run, its own violation counters must stay at zero, and it must keep coming back to a quiet state.
Proof that the walk went somewhere
A fuzzing run can pass by doing nothing. If Bombadil waits for a minute, every rule holds.
So each Soundfish scenario also has to show it explored. A run needs at least three actions that are not waits, and at least two different editor states. It must also show a change in the editor right after the action that scenario is about: a drag in the pointer-editing scenarios, an arrow key in the keyboard ones, a click in the playback and compact views. A window-size change has to change the full snapshot on its own, and it cannot count toward the editing requirement. A drag scenario whose drags never changed anything fails.
The Soundfish README limits what this shows. A change right after an action shows that the two happened in sequence. It does not show that the action caused the change, because queued editor or playback work can also change a snapshot.
When a rule breaks
Each run keeps its full trace: every action, every snapshot, screenshots, and logs. A developer can open the trace in Bombadil's inspector and replay it against the same scenario:
bun run fuzz:direct -- --campaign <scenario> --replay <path-to-trace.jsonl>
The README then asks for a small deterministic regression test written from the failure before the product is fixed, and for the trace to stay with the change when it explains something a fixed test cannot.
The README also says browser replay can diverge from the original run, there is no fixed random seed, and there is no automatic shrinking of a long trace to a short one. A person cuts the trace down by hand.
Raw traces can contain rendered text and local details, so they stay on the machine that ran them. The scheduled run uploads only a sanitized summary.
How often it runs, and what a pass means
The campaign is set to run in a GitHub Actions workflow every Wednesday morning UTC, and whenever someone starts it by hand. It is meant to run all 12 scenarios one after another, 60 seconds each, in Chrome on a hosted Linux runner, and it keeps the summary for seven days.
That scheduled run has not passed yet. Each of the three scheduled runs so far, on September 9, 16 and 23, 2026, stopped at the first scenario when Bombadil exited with an error before taking a single non-wait action, so the weekly job has not yet explored the editor. The cause has not been diagnosed, and these runs say nothing either way about the editor itself.
The job is not a check on each pull request, so a change can reach the main branch before the fuzzer sees it. Developers can also run shorter or longer passes locally, from 12 to 300 seconds per scenario.
A passing run will mean that a one-minute random walk from each of 12 pinned states, using the actions on the menu, broke none of the rules above. The README calls a pass diagnostic evidence, not a coverage claim. Paths longer than the walk, actions left off the menu, and rules nobody wrote are outside it. Bombadil drives Chrome only, the phone scenario is emulated rather than real touch, and none of this shows audible sound, timing on a real device, or behavior in other browsers. Soundfish is in beta, and the fuzzer adds to its deterministic browser tests and written promises rather than replacing them.