Case study · Oct 2026
thrum
A string lab in the terminal, played with your hands. Pinch the air in front of a webcam and a particle appears between your fingers; flick, and it flies. Pinch with both hands and pull, and a string stretches between them, tuned by its length. Everything on screen is a character.
From a blank lab: two strings make a valley, two more hang above it as chimes, and particles thrown in ring them on the way down, pile up and bond. One is picked up and held still, which cancels it. One is held until it charges, then thrown: it bursts where it lands and scorches the string. Then an EMP. Recorded from the app itself through a pseudo-terminal, played with a scripted mouse because a camera cannot be scripted, and silent, though the lab is not.
The pinch
Everything starts with knowing when two fingertips touch. MediaPipe returns two sets of hand landmarks: one in the image, and one in metres, estimated in 3D. Metres looked like the obvious choice, because a pinch is a physical distance and it would not care how far the hand is from the lens. It was wrong. On photographs of people making the OK sign, the metric landmarks put fingertips that are touching 6 to 8 cm apart, and how wrong depends on the pose, which is why one hand could pinch and the other never did.
The fix measures the gap in the image instead, divided by the palm’s size, taking whichever palm measure foreshortening shrinks least. On the same photographs, and on their mirror images so both hands are covered:
| hand | gap ÷ palm | metric gap |
|---|---|---|
| pinching | 0.10 to 0.19 | 1.6 to 8.1 cm |
| open | 0.85 to 1.45 | 6.7 to 10.7 cm |
The ratio separates them with room to spare, so a pinch starts below 0.25 and ends above 0.40, and that gap between the two is what stops a wobbling hand from dropping what it holds. The metric column overlaps itself. The landmarks from those photographs are now a test, so the bug cannot come back quietly.
Real gravity
A thrown particle leaves at the speed your hand was moving, measured in screen pixels. The first version fell under a gravity picked because it looked nice, which turned out to be about eighteen times weaker than real at that scale. A gentle 1 m/s flick upward rose two and a half screen heights and stayed in the air for nearly four seconds. In a real room it rises five centimetres. Nothing was buggy and everything felt wrong.
So the world takes its scale from your hand. A palm is about 9.5 cm from wrist to knuckle; its length in the camera gives pixels per metre, refined slowly while you play, and gravity is 9.81 m/s² in those units. Stand further back and the hand shrinks, its speeds shrink with it, and gravity shrinks to match. In a simulation of human throws through the whole tracking pipeline, a particle leaves within 3% of the hand’s speed and 1° of its direction.
Throw or cancel
Letting go does one of two things. Moving, it throws; still, a particle you just made is taken back. The first version read your speed at the instant the release registered, which is a frame or two after the fingers start to open, and by then most people’s arm has already stopped. Every throw read as a cancel. A throw is now judged by the fastest moment of the last quarter second, so a whip that stops before the fingers open still throws, and a push toward the screen counts too: the hand grows in the camera, and the particle goes up. Both cases are tests that fail on the old code.
The decision is also shown before you make it. While you hold a particle it says either throw, with an arrow the way it would fly, or let go: cancel.
Charge
Hold a particle and it charges over three seconds: a meter fills, it grows, its electrons multiply and speed up, and the hum underneath it swells. Thrown, it is faster, heavier and louder. At full charge its first hard hit bursts in a small shockwave and spends itself, and a charged hit scorches the string where it lands, a white-hot mark that cools to a scar, and energises it for a few seconds so it bounces things higher. The first version of that bounce returned more energy than it took, and a ball on an energised string climbed forever; a test caught it, and the extra bounce now stops just short of one.
Drawing in a terminal
The lab is simulated in pixels and drawn into a grid of character cells. A line takes its glyph from its slope, so strings are made of -, \, | and /. Particles are metaballs shaded with .:-=+*#%@, so two that bond melt into one shape. A cell stays warm for a moment after it is drawn, which is where the trails come from. Each frame rewrites only the cells that changed, inside a synchronized update, so nothing flickers. The sound is two operator FM bells mixed in NumPy: a longer string rings lower, and a molecule of particles rings a chord.
Limits
The tracking is tested on photographs and recorded video of real hands, and on synthetic hands for everything timing-dependent; it has not been tuned on many people. The pinch thresholds are fitted to a handful of photographs. Real gravity is honest and also fast: a particle falls the height of the screen in about a quarter of a second. It needs a terminal with mouse support and Python 3.10 to 3.12, which is MediaPipe’s range.
It began as a browser toy called nanoforge, went green phosphor, then ASCII, then moved into the terminal. A thrum is the low hum of a plucked string.