The Emergence of Micro-Resilience in a Physical–Virtual Interactive Installation-Development and Production
458V
When Block 2 started, I thought the hardest part would be turning the concept into something real.
I didn’t expect the first thing to stop me would be voltage.
EL wire needs an inverter to convert low-voltage DC into high-frequency AC. Chris in the lab measured it for me — 458V.
UAL’s maximum permitted voltage is 24V.
I stared at that number for a moment. Not a dramatic collapse — just a quiet “oh, that road is gone” feeling. Then I started thinking about what came next.
That’s micro-resilience, actually. Not a heroic pivot. Just: okay, what else.
I tried LED strip as a substitute. Did several rounds of testing, thought it might work, then attached fabric to the projection screen — the image immediately distorted. Mounted the LED strip in front of the screen — something felt wrong at first glance. Like forcing two things together that belong to different worlds.
That’s when I understood: I had been imagining something closer to a VR-hybrid mode, physical combined with virtual. But by this point, going back to add mannequin frames or physical supports wasn’t possible anymore.
The e-textile layer was gone. I had spent most of Block 1 imagining it. It was abandoned before fabrication really began.
The First Time I Saw Particles Form a Human Shape
After the system was built, there was one afternoon when I first watched Kinect data turn into a particle figure on the screen.
I stood there. On the screen, something stood there too — made of particles, slightly blurred, drifting slowly.
It’s hard to describe that feeling. Not “wow, that’s cool.” More like a quiet, slightly strange sense of being seen.
I thought: if this can give someone else even a small version of that feeling, the work will have meant something.
The Long Argument with Kinect
In the essay, the gesture detection section reads cleanly: “multi-layered condition logic, each condition addressing a specific failure mode.”
Here’s what it actually looked like:
I thought a single distance threshold would be enough. It wasn’t — the hand at natural rest would trigger it too.
I added a fist-closed condition. Kinect’s hand-closed recognition was unstable — sometimes I was clearly making a fist and the system didn’t respond at all; sometimes my hand was open and it said I’d closed it. Abandoned.
I added a spatial activation zone. During testing I found that the 2.6m I’d measured by hand came out as 3.09m in Kinect — because the Kinect was on the floor pointing upward at an angle, measuring diagonal distance not horizontal. Re-measured, re-set.
I added a tz condition (the hand must be closer to the camera than the body to count as a touch). Ran the numbers: at natural rest, the tz difference was 0.17 to 0.20; arm swinging while walking peaked at 0.34; actually touching the chest was 0.357.
A difference of 0.013 metres. Just over a centimetre.
I looked at those two numbers for a long time. The solution was to combine the tz condition with Filter Width — arm-swinging is momentary, a real touch is a sustained hold. Use time to tell them apart.
But those are patterns I found in debug logs afterward. While it was happening, it was just: test again, fail again, figure out what went wrong.
Several times, a person would walk to the screen and all the wounds would trigger at once. Or no body part was touched at all and the system moved on its own. I’d open Textport, watch the data, adjust while looking, and sometimes fixing it one session meant it broke again the next.
partial pic of debug processing
The Particle and Shadow That Never Lined Up
The body particles and the shadow run on two separate chains, both from the same Kinect, but using different cameras — the colour camera at 1920×1080, the depth camera at 512×424. The two cameras have a physical offset in hardware, so their viewing angles don’t quite match.
The result: the particle figure and the shadow on screen never aligned. They’d sit at a slight distance from each other. No amount of parameter adjustment could fully close that gap.
At first I treated this as a bug that needed fixing.
Gradually I came to see it differently. Shadow and particles represent two different states of the body: particles are the labels the outside world has put on you; shadow is your most essential physical presence. The fact that they don’t overlap starts to feel like a statement — these two versions of you have never been fully the same.
A technical limitation became part of the work’s meaning. I didn’t solve it. I accepted it. Then it became an argument.
There was also the resolution problem — different layers with different resolutions, composited without adjustment, producing distortion. I only encountered this because my project stacks several TOP layers. You don’t know until it happens to you.
The Singing Bowl and the Dissolving Particles
My original idea was optical flow — let participants “wipe away” the particles with their hands, the path of the gesture revealing the shadow underneath.
It sounded tactile. Ritualistic.
Testing showed it wasn’t workable. Optical flow is better suited to detecting global motion across the whole screen; it couldn’t be applied precisely to the local area inside a body silhouette.
So it became: singing bowl audio triggers a full dissolution.
I’ll acknowledge the loss — that direct tactility of actively erasing. But what came in its place was a different quality: the dissolution isn’t something the participant does, it’s something that happens to them when the sound comes. The body receives before it acts.
That’s actually closer to micro-resilience. The adaptations that really work aren’t usually ones you consciously decided on. You notice at some point that you’ve already changed.
Sometimes what you lose brings you to something more accurate.
The Wounds That Couldn’t Follow the Shadow
I wanted the wounds to be bound to the shadow — to move when the body moved. Tried using Blob Track TOP to follow the body’s displacement. Couldn’t get the coordinate systems to connect stably.
For now, position is set manually. Participants are guided to stand in a marked spot on the floor. A technical constraint became a spatial design decision.
A little disappointing. Accepted.
Not Enough Ports. Not Enough Power.
Nobody told me this project would need so many things plugged in at once.
One charging cable. One Kinect. One Arduino. One projector. And one TouchDesigner education licence USB key — without it the resolution is limited, so it has to stay plugged in.
Five ports. My laptop doesn’t have five ports. So I borrowed a USB hub.
The hub didn’t always have enough power. Devices would drop — Kinect disconnects, Arduino disconnects. Re-plug, re-recognise, start again.
Then there was the battery. TouchDesigner plus Kinect Studio plus rendering — the fan running constantly, battery draining fast. Hit empty mid-session, plug in the charger, the charger takes a port, the hub loses a slot. Sometimes charging and working couldn’t happen at the same time. Just wait. Wait until there’s enough charge to continue.
My desk during that period always had a pile of cables on it. I was always counting: how many things are plugged in right now, how many ports are left, how much battery is there.
During the e-textile phase it was worse — Arduino, Kinect, licence key, charger, projector, fabric and circuit boards all spread across the desk. That period I felt less like I was making an art installation and more like I was managing a small server room.
None of these are technical challenges. But together they interrupt you — right in the middle of concentration — with: disconnected again, out of battery again, need to restart again.
Mirroring Correction in Kinect Skeleton Tracking
The mirroring of Kinect skeleton tracking went unnoticed until I reviewed the recordings, revealing that the small wounds on my left and right shoulders had been displayed in reverse. I have since inverted the hand mapping for touch interactions; accordingly, audience members must now touch in corresponding mirrored positions for their gestures to align correctly with the on-screen image.
The Way Someone Stood There
During Dark Lab testing, I watched a classmate walk up to the screen.
They stood still for a moment, looking at their own particle figure on screen. Didn’t move. Then slowly placed a hand on their chest.
A wound appeared.
They didn’t say anything. Just looked at the glowing particle pattern inside their own body’s silhouette on the screen.
I stood nearby and didn’t say anything either.
That was probably the moment this whole semester felt most worth it — not because the technology was running, but because someone was there, quietly looking at their own small wound, and being seen for a moment.
One Last Thing
This work is called Making Resilience Tangible.
But the process of making it was itself micro-resilience in practice — every abandonment wasn’t a collapse, every adjustment wasn’t starting over, the whole semester was just moving forward one step at a time under constraint.
Not dramatic. Not heroic. Just continuing.
What is my own ordinary magic? Probably this.
Leave a Reply