Binding Guide

A binding is a single rule that says: "read this number from the DataBus, transform it, and apply it to that property on that object."

One binding drives one property. If you want a light to react to your arm, a material colour to cycle with music, and a particle system to burst on beat — that's three bindings. They're cheap, and the editor UI makes them fast to create.

How to think about it

The 4E Data Flow System is a giant post office. Your body, the music, the keyboard — all write numbers into named mailboxes on the DataBus. A binding is a little delivery worker that stands by one mailbox, grabs whatever number is in it every frame, transforms it, and hands it to one Unity property.

The worker can do smart things too: hold the value, reshape it, only work on certain days, or fire a pulse when something important happens. That's what binding modes are for.

Every binding, regardless of what it drives, has the same five-stage pipeline. Most stages are optional — a minimal binding is just a source and a target.

Scene Bindings — recommended for most cases

Bindings are stored directly on the DriverMappingRunner component in your scene, under the Scene Bindings list. You drag scene objects straight into each binding's target field.

• Scene references persist when you save the scene — you won't have to rewire anything.
• No extra assets to manage.
• Easiest to debug — everything lives on one component.

Use when: you're building a single scene, or each scene needs its own unique mapping.

Asset Mappings — for reuse across scenes

Bindings are stored inside a DriverMapping ScriptableObject asset (create via Assets → Create → 4E → Driver Mapping). The asset holds the recipe — channels, ranges, modes, curves. Scene references are supplied separately through the Asset Targets list on the runner.

• One asset, many scenes — share a mapping across an entire show.
• Swap multiple assets at runtime to change the entire performance feel (see Runtime Swap).
• Scene references do not persist inside the asset — they're re-wired at Start() from the runner's Asset Targets list.

Use when: multiple scenes should react the same way, or you want to A/B test two mapping presets on the same show.

This is the literal order of operations, taken from DriverBinding.cs:

Body Pose (33 landmarks)

MediaPipe tracks 33 body points. The PoseChannelSource writes the raw landmark positions plus several derived channels — joint bend angles, limb raise angles, overall body centre, and movement velocity.

Joint names for bend / bendNorm

leftElbow, rightElbow, leftWrist, rightWrist, leftKnee, rightKnee, leftHip, rightHip, headTilt

Limb names for raise / raiseNorm

leftUpperArm, rightUpperArm, leftForeArm, rightForeArm, leftThigh, rightThigh, shoulderWidth, hipWidth

All 33 landmark names

nose, leftEyeInner, leftEye, leftEyeOuter, rightEyeInner, rightEye, rightEyeOuter, leftEar, rightEar, mouthLeft, mouthRight, leftShoulder, rightShoulder, leftElbow, rightElbow, leftWrist, rightWrist, leftPinky, rightPinky, leftIndex, rightIndex, leftThumb, rightThumb, leftHip, rightHip, leftKnee, rightKnee, leftAnkle, rightAnkle, leftHeel, rightHeel, leftFootIndex, rightFootIndex

Face (478 landmarks + 52 blendshapes)

MediaPipe's face mesh gives you 478 raw landmark points plus 52 ARKit-compatible blendshape values that capture expressions like smile, blink, brow raise, and jaw open. Blendshapes are the usual choice — landmarks are more useful for bespoke measurements.

All 52 blendshapes

browDownLeft, browDownRight, browInnerUp, browOuterUpLeft, browOuterUpRight, cheekPuff, cheekSquintLeft, cheekSquintRight, eyeBlinkLeft, eyeBlinkRight, eyeLookDownLeft, eyeLookDownRight, eyeLookInLeft, eyeLookInRight, eyeLookOutLeft, eyeLookOutRight, eyeLookUpLeft, eyeLookUpRight, eyeSquintLeft, eyeSquintRight, eyeWideLeft, eyeWideRight, jawForward, jawLeft, jawOpen, jawRight, mouthClose, mouthDimpleLeft, mouthDimpleRight, mouthFrownLeft, mouthFrownRight, mouthFunnel, mouthLeft, mouthLowerDownLeft, mouthLowerDownRight, mouthPressLeft, mouthPressRight, mouthPucker, mouthRight, mouthRollLower, mouthRollUpper, mouthShrugLower, mouthShrugUpper, mouthSmileLeft, mouthSmileRight, mouthStretchLeft, mouthStretchRight, mouthUpperUpLeft, mouthUpperUpRight, noseSneerLeft, noseSneerRight, _neutral

Hands (21 landmarks per hand)

Each hand gets 21 tracked points: wrist, four joints per finger (MCP, PIP, DIP, tip). Hand labels are already swapped to user perspective — hand/right is your right hand, not the camera's.

Useful landmark indices

0 wrist · 4 thumb tip · 8 index tip · 12 middle tip · 16 ring tip · 20 pinky tip. Indices 1–3, 5–7, 9–11, 13–15, 17–19 are the intermediate finger joints.

Audio

Requires an AudioAnalysisSource component in the scene with an AudioSource playing music. FFT analysis splits the spectrum into low / mid / high bands and runs a beat detector.

Time (LFOs and clocks)

The TimeSource writes automatic oscillators and a BPM clock — useful when you want something to pulse or cycle without any performer input. Three pre-configured LFOs ship with the system.

Keyboard & Mouse

The InputSource writes keyboard and mouse state to the DataBus using Unity's new Input System. Great for dress-rehearsing effects before you've got MediaPipe running.

A channel usually speaks in its own units — degrees for a joint bend, 0–1 for a blendshape, −1 to 1 for a landmark position. Your target usually wants something else. Remap is the built-in converter.

A response curve is a Unity AnimationCurve that reshapes the remapped value non-linearly. It operates in normalised 0–1 space between outputMin and outputMax, so swapping your output range doesn't invalidate the curve. X axis = input, Y axis = output.

Common curve shapes

Direct — default, passthrough

The remapped value passes through unchanged. Source → Remap → Target.

Settings: none

Switch — binary on/off

Above the threshold → outputMax. Below → outputMin. No in-between.

Settings: modeThreshold

Gate — two-channel AND

Passes the signal only when a separate gate channel is above 0.5. When the gate is closed, output snaps to outputMin.

Settings: gateChannel (a DataBus channel key)

Latch — capture and hold

On rising edge above the threshold, capture the current value and hold it forever — until a separate reset channel fires.

Settings: modeThreshold, resetChannel

Sequence — step through a list

Each time the signal crosses the threshold from below, step to the next value in sequenceValues[]. Wraps back to the start after the last value.

Settings: modeThreshold, sequenceValues (float array)

Pulse — hit and fade

On rising edge above the threshold, snap to outputMax. Then decay back to outputMin over pulseDecay seconds.

Settings: modeThreshold, pulseDecay (seconds)

Mode-specific fields reference

These fields only matter when the corresponding mode is selected — they're hidden in the inspector otherwise.

The final stage before the target. A lerp-based low-pass filter: output = lerp(newValue, lastValue, smoothing). A value of 0 is instant; values approaching 0.99 feel molasses-slow.

Smoothing level quick reference

1. 1
   Confirm the runner exists. Your scene needs one GameObject with a DriverMappingRunner component. If you're starting fresh, create an empty GameObject called 4E_Runner and add the runner component. Also make sure the 4E_DataBus and a source (like 4E_PoseChannelSource) exist — see SetupGuide if you haven't done the initial wiring.
2. 2
   Press Play briefly. Press F12 to open the DataFlowMonitor. Confirm that pose channels are being written. If nothing is flowing, fix the source first — bindings reading from an empty bus will silently do nothing.
3. 3
   Stop Play and pick your approach. Scene binding for a one-off effect in this scene; asset mapping if you want reuse or runtime swap. For this walkthrough we'll use a scene binding.
4. 4
   Expand the Scene Bindings list on the runner in the inspector. Click + to add a new binding.
5. 5
   Pick a channel. Click the channel picker. Filter by category (Joints / Body / Face / Hands / Audio / Time / Input). In Play mode, each row shows the current live value so you can verify you're picking a channel that's actually changing.
6. 6
   Pick a target type. The inspector collapses unrelated fields so you only see what's relevant.
7. 7
   Drag the scene target into the relevant field — transform, mesh, renderer, light, particle system, audio source, or reflection target GameObject.
8. 8
   Adjust the remap. The editor auto-fills sensible defaults but you'll usually want to tune. Watch the live value while you move, then set inputMin = your "rest" value, inputMax = your "full range" value.
9. 9
   Press Play and test. Iterate on remap, smoothing, and response curve until it feels right. You can edit while in Play mode — changes take effect immediately.
10. 10
    Exit Play mode. Unity automatically saves the binding configuration with the scene. But: edits you made during Play mode do NOT persist. Write them down, or use Copy Component → Paste Component Values.

Asset templates vs scene bindings

DriverMapping ScriptableObjects are template libraries — they store the recipe (channels, remap ranges, modes, curves, target types) but hold no scene references. They're reusable across any scene in the project.

The DriverMappingRunner in your scene has a sceneBindings list where actual scene object references live and persist with the scene. To get bindings from an asset into the runner, you deploy them.

How to deploy

1. 1
   Select the DriverMapping asset in the Project window. Its custom inspector shows all the bindings stored in the asset.
2. 2
   Click "Deploy to Scene" to copy a single binding into the Runner's sceneBindings list. The binding's recipe (channel, remap, mode, curve, target type) is copied — but the target field is left empty for you to assign.
3. 3
   Click "Deploy All to Scene" to copy every binding from the asset that isn't already deployed. Bindings that already exist in sceneBindings (matched by channel key + target type) are skipped to avoid duplicates.
4. 4
   Assign scene targets on the Runner. Select the Runner in the Hierarchy, expand sceneBindings, and drag your scene objects into the target fields. These references persist with the scene — the asset keeps its templates clean for reuse.

The Sample Dashboard — 6 tabs, 48 widgets

The generated dashboard lives under [4E_Dashboard] in the scene. It has a header with six clickable tabs and a body that shows one tab panel at a time. Each panel is a 4×3 grid of 12 cards, and the widget choice inside each tab is motion-matched to the data it visualises — vertical bars for vertical motion, horizontal meters for side-to-side motion, radial gauges for rotational motion.

Click a tab in the header to swap panels. Press F9 to show or hide the whole dashboard — the toggle component lives on the Canvas so it keeps working even while the root is hidden.

Why tuning matters

A channel rarely spans the exact range a widget expects. A blendshape is 0–1, a bend angle is 0–180, a body velocity hovers between 0 and about 0.1 in seated use. A donut wants 0–1 fill. A big-number text might want 0–9999. A colour image might want 0–0.33 (hue red→green).

The remap stage is where you tell the binding: "when the channel is at this, the widget should show that". Get that right and the widget feels alive. Get it wrong and the widget is either pinned at full, frozen at empty, or jitters between two values with no middle.

The four knobs you actually tune

That's it. Mode, response curve, invert — all useful, all optional. For 95% of dashboard widgets, these four knobs are the whole job.

Worked example: 0.3–0.6 of a channel mapped onto a 0–100% donut

Say you're driving a donut chart from face/blendshape/mouthSmileLeft. The channel technically goes from 0 to 1, but in practice the blendshape only ever reaches 0.6 with a big grin, and anything below 0.3 is just resting face noise. You want:

• Neutral / small smile → empty donut
• Medium smile (0.45) → ~50% donut + centre label reads "50%"
• Big grin (0.6 or above) → fully filled donut + centre label reads "100%"

Binding 1 — the donut ring (Image fillAmount)

channelKey

face/blendshape/mouthSmileLeft

useRemap

true

inputMin

0.3 — the dead zone's upper edge

inputMax

0.6 — the realistic peak for this user

outputMin

0 — donut fillAmount empty

outputMax

1 — donut fillAmount full

clamp

true

smoothing

0.2 — face blendshapes need only light smoothing

targetType

UGUIImageFillAmount

targetImage

The donut ring Image

Binding 2 — the centre label ("50%")

channelKey

face/blendshape/mouthSmileLeft — same channel

inputMin

0.3 — same slice

inputMax

0.6

outputMin

0 — number range, not fill

outputMax

100

smoothing

0.15

targetType

UGUITextValue

targetText

The centre-of-donut Text element

textFormat

{0:F0}% — zero decimals, trailing %

Both bindings share the same channel and the same input range. Only the output range differs, because the donut ring natively wants 0–1 and the label natively wants 0–100. This twin-binding pattern — one for the visual, one for the number — is how every gauge / donut card in the sample dashboard is wired.

Finding the right input range for a new channel

Guessing inputMin / inputMax is the most common source of "my widget doesn't move" bugs. Don't guess — measure.

1. Press Play. Open the DataBus Monitor with F12.
2. Filter to the channel you care about (e.g. click the "Face" filter button, or type smile into the search field).
3. Do your full range of motion — rest, halfway, maximum. Watch the value column.
4. Note the lowest useful value (anything below is "noise / rest") and the highest comfortable value (anything above is "asking for too much"). Those are your inputMin and inputMax.
5. Plug them into the binding, stop Play, Play again, re-check.

Common widget output ranges — cheat sheet

Smoothing recommendations per widget type

When to reach for invert and response curve

The four knobs above cover most cases. These two are for when a linear 0→1 response doesn't feel right.

• invert — flips the mapping. "Hand down = full" instead of "hand up = full". Easier than swapping outputMin and outputMax, and it plays nicely with clamp.
• response curve — a Unity AnimationCurve that reshapes the 0–1 space after remap. Use an ease-in curve (quadratic) when you want small motion to be subtle and big motion to be dramatic. Use ease-out (square root) when small motion should already read strongly.

Where to actually change these values

1. Find the [4E_Runner] GameObject in the scene Hierarchy.
2. In the Inspector, expand DriverMappingRunner > Scene Bindings. There's one entry per widget — the list is long.
3. Each entry has a channelKey at the top and an expandable Value Mapping foldout with the four knobs.
4. Tune in Play mode to see the effect live, but copy the final numbers — Play mode edits are lost on stop. Right-click the component header → Copy Component, stop Play, Paste Component Values.

1. Body-driven lighting show

A single spotlight reacts to your movement, music, and a slow colour drift all at once.

Note: bindings 1 and 3 both drive LightIntensity — the Pulse binding writes after the continuous one each frame, so the flash overrides. If you want them to add instead, route binding 3 through a Reflection target on an intermediate script.

2. Audio orchestrator

Gesture-controlled audio mixing. Hand crossfades between two tracks, body velocity drives tempo, right-hand visibility plays or pauses the whole thing.

3. Interactive character

A rigged character mirrors your face and head in real time. Bindings 1 and 2 drive facial blendshapes; binding 3 tilts the head with yours.

4. Particle explosion controller

Particle emission layered from continuous motion (velocity), discrete pulses (bass hits), and hand-controlled size.

DriverMappingRunner has a Mapping Library list. Drag multiple DriverMapping assets into it, set the Swap Key Path (default m), and each press cycles to the next asset. The active asset replaces the assetMapping slot and its bindings take effect immediately.

Nothing happens when I press Play

1. Open F12 (DataFlowMonitor). If no channels are flowing, the issue is upstream — fix the source (MediaPipe / WebSocket / AudioAnalysisSource) first.
2. If channels are flowing but your binding's channel name isn't there, check for typos in channelKey. Channel names are case-sensitive and use forward slashes.
3. Check the binding's active toggle is on.
4. Check DriverMappingRunner exists in the scene and the binding is in its Scene Bindings list (or in an asset wired to its Asset Mapping slot).

Target moves but the range is wrong

1. Open F12 and watch the source channel while you move. Note the min and max values you actually reach.
2. Set inputMin = your observed min, inputMax = your observed max.
3. If the output is still wrong, check outputMin / outputMax match what the target expects (remember: blendshapes are 0–100, not 0–1).
4. If values are the right magnitude but flipped, toggle invert.

Movement is jittery

1. First, check the source itself via F12. Is it jittery at the source? MediaPipe is noisy — that's normal.
2. Raise the binding's smoothing to 0.4–0.7.
3. If smoothing enough to kill the jitter makes motion feel laggy, chain a SmoothProcessor upstream (spring-damper mode) and leave the binding smoothing at 0.

Latch / Gate / Pulse / Sequence never triggers

1. These modes detect rising edges. If the channel starts above the threshold, the first trigger only fires after the channel drops and rises again. Use F11 to drag the slider down then up.
2. Check modeThreshold is within your remapped output range. A threshold of 0.5 makes no sense if your output range is 0–100.
3. Gate / Latch reset channels use a raw > 0.5 test on the DataBus value, ignoring your binding's remap. Pre-process with a ThresholdProcessor if needed.

Material changes don't appear

1. Make sure the material property name is correct. URP Lit uses _BaseColor, not _Color. Emission is _EmissionColor.
2. For emission to show on URP Lit, the material's Emission keyword must be enabled in the material inspector.
3. The binding writes to Renderer.materials[materialIndex], which creates an instance. Changes won't reflect in the shared asset — that's fine for runtime but can be confusing when inspecting.
4. MaterialColor needs a colour property, not a float. If the named property doesn't exist, it falls back to _BaseColor, then _Color. If none exist, a warning is logged.

Reflection binding silently does nothing

1. Check the Unity Console for [4E Binding] warnings logged during Initialise(). They tell you whether the component was found and whether the member was found.
2. Component type name: try both short (Light) and fully-qualified (UnityEngine.Light). The match is case-insensitive and uses Contains, so light also works.
3. Member name must be a public field or property. Private and serialised-but-private fields aren't supported.
4. Only float, bool, int, and double members are supported. Strings, Vector3s, Colors etc. won't be written.

Keyboard shortcuts

Building bindings — general tips

• Start with F11 sliders to test bindings before involving MediaPipe. Prove the binding works with a synthetic input, then swap in the real channel.
• Use BendNorm / RaiseNorm channels (0–1) when you don't need the angle in degrees — they save you a remap step.
• Always add some smoothing (0.1–0.3) to body-driven bindings. MediaPipe is jittery; the eye reads raw values as "broken".
• Use Gate mode with hand/*/detected to create effects that only work when a specific hand is visible. Simple, reliable "kill switch".
• Use Sequence mode to cycle through presets with a single repeated gesture — much cleaner than having one binding per preset.
• Combine Pulse with audio/beat for beat-reactive flashing effects. Pair with very short pulseDecay (0.1–0.2s) for snappy strobes.
• AudioCrossfade with smoothing 0.3–0.5 creates cinematic transitions between two tracks. Set both clips to Loop.
• Use Response Curves to add dead zones (ignore small values) or exaggerate extremes (make the full range feel more dramatic).

Tips for live performance

• Keep the core rig as scene bindings. Reserve asset mappings for variation, not foundation. If the swap logic misfires mid-show, you want the base effects to still work.
• Always leave yourself an exit. Build a "neutral" asset mapping as position 0 in the library, containing no bindings (or very subtle ones). If something goes wrong, press the swap key until you're back to neutral.
• Pre-flight with F12. Before every show, press F12 and watch the channels while you do your full range of motion. If any expected channel is missing or stuck, fix it before house opens.
• Don't tune in Play mode and forget to copy the values. Play-mode edits are lost on stop — use Copy Component → Paste Component Values, or write the numbers on a sticky note.
• Smoothing is cheap in CPU, expensive in feel. The more you smooth, the less responsive the effect. Audiences read lag as "broken". Find the lowest smoothing that kills the jitter.
• Test triggers with F11. ChannelSliderPanel lets you manually fire any channel while MediaPipe is off. Use it to dress-rehearse Pulse / Latch / Sequence modes without warming up.
• Name your assets after gestures, not effects. "big-wide-stance" is more useful live than "fog-and-bass-boost" — you can remember what to do, not always what should happen.