Real-time body-pose-driven effects in Unity.
MediaPipe → DataBus → DriverBinding → scene.
Documentation, references, and interactive demos.
Initial wiring, Chrome browser path, channel cheat sheet, processor overview. Start here if you're new to the system.
Read guideComprehensive reference for MediaPipe in the 4E system — landmark anatomy, coordinate spaces, tracker behaviour, and integration patterns.
Read guideMediaPipe-specific setup notes — webcam vs video file, calibration flags, hand-label swaps, the IFrameSource contract.
Read guideField-by-field reference for every DriverBinding property, plus 7 workflow recipes and a troubleshooting checklist.
Read guideHow the runner orchestrates everything. Every BindingMode (Direct / Switch / Gate / Latch / Sequence / Pulse) explained with signal diagrams and recipes.
Read guideEvery runtime panel and its keyboard shortcut: F3 BindingLab, F8 Calibration, F10 Preview, F11 Sliders, F12 Monitor.
Read guideThe flagship instrument — Embodied Kinetic Orchestra. Webcam + MediaPipe in the browser turn body movement into a live orchestra. The thing this whole system is built around.
Open demoHand-position theremin. Pitch on horizontal, volume on vertical — classic synthesis, body-controlled. A small standalone version of the Echo audio path.
Open demoThe coordinate-space practice ground. Pick a hand landmark, set grid increments, snap-to-grid, motion trail. The lab that inspired the in-Unity grid overlay (F10 preview).
Open demoFacial Action Coding System studio — capture and review facial action units from MediaPipe Face Landmarker. The expression-coding companion to body tracking.
Open demoSession dashboard for FACS data — review captured sessions, browse action-unit timelines, compare takes. Pairs with FACS Studio.
Open report viewer