Official Post here
A sculpture of disassembled music hardware and 8 channels of generative video that encapsulated a NY based jazz duo performing live from inside. 2 live camera inputs were run through a facial landmark tracking algorithm to break apart the musicians facial features and disperse them across the sculpture on the 8 screens which was further computationally manipulated to appear as if they were swapping and moving in between the screens (all realtime rendering). Eyes, keys, mouths, sticks, poles, noses, hands, the elements all together, but without order in a cybernetic noise-making pile.
This installation and performance ran for 9 days at DOMICILE a boutique shop in Tokyo.
2 cameras + 8 screens + 1 drum set + 4 keyboards + 2 musicians (or 1 Human Squared) + 40 drum stands + 5 mic stands + 6 music stands + 5 monitor arms + 1 RTX 3070 computer + 1 face tracking video distortion program + 1 pair of speakers + 384774 cables + gravity + balance + some gaff tape = one live noise making pile.
The media distribution system was a small-form computer with a RTX graphics card with kind of a hacky mod on one of the m.2 slots that I built specifically to capture the two live video inputs and process video which was then distributed on two video splitters for 8 channels of live video output.
The main software was a custom program I made in Touchdesigner to take the two camera inputs, run facial landmark tracking on them, and crop, zoom in, and split the video into 8 streams of left eye, right eye, nose, and mouth for 2 faces. The program also had a modular effects patch I built to distort, rearrange, swap, and run all kinds of visual manipulations on the 8 channels of face part videos.
I carried three computers and all expensive video hardware on my back to Japan, scavenged recycling stores for remaining video equipment, collected most of the music hardware for the pile on the Japanese equivalent of eBay, and rented some AV equipment for the band. I installed the pile of music hardware and wired the video distribution network by myself in two days.
Funding for this project came from a Kickstarter campaign I ran after my curator secured the venue at DOMICILE as well as a donation from PROOF and non monetary support which included monopo Tokyo who provided speakers, Shibuya TV that streamed a video advertisement of the show on two huge billboards in Shibuya for two weeks, and friends who donated and lent equipment.
With Human Squared
Curated by Kazuki Chito
Supported by PROOF, monopo Tokyo, Shibuya Television, and all Kickstarter supporters
Photos by monopo Tokyo and Rin Nagamoto
Graphic design by Nafhan NQ