Marco Tempest is a Creative Technologist at the NASA Jet Propulsion Laboratory, a Director’s Fellow Alumni at the MIT MediaLab, and the Founder and Director of the MagicLab in New York City. With his background in technology and stage-magic, Tempest is the embodiment of Arthur C. Clarke’s famous dictum: “Any sufficiently advanced technology is indistinguishable from magic.” Tempest’s gift is to the use of illusion in prototyping future technologies. This involves creating immersive user experiences so users can, in effect, experience tomorrow’s technology today. This gives valuable insights into the way people interact with new and emerging technologies.
Last week Tempest quietly released his newest work, the story of Ada Lovelace, the first computer programmer, brought to life through the lens of real-time AI storytelling. “The piece is rendered on a laptop with a webcam using Nvidia’s Ada Lovelace architecture, showcasing the very essence of her vision — machines capable of creating art,” Tempest wrote in an email.
Lovelace is a key figure in the development of the computer, and the perfect subject to be reanimated in this way. In 1833, at the age of 17, the young aristocrat met Charles Babbage, an eccentric scientist trying to finance a large and complex calculating machine, which he called “The Difference Engine.” It would be programmed by punch cards, like the industrial looms of the time. Lovelace took on the job of writing a program for those cards. She envisioned machines that would someday be able to create art, and now, nearly two hundred years later, they can.
“I am utilizing the power of open-source software (ComfyUI, Stable Diffusion, OBS, OSC, NDI, and models operating locally on my laptop, enhanced with custom modifications to facilitate live storytelling with AI.” The system captures Tempest’s performance through an iPhone equipped with a custom app (programmed by Tempest) designed for video segmentation. “This app efficiently removes the background, tracks my hand movements, and streams both video and positional data to my live performance setup.”
For generating images and video sequences, Tempest used ComfyUI, a high-speed open-source LLM with a web interface. The magician then created a bespoke workflow that could manage pre-programmed prompts generating AI images, and their integration of computer vision, all orchestrated by a show control software called Aximmetry. It’s responsible for the video compositing and managing the overall timeline. This includes gesture recognition, music playback, voice cloning cues, and sending specific parameters back to ComfyUI for dynamic adjustments. Tempest says “The system is smartly configured to make decisions on its own, autonomously selecting and rendering the content that will be displayed during a performance.”
Marco Tempest has a proven record of using digital technologies to communicate with audiences in astonishing ways. As a regular speaker at TED conferences he is known for his work with digital technologies to create immersive, and often astonishing, experiences. One constant goal throughout Marco’s career is to inspire. To take difficult topics and find ways to explain them. To communicate a message effectively. To enthuse and astonish. To create and innovate. He believes that good ideas, like good people, are nurtured and with the right amount of care and support nothing is impossible.
Read the full article here
