Temporal Shadows:

An Exploration of Ephemeral Narratives through
Real-Time Motion Tracking and Interactive Projection

Tools:
OpenFrameworks, Xcode,
Kinect

---

Overview

"Temporal Shadows" is a projection-based interactive installation designed to immerse audiences in the dynamic interplay of shadows, time, and personal narrative. By utilizing Kinect for motion tracking, the installation captures the audience's position and silhouette, translating their movements into ephemeral shadows on the temporal canvas. Participants are invited into an engagement with their own temporal essence as they witness the ever-shifting interplay of shadows, serving as a metaphor for the passage of time.

Employing OpenFrameworks for graphics processing and a ceiling-mounted projector for visual projection, the project offers a unique experience where participants actively shape the unfolding narrative through their presence and movement. Through the integration of motion-tracking sensors and engagement with projection-based displays, “Temporal Shadows” delves into topics including the philosophical implications of temporality and the pivotal role of audience interaction in shaping narrative experiences, exploring relationships among the body, data capture, feedback, and usability.

Introduction

The profound intertwining of human existence with temporal dynamics has been a subject of philosophical contemplation throughout history. As Merleau-Ponty suggests, our very "staying in place" is inherently temporal, highlighting the transient nature of each bodily stance[1]. Grounded in this philosophical exploration, “Temporal Shadows” seeks to innovate within the realm of interactive installations by merging the concepts of human bodies, temporality, and time through the prism of technology. It aspires to challenge preconceived notions by creating a digital canvas where the transient nature of human existence is visually manifested, allowing individuals to engage in an aesthetic encounter that goes beyond the subject-object dichotomy.

Moreover, Kühnapfel et al. introduce the concept of "triggers" as catalysts for attention redirection towards the self, fostering moments of reflection [2]. Our project, influenced by this notion, incorporates elements that may trigger specific responses, such as the sense of body movements being observed and being projected. These triggers are designed to shift participants' attention inward, inducing moments of self-reflection amidst the dynamic interplay of shadows and time.

In the synthesis of philosophy and technology, our project represents a pioneering endeavor. Incorporating techniques like motion capturing and computer vision using Kinect and OpenFrameworks, we employ a technical framework to capture the nuanced interplay between the human body, temporality, and time. By translating Merleau-Ponty's philosophical insights into a tangible, interactive experience, our installation not only reflects the transitory nature of human presence but also actively involves participants in shaping the temporal narrative. Through the creative use of technology, we aim to demonstrate that the beauty of art, in this context, lies in its ability to fuse philosophical concepts with experiential aesthetics, challenging conventional boundaries and fostering a deeper understanding of the temporal dimensions inherent in our very existence.

System Design & Implementation

We developed the system using Microsoft Kinect for Xbox One, a motion sensing input device, OpenFrameworks [5] , an open source C++ toolkit for creative coding, along with its Computer Vision libraries including ofxOpenCv[4] , and ofxCv cite{McDonald24}, as depicted in Figure [3] The physical setup in Figure ref{fig:setup} features two Kinects, one overhead to capture body position and another angled to capture silhouettes, along with a projector mounted on the ceiling.



By filtering and processing the real-time data using depth thresholds, the program created in OpenFrameworks retained only the essential information for capturing the user’s position and silhouette. Subsequently, the program stored the drawn representations as buffered textures based on the processed data as shown in Figure ref{fig:system}. In the final stage, these textures were seamlessly rendered on the same canvas as a clock and were passed to the projector.

Summary & Future Work

In our exploration of the intersection between philosophy and technology, "Temporal Shadows" has illuminated the transient nature of human existence through innovative interactive installations. Grounded in the philosophical insights of Merleau-Ponty and inspired by Kühnapfel et al.'s concept of triggers, our project dynamically merges the concepts of human bodies, temporality, and time. By leveraging motion capturing and computer vision technologies, we have created an experiential canvas where participants engage in a visually manifested exploration of the temporal dimensions inherent in their very being.

Looking ahead, "Temporal Shadows" holds potential for further development and refinement. Future iterations of the project could explore avenues for deeper individual and collective engagement. Introducing choice points and subtle hidden elements progressively revealed over time to the narrative could empower participants to shape their experience, fostering personal connections and individual reflection, also could encourage repeated exploration and deeper engagement, inviting participants to unravel the artwork's meaning over time. Additionally, collaborative shadows could allow for multiple participants to interact simultaneously, creating interconnected narratives that reflect the concept of shared temporality.

By implementing these enhancements, "Temporal Shadows" has the potential to evolve into a multi-faceted experience that resonates on both personal and collective levels, leaving a lasting impression on its audience. Through continued innovation and exploration, our project aims to foster a deeper understanding of the temporal dimensions inherent in human existence.



[1] Daniel Harland Briggs Jr. 2018. Body and Time: The Temporality of Human Embodiment. Ph. D. Dissertation. University of New Mexico, Albuquerque, NM, USA.

[2]  Corinna Kühnapfel, Joerg Fingerhut, and Matthew Pelowski. 2023. The role of the body in the experience of installation art: A case study of visitors’ bodily, emotional, and transformative experiences in Tomás Saraceno’s “In orbit”. Frontiers in Psychology 14 (Jul 2023). https: //doi.org/10.3389/fpsyg.2023.1192689

[3]  Kyle McDonald. 2024. ofxCv. Retrieved February, 2024 from https://github.com/kylemcdonald/ofxCv

[4]  openFrameworks.cc. 2024. ofxOpenCv | openFrameworks. Retrieved February, 2024 from https://openframeworks.cc/documentation/ofxOpenCv/

[5] openFrameworks.cc. 2024. openFrameworks. Retrieved February, 2024 from https://openframeworks.cc/