Note: sound on
PERCEPTION,
PREDICTION,
DECISION,
TOWARDS THE UNCERTAIN IMMEDIATE FUTURE.
SPATIAL INSTRUMENTS
GAP +
2021
Architectures of Sound
Centered around the criticism of linear history, time is considered as a re-generating process of presence.
Upon the topics of post-colonial and communism identities of Kulangsu island, from a standpoint of invisible architecture, “GAP+” emerges as a comprehensive synthesis of my explorations in algorithms, sound/media art, and installation. It embodies an experimental spirit that seeks to dissolve the boundaries of traditional architectural design. I envisage architecture as a generative system, a crucible for new forms and possibilities, fueled by diverse theoretical frameworks.
Drawing inspiration from the “live set” music concept — where the mechanisms of sound mixing shape the aural space — GAP+ offers a critical lens on the conservation of “historical” urban spaces. It constructs an auditory landscape, a soundscape that engages with the commercial-dominated urban soundscape. GAP+ redefines architectural presentation, introducing novel concepts of “real-time” and “remote” experiences within the built environment, in form of spatial instrument.
Through the spatialization of the Live Set framework in electronic music, it delves into the mechanisms of spatialized mixing within an acoustic space, conceptualizing it as an interactive acoustic museum. In this installation, gestures unlock the potential for real-time sound generation, transforming each audience into a performer, to re-interprete the sounds collected on Kulangsu. The narrative gives rise to two distinct sets on different scales: “GAP+” embodies architecture, offering a structural expanse, while “GAP-” serves as an accessible instrument/installation, inviting engagement from all.
ACOUSTIC GARDEN
2021
Augmenting Decisions and Experiences at Walkable Scale through AR Spatial Audio
Focusing on peripersonal space and our immeidate future, we implement space as a medium to define boundaries and decisions.
In this project, inspired by the progressive structure of electronic music, we introduce a spatialized sound synthesis method based on distance-related audio effect modulation combined with binaural spatialization. This approach is intended to navigate users through space without depending on visual indicators, utilizing auditory cues from a multitude of virtual audio objects instead. These objects are responsive to user movements, providing immersive musical experiences on a walking scale. Interaction with specific audio objects enables users to dictate different musical progressions, which follows a self-similar spatial structure.
Distinct from conventional audio spatialization and sonification techniques which cater to simulation and direct translation of data, our method emphasizes the affective feedback of the translated auditory messages. It reveals the potential of interactive, multi-faceted musical narratives driven by spatial interactions, allowing audiences to participate as performers. During our experiments across desktop applications, handheld AR and AR headsets, we encountered challenges from sound design, hardware constraints, and the human brain’s capacity for information processing.
MAELSTROM
2023
Survival in the Ever-evolving Whirpool of Message
Focusing on peripersonal space and our immeidate future, we explore decisions through the uncertainty of our current tendencies.
How can we perceive patterns in otherwise seemingly chaotic and unpredictable streams of activity as a means of not only surviving but also embracing the inevitable interdependence of an unpredictable future? Marshall McLuhan recognized parallels of today to Edgar Allan Poe’s novel “A Descent into the Maelstrom.” In this short story, a sailor becomes so absorbed in thought that he is caught in a maelstrom. Realizing he cannot row his boat out of the situation, he must observe the objects within the maelstrom and grab onto the recurring ones as a means of escape. As an analogy: only through pattern recognition can we cope with information overload. This vortex embodies the challenges of our information age today, and the exhibit replicates the process of “detachment” through information overload, yet it possesses cyclical patterns in time. Through such sensory patterns, collective memories and habits become an inheritable feature of environment, offering a way out of the maelstrom.
SPATIAL DAW/MUSIC STRUCTURE
2019-2023
Musical Narrative as a Means to Tuning Data & Perception
From 2019 to 2023, my endeavors have focused on spatializing music systems and narrative structures across various projects. The objectives are twofold: firstly, to create intuitive instruments that reflect general audience’s presence and enable them to engage in performances and steer narrative directions; and secondly, to evolve narrative forms and hierarchies that are innately tied to such interactive experiences. This journey has led to the development of a unique methodology rooted in this system, aiming to forge new avenues for audience-driven performances and immersive experiences. These endeavors are set against the backdrop of urban history and accessibility, striving to unveil potential cultural shifts that move beyond the prevalent ocular-centric paradigm.
The next step is targetting a musical system as an universal philosophy for message and data processing, where temporal pattern functions and accessed through spatial design, taking impact on human perception and affective process. The system will be further combined with generative human-input interfaces (WIP), which is pictured as a braile interface for informative touches, extending the temporal-spatial translation into the material world.
THE UNKNOWN CITIES
2019-2020
The project is series of 3d-printing based on fiction “9 City, Millions of Future” by Yan Wu. The fiction discribed a process of returning territory from city to nature, transitting the earth to a planet of nature. After several fascinating and upset seasons, we finally find a proper way of “returning” and ways to compress, transmit and expanding cities on new planets.
In our story, when the ending of earth – Crystal Season starts, artifacts of human era begin to shrink, fade and crystalize, and we human will also fade away. But this process is a new birth, instead of death – which creates new realtionship between human and nature, and new forms of city and architecture in new space.
PUBLICATIONS
2022 CDRF: A Slime Mold System Driven by Skeletonization_Errors (Check full blog HERE, source code available at GitHub)
ARTICLES
A/V Performance
Since 2017, I’ve been performing as a VJ and media artist through concepts of visualization & sonification, exploring sensory experience through architectural mediums. From the scene of performances, I initiated my very first idea of spatial music and audience-directed performance.
WORLD BUILDING FOR THE ACOUSTIC FUTURE
IML 475 - JUNK World Building
IML 521 - Looks Good, Sounds Great
Work at Refik Anadol Studio
Since 2021, I’ve been working as an architect and computational designer at RAS, participating series of architectural projection, media facade and interactive installation projects through visualization, fabrication and generative algorithms. Providing consulting services of projection mapping, architectural visualization and data-driven geometry generation/optimization, I consider interactive technologies as a means to access imaginary space.
Sense of Space : Connectome AI Architecture
Computational Designer (Form Study & 3D Printing)
Casa Batllo - Living Architecture
Computational Designer (Geometry Generation & Animation of Chapter 2), unreal visualizations.
Zaha Hadid - Architecting the Metaverse
Computational Designer (Unreal Engine visualization and geometry reconstruction of DALLE 2 Image), curation design & projection mapping simulation.
MoMA - Unsupervised
Previs and visualization