We combine ubicomp and biofeedback technologies to immediately and affectively connect and relate an artistic responsive environment with a visitor. By doing this we consider both sides – the human and the technological system – as equal actors that are connected and interplay in real-time in a human-in-the-loop system (Fig. 1). Although the technical system has no independent intelligence and currently forms a simple feedback loop, its physical responses follow dynamic mappings to physiologic human reactions, manifested through light, sound and wind. These responses are expressive and can cause new physiologic reactions, creating a human-machine interdependency.
Ubiquitous Computing is an interaction paradigm and technological approach where digital sensors and actuators are incorporated into physical space by introducing a networked layer of intelligence into things. Compared to Virtual Reality which immerses a visitor in a digitally constructed synthetic world, isolating her from the physical surroundings, the aim of ubicomp is to seamlessly blend virtual sensing and control into the real world: “…the singularity of “the interface” explodes into a multiplicity of more or less closely aligned, dynamically configured moments of encounters within sociomaterial configurations, objectified as persons and machines.”
Biofeedback technology and corresponding body worn sensors measure personal body functions and physiological parameters in sports and daily life, with the aim to improve health and perhaps even more importantly to increase mental and physical performance. Biofeedback techniques are developed to gain more awareness and control of physiological functions influenced by thoughts, emotions and behavior.
We measure psychophysical signals for breathing (in-point, out-point, regularity), heart activity (beats and rate) and body movement of visitors in a responsive space and record primary, preconscious reactions and emotions. These parameters enable a spectrum of control: While body motion is under direct and voluntary control, breathing is much less direct and not commonly used for explicit interactive control. However, humans have some degree of voluntary control over their breathing. In contrast, while heart rate is coupled to body motion and deep breathing, we have no awareness and no common experience of voluntarily controlling our heart beat.
Accelerometers, an elastic respiratory sensor and a photoplethysmograph are used as input modalities. The elastic chest belt that is worn by a visitor is equipped with the breathing sensor, the motion sensor and a wireless transponder. A photoplethysmograph is attached to the index finger of the right hand to measure blood-oxygen concentration and heart beat. A second wireless biofeedback interaction device can be added to enable the participation of a second visitor.
We record the measured bio-physiological sensor data. The sensor recordings are merged with the video and audio track of a recording of a visitor in the Evaluation Viewer (Fig. 2). This allows us to analyze the recorded behavior together with the appearance and sound of the installation at a glance.
