Katharina Gsöllpointner

Katharina Gsöllpointner: Digital Kinesthesia. How Kinesthetic Features in Digital Art Integrate Sensory, Semantic, and Social Perception.


Kinesthetics as a unique sensory domain has been rediscovered by the sciences in recent years, although its existence has been under investigation since as early as the 19th century. The exact definition of kinesthesia (the kinesthetic sense) is still controversial and therefore varies in literature. In general, the following very brief description of the Merriam Webster Dictionary (2015) provides a good conclusion of the basic points: there kinesthesia is defined as “a sense mediated by receptors located in muscles, tendons, and joints and stimulated by bodily movements and tensions;” but it is also at the same time the “sensory experience derived from this sense.”

This conceptual duality of kinesthetics can be considered as exemplary for all sensory domains, taking into account the important aspect that, instead of the dichotomy of a (conservative) concept of five single distinguished senses on the one hand and the elimination of the idea of distinguishable senses on the other, science today rather suggests a model of cross- and multimodal sensory domains and therefore of a cross- and multisensory perception (c.f. Gsöllpointner 2016). However, a consistent and standardized definition of kinesthesia still is lacking, although there have been made several suggestions that considering it an all-integrating sensory domain might be a good approach (Berthoz 2014 and 2000, Gibson 1966 in Fingerhut 2011).

In my talk I will show by the example of 15 interactive, responsive, immersive, augmented, and dynamic digital art installations — which have been produced in the course of the transdisciplinary, arts-based research project “Digital Synesthesia“ (2013-2016 www.digitalsynesthesia.net) — how kinesthetic features in the media aesthetics of digital artworks allow for cross- and multimodal sensory experiences in the user/spectator. I claim, that (artistic) media like e.g. film, image or sound not only provide sensory experiences within their primary sensory modality but each consist of multimodal sensory features which are “integrated“ by the kinesthetic domain. This is true no matter if we observe dynamic (e.g. film) or static (e.g. pictures or objects) artifacts. This assumption is supported by a number of neuro- and cognitive science findings which say that perception per se is basically built on the physical (the body and its parts like the extremities or the eyes) and mental (literally and metaphorically e.g. taking a position, changing perspectives, understanding, empathy) ability to move (Spering 2016, Spering & Carrasco 2015, Berthoz 2014, Robles-De-La-Torre 2006, Berthoz 2000).

It is important to stress the fact that the kinesthetic sense not only is considered to integrate the other senses, but moreover that it also incorporates semantic concepts like time and space, as well as what is called the “sense of empathy”. This has been shown in recent research on synesthesia (Ansorge et al. 2015, Berthoz 2014, Mroczko-Wąsowicz and Nikolić 2014, Hale et al. 2014, Cuskley 2013, Rothen et al. 2013, Nikolić et al. 2011, Amin et al. 2011, Ward et al. 2008, Banissy and Ward 2007, Sagiv and Ward 2006) and will be an important part of my presentation.

I claim that the cross- and multimodal integration of perception through the kinesthetic sense happens by means of the media aesthetics of the digital artworks, and beyond that also of any other objects. The focus of my talk will be put on the question, how the cross-modal transfer of kinesthetic features from one sensory domain to another (e.g. vision to audio, audio to smell, taste to kinesthetics etc.) takes place in the 15 above mentioned, exemplary digital artworks. I will show that this is undertaken by means of material, media, metaphorical, and form decisions which I will explain in detail. As exemplary indicators of such features the following will be presented:

  • verbal indicators in the artwork title,
  • primary sensory modalities/domains which are addressed by the artworks,
  • certain formal indicators which provide information about the semantic field which encompasses the artwork,
  • exemplary kinesthetic verbal expressions and metaphors from the semantic field,
  • cross-modal media aesthetics (media forms in the artwork which suggest the transfer of either kinesthetic feature into another sensory domain, or vice versa).


Katharina Gsöllpointner: She is a media art theoretician with a special focus on media aesthetics and the cybernetics of art. She has taught media and art studies at international universities and has worked as a researcher at the Dept. of Digital Art at the University of Applied Arts Vienna. Since the 1990s Gsöllpointner has conceived, led and carried out a series of inter- and transdisciplinary research projects on issues of media aesthetics and media arts, e.g., AESTHETIC KNOW-HOW. Language – Technology – Media (2007–2009) or Digital Synesthesia (2013–2016). From 1991 to 1995 she was manager of the Ars Electronica Festival in Linz (w/ Peter Weibel). Since the 1980s she has published extensively about media and digital art history, media aesthetics and on the transdisciplinarity of art and sciences. In her habilitation treatise Intermedia Production and Multimodal Perception. On the Aesthetics of Digital Art (2015), she formulated a media theory of digital art as an aesthetic model for the multimodality of perception. She received the Venia Docendi (Habilitation) in media theory from the University of Applied Arts Vienna in 2016.

Comments are closed.