Philipp Gschwendtner

In the tradition of optical instruments, algorithmic models are considered as instruments of knowledge magnification (Pasquinelli, 2019) reaching beyond the scale of human perception. The interactive instrument and its accompanying manual open up questions of scale, objectivity and the limits of algorithmic models. Through the mismatch of GAN- generated images analysed by an object detection model, the installation highlights an algorithmic culture of “automatic correlation”.

The experimental project considers the entanglements between algorithmic practices and those in the natural sciences, especially the field of microscopy. The installation places algorithmic models in the tradition of optical instruments, highlighting their ability to reach beyond the scale of human perception. To consider the history of optical apparatuses, as well as the effects of “zoom” they produce, allows a pointed critique of algorithmic culture and the different scales and dimensions algorithms operate in/at. As representations of unrepresentable objects, algorithmically derived knowledge stands in close relation to satellite and microscopic images. Just as the microscopic image is not a photographic representation, but rather an information-matrix mapping the reaction to light or radiation, the “images” and knowledge algorithmic models produce are removed from reality as causality is replaced with a form of automatic correlation. Increasingly algorithmic systems and microscopic instruments are deployed together to navigate across these different scales of compression and complexity. As experiments from IBM with large-scale AI-powered microscopy of plankton particles show, planetary change is sometimes the most perceivable at the smallest orders of magnitudes. The changes sensed in theses instances are not noticeable in the context of a broader, more compressed model, and are easily filtered out in the moment of complexity reduction.

Exhibited @ CTM / Transmediale Vorspiel (MaHalla Berlin)

The artistic research project Interwoven Sound Spaces investigated the creative possibilities of telematic music performance in new ensemble music through textile and network technologies. The aim was to enable rich and tangible interaction between musicians who are spatially in different places. The results of this artistic and technical research were presented in a joint interactive workshop concert with the ensemble KNM in Berlin and ensemble Norrbotten NEO in Piteå, Sweden. The project was motivated by the need to enhance communication and a sense of co-presence between musicians and audiences during live concerts happening concurrently at multiple, distant geographic locations connected via telematic means. To achieve this, we commissioned new works that combined textile wearable technologies, interaction design, machine learning, and spatialised sound in performance, thereby extending the experience of playing and spectating music telematically beyond screen-based applications.

In addition to providing tools for telematic ensemble play, we were particularly interested in the roles and dynamics of socio-cultural spaces that characterise live music performance. Factors other than music – such as dress cultures, communication between musicians, interaction between audience members – contribute to the enjoyment and success of live music performances [ref needed]. Our project and technical decisions were guided by these considerations through close collaboration between musicians, composers, researchers, venues, and developers.

The final performance took place in December 2022, simultaneously at Konzertsaal in Berlin, and Studio Acustcum in Pitea. Three musicians as well as live audiences were present at both locations. In addition the concert was streamed on the internet, where the audience was able to join two separate streams for the locations.

Affirmative non-exploitative abstraction can be poetic and political, and translucently feeding through the local, contribute to the illumination of global notions and ways to address large-scale problems. Olga Goriunova: Translucency (2001)

Der Wunderblock ist ein Lernspielzeug, eine Schreibtafel, die es erlaubt, Geschriebenes wegzuwischen, um darauf Neues zu überschreiben. Dennoch bleiben Druckspuren. Diese Gedächtnismetapher, als das Überschreiben von Erinnerungen, steht für das Unbewusste, das Nichterfasste, das Nichtgewollte oder auch bewusst Ausgelassene, anders gesagt, für die Lücken des Archivs.

Dazu entwickeln wir die Kunstfigur des „Ewigen Archivar“ (bewusst gewähltes masculinum), der mit seinem Archivregal alleine in seinem Arbeitsraum haust. In seiner unermüdlichen Arbeit kommt er in Berührung und konfrontiert sich mit den Auslassungen, die im übertragenen Sinne durch die Jahrhunderte für die Ausgrenzung und die Legitimität der Ungleichheitserfahrungen von marginalisierten Gruppen stehen. Archi­­­­­­­­­­vieren stellt eine Ausübung von Herrschaft über die Erinnerungskultur dar und ist daher politisch:

Wer bestimmt, was erinnert wird?

Angeblich sollen durch den Einsturz des Historischen Archivs in Köln vor 13 Jahren 5% der Archivalien unwiederbringlich zerstört sein. Tatsächlich wird man erst in 30 – 40 Jahren, nach abgeschlossener Restaurierung, wissen, wieviel verloren ist. Der Bezug des neuen Archivgebäudes des Historischen Archivs Köln 2021 gab uns den Impuls, über das Verwalten und Verwahren von Erinnerung und der daraus resultierenden Kultur nachzudenken. Mit dem Projekt Wunderblock oder Das Imaginäre Archiv, werden diverse Themen durch die Reflexion über die Methoden des Archivierens öffentlich sichtbar verhandelt.

Text: mythen der moderne
Fotos: Tobias Schmücking

As argued by Pasquinelli, machine learning models have emerged as a form of magnification of knowledge, a nooscope, which in tradition to optical instruments has brought forth not only a different form of perception but also of cognition and knowledge production.

In a similar fashion to the lenses of optical instruments, the „lenses“ of the nooscope are surfaces that are irregular, that diffract signals passing through them, that produce a different scale, of magnification for example through dimensionality reduction.

Optical instruments have led to the production of images of spaces beyond human perception, be it micro- and nanoscopic scales of microscopy, or the space-dissolving quality of telescopic lenses. If considered as a form of „knowledge instrument“ that stands in tradition to optical instruments, machine learning operates on a scale that is beyond human perception.

Often, these beyond-human scales intersect and merge with one and other. In the COVID-19 pandemic, global policies and borders intersected with a virus spreading on a cellular level.

In the context of climate change, planetary models and global initiatives propose a global perspective as a way of inducing change. Yet, as experiments from IBM with large-scale AI-powered microscopy of plankton particles show, planetary change is often perceivable at scales beyond-human perception.

While the title „scope“ on the one hand refers to the naming of optical instruments such as microscope or telescope, it can also be understood as meaning ‚area of application‘. The installation shows the „mismatch“ that occurs when algorithms trained on specific data are confronted with data input that is outside of their scope. Algorithmic lenses are molded based on specific training datasets become comically useless in other areas of application that reach beyond their intended scope.

The GAN generated satellite images that are projected on the glass surface are hard to tell apart from real satellite images at times, as the planetary perspective is one that is outside of non-mediated human perception. The instance of object detection on the other hand is trained on more ubiquitous categories that are within a human scope.

On a third level, the translation of scope as margin or leeway reflects a point of critique that moves past the demystification of algorithmic models through the metaphor of the lens. By making visible the dependence of models on their scope - their scale and magnification - allows for a way to hack or „trick“ the algorithmic apparatus.

Exhibited @ Ars Electronica 2022