An Embodied Experience of an Algorithmic Art Installation in the Time of Information Crises: LAWKI—Alive

LAWKI—Alive is an immersive installation which collects online videos and combines them with a sound composition that responds to visitor’s movements.This mechanism is directed by an Artificial Intelligence and a Machine Learning-led algorithm. The video clips are the result of a long list of terms submitted by YouTube and Dailymotion in the ten most widely spoken languages in the world, with the intention to represent the widest audience possible. The top ten videos of each search are then ordered based on their meta-tags and description. They are not grouped according to title, year or genre, but by what the video shows, its content. The audience gets an embodied experience in the endless stream of audiovisual material by wandering around the installation. This intends to resonate with our daily life experience where we lose ourselves entirely in the stream of information coming from the screens  connected to the globalized world. The installation raises many questions such as “is the ‘truth’ portrayed on the  Internet taking over our reality?” “How do humans and machines collaborate in the information age?”

Project Website:  http://www.ark.amsterdam/#project-page, https://lawki.online/about

1. Could you introduce the core of your collective and how did you form a collective?

The ARK group is an example of the generation, within its projects it merges different artistic disciplines. It also  collaborates in shifting constellations across various other projects through technological convergence. The collective explores a collaborative working method as a mechanism to reflect, refract, and speculate upon alternative frameworks for productive engagement and exchange. ARK feeds quite literally off the socio-political urgency of our current reality by using media as a source to create new narratives.

The members of the collective share a curious way of working: mixing human and non-human actors with technology to seek new, unexpected outcomes. In ARK, we play. It is a space to exercise/explore/enact a ‘post-signature practice’ as a collaborative methodology.

For the project LAWKI, the collective consists of eight people: Arran Lyon (computational scientist), Federico Campagna (philosopher), Louis Braddock Clarke (crossmedia artist), Roosje Klap (designer, educator and researcher), Senka Milutinovic (crossmedia artist), Teoniki Rozynek (composer), Valentin Vogelmann (computational

linguist) & Zuzanna Zgierska (crossmedia artist). LAWKI also has one computer that stores all the artwork’s data. This is the part of our collective that will travel to the Digital Art Festival Taipei in 2022.

2. Could you explain how the AI selects and decides the sequence of videos that we see in the installation LAWKI—Alive? We understand that the material is categorized and grouped according to formal and thematic similarities or a particular sentiment, but what is the first video clip that pops up? Where are the content of the meta-tags and descriptions coming from?
   

After downloading the database of videos and their metadata, we use the meta-data as a proxy of the videos’ content and sentiment and organize them in such a way that similar videos become “neighbors”. This arrangement is achieved by having a neural network language model capable of reading titles, descriptions and other meta information and judging which ones are similar to one another. Technically, the language model represents each video as a coordinate in a space, one that is not 2D or 3D as we are used to in the real world, but a literally unimaginable space of 768 dimensions. This allows to create very complex “neighborhoods”/”districts” and therefore  generating intricate relationships of similarity between videos as well. To make an actual stream of video content, an algorithm which we call “random walk” explores the space created by the language model and selects videos whenever it passes a point in the space that corresponds to a video. Because of how the space is organized, successions of videos will actually have semantic relationships.

As an analogy, think of browsing through a grocery store: products are arranged (on shelves in a 3D space) in a way that related items are close to each other, so that oils might be next to sauces which are close to canned vegetables, etc. Even if you are unsure about what to cook, by wandering through the aisles in a random path you could discover a recipe just by the products that you encounter.

3. How does the methodology resemble the Bilderatlas Mnemosyne by German Historian and Cultural Theorist Aby Warburg in 1927? Why do you want to make a connection between the system in network culture and iconology?

We draw inspiration from  the special connections laid bare in the (unfinished) Bilderatlas Mnemosyne by the German Historian and Cultural Theorist Aby Warburgand the Warburg Library, where the books are arranged along the lines of general themes allowed to construct connections with their neighboring books. The interconnection between the various disciplines is generated in a way that ‘the history of art may not be studied independently but rather in its interaction with other branches of learning which in their turn only receive their proper significance if taken as a whole.’[1] The systematic yet intuitive approach parallels how most modern Artificial Intelligence works and is in fact what makes it work: models predict the closest neighboring input to create a string of actions. What is nicely demonstrated by the Warburg Institute is that all topics are intertwined. Their interrelation connects everything, like ‘good neighbors’.[2] This idea provides open paths to challenge the conventional classifications of human culture. Could Warburg’s approach be helpful to connect rooted artistic authorship with creative technology developments? And if AI is working with libraries as well (datasets), how can AI form interrelations with the people that steer the technology?

4. For most people, the selection process powered by Ai is a kind of “black box.” What is your view on this, and why do you choose to use Ai in the artwork in the first place?

The work of LAWKI originated in the form of a short film in September 2020 with the same title (see LINK)[3], where each video clip was searched, selected, and edited together all by hand. This left us with several questions as: what if an algorithm could do this work?, what videos would it select and which new narratives could it construct? From then on, the installation versions of the film utilized AI to drive a non linear storytelling construct. Even though we try to communicate the processes and ways in which LAWKI works in an attempt to demystify it, the AIs that are at work cannot be opened up and understood easily and to satisfactory levels. But in our view, this is not a problem: since we focus mainly on aspects of narrative and perspective, the AI’s role in LAWKI is also just that of an observer, simply another perspective onto our human digital lives and one that is equally valid as the one of a human curator. And in the same way that we wouldn’t ask human curators to explain each detail of their decisions (which they probably couldn’t), we accept the decisions of the AI as simply based on the data of human activity it has observed. Coming to  such an understanding of AI allows us to unlock the great potential that lies on them. Only computers have the power to explore and (re-)present the almost limitless vastness of the modern digital world while being on neutral ground and not tied to the perspective of individuals.

5. In the statement of this artwork, you mentioned some features of our information age such as attention economy and the circulation of fake news. Could you explain a bit further how the artistic languages you work with relate to these concepts?

There are different layers of artistic languages at play. In terms of attention economy, one of the most evident conceptual decisions we’ve implemented is a kind of ‘censorship’ of popular videos. If a video has millions of views, it is covered with a colour that obscures much of its content. On the other extreme, the large group of videos which fall through the cracks of this attention economy were only the ‘rich get richer’ remain mostly visible. This decision was made in order to confront/go against the notion that algorithms should feed users with what they predict they want, as e.g. the Google or YouTube algorithms do. On the other hand we have written language. The process of constructing the databases which are at the core of LAWKI starts by searching phrases from imaginary users which are then machine translated into the majority of existing languages. When searching visual content, the process therefore mimics the process  of a user navigating the online world and “naively” ingesting whatever it finds,  any event and topic that are  the centre of attention. In the same spirit, there aren’t any mechanisms to avoid fake news being platformed. A sincere story, a staged vlog, and overdramatizing fake news will be played back to back and without judgment or hierarchies in the way they are organized. Instead of falling into a rabbit hole of conspiratorial thoughts, LAWKI offers cross-pollination between the echo chambers of online communities.

 

6. The images shown on the installation respond to the visitor’s movement, and therefore creating an “embodied viewing”. I am wondering how this choice relates to your concept?

Currently, a lot of us are leading screen-based and media-oversaturated lives. It often feels as if we are physically detached from the screen and we don’t have much agency over it, but in reality the curated content we watch on the screen is there precisely because of our interaction with it. At this point, it is an extension of our body and a mirror of our society. Yet this doesn’t seem so apparent when simply scrolling and clicking on our phones. Because of this

disjuncture, LAWKI is activated by visitors’ movements. This gesture, which creates a causational viewing,urges us to remember that we are all active participants in the processes of content curation developed by the algorithms. There is almost non passivity in our spectatorship. In a sense, LAWKI puts the body in the foreground because it is an instrument the visitors activate. The embodied experience of the installation- physically moving through and across narratives and activating them at the same time-also serves as a reminder that the digital world is still situated in our real world. Even though it is literally virtual and therefore not embodied, the digital world actually interacts in very massive, real and complex ways with our reality.

7. Could you explain your thoughts on the spectatorship of the artwork or how you hope to communicate with the audience? (For example: what are the differences between sitting in front of a computer or scrolling your phone and the experience of walking inside an immersive installation?)

LAWKI uses the architecture of a space to create an experience that transcends the two-dimensionality of screens. While walking through the installations, the visitors are performing a choreography. Our aim was to go beyond mere spectatorship and to ask the audience to be part of the work through their interaction. That’s why if they remain static in front of a screen for too long, the image they observe will get flooded with one colour that eventually covers everything. Only while conversing with the screens: by walking, dancing, through movement, will it unravel its wonders. In this way, and one that reflects how narratives continue being created  in the digital realm and the fact that humans and technology become co-creators.

                             

[1] Bing, G. 1934. The Warburg Institute. Offprint from: The Library Association Record. p4
[2] Gombrich, E. H. 1986. Aby Warburg: An intellectual biography / with a memoir on the history of the library by F. Saxl. Phaidon: Oxford.
[3] Work of the short film LAWKI-Alive link: https://vimeo.com/462117359