Squaring and scripting the ESP game

François Bry, Ludwig-Maximilian University of Munich
Christoph Wieser, Ludwig-Maximilian University of Munich

ARTigo is a gaming platform (run in German since four years and beginning in English and French) with two goals

  • The development of a semantic search engine for art works
  • An (algorithmic) study on art work reception.
ARTigo offers an ESP game, a game called Karido collecting more specific tags than the ESP game, and squared and scripted versions of the ESP game we devised for collecting deep semantic tags.

Squaring consist running the ESP game with an image and a (validated) tag formerly assigned to this image. We call the game a squared ESP game because it consists in applying the ESP game to an outcome of that game. With a squared ESP game, Holbein’s portrait of Hans Schwarz and its tag “man” are further described by “young”. Heinrich Wilhelm Trübner’s self-portrait of 1882 has been tagged with “look”. The squared ESP game further yielded the qualification “melancholic” for that look.
Scripting consists in explicitly or implicitly giving a semantic context to the ESP game. The approach is inspired from a technique with the same name of pedagogical psychology. One of the various forms of scripting we experimented with consists in displaying two images similarly tagged and asking for what the images have in common or for what distinguishes them. A photograph of Oxford Bridge in Stowe Landscape Gardens, Stowe, Buckinghamshire and the painting of James Abbot McNeill Whistler “Nocturne in Gray and Gold” of Westminster Bridge yielded “old” as answer to the first question, and “size” as answer to the second.
Scripting is also done on ARTigo by daily teasers suggesting a critical look at art works. For example, Jacques Louis David’s “Napoléon crossing the St. Bernard Pass” of 1800 has been once commented by our partner from Art History as follows, stressing the ridiculous of this ordered painting’s: “Napoléeon rides a fiery black horse over the Alps. In reality, he sat on a donkey.”
Squaring and scripting turn out to be very effective at collecting deep semantic tags. Both approaches do not require from players to learn new game but nonetheless provide distinct playing experiences. Squaring and scripting are easy to deploy since they do not require implementing novel game logics.
For more, see the paper, “Squaring and Scripting the ESP Game: Trimming a GWAP to Deep Semantics“, Proc. of Intl. Conf. on Serious Games Development and Applications (SGDA), 2012)
François Bry
About the authors:
François lives in Munich, Germany, and Vienna, Austria. In the one city, he mostly go to work (with the Institute for Informatics of the University), in the other often to art exhibitions (amongst other to those of the Belvedere national gallery where the afore mentioned glimpse of past French grandness can be seen.) François blogs (in German, search for “erlebt”) on the experiences he make as a scientist and as a teacher.
Christoph is known for his skills at playing the trumpet, sailing and programming. He is researching for a doctoral degree on serious games, data analysis and semantic search.

About the author


I am a computer scientist working with the University of Munich, Germany. I am interested in declarative programming, automated reasoning, social media and crowd sourcing, semantic search, and novel forms of teaching (and a few more issues). I blog (in German) in erlebt on my experiences as a scentist and university teacher. I live in Munich, Germany, and Vienna, Austria, I am French and German, I like to travel in Europe and Asia and to read about history and science.

View all posts