In reading Bogost’s text on different types of rhetoric (verbal, written, visual, digital, procedural), I found myself wanting to push back on the limitations of the visual (pp. 21-24). According to Bogost, “Visual rhetoric simply does not account for procedural representation” and images are subordinate to process because they are constructed in media through code. If procedural rhetoric makes arguments through coded/programmed processes and behaviors (rather than solely words and images), why can’t images also make arguments through processes? Reading through Blair, Birdsell, Broarke, Hill, Lake, Pickering, Deluca felt confusing and overly complicated. Going by the definitions that procedure is a way of doing something but also an execution of rules for behavior, I can make an argument (albeit messy and somewhat haphazard) that images, particularly symbols are forms of procedure. Symbols as icons placed on street signs help us navigate the built environment, both lawfully and informally. As emojis, they have become an essential behavioral tool of communication. As signifiers such as the “raised fist” graphic, symbols function as a method of protest and civil disobedience, a way to provoke behavior. Like a computer, these image processes also reflect the material world.

Earlier in the article, Bogost brings up symbols and symbol manipulation which might be why I was so focused on this subject. I thought of the example of The International Symbol of Access, originally designed in the 1960s by Susanne Koefoed. At the time, it was a radical and historic gesture to display the symbol in areas that could be accessed by people with wheelchairs (parking spaces, entrances, etc.). It created awareness and established procedures for spaces, streets and buildings. The symbol became part of our visual lexicon, easily recognizable, instantly sparking behaviors and understanding.

However, it was designed to be static, the person represented was passively seated in the wheelchair with their arms stagnant on the armrests and their feet positioned firmly on the footrest. In 2011, the Accessible Icon Project used design activism by placing a newer, more active icon sticker on top of the older symbol on existing street signs in Boston. The new icon showed a person leaning forward in motion, arms which actively control the wheelchair and the removal of the footrests. The act of placing the new icons on noticeable street signs provoked the public and actively advocated for better disability rights. Today, the new icon is used globally in various iterations, by governments and citizens. Perhaps, the symbol change also changed public perception of those with disabilities, no longer resigned to the wheelchair, instead a person with agency and mobility.

This may be a stretch but I see this as a type of procedural argument- the symbol, both old and new, constructed a model of how the world works and in the process of updating the symbol, maybe even improved that world with refreshed ideas and promoted policy change. There is also some form of persuasion happening, a characteristic symbols share with images. Certainly, the procedural rhetoric is not computational and not related to videogames in the same way the McDonald’s game implies societal wrongdoings, but I think the symbol has the capacity to reveal how things work in a rhetorical way- through our experiences and interactions with them.

In an effort to educate (or re-educate) myself on the Israel-Hamas war and its associated complications, I’ve been absorbed in the news media cycle. Reading and watching videos on the topic of surveillance intelligence and facial recognition and the Israeli military’s use of it in the contested city of Hebron in the southern West Bank has been both horrifying and eye-opening. Certainly in the U.S. and other countries, the use of facial recognition brings up issues of privacy, racial discrimination, policing and politics. While definitely not a game in the pleasurable sense, there are features of Israel’s surveillance technology and the way it’s being used that are game-like and relate to Zuboff’s description of behavioral modifications and surveillance capitalism as a form of state power. Obviously, I don’t have first-hand experience of the technology but I will try to summarize how it works based on the limited information I could gather (details of this program have not been released).

Blue Wolf is an app designed to work on specific models of Samsung devices handed out to the Israeli military (it does not work on all devices nor can it be downloaded). The purpose of the app is to capture photos of Palestinians’ faces and match them against a database of other Palestinians’ faces. The app is part of the larger surveillance program called which includes face scanning cameras and closed-circuit television cameras for monitoring planted in the city streets, along with checkpoints that control areas Palestinians can enter. The database is referred to as the “secret Facebook for Palestinians”. The military’s justification is that this is necessary to thwart terrorist activity in Hebron. Once a soldier takes someone’s photo and a match in the database is made, presumably through some type of artificial intelligence, the app shows various notifications: it will light up yellow if the person is to be detained, red if the person is to be arrested, and green to release the person. However, the notification does not tell the soldier what the grounds are for the arrest or detention.

When these programs first launched in 2020, there was a need to actually build the database with photos and other personal information like family history, education, drivers license, and security rating. So, the Israeli military launched a competition to incentivize soldiers. Each army unit was expected to take at least 1,500 photos a week, the unit with the most photos would win prizes like a night off. On their daily patrols, the soldiers were allowed to stop any Palestinian and take their photo whether they consented or not. Children, unaware of the circumstances, posed happily while the elderly and women tried to resist. The soldiers are participants in an involuntary crowdsourcing scheme of big data.

I would argue that Blue Wolf was not designed to control the behavior of Palestinians, the ones being surveilled, who have not consented or opted into this program. If it was designed for this purpose, it fails – the app does not make them more compliant or docile in the eyes of the government. If anything, the use of this app gives them further reasons to resist. The behavior modification occurring here is on the part of the soldiers, the users of the program. They have been trained to respond to the colors of the interface. They are not given more detailed instructions or are allowed to ask questions. To further complicate the scenario, the soldiers also have not consented or opted into using the app. Whether the soldiers have any sense of fulfillment or emotional engagement (Sicart) using the app (other than maybe winning a prize) is unclear. Situating Blue Wolf with Zuboff, both the users and the surveilled have been tuned, herded and conditioned. However, instead of a company profiting off of this behavior, it’s the Israeli government who retains and increases control of both the Palestinian population and its own military by taking away their rights and autonomy.

Sources:

1. How Israel automated occupation in Hebron
2. Israel escalates surveillance of Palestinians with facial recognition program in West Bank

At first, I was very skeptical about turning Tristram Shandy into a game. It’s long-winded, confusing, told by an unreliable narrator, frustrating but also funny and unpredictable. The process of reading and annotating it with the group was helpful to highlight our interests and observations. We all agreed that the digressions stood out—it takes Tristram so long to get to his story and how could he possibly start before he was even born (Book 1 does not even cover his actual birth)? Our early idea of looking at the punctuation and Kai’s first python outputs of text and experiments with motion really got us going. The fact that NLTK does not consider the dash as punctuation made us want to focus on the dash even more! Sterne in particular uses the dash seemingly with abandon but he may have also placed them intentionally within the text. By doing the opposite of a typical text analysis, and seeing only the dashes, the digressions are emphasized even more in our game. The dashes page is my favorite! Perhaps you don’t know what they say or mean but you’re keenly aware of their existence and the author’s attempt at diverting attention and creating tangents. And, thinking about the different variations and uses of dashes—a pause, an omission, a conclusion, an intrusion, an insertion, a separation, an exclamation, an attention-grabbing symbol, etc.—adds a level of meaning and understanding to this story that you wouldn’t normally get with a close read or conventional text analysis. In printed form, the dash is a typographic, visual expression. On a digital screen, the dash retains this characteristic but can also become interactive (we made them hyperlinks). If we ventured into audio, the dashes would have been a ripe area for sound experimentation.

Another area of Tristram Shandy that interested me was the black page representing Yorick’s death. When confronted with this on Project Gutenberg, I wished I was reading a printed copy instead. It must be a bit unnerving to see this physical page after reading so many long-winded sentences printed in a serif font of the time period. I immediately thought of Kazimir Malevich’s Black Square painting of 1915, the seminal starting point of non-objective art, which Tristram Shandy pre-dates by more than a century. Sterne had profound foresight including this in the novel before the emergence of abstract art. Bri pointed out the black page was turned into an exhibit in 2009 where artists rendered their own interpretations. Our digital interpretation, albeit hidden unless you come across the line “Alas, poor Yorick”, allows the viewer to create their own version of the black page by playing with the mouse. Coloring areas of the digital page with black text generates a void and relates to other areas of the game where we purposefully “erase” text. I really enjoyed taking this discursive text and turning it into voids, erasures and empty spaces.

My specific part of the game creation involved taking our versions of the text (punctuation, characters, TLDR phrases, commands to the user, meta-discourse) and turning them into html pages. There was a lot of searching and replacing involved and dealing with the intricacies of Microsoft Word and Google docs. It wasn’t a perfect process (grammatical errors abound), but seeing Sterne’s text in code was eye-opening. It was like I was inserting my own digressions to the text (code needed to be added to in order to hide areas or link other parts of the text). The end result when viewed as html is completely headache-inducing and unintelligible (reminiscent of Kai’s morse code page). Here’s a snippet:

Lastly, I applaud Maria and Bri’s skill in writing the dedication and instruction pages. It adds a level of humor akin to Tristram’s tone as the narrator. I appreciate the task of writing as you normally wouldn’t, like drawing with your left hand when you’re right-handed. This speaks to that zone of uncomfortableness, frustration, but also amusing experience of reading Book 1. We hope we were able to translate these feelings into our game!

This week’s readings emphasize play in contrast to work or “ordinary life”. They also leave open the possibility of these boundaries being blurred and it is within these boundaries that I can find many correlations with my work as a graphic designer. Design (as in “to designate, make a plan with intent, or solve a problem”) may sound different from the free and non-serious characteristics of play but many of the same factors go into each: competition, observation, improvisation, motivation, curiosity, skill, etc.

Huizinga describes the “rules of the game” as an important factor in holding the game together – break the rules and the whole game falls apart. Similarly, Bogost points out that the limitations within games are what actually makes them fun. With design work, the rules, restrictions or limitations placed on you (either by the client or yourself) are necessary: print publications must be a certain size, brand design must contain certain colors, advertising must show specific copy, layouts might use specific grids or the golden ratio. These restraints provide a designer the freedom to play, to develop creative solutions with meaning. My design students often misinterpret rules as stifling, not understanding that if they were presented with a blank sheet of paper and a project that had no rules, no purpose, they would be frozen with fear and confusion. Relating the design process to Caillois’ continuum, it would fall somewhere between paidia and ludus. The designer is spontaneous (paidia) within a structure (ludus).

In terms of Caillois’ classifications of games, design is perhaps most associated with alea and mimicry. Randomness and chance (alea) play a factor in design, although it is not a passive activity. Whether using computer tools or physical tools, “happy accidents” happen when you mean to do one thing but it ends up looking totally different but perhaps that result is better than anything you could have thought of intentionally. Too many times, I’ve played with an Illustrator tool I’m not familiar with and ended up with solutions that inspired new ideas, or made some mistakes with code that turned out to be the best thing that could happen.

With mimicry, it’s not a direct tie-in with design, it’s more conceptual. Designers don’t pretend to be someone else, however, the work can certainly mimic other or false worlds. An advertisement may be designed to convince a consumer that their beauty will be enhanced by using a certain skin product, highlighting images of models which have been manipulated by digital means. Sure, this may be unethical, as advertising often is, but it calls attention to a reality of design – it can be used to create illusions, to deceive. And this is certainly the case when design works in conjunction with technology. Huizinga also uses the word representation as the display of something. Representation needs an audience; while not quite a performance, there is no design without it being seen. In fact, the audience is an important consideration, a design can’t function for all demographics or cultures.

Lastly, I was thinking how Bogost’s magic circle relates to design. In order to be seen or experienced, design must exist in reality, in space and time, not outside of it. The magic circle might consist of the audience who is targeted by the design or the audience who is affected by it. But then, there are also multiple playgrounds a designer might use in the process. They might conceive of ideas in a mindspace, which is one type of playground, playing with concepts, scenarios. Then, the blank canvas on which they design is another type of playground, sketching, doodling, experimenting. In all of these playgrounds, the designer attempts to find order in the form of the design solution.