This episode we speak with Adrienne Shaw about the paper “Encoding and decoding affordances: Stuart Hall and interactive media technologies“. This paper brings Stuart Hall’s concepts of encoding and decoding into proximity with ideas of affordance and technology.
Adrienne Shaw is an Associate Professor in Temple University’s Department of Media Studies and Production and a member of the Lew Klein College of Media and Communication graduate faculty. From 2019-2022 she will serve the first director of Temple’s new Graduate Certificate in Cultural Analytics. Shaw is author of Gaming at the Edge: Sexuality and Gender at the Margins of Gamer Culture (winner of the 2016 International Communication Association’s Popular Communications Division’s Book Award). She has co-edited three anthologies: Queer Game Studies (2017, University of Minnesota Press), Queer Technologies: Affordances, Affect, Ambivalence (2017, Routledge), and Interventions: Communication Research and Practice (2018, Peter Lang). She is also the founder of the LGBTQ Game Archive and co-curator of Rainbow Arcade, the world’s first exhibit of LGBTQ game history (Dec 2018-May 2019 in Berlin, Germany). From 2011 to 2015 she was also part of the multi-million dollar and award winning CYCLES project, which developed games to train users to identify and mitigate cognitive biases. A full list of her publications is available via Google Scholar.
As a joint venture, “Keywords in Play” expands Critical Distance’s commitment to innovative writing and research about games while using a conversational style to bring new and diverse scholarship to a wider audience.
Our goal is to highlight the work of graduate students, early career researchers and scholars from under-represented groups, backgrounds and regions. The primary inspiration comes from sociologist and critic Raymond Williams. In the Preface to his book Keywords: a vocabulary of culture and society, Williams envisaged not a static dictionary but an interactive document, encouraging readers to populate blank pages with their own keywords, notes and amendments. “Keywords in Play” follows Williams in affirming that “The significance is in the selection”, and works towards diversifying the critical terms with which we describe games and game culture.
Please consider supporting Critical Distance at https://www.patreon.com/critdistance
Production Team: Darshana Jayemanne, Zoyander Street, Emilie Reed.
Audio Direction and Engineering: Damian Stewart
Double Bass: Aaron Stewart
Darshana: You’re listening to Keywords in Play, an interview series about game research supported by Critical Distance and the Digital Games Research Association.
As a joint venture, Keywords in Play expands Critical Distance’s commitment to innovative writing and research about games while using a conversational style to bring new and diverse scholarship to a wider audience.
Darshana: Adrienne Shaw, we’re really happy to have you here. Can you introduce yourself in your own words?
Adrienne: Sure. My name is Adrienne Shaw. I’m an associate professor in media studies and production at the Klein College of Media and Communication at Temple University. I study video games, specifically LGBTQ representation in video games, as well as marginalized video game players and the video game industry.
Darshana: We’ve got a specific text of yours that we’re going to be chatting about today. That’s ‘Encoding and Decoding Affordances: Stuart Hall and Interactive Media Technologies’, which is published in ‘Media, Culture and Society’ in 2017. Really kind of interested in your use of Hall here, in particular, Stuart Hall is highly influential in cultural studies, but you tend to hear less about, you know, referencing of his work in new media and game studies. So, you argue in the paper that new media studies needs to engage more with questions of hegemony, can you tell us a little bit more about why you turn to Hall’s work to deal with this problem?
Adrienne: Yeah, I mean, I’ve always found Stuart Hall’s work particularly useful in thinking through how power works, and how people resist power simultaneously. And sort of thinking about not just sort of resistance as like a constant state that people are in, but how negotiations with media produce moments of potential resistance, and his work is especially good at thinking through that in terms of people’s positionality. And so like, thinking about what my personal context and relationship to the media systems and the class positions, specifically, of the people who sort of make up that media system, how that will impact my response to media messages. And those responses aren’t just about I mean, one of the things I like about encoding/decoding and Hall’s work generally is that it’s not just about meaning-making, it’s also understanding the intentions behind media producers is part of the audience reception practice. And I think that’s something that people don’t necessarily pay much attention to when they when they first learn about the reader positions in the encoding/decoding model. They think of those in terms of just interpretation. But one of the things I like about his piece is that he’s also thinking about how the audiences are thinking about what the media makers are trying to make them think and that there’s an unpacking of intention that his work brings, that I think would be useful in new media, because we have a robust body of work that critiques algorithms, for example, right, and the underlying racism of those algorithms. And a lot of, especially popular discourse around that thinks about the encoded bias of those algorithms as something that was not necessarily intentional but is built into the systems that sort of produce those structures in the first place. And I think that people’s understanding of their position within algorithms, and how they think those algorithms function is actually just as important as unpacking the sort of underlying computer science of how they worked in the first place. And I think that’s something that this idea of encoding/decoding that and Hall’s version of addressing hegemony sort of brings to new media studies. It’s not just understanding how the systems work, but understanding how people live in those systems and think about those systems. You don’t have to be a media scholar to try to unpack how media is working. Right, and he takes that very seriously. And that those everyday interpretations of how it’s working is just as important as what was meant by the people who made it or what, you know, scholars might think people might make of those things, he sort of brings the everyday experience back into the equation, which I find particularly productive and useful.
Darshana: So, you see encoding and decoding as two separate processes.
Adrienne: Yeah, I mean, I think that you know, and in addition to the text, which can be its own standalone object, but understanding encoding/decoding is sort of this circuit of production and meaning-making practices and so they’re related to each other right? Like they don’t exist in a vacuum from one another. The modes of production of the media text itself is its own process. and then the sort of reading side of that is its own separate process after the fact. And they’re aware of each other and they’re working within sort of similar systems of meaning as each other, but they don’t predetermine each other. And I think that that’s one of the things that I found useful in applying encoding/decoding to this idea of affordances is, it’s not pre-determined, it’s not something that you can, as the maker of a text, say, this is how people will understand my text, right? It’s one of the hardest things for any maker to get over is you don’t control it, once it leaves your brain, right, you can’t control how people understand what you did. But you do have a, you do have a general sense of how meaning works, and how like if I put this camera at this angle, it will evoke this sense in the audience. And the audience might get that and might see it and the audience might, you know, you’re trying to make a very serious scene, and then the audience might laugh at it because they’re just making fun of your movie. But like, they, they know what you meant, right? They know what you meant at the end of the day. And that’s part of why they are related processes, but they don’t predetermine each other, you can’t force a specific meaning. And you can’t assume people understand the codes of production that you use, especially if you’re working in a medium that’s new or medium that hasn’t existed before, something that audiences aren’t familiar with. If people understand the underlying tricks of the trade, don’t get what you did, if they don’t get it, then they, you know, there are different kinds of reasons, they might not get it that you don’t have control over. Right?! Either they don’t understand the techniques you use, or they don’t understand the, you know, the backstory of the trope that you’re drawing on. Or they get it and they just reject it and they might be like, “I get what you’re trying to say, but that’s bullshit!”
Darshana: This might be a good time to kind of like go through the three modes of repeating that Hall identifies which dominant, resistant and the other one I always forget.
Adrienne: There’s dominant and then negotiated and oppositional and one of the most maddening things about trying to teach encoding/decoding is that dominant hegemonic isn’t really clearly defined in Hall, right? It’s basically if you understand that media system, that’s probably what the producer meant by it. It’s very much assuming that the hegemonic reading is the preferred reading of the Creator. And an oppositional equally not clearly defined, like does, is that gross mix is understanding or is it an absolute rejection of the worldview, which my understanding of Hall’s politics, I think it’s a rejection of the worldview of that piece. The vast majority of all experiences are somewhere in that negotiated, which is like, you get what the Creator is trying to say. You disagree with aspects of it, you’re not you don’t misunderstand it, but you do reject underlying implications of the text in some way or aspects of it, or there are things about it that don’t quite fit with what you would have liked to do? Right? When it comes to sort of dominant, hegemonic use, like one of the easy ways to think about it is like for most new media, what they want us to do is make the money, right, what they want us to do is do the right kind of clicking, the right kind of reactions, the right kind of consumption behaviour that will reproduce the version of capitalism that those systems are built upon. Oppositional can range from sort of hacking it to trying to break the algorithm, which some people try to do, to not using it at all, or not using it correctly, right?! Whereas I think for the vast majority of us, we’re doing some version of negotiated use, we like, I know that my eyeballs are the thing that’s making the site money, but also, this is a good way for me to keep up with my work colleagues. So, I’m going to continue to go to this site and use it so that I get enough of the kind of content that I’m looking for. But you know, maybe not, like give over all of my personal information at the most recent meme so that somebody can easily guess all my passwords, right? Like, those are, those are the versions of engaging with new media that I think sort of aligned with this idea of encoding/decoding, this idea that these sites are structured in a way that we understand under how they work to some degree, and we understand how they want us to use them. But within that, there’s a lot of flexibility for how we actually do use them.
Darshana: You’re also making a sort of like interdisciplinary argument here, in many ways, which is where Hall’s work, because it is what has been read as about reading, you know, about reading texts, whereas in games and new media studies, we’re more talking about interactivity, there may be a moment of interpretation. Of course, this has been like the source of a lot of like, kind of like, back and forth which we won’t go into. But, that kind of like, do you see that as being one of the reasons that Hall’s influence has been relatively limited to date in something like game studies? And is that part of the kind of like the intervention that you’re trying to make in bringing these, these concepts together?
Adrienne: I mean, I think that there are two things there. One is, is that it does sound sort of very traditional reader text-based. And that probably is why it hasn’t, it doesn’t leap off, it doesn’t leap into the syllabi of people teaching about interactive media, media, necessarily. And it’s, you know, one of the things I hope I accomplished by writing this piece is I that think, this is a way for people to think about it, and to think about it in a way that’s tied to a longer history of media studies. And I think that’s, that’s, that’s also one of the bigger points I’m trying to make is that like, I feel like a lot of game studies seems to have forgotten Media Studies existed when game studies started, like games, game studies formed and coalesced around the late 1990s, early 2000s. But media and cultural studies had existed for a long time before that, asking a lot of similar questions, maybe in a different context, but a lot of similar questions. My first book was largely informed by media and cultural studies, game studies was there, obviously, but like, I found that most of the, the questions I wanted to ask and the phenomena I was interested in, could be better explained drawing on media and cultural studies, theories, and sort of building on those. And understanding that games are a media text. I don’t think that’s controversial to say, I’m always surprised by things I say that end up being controversial, but like, media texts have systems of production that underlie how they get made in the first place. There are norms that go into how they are created, and audiences learn to interpret those in particular ways. Even if it’s interactive, right? I know when a game is trying to make me make a decision that will influence the story versus Is this a one-off decision? Right? Like we have created enough games built around that, that I know that that, you know, me deciding to kill that wolf in Assassin’s Creed Valhalla, which is what I’ve been playing this weekend, right, will not affect the end of the game. We decided to kill that rando guard, in that one town will not affect the end of the game. But this big cinematic build-up to this specific question, somebody is asking me where there are three options. And it’s clear that two of them are very different options, that one will always produce some outcome later, though, that is meant to be a pivot point in the game. And those are, you know, those are norms that are achieved over decades, over time of game production, they get retaught that’s how people who make games in the future learn that that’s how you make that decision. You know, you don’t, you know, I think you’d actually be really funny to create a game where all the like little meaningless decisions that we’re used to doing tend to have the most impact on the outcome of the game, like, how many times did you make soup for your dog ends up having like the biggest outcome on the back of the game, and all of the like big dialogue tree choices you make have absolutely no impact whatsoever. Like, I think that’d be a really fun game to play. And I hope somebody makes it. Understanding encoding/decoding, specifically being about questions of power is something that I think hasn’t been fully incorporated into games necessarily. If you are doing text analyses of games, there’s this idea that there’s one-way players interact with the thing. And any audience-based studies with game shows that we know, we know that’s not true, there are multiple different ways people might approach it. I think the encoding/decoding helps bridge the gap between those two by acknowledging that there are different ways people are coming to the game, there are different knowledge sets people are bringing to the game that will change their understanding of what the game is doing, or what their options in the game are. And I think that understanding the power implications behind what is the right and wrong way to play a game isn’t just I mean, I can sell those books on cheating is really useful. But I think that understanding it just as a normative practice is one way of going about it, but also understanding who’s version of not doing what the game expects you to do is seen as part of game culture versus not game culture, is a hegemonic question that we could unpack by taking more of a media studies approach to it.
Darshana: And that kind of like brings us to the flip side of the paper, which is this question of affordances. It seems like you’re opening up the space in that negotiated concept to bring in these these questions of interactivity.
Adrienne: Absolutely. I mean, I think that understanding because I mean, we’ll get to affordances but understanding affordances as a site of interaction between an object and a person or an object and the user, right? That that is not all that different to my mind from understanding meaning-making as this position between the producer and the text, right? That we’re always negotiating how we understand we’re supposed to interpret this text or how we’re supposed to use this technology and what we actually want to do with it. There are oppositional and hegemonic uses, but the vast majority of our experiences is in this negotiated space. And especially when we’re talking about texts that are consumed by a wide variety of people on a global scale, that this idea, that this game means one thing to all people just, it just doesn’t map with what we understand about media consumption, in a broad sense. And so I think that understanding how even the interactive possibilities aren’t pre-determined. You can encourage certain types of play, but you can’t force people to play it that way, is something that I’m particularly interested in, because I think that it opens up questions of, it opens up questions of power, it opens up questions of how people can make games what they want, even if games aren’t being what they need them to be.
Darshana: Yeah, I mean, you just need to think about fortnight, and that wasn’t developed to be what it is now, you know, the original kind of like idea behind what they were developing and you know the, the affordances in that game, it only became what it was through cultural phenomena as much as sort of like, you know, the gameplay and stuff like that. So that does bring us to affordances, or action possibilities, I think Gibson calls them and this is kind of a drawing from cognitive psychology, which is we you come at it from. Sometimes, what I teach affordances is through Don Norman ‘Design of Everyday Things’. How does the, the concept of the affordance? Like, how would you build that out and explain that concept?
Adrienne: Yeah, I mean, I think, you know, the idea of affordances came out of developmental psychology is like this, basically, at what stage do humans specifically, I don’t think it was used to apply to animals, but at what stage so humans figure out how to use the world around them? Right? Like, when does it become obvious to you what things you should put in your mouth and what things you shouldn’t put in your mouth? What are the like moments in which you realized, like, this stick can be used in x different amount of ways, but it really should be used in this way? And then, you know, Norman, and other designers sort of adapted that into design theory, which is built on the idea that how do you tell somebody through design how to use this thing? Like, how do you make it very clear from the outside how you’re supposed to use this thing? Right? You know, the classic example being things like the shower controls, or the ‘Norman Door’, right, the door that looks like it should be pushed, when it should be pulled, right? Whenever I explained affordances that way to students, it’s like, oh, so it’s not something that’s wrong with me, the door is wrong. And I was like, exactly, the door is designed wrong, the door is encouraging one motion, whereas it actually is requiring a different motion. And I think that that’s one of the ways of understanding affordances as not just features, right? Like that’s, it’s a pet peeve of people who do design studies, or people do even new media studies. Now, it’s like affordances became a buzzword. And now people use it to describe basically every feature of a product as opposed to a specific negotiated interaction between the person and the object, right? One way of thinking about it is sort of like an affordance is in the using of something just like Interpellation is in the turning like the moment you turn, when the cop call on you, that is, when you’re being hailed by the ideological state apparatus, you’re acknowledging your existence as a subject is the moment you’re turning. The moment you’re using something, that’s when the affordance happens. It’s not something that’s sort of inherently in the object and not inherently in your use of the object. It’s the negotiation between the two of those, which again, that’s why I feel like it maps to Hall’s encoding/decoding pretty well is because there is this idea that there’s a correct way to use something and there’s a correct way for it to indicate that’s what it’s meant to be used as. But and this is what I try to explain in the piece that doesn’t predetermine how you use it. Just because it’s supposed to be used in one way doesn’t mean it has to be used in that particular way. There have been lots of different people who have explained this in terms of sort of imagined affordances or false affordances or hidden affordances. Right, this idea that there are things a designed object can do that doesn’t seem obvious on the surface, or there are things that it seems like it should be able to do, but it can’t are there ways you want to use it, even if it doesn’t necessarily on the surface show that that’s a way you could use it, but you want to use it that way. Or it seems like it should be able to do a certain thing. And so, I think affordances is one of those things where it’s, it’s easiest to explain via examples. And so, like, like the case of the door like it seems like it should be pushed one way and when it should be pulled interfaces indicate to you what you should be doing in a given space. And again, like the heads-up display will tell you what you’re supposed to be looking at and when you’re supposed to be looking at it, and it tries to direct you. And it’s why games that are a little too open-ended that don’t give you much instruction end up frustrating people, right? I used to use the game, ‘Gone Home’ in my LGBTQ media representation class, there are various reasons in which I couldn’t assign it anymore. But one of the, one class, a good two-thirds of people couldn’t figure out how to open the front door like they’re just wandering around on the porch for hours not being able to figure out how to go in eventually, they watch the Let’s Play video on YouTube instead. And just so that they could get to the part that had a story, right. And that’s one of the things where it’s like if you are used to that kind of game if that’s the kind of thing you want is to just wander around and click on things until something happens, then it works for you. But if you’re used to having things tell you what to do with them, then it becomes very frustrating. And you can’t engage with it in the same way.
Darshana: So, in a way, like that question of Interpellation, which is from Louis Althusser. And that’s kind of like a moment, momentary relation that gets established in an encounter. Would you say that’s kind of like it’s the affordance doesn’t preexist the encounter between user and designed object?
Adrienne: Yeah, I mean, I think that you know, we can map like what the affordances of something are by studying how people have used it. But I think that attempts to sort of say, I’m going to unpack the affordances of this by just looking at it don’t really get sort of the, the underlying nuance in affordances, right? Like, you can say, these are the features of this app. And I can walk through how it’s used. But until you sort of go through the everyday experiences of it, and look at how you use it, but also how other people use it, do you start to understand what the affordances are as those decisions people make in their interaction with that interface. Again, to your earlier question is one of the things why perhaps interactive media has avoided thinking about it through sort of a cultural studies lens of thinking about it as something that’s not pre-determined something that is, you know, ongoing negotiation of how do we use this thing, and that that’s something that affordances was meant to get at? People have used it in a lot of different ways. But a lot of the versions of it have sort of narrowed down to, I can point to this and say these are the affordances of this particular platform, as opposed to this is a platform, I’m going to look at the ways people have used it and use that to identify what the affordances of it are.
Darshana: So we have encoding/decoding on one side. And then we’ve got affordances on the other. And putting those together, you seem to be like pointing towards a future research trajectory, or some questions that we can kind of like ask.
Adrienne: Yeah, I mean, I think cheating is a great example. Because you know, one of the things that Mia Consalvo’s book on ‘Cheating’, which I think is 2007, one of the things she explores is how some versions of cheating might be seen as unfair, and add-ons that people have added to the game to give them an advantage. But other versions of cheating are just exploiting existing built-in things to the game in ways that other people didn’t know they could do. Right. So, like cheat codes, most cheat codes already existed in the game, they’re basically designed speed ups, it’s like, I need to jump from this part of the game to this part of the game where I need my character to have all the powers, so I don’t spend, you know, hours of troubleshooting trying to level up, right? And that those are things that you can uncover access to the game through, and suddenly you’re playing a different version of the game, right? I wouldn’t call those necessarily oppositional uses, I would say that you know, you’re not, you didn’t hack the game, you didn’t change anything underlying about the game, you just accessed an exploit that exists within the game, to play it in a different way. Or accessing locked content, right? I mean, in the paper, I use the example of the hot coffee mug from Grand Theft Auto, which feels like 1000 years ago at this point. But it was like a scene in the game that clearly before shipping, they decided they couldn’t include for, you know, probably the parental ratings and, and censorship concerns. But it was still there, all the code was there, somebody just had to unlock it so that you could play it in the game versus other mods, which, you know, I’ve been working on the LGBTQ game archive for several years now. And a lot of those mods are adding in same-sex relationship options that didn’t exist in the game in the first place, right? There you know, there are lots of examples of people using mods to add themselves into games, so mods that will change the race of characters, mods that will make certain characters playable that weren’t playable in the original game. There are versions that are, you know, mods that are allowing you to play the game in a way that you would prefer to play the game that’s not inherently built into the game or even you know, like Robert Yang’s games, you know, mods that create a new game out of the existing game assets. And that those are, those are closer to what I would say are oppositional uses, are mods that aren’t just unlocking content of the game aren’t just, you know, making it easier to, to accomplish what the game wants you to accomplish, which, at the end of the day is what cheating is, right? Cheating is how do I do what the game wants me to do fastest? Rather than how do I do something the game didn’t intend for me to do in the first place, which I think not all mods are that but a lot of mods are and I think especially mods that are representational mods, mods that are trying to make a space in the game that the game industry wasn’t making available. I think that those are a kind of mod that the power involved in that, like the, the representational hegemonic power of using a mod to reject the game’s like refusal to include you. I think that that’s what Hall gets us that just talking about mods as like, just a general part of game culture doesn’t quite get at, right? Not all mods are the same. Not all mods are oppositional. Not all mods are pushing back on power. But I think if we think about it through questions of power, we can point to the mods that are, right? Same with cheating, is cheating, trying to do something the game doesn’t want us to do? No, not necessarily. So, what is cheating? It’s, it’s a social arrangement. It’s not about the text, right? Like, it’s not a rejection of the text, as the creators wanted us to play it. It’s a violation of social norms in a given space. And that’s a different understanding of what it is in relationship to the text. I think that there, there is also, you know, there are games that make it easier for you to not do what it expects you to do. So, you know, there are versions, I can’t remember which, which game in the series it was, but you know, I had a friend who really like playing Metal Gear Solid. Well, he just went through and shot everybody with darts that put them to sleep, right? I think it’s called the Sphero badge or something like that. Like, if you go through the game, and you just put everybody to sleep, you don’t kill anybody, that’s built into the game. Like if there’s a badge associated with it, you know, somebody made it into the game. Versus you know, there’s stories of people who want to play through all of World of Warcraft as pacifists, it’s actually really hard to do that unless you’ve matched up with people who will do all your killing for you. And so like, that’s something that’s not built into the game, and that you have to find a way around what the game wants you to do to accomplish. Versus other things that are not necessarily the thing they expected you to do, not necessarily a thing that would make it the most fun game. But if you want to just drive around and deliver pizza and ‘Grand Theft Auto’, you can do that, you’ll never get very far. But you could do it if you wanted to, right? That’s something that’s built into it, even if it’s not intended. I am particularly interested in the moments where if we understand what the game is allowing us to do versus what players make possible for themselves within the game, we can understand a little bit more about who’s being included and who’s being excluded, right? I think that that’s the, whose perspectives are being included and whose are being excluded based on how much work goes into resisting the dominant affordances of it. Right, how much harder it is to do a negotiated or even further oppositional play of that game tells us who’s being left outside of that game.
Darshana: Thanks so much, Adrienne. Where can people find out more about what you’re up to at the moment?
Adrienne: I update my website adrienneshaw.com fairly regularly. But the LGBTQ game archive is my big ongoing project that I’m trying to do a lot of updates on in the coming months. And then obviously, you can follow me on Twitter, although I tweet less than less these days @adrishaw.
Darshana: Awesome! Thank you so much.
Darshana: We hope you have enjoyed this episode of Keywords in Play. For more great ideas around games check out criticaldistance.com. Or take a dive into the DiGRA archives at DiGRA.org.