Some Strategies for CDA Research
I would like to briefly sketch some theoretical resources for extending CDA to be able to undertake effective critical analysis of inter-media franchises: first a multiplicative, heteroglossic model of meaning effects across media and then more briefly two cross-media strategies for critical analysis.
The first is my approach to cross-media analysis of inter-discursivity, which builds on Halliday’s meta-functional principles for language and on my interpretation of Bakhtin’s notion of heteroglossia. I take inter-discursivity to be most interesting when it is about relationships between different discourses and their ideological effects as instantiated in different registers and different genres in different communities or sub-communities. In this sense it is much the same notion as Bakhtin’s heteroglossia, in which different “social voices” speak differently about the world, and these differences are correlated with the differences in their social positions. In Bakhtin’s original formulation there was less of modern critical sociology and more a sense of how these different voices are arrayed in relation to one another in the modern novel. Elsewhere I have tried to stress the potential of this notion for collocating our practices of social positioning with other discursive practices (Lemke, 1995). It is not difficult to extend such a proposal to non-textual genres and non-linguistic semiotics. We need then to ask how all media artifacts present (and allow us to perform) different social identities to different degrees, or not at all, or how they oppose such identifications. Said in another way, we need to ask which groups of people identify with which media artifacts and qualities (types of music, types of art, types of videogames; visual styles, musical styles, gameplay styles), and then discover what principles are at work for differentiating and hierarchizing these groups that can be discerned from the affordances of the media artifacts themselves.
Such a project can only succeed of course in the case of multi-media, (i.e. those which coordinate the use of different semiotic resource systems such as language, images, music, etc. to produce meaning effects and ideological effects for various groups of users), if we also have a way to approach the coordination of meanings across media. For this, I generalize (Lemke, 1995), as have others (Kress & van Leeuwen, 2001; O'Toole, 1990), from Halliday’s three linguistic metafunctions (Halliday, 1976, 1994) to see these as generalized semiotic functions, each of which is in operation simultaneously in every meaning-making act, and each of which must be supported by every meaningful text or artifact. They constitute the well-known triad of (1) ideational or representational meaning content, which I generalize as the Presentational function and Presentational meaning, (2) interpersonal-attitudinal meaning effects such as speech act relationships and evaluative semantics (the general Orientational function), and (3) textual-textural or structural-cohesive principles (the general Organizational function). So, in the traditional terminology of art history for images, these would encompass the iconographic, perspectival, and compositional aspects of images, respectively. Van Leeuwen has also done important work to extend these notions to the domains of music and other socially meaningful sound effects (van Leeuwen, 1999).
What particularly interests me, however, for the purposes of this project is not the generalization of the semiotic functions themselves across media, but rather their inter-coordination within any particular multi-media genre. While each such genre clearly achieves such inter-coordination in its own way, we can begin by asking how the overall meaning effect is constituted by:
(1) the product of the Presentational (or Orientational, or Organizational) affordances and effects from each medium: e.g. the Presentational meanings from each of the media cross-contextualizes those from the other media, in general either creating a single more specific Presentational meaning effect (mutual narrowing for mutual consistency) or blurring the definitiveness of the meaning from any one medium by lack of perfect consistency with that in the others (see below on incommensurability), and
(2) the cross-influence of Orientational and Organizational meanings from each medium on the net Presentational meaning effect (and of Presentational and Organizational on net Orientational meaning; and of Presentational and Orientational on net Organizational meaning effects).
I call this a “multiplicative model” of multi-media meaning effects (Lemke, 1998), because it assumes that meaning effects are not simply additive, but “multiply” insofar as the meaning potential, or set of possible meanings from each component multiplies that from each other component, creating in principle a vast combinatorial space of meaning possibilities. Any particular text-specific, constructed-interpreted meaning is then one intersection in that very large space, and so correspondingly more specific in its meaning than it would be as one instantial meaning out of the much small set of possibilities obtained by adding each dimension. The basis of this model is the general notion of cross-contextualization: that the meaning of any word is made more specific by the context of words (or situation) around it. So also the meaning of any text is made more specific by our assumption that it is consistent with (in some local sense) the accompanying image. And conversely the meaning we make with any image is more specific when we make the meaning in such a way as to be locally consistent with an accompanying text. Provided, of course, that the multimedia genre gives us recognizable cues through its Organizational functions to tell us that the text and the picture are indeed meant to go together. I have been speaking here implicitly of the Presentational meaning of the text and image, and this simple example shows how the Organizational function from another semiotic (say page layout of text and image) influences Presentational meaning.
I have found this an extremely rich heuristic for cross-modal analysis of multimedia genres such as scientific publications (Lemke, 1998), NASA websites (Lemke, 2002b), and political cartoons (http://www-personal.umich.edu/~jaylemke/papers/polcart.htm ).
There are still two more strategies I want to recommend, particularly for their critical potential.
The first is to look for instances of cross-modal subversion of consistent meaning effects. (Note that I am using the term modal here to refer to semiotic modes or media such as language, images, music, etc. Sometimes the term “media” is also used for the technologies, or for the multi-modal genres.) No two semiotic resource systems are capable of producing exactly the same meaning potential in a text or artifact. There is no image that corresponds exactly in meaning to a word, phrase, or sentence. And conversely, there is no perfect description of an image in words that captures all of its meaning affordances. This is the principle of incommensurability. It derives from the simple fact that the meaning potential of any sign within a particular semiotic depends on its relationships with other, alternative signs available within that semiotic. And its concrete meaning-in-use depends on how that meaning potential is further cross-contextualized, as discussed above, with culturally significant features of the context of situation in which the sign appears for communicative or other purposes. The practices of a community may well construct conventional one-to-one relationships between forms from different semiotics (e.g. an equation and its graph in mathematics), but each form retains meaning potential or affordances not available in the other. Not even a thousand words can exactly tell what a picture means.
A consequence of this incommensurability is that even a matched text and the picture (e.g. a newsphoto and its caption) each can easily be interpreted in ways that subvert or undermine the meaning of the other. A critical reading, or a critical analysis can often exploit this incommensurability to disrupt the ideological effects of the joint text-image construction, or to identify those effects and how they are being produced. As we move to more complex multimedia, with many cross-modal ideological effects, the opportunities for subversive readings “against the grain” also multiply. And in the case of franchise-world products, where the ideological effects may be subtly distributed across completely different media (television programs, websites, videogames, calendars), the analytic possibilities of such a critical cross-modal analysis seem greater still, if not necessarily easy to carry out.
The final critical strategy I wish to recommend is one I have called “traversal” analysis (Lemke, 2002a, 2003). It relies on the fact that, by and large, ideological effects of individual texts and media tend to be confined to particular institutional sites, those where the genre and register of the text or media product are appropriate. Over time, however, people increasingly find themselves moving among different institutional sites, where we encounter different genres, in which some of the same ideological principles of the wider community or culture may also be operative, but in which they are once again never quite the same as in some other genre of ideological context. If we do not connect meanings from one context and site to another, we may not be aware of the potential inconsistencies and contradictions across sites and genres. But the customary experiential separation of institutions, sites, and genres, particularly their separation in time, is being undermined by new information and communication technologies. When we channel-surf on cable television, or web-surf online, or get a personal cellphone call, email, or IM/CHAT message during a business meeting, we find ourselves cycling our attention much more rapidly among various real and virtual institutional spaces and their genres. We find ourselves far more likely to start cumulating meanings along our daily traversals across multiple institutional worlds, and so making meanings in which contradictions may get noticed. This is good for us all, but a special boon to critical discourse analysis.
Except for two difficulties. The first is the practical one that it is quite difficult to follow people, other than ourselves, around through their daily or weekly traversals across institutional and attentional spaces. The second is that the new phenomenon of inter-media franchise-worlds is in fact something of an effort on the part of capital to create more consistent meaning-worlds across sites, contexts, institutions, and media. Assuming, however, that the franchises are not yet entirely successful in this regard, as analysts we ought to be able to start discovering some of the new strategies of this ideological inter-medium by following such traversals.
How to do it? The ideal approach is to combine multi-site ethnography with critical discourse analysis of the texts and media encountered by subjects in the course of their life-traversals (this would of course need to include interviews to assess subjects’ interpretations of each text/product and also across sites and texts). This approach is however quite difficult in practice because of the need to coordinate two levels of data collection and two levels of analysis: one on the timescale of short encounters with media, and the other on the timescale of lived days and weeks, many orders of magnitude greater.
There is still another important pre-requisite for the conceptual analysis of meaning constructions along traversals, whether for the study of inter-media franchise-worlds or not. We need a better understanding of the phenomenological aspects of meaning construction in real time and across media- and artifact- rich spaces. Meaning is made in time, and along traversals it is also made dynamically across real and virtual attentional spaces. What meanings we make and what feelings we experience as we interact with semiotic artifacts depends not just on their affordances and meaning potentials, and not just on our own interpretative stances, but also on the dynamical qualities of the interaction itself, such as pacing and spacing. Whether we interact leisurely or urgently, whether an interactive response from the medium comes quickly or tardily relative to our expectations, whether we encounter media arrayed in real space as we move through that space, and just how we move from one attentional focus to another in time and through space … all these matter to the meanings we make and the feelings we experience, even for otherwise identical media and artifacts. This is a key aspect of meaning making that we need to more completely understand if we are to successfully analyze traversal-meanings in general, and cross-medium franchise-worlds and their distributed ideological effects and relations in particular.
I believe that we can learn a great deal about these processes, and the typical and newly emergent “chronotopes” (Bakhtin, 1981b) of present-day and new cultural orders, not just by difficult, direct study of people-in-action across multiple timescales, but also in a case that is much easier to study but should exhibit many of the same general features. This is the case of people making meaning across semiotic media/modalities, in real time, and through virtual spaces, in interaction with virtual semiotic artifacts in the new meta-medium of 3D computer gameworlds.
Clearly, not all the complexity of either real-life or of inter-media franchises (which include such games but also other media) is available in gameworlds, but on the other hand, activity in such worlds on all timescales can be readily recorded for analysis. Moreover, such gameworlds do often contain virtual replicas or instances of other media and multimedia genres: books, films, toys, music, etc. There are even games-within-games. In multiplayer game genres, players interact with avatars of other players as well as with the game program, and they may well do so outside the game (e.g. via CHAT or IM) as well but still on the same computer, which can be synchronously recorded. Many massively multiplayer (hundreds to thousands of simultaneous players) gameworlds are also persistent worlds: architectural, historical, and artifactual changes and effects there persist for players across different logins, adding additional dimensions of ordinary life traversals on still longer timescales.
For the single player in a gameworld, we can see the effects of meaning construction across media and across a range of timescales within a fully designed virtual semiotic environment. We can see events and meaning construals that will work with, against, and obliquely to the ideologies presented within the game, foregrounding these for analysis. We will need of course to also interview players to better understand their interpretations and interpretative stances. In multiplayer, and especially in massively multiplayer persistent worlds, we will see ideological conflicts arising from the heteroglossic diversity of player viewpoints and in-game and out-of-game social positioning and trajectories, as well as from social and cultural divergences between players and game designers. This research can then be extended outward into the lifeworld of the players, to their encounters with other instantiations of the franchise, and with all the complexity and practical difficulties of studying the fully general problem we have posed.
I hope that this sketch of phenomena, issues, and strategies for research goes some way toward formulating a research agenda for the extension of the project of critical discourse analysis to an important class of new media and potential new sites of ideological effects.