Action Stars and Smoke-veiled fights

I've posted every now and again about a convention in comics that I've called "action stars", where a whole panel is replaced by a star shaped "flash" that essentially represents "event happens here!" but doesn't show that event. I've likened this to being like a pronoun in the visual grammar, since it can replace the Peak events of the sequence, just like a pronoun can replace a noun (or noun phrase).

Over the past year I've run some successful experiments using action stars, and am planning a few more of them. But, I've also had the lingering question whether there are any more of these "visual pronouns" out there...

And I think I've found one.

Another common piece of visual morphology is the "smoke-veiled fight" (alternative names welcomed), where a big puff of smoke is shown with arms and legs sticking out, to stand for a fight occurring, which can also take up a whole panel:



Some interesting contrasts can be made between the smoke-veiled fight (SVF) and action stars. First off, SVF panels are much more restricted; they can only appear for fights, whereas action stars can go on almost any Peak panel. We might write this out this difference in meaning formally as:

Action Stars: [Event: X]
SVF: [Event: FIGHT(A,B,...n)]

This basically says that an action star carries the unspecified meaning that an Event "X" occurs, but SVF panels show an Event of "Fighting", consisting of at least characters A and B up to "n".

Also of interest is that while both depict "events", the nature of those events is intrinsically different. Action stars show a single event, while SVF panels show a duration. Notice, you can't glean the sense of duration from an action star, nor can you interpret the SVF as a single event. But, the difference is there — even though in neither one can you actually see what events are "actually" happening!

"Defining Comics" video

Patric Lewandowski offers this video lecture of his attempt to define comics, based on his earlier column from Comixtalk. He covers a lot of ground, meshing numerous memes of comics theory.

Ultimately, I do disagree with most of his points, for reasons I've described elsewhere**, but it's at least interesting to see him present it all together, and I do like that other people are at least trying to address these issues.

While I've defended people's attempts to define comics before, from the visual language perspective, the whole issue of "defining comics" does seems a little strange, and likely stems from McCloud's big thrust to do so in Understanding Comics.

McCloud's guiding rhetoric was a division of form and content. For him, the form was comics and the content was the genres that appear in comics. But, you can take this one further, since comics are only a "form" if you presume them to be.

Really, comics are made of two mediums: text and sequential images. These can be the "form" and the notion of "comics" is the content. Truly, as Patric tries to get at, text and/or images appear in lots of places, but only sometimes are they called "comics." This says to me that "comics" are not a thing definable by those elements at all as a type of "form."

Rather, comics are written in text and images the same way that novels or magazines are written in text. From this perspective, debates over "what are comics?" are rendered similarly to "what are novels?" or "what are magazines?"


-----
**Though I will add emphatically that my theory of visual language as he presents it there is fairly misleading. Visual language is NOT just about iconography!!

5 Card Nancy and Panel Transitions

One of Scott McCloud's more wacky inventions is the game Five Card Nancy which is based on the old comic strip Nancy. The basic premise of the game is that you can create lots of different (and fun) novel strips by combining random panels together. Scott recently posted an old collage he did that led to the game.

Of immediate note in his collage is that the sequence doesn't exactly make much sense, despite some cohesion between the panels. I'd say that it may have a narrative structure (i.e. visual grammar), but no meaning (semantics).

In some cases though, the juxtaposed panels do make sense, but the global meaning does not. In linguistics (borrowed from math), we'd call this a "first-order Markov chain", since only the units right next to each other have a connection. If a panel had a connection to two panels next to it, it'd become a "second-order chain", etc...

Markov chains were the primary way that people thought about language's grammar up until the 1950s, when Noam Chomsky then showed that grammar needed to account for connections farther than just countable individual word relationships (an approach I then applied to comics' sequences).

Essentially, McCloud's theory of panel relationships is a first-order Markov chain theory. It only looks at juxtaposed relationships. Interestingly, his Five Card Nancy game follows the same characteristic. Since players put down one panel at a time, it appears as though they are just making choices linearly. However, I'm guessing that the higher scoring combos are all ones that gel on a global scale, not just a local connection.

Also, the limitation of the panel transition viewpoint is really highlighted by McCloud's Nancy collage. How can panel transitions be correct if only local connections make sense but ones further down the sequence do not? Though we may draw and read comics one panel at time, it doesn't mean we don't build or project a bigger structure in our minds beyond the linear relations.

Storycards and visual grammar

My friend Alex sends along this link to a gift pack of "storycards". Basically, you can use these cards in sequences to create lots of different novel stories. The idea is similar to McCloud's Five Card Nancy game.

I'm interested in it for a few theoretical reasons. For example, having a stock set of units that can be combined in different ways is similar to language, where you have a set of words (vocabulary) that are combined in various ways (grammar). As the main thrust of linguistics in the last 50 years has told us, infinite possible sentences can be made with just a small set of vocabulary items, and that's basically the fun of such a card set!

However, even more interesting, its very similar to a study I finished running a few years ago and am still working on getting written up. In it, my participants were given four panels from a Peanuts strip and asked to arrange them in an order that makes sense.

People were very good at getting the original order of the strip (around 90% if I remember correctly), though that's not what I was interested in. I was more interested in the errors that people made, and whether there were patterns to them. And, indeed, there were. Prior to testing, we had coded the panels for numerous narrative properties, and found that certain narrative categories got moved around in particular patterned ways.

What this showed was that people don't just make up sequences one panel at a time as this game suggests, but that elements of that order are conditioned by roles of panels. These roles are determined both by properties of individual panels, and the relations between images.

So, one-by-one reading/drawing, but guided by underlying complexity (grammar) beyond just linear relationships.

Almost back...

Okay, one more status update then we're back to some real posts...

My brainwave study has nicely concluded and I've now moved on to analyzing the results. Things look fantastic, so I'm very excited about working to get these written up and submitted to a journal in the coming months. This whole project has been a very long one: 1.5 years making stimuli, 7 months running the experiments, probably another 3 or so writing it up. It'll be a relief once its done (and the next beginning), but the results seem to be worth it...

Otherwise, I've finally had some time to tinker with some decent blog posts. So, I'm just going to let them accrue until after the holiday weekend then come back and start posting again. It's been a busy busy few months, but I think some more blogging will be back in the mix shortly. Phew!

Not MIA!

Unfortunately blogging has had to take somewhat of a back seat lately, as my workload has been just crazy.

At the end of this week I should finish running participants through my brainwave study using comics, though I can already confirm that the results are just fantastic. I think I can confidently characterize how the brain processes narrative sequential images, and its really quite exciting (with pretty solid evidence against panel transitions). Stay tuned... (though I'll say already that this will be the topic of my ComicCon talk in summer).

Otherwise, I've been working on writing up several other studies that have been ruminating in my computer. What with the multiple articles currently under review for journals and books, over the next few years I should have a steady stream of new papers emerging.

Beyond that, the publisher De Gruyter has been nice enough to provide me with a review copy of the recent volume Multimodal Metaphor which has lots of chapters on comics. Once I have a little more time I'll be writing reviews of the book and relevant chapters.

Transition Overload!

I've frequently heard it said that every panel in a comic has to connect to every other panel. I've tried to go about showing the problems with individual transitions or McCloud's closure, but I have yet to tap into this issue on the blog.

Potentially, this could be at least somewhat the notion behind Groensteen's ideas of braiding and arthrology. "Restrained arthrology" says there are meaningful connections made between all juxtaposed panel relationships (i.e. what McCloud would call panel transitions), while "general arthrology" pushes this up to possible connections between all panels in a book ("braiding").

In my book, I toyed with a similar idea of multi-connected transitions for very specific examples, but cast it aside before proposing my alternative approach based on Chomsky's generative grammar. However, the "every panel with every other" viewpoints are far more unconstrained than my approach ever was.

One of the biggest problems with this "every panel with every other" as a theory of comprehension is that it would just overwhelm a person's working memory to keep that many things active in their head with no guiding structures. So, I figured it would be worth the exercise of showing how ridiculous such an assertion might be...

For an average book that has 6 panels per page for 24 pages, this would give 144 panels in a book. Connections between any two panels in those 144 would be calculable as 144!/(2!•142!). This would build up to 10,296 possible transitions as every possible combination would additively create with each successive panel read, as the mind continuously retained them all in memory. Granted, not all panel relationships might need to establish an explicit "transition", but all connections would be necessary to at least confirm or deny the need for an explicit transition.

Without any underlying structure to guide such connections, this would be overwhelming for human memory to handle. Rather, there needs to be something explicit provided by the mind to manage (and group/subdivide) such connections— just like a grammar for language. Transitions and general principles of "arthrology" just won't do it.

Learning to read your brain(waves)

So, today marks a minor milestone for me, as I ran my very first study of comics looking at people's brainwaves. The image here to the right is from that first participant, and each of the lines is of a different type of sequence that we are experimentally testing.

So, what does this tell us?

Absolutely nothing.

Yet.

Data from one participant doesn't say much, but give me a few more weeks and these waves will be (hopefully) showing interesting information about how the brain processes sequences of images.

Review: Adventures in Cartooning

Adventures in Cartooning is a fun and creative book by James Sturm and two of his graduates from the Center for Cartoon Studies, Andrew Arnold and Alexis Frederick-Frost, and published by First-Second.

It is designed as a how-to create comics book, though the lessons almost wholly come in narrative form as the Magic Cartooning Elf and other characters discuss the properties of comic creation while carrying out a simple and fun story. The book is aimed largely at children, and the humor reflects it (though did make me laugh aloud at parts — particularly the "Warning" on the back cover, which is just the sort of thing to get kids to pick it up).

Most of the overt instruction is fairly simple — things like what is a panel, how text can enrich images, orders of word bubbles, and the nature of different graphic devices like motion lines or dotted panel borders. The last several pages of the book also contain sections on cartooning basics that make explicit several of the lessons as well as some additional instruction.

However, because most of the instruction comes narratively, there ends up being only a limited amount of things instructed. This is a shame, because several techniques the authors use are very clever, elegant, and well worth instructing learners if they don't notice them.

For example, in one section the characters climb and then descend a mountain. On the climb, the three small square panels of the page are positioned climbing upwards left-to-right so that the line of the mountain-side is retained, while the opposite configuration occurs for the descent. On another page the characters sink down into water, with the length of the panel growing away from the top of the page in each panel to show falling deeper and deeper.

These are fun, simple, and effective techniques that comic creators can put to great use, though without the explicit instruction I fear they might be lost by less observant readers. Perhaps allowing for some additional non-narrative instruction would allow for even more of these aspects to be brought out explicitly. It would be easy to do this with the little labels and arrows used on the cover (which appear nowhere else in the book), or with the Elf character hovering outside the panels to point things out as well.

(On the flipside, I can understand why the authors might feel they just want to give kids the basics and not overwhelm them with too many concepts, though I'm inclined to think kids can handle it).

Additionally, I greatly liked how much of the book was carried out without text and the implications that text is only used to enrich visuals. This subtly reinforces the development of the visual language grammar in learning — which is no doubt the intent.

Overall, the book is a good read and would likely be a useful tool for helping young creators get on their way to creating graphic stories.

Review: The Power of Comics

Duncan, Randy & Matthew J. Smith. 2009. The Power of Comics. New York: Continuum Books

The Power of Comics is a recently released “first textbook ever” for “comics studies”, authored by Randy Duncan and Matthew J. Smith (book website here).

Perhaps to be expected from a book on the general “comic studies”, it includes a broad range of topics, from the history of comic books to comprehension of the medium, to creators and fandom. In many ways, the vastly disparate topics brings into question the overall utility of conceiving of a singular field called “comic studies.” Indeed, a person interested in one particular subtopic may find the entire rest of the book irrelevant. No doubt though, the scope of the book is meant to be inclusive and to at least cast the possibility that such a broad field could exist.

As a textbook, it succeeds in content, scope, and execution. The chapters are well laid out, have thoughtful questions at their ends, and several chapters end with very practical examples of analysis that serve as models for students. Chapters also reference a broad range of experts (discussed below), which further validates breadth and depth of this growing scholarship.

Given the nature of my own interests, I’ll focus primarily on the chapters dissecting the medium, Chapters 6 and 7. These chapters on formal analysis largely expand and refine Duncan’s earlier papers on “Comic Book Communication.” The theories stick largely to the expected status quo of theory: nothing overly radical or surprising jumps out, which is perhaps to be expected from a textbook.

In some ways though, this is a detriment. Despite growing works on formal aspects of the “comic medium”, most of these chapters rely on concepts inspired by Eisner and McCloud, supplemented by Groensteen and scattered others. However, much of these ideas go by with little regard for debates to their legitimacy. For example, “closure” is assumed to be true and never questioned as being valid at all (though multiple interpretations are presented). Others include the (erroneous) belief that the “gutter” somehow contributes to meaning, and the idea that each panel must be connected with every other panel in comprehension (can you say “working memory overload”?).

The primary focus of these chapters describe aspects of meaning-making, providing summaries of overview notions that intertwine across numerous levels of comprehension (sequence, layout, etc). The chapters are chock full of information, much of it useful to a beginner and some likely useful to more advanced students. I particularly liked the idea of an interplay between the reduction and expansion of information in the medium as a nice simple way to describe the status quo of considerations about the medium.

However, on the whole, numerous broad theoretical concepts are discussed without much real theoretical grist to them. Again, this might not be bad since the format is a textbook — elaboration on numerous topics would be impossible for the space.

Nevertheless, some aspects describing theories are a bit roughshot — for example describing “icon/index/symbol signs” could have benefited from better explanations, and proper terminology (it should either drop “signs” or be “iconic/indexical/symbolic signs”) and attribution to their originator (Charles Sanders Peirce) would at make for a good mention that these theories are roughly 100 years old.

Additionally, while McCloud and, at least somewhat, Eisner, are recognized for their theoretical insights and contribution to the canon, the realms of theory and praxis get blurred further in the chapters with quotes from numerous comic creators and an unheard of “comics art collector,” which among the few experts seems curious as legitimate sources. It also brings into question just what and who these chapters are aiming at: Theory? Analysis? Praxis?

Despite its limitations, The Power of Comics marks an accurate state of the field (whatever it might be) for studying comics. For good or bad, the theories in Chapters 6 and 7 reflect a particular paradigm of thinking about the medium. While it is my personal belief that formal theories have moved into a more sophisticated state, the views expressed in this book reflect what will someday be viewed as a nascent growth stage of considering the medium. For that, it almost seems like a “living history”, saying where we’ve been while knowing bigger things have and will appear.

Overall though, the book — including the theory chapters — is reasonably good for a “first textbook on comics,” and I would imagine it will fast become a standard text for those sorts of classes.

Panels connected by sequentiality

Derik has a short post that makes a nice note about how understanding of individual panels is sometimes conditioned by their context in a sequence.

I think this is a very important point that is well illustrated by his example. Sometimes, understanding of the elements in an individual panel relies on the information in other panels.

Most all cultures and individuals have little trouble decoding most propositional information in images (i.e. that an image of a horse means "a horse", or that an image of person is "a person", etc). However, certain individuals may have trouble comprehending the objects if their meaning is conditioned by a sequence. For example, this sort of meaning by context is often what children under four and other "non-visual language fluent" readers (or those fluent in a different type of system) struggle with.

Why is this important/interesting?

1) It lends validity to the idea that there is a fluency required for sequential image comprehension (and thus that there is a "system" guiding understanding to be fluent in).

2) It implies that even perceptual understanding (i.e. vision and object recognition) might rely on sequential understanding in these contexts, meaning that mere perception alone isn't enough to explain sequential image comprehension (i.e. again, a system for sequential images is necessary).

3) It hints that these sequential images were created to be in sequence and not just as random images strewn together. This is also a support against an image-to-image system of understanding like panel transitions, since transitions could function no matter what is thrown next to each other. This sort of execution has a more global scope: it's a whole sequence made to be a whole sequence, not just one after another.

Review: Metaphor and Metonymy in Comics Storytelling

Kukkonen, Karin. 2008. Beyond Language: Metaphor and Metonymy in Comics Storytelling. English Language Notes 46 (2):89-98.

This paper from the literature point of view explores meaning-making in comics, particularly from metonymy and metaphor. It argues that the "semiotic" approaches of European comics scholarship that dissect parts into structrualist "minimal units" are insufficient to capture the complexity of comics' meanings, and is thereby a tacit argument against viewing "comics as a language" in the semiotic sense.

(Groensteen takes this same perspective against minimal units, though maintains the "comics as a language equation. I actually think that "minimal units" are *kind of* there, but it's beside the point, since linguistics hasn't really been concerned about "minimal units" since around the 1950s...)

While she does explain and support the cognitive linguistics view of metaphor taken from Lakoff and Turner, she does not actually use it in exposition. Most of the examples of metaphor and metonymy she cites are through a close reading of Watchmen, involving large scale metaphors on the scale of plots, themes, and motifs, and doesn't ever cite the correspondences of one "coneptual domain to another" that conceptual metaphor entails.

Her view of metonymy is equally broad. For part-whole metonymy she cites scenes where a whole understanding of an environment is given across multiple panels. This would imply that all instances where multiple characters are shown in their own panels but part of the same broader environment (what I call "Environmental-Conjunction") are metonymic, because they construct a broader whole by only seeing the parts. This is a curious proposition that I (mostly) like, though one that seems at least partially limited by not having a robust view on the broader narrative grammar for how sequential images are comprehended.