The Latent Grammar of Comics: What Happens Between Panels
What happens in the blank space between two comic panels? I used a vision model to compare thousands of panels and look for the visual patterns that make a story feel fast, quiet, or surprising.
Looking for patterns between panels
I’m interested in whether a vision model can pick up a kind of latent grammar in comics: patterns in how one panel leads to the next. Can the distances between image embeddings tell us anything about pacing, continuity, or a sudden change of scene?
A useful way to make that question concrete comes from Scott McCloud’s 1993 book Understanding Comics. His example shows how much of a sequence a reader supplies between the images:
In Panel 1, an axe is raised above a cowering victim screaming: "No! No!"
In Panel 2, a silent nocturnal city skyline echoes: "EEYAAAH!"
Neither panel shows the blow. The reader connects the threat and the scream to infer what happened in between. McCloud calls this closure: our habit of turning separate pictures into one continuous event. The space between the panels is called the gutter.
That gives a useful vocabulary for this experiment, though measuring visual change is a much narrower task than explaining how readers understand a story. Here, I focus on three questions:
Can a vision model tell the difference between a small beat and a major scene change?
Do American comics and Japanese manga create rhythm in visibly different ways?
Is a page turn really used to hide bigger reveals?
How the comparison works
DINOv2 turns each panel into a compact description of its visual appearance. It does not read the dialogue or know the plot. It mainly notices shapes, composition, subjects, lighting, and texture. I compare those descriptions for each pair of neighboring panels.
Golden Age American comics and Japanese manga
Each pair asks: how much changed between these images?
How much changed?
Two similar panels create a small step. A new place, subject, or composition creates a large step. The charts call this visual distance “velocity.”
Did the sequence turn?
A chase may keep moving in one visual direction. A reaction shot or scene change bends the sequence. The charts call that change of direction “curvature.”
What stayed in place?
The model also checks whether parts of the background remain recognizable. That helps distinguish movement within one scene from a jump somewhere new.
The subject barely changes. The reader fills in a moment.
The same subject moves through one scene.
The visual world changes and the reader makes a larger leap.
Two different page rhythms
The comparison uses 15 Golden Age American comic issues and five Japanese manga series. They are not stand-ins for every comic ever made; they are two contrasting samples with visibly different habits.
Regular grids often carry one action into the next: punch, recoil, chase, reply.
More varied layouts pause on a place, an object, or a quiet reaction before moving on.
What the model noticed
Finding 1
Small beats and big jumps look different
McCloud described several ways one panel can lead to another: a tiny change in time, the next step in an action, a switch to another subject, or a leap to a new scene.
The model was not given those labels, yet the transitions still formed recognizable groups. Near-identical moments cluster tightly; major scene changes sit farther away. That is the central result: a generic vision model can recover part of the visual grammar that readers experience intuitively.
Finding 2
The manga sample pauses more often
McCloud argued that manga more often moves between different aspects of one place: a face, a doorway, rain on the street, a hand holding a cup. These panels may not advance the plot, but they build mood.
That pattern appears clearly here. The manga sample spends about twice as much of its sequence on these atmospheric transitions, while the American sample more often moves directly from one action to the next.
Finding 3
You can see pacing as a shape
Plot the visual change from panel to panel and the contrast becomes easier to see.
The American pages tend to hold a steadier pulse. The manga pages move between longer quiet stretches and sudden jumps. Neither rhythm is better; they create different reading experiences.
Finding 4
Page turns really do hide bigger changes
Comic lore holds that the physical turn of a page serves as a suspense curtain drop. Does DINOv2 detect this?
The data supports the intuition. On average, the visual jump is larger when the next panel is hidden on the following page. Cartoonists use the page edge as a curtain: the reader has to pull it aside.
What this does—and does not—show
The model does not understand a character’s motives, recognize a punchline, or experience suspense. It measures visual change. That limited signal is still useful because cartoonists use composition, continuity, and page layout to control how a sequence feels.
The interesting part is the overlap between a simple visual measurement and a reader’s vocabulary: quiet beat, continuing action, reaction shot, scene change, reveal. The numbers do not replace close reading; they provide another way to see the craft already on the page.
References
- comic-latent-grammar: GitHub repository with pipeline code, DINOv2 extraction, UMAP models, and speedometer CLI.
- Scott McCloud, Understanding Comics: The Invisible Art (1993), William Morrow / HarperCollins.
- Oquab et al., DINOv2: Learning Robust Visual Features without Supervision, arXiv:2304.07193.
- Vivoli et al., CoMix Dataset: public domain Golden Age comic page scans and panel annotations.
- Mantra Inc., OpenMantra Dataset: multilingual manga dataset with authentic reading order panel annotations.