“What are you working on this summer?”
Usually, at this point, the person I’m talking to puts a huge grin on their face and exclaims how awesomethat sounds, and I get to bask in the satisfaction of having found a really catchy way of summing up my internship project at FUSE Labs.
Then, they think for a moment and ask the inevitable question: “So what does that mean?” That would be the point where I’d admit that I actually had no idea but doesn’t it sound cool?
However, a couple weeks into the internship and several exploratory prototypes later, I’d like to share some of the thoughts we have on how data and comics can work together to help people make sense of what is going on around them.
Comics are often used to tell stories about how events were perceived.
Maus by Art Spiegelman; Stop Paying Attention by Lucy Knisley; Johnny Wander by Yuko Ota and Ananth Panagariya; Footnotes in Gaza by Joe Sacco.
The problem with social media data: there’s a lot of it
Developing mechanisms to deal with the huge amount of content we encounter every day is an ongoing problem. When Facebook revamped its News Feed feature in 2011, they focused on helping users see only the best and most important content to them. An ecosystem of third-party Twitter apps help reduce noise and sort through only the most important posts. But even with all of this, it’s hard to put together the bits and pieces of information we see into a coherent picture of what’s really going around us. Current visualization techniques can provide us with graphs or timelines that reveal the trends, patterns, and connections within this data, but we lose a sense of the emotional ups and downs of an event or the details of what people notice or find interesting. What would it look like if we could visualize interpretations of what is going on?
In contrast, comics journalism and other non-fiction comics are examples of media that combines data collected from multiple channels – interviews, the author’s own observations, photographs, prior research – to capture real-life experiences. One famous example is Maus, a graphic novel by Art Spiegelman that depicts both his father’s experiences as a Holocaust survivor and his present-day relationship with his family. Other examples are webcomic diaries like Johnny Wander, Stop Paying Attention, and Today Nothing Happened, which depict thoughts and events surrounding the artist’s life. FUSE Labs has also been exploring this concept of “humanizing data”; HereHere grabs data from 311 calls in NYC represents them as cartoons.
Not only do these types of comics visually express a sense of “being there”, but they also expose the point of view of the journalist or storyteller (often by including the author within the story). And unlike charts and graphs, comics tend to focus more on expressing a particular viewpoint as opposed to enabling exploration of data. However, creating these stories require special skills and for the author to be present at the moment being depicted (or extensive research).
Joe Sacco appears in his graphic novel, Footnotes in Gaza.
So on one hand, we have tons of data without a way to discover the underlying story. On the other hand, we have a medium good at depicting narratives around real-life experiences. Is there a way we can combine these two things to create data-driven visual stories?
Finding stories in data
Creating stories is hard; there’s no magic formula for it (well, not for non-Hollywood stories, anyway). Thankfully, there are a couple things in our toolbox we can make use of:
- Communication theories. There’s a surprising amount of theory that has been developed in the past 10 years regarding how people interpret and understand comics and the techniques comics use to communicate stories. We can look to these theories for clues on how to structure data to fit a visual narrative. Such theories include work by Scott McCloud and Neil Cohn on the visual language of comics and the design choices that affect how readers interpret action and meaning.
- Crowdsourcing. Not everyone can write an award-winning screenplay. However, people might be better at finding stories than creating them. With a little help from the communication theories mentioned above (and some past work), we might be able to structure crowd work around the task of surfacing interesting narratives in data.
- Social media is multimedia. Social media are often not just text, or just photos, or just links – tweets, Instagram posts, and Facebook statuses are often a combination of many things. Comics similarly make use of both text and images (with online comics even making use of animation, hyperlinks, and interaction) and may be a good way of making use of all of that content.
Currently, the rough plan for this project is to create a story creation tool that combines the scalability and human perspective of crowdsourcing and the visual language of comics to convey events based on perspectives present in social media data (which we’ll pull from Seen.co – thanks guys!) to generate summary comics based on a hashtag. We’re also planning to investigate the strengths and limitations of creating stories in this way. Will people gain insights they otherwise wouldn’t? Are data-driven comics effective at facilitating the sharing of knowledge or personal viewpoints?
I’m looking forward to what we might find through this project: the role of crowds in creativity, a method for structuring data with respect to narrative, and the value of comics as a data visualization technique. Our online world is evidence that we’ve figured out how to create social media. Maybe we can start figuring out how to support telling stories.