Abstract

‘Snowfalling’ is one way to go. Done right it results in an engaging and immersive narrative that transfixes readers. It marries the best of print aesthetics to digital media. But, what if you want to do something more digital? Something interactive? What if you don’t want to do a linear piece all glossed up in fancy magazine-inspired decoration and layout? What if you want to go web-native? Find out what is possible; what narrative and design tactics the web lends itself to; what interactive forms, structures, and tactics aid the exploration of complex stories.

Download Slides (KEY)

Transcription

For those who don't know me, I'm Baldur Bjarnason. I'm a long time sort of opinionater online on anything regarding e-books. I'm working currently for Unbound as part of the digital production team, and like many of you, my first experience with creating interactive media was hypercart. [video cuts ou]

[Video resumes] so many of us focus our creative energies and how we create these works and that is by focusing on the experience that we are creating for the audience. As in, when it comes to interactive media, interactivity is what you do. It's not what you watch, not what you hear, not what you read. The text, in a piece of instructional media, that's not the work. To be more specific, interactive media is composed of the meaningful actions that the user takes while interacting with the work. The stuff that you see on the screen, that's a product, which is an important thing, but it doesn't actually become anything until it is experienced by the audience. In terms of interactive media, that experience is their actions. I'm completely agnostic about the idea of whether interactive media is a hypercart stock or an e-book or an app or a website. You can think of this like duck typing. If it quacks like interactive media, looks like interactive media and flies like interactive media, it probably is interactive media. Format be damned.

You're probably familiar with this idea because this is basically what Marshall McLuhan meant. The experience of the work is what is the work itself. To be more specific, he actually knicked the idea from John Dewey who was writing in 1931. The basic idea is the audience has no way of reading our minds and knowing what our message was or our subject matter was, so their entire experience of what we want to convey to them is mediated through the structure and the form of the interactive media. We have to think about the work as the experience that the user has when they're engaging with the work and not as the piece of metal or the pixels that you see on the screen.

Breaking it down a little bit further into the bits, the first obvious thing that you see in interactive media are the objects and widgets on the screen, the bits that you interact with. The important thing to think about there is that it's object all the way down. Almost every interactive thing that you interact with is composed of other bits that that you interact with as well. A dialog box is made out of a series of buttons. The button is made out of a series of states that each responds to your actions. It's like a loosely connected structure because it's not tightly connected like a novel is where everything happens one after another. You can't necessarily predict exactly how the relationships between all the interactive bits would play out.

The second thing to think about is context. The best example of why this is important came from a game designer called, and I hope I'm pronouncing the name correctly, Ralph Coster. His example was about two different games. One is Tetris, the one that we all know and love about stacking bricks and making them disappear, but he posited an idea where you could have another game that has exactly the same game play, exactly the same design, the same interaction, the same everything except by virtue of the desired context around it, you're playing a Nazi trying to stack bodies or corpses in a pit and trying to maximize the amount of dead people you can get into a pit. That changes the game that is otherwise identical into something horrifying, just by virtue of context. You haven't changed any of the actions. You haven't changed anything that anybody does. Just by virtue of the context, you've completely colored the entire experience.

The sort of third major bit is obviously any piece of interactivity requires direct feedback. Otherwise, you won't know you've clicked a button. Otherwise, you won't know you've paged through something or interacted with something, but notifications or prompts, basically feedback, are something you can disconnect from the direct action and use it to space out the structure in time or space depending if you want to use GPS or smartphones.

The final major bit is play. This definition, play being free movement within a more rigid structure, plays with the other meaning of play being a little bit of flex. It comes from the Rules of Play by Katie Salin and Eric Zimmerman. The basic idea there is that a non-playful slider would be a slider where you can slam it to each end and get the minimum and maximum values. A playful slider would let you go above or beyond and give you direct feedback about how far off or by how much you missed. If you wanted to make it a game, then you’d start letting the user collect points based on how often they actually managed to hit. So, the point about play is that every single bit of interactive media that you create can either be playful or not. It doesn't have to be a game. You can turn any piece of interaction into a playful experience just by adding a little bit of flex around it and giving people feedback depending on how closely or not closely they hit it.

There are pitfalls, major pitfalls. These are pitfalls that a lot of people in publishing run into. This is kind of the biggest one: you can't make something that isn't interactive, interactive. You can't take a novel and glob interactive bits onto it because the novel is a completely linear structure. It doesn't have the structure that allows for interaction to be integrated within the experience. It'll be two separate experiences joined at the hip.

A piece of interactive media is something that’s composed of actions, of meaningful actions that have an effect on the experience. It's not a decorative piece of other kind of media.

This is the thing that annoys me greatly. Perspective didn't appear in painting because a mathematician invented the idea. “I've got this mechanism and concept of creating perspective in painting, but I've never painted before. I've got this invention that I am going to go around and sell to painters. This a method that you can now use to create perspective in painting.” Perspective came from a demand for experience, for new experience. It was painters that wanted to represent their ideas, their needs for new experiences and new art. So, it was painters that learned mathematics, and it was painters that experimented with mirrors and trying to build new techniques. The point is techniques do not appear in isolation from the creative process, and this is why iBooks Author, for example, is an atrocious idea for creating interactive works. It's literally paint by numbers for interactivity. You are asking your designer to fill in boxes created by a programmer probably somewhere here in San Francisco, and you are abandoning a large part of the authorship of your work. That's why the programmer is your coauthor. He needs to be treated as such because they have such a control over the experience of the interactive media work. They affect the work more than a cover designer, more than the illustrator, more than the typesetter, more than the designer. Leaving them out, outsourcing it, or even worse, standardizing it in the form of apps that are used to author prebuilt blocks of interactivity means that you are abandoning a large part of the authorship of the work and your control. It means that you're going to end up with cookie cutter media artifacts.

Another big pitfall is the tragedy of design fads. The issue of skeuomorphism versus flat is that it’s about ornamentation. It's not about functionality. The biggest tragedy about iBooks sort of copying the book metaphor in it's design isn't that it copied the book metaphor, it's that is was non-functional. [video cuts out]

[Video resumes] A big problem with the flat trend is that is assumes expertise. It assumes that you know that the thing that's in the upper left hand corner of the screen is something you can click on or press, when nobody's born, nobody pops out of the womb, with an ingrained knowledge of how interactive media or user interfaces work. If everything goes to flat design, how are the new users going to learn the basics of interactive media. It's a pox on both their houses.

The final pitfall is from the perspective of the user, all of their experiences are linear. We are linear beings. We experience things in time, but it is not necessarily something that has an end. It's like when we experience painting. There's a point when we start watching or inspecting the painting, and there's a point where we stop. It's not really an end. It's a suspension because we know we can always revisit it, and every time we revisit it, it will be a different experience.

The same thing applies to interactive media. It's not, strictly speaking, a linear artifact like a novel or a book would be, but it has a linear experience to it. It's the end bit that gets people the most, realizing that people can actually leave the work and revisit or even not.

Now I think we should move on to the more concrete examples instead of these abstract theories that everybody is nodding off to. This is one of the more interesting works I've seen lately made by friends of mine, Tom Abba and Duncan Speakman. It's a part of a UK research project called React that's supposed to promote collaboration between artists, academics and industry. They had a few helpers on it called Nick Hathaway and Neil Gaiman, I don't know if you know them. The idea there is that the work was split in two. One of them was a gloriously designed printed book that was an incomplete experience. There was no way to make sense of the printed book without the interactive bits, and the interactive bits were spread out over the center of Bristol in wi-fi hot spots. So, to experience the whole work, you have to wander around the center of Bristol with smartphone in one hand, a book in the other hand, and you had to plug the two things together to make sense. The text had cues for what you had to do with the book. The story was about an ultimate Bristol that surfaced every few years that was different from ours, and the text was messages from the other side. The idea was that we were trying to explore and discover what this other Bristol was from the perspective of ours.

An important part of what they did is that everything was a little bit playful, especially the text responded to your touch. As soon as you tried to read the text, you'd realize that this isn’t behaving like normal text. Some of it moves, some of it disappears once you react it, so you know it actually can be acted on. It's a bit academic. It had a few issues and was a bit difficult to use, but it actually made money because people like buying things that are souvenirs. The idea creating an interactive experience that was a mix of print and digital was something the people were more willing to pay for and having a souvenir for it, like there are for many apps.

Also, I think that the exclusivity appealed to a lot of people because it was like a fixed time event. They're planning to do it again in other cities, and each city will be different because it needs to be adjusted and rewritten to fit each location. Even though this is not purely digital, this is purely interactive media, so that is part of the reason why, at the start, I said I'm a bit agnostic about what [audio cuts out]

[Audio resumes]...pretty much everything I was talking about earlier: spacing things out in time, and it's not just a single object. It's composed of dollar boxes, app-like things, all down to audio media. You can see that you have access to his dropbox as well, so you can access the secret government files where he's plotting to cover things up. Overall, it's one of my favorites.

Here's the point of it. This entire thing is hard. The makers of the Malcolm Tucker app closed just about a month ago after a lengthy analysis and thought where he talked about, where Dave Addey, who ran —-, basically outlined that this isn't sustainable. Creating apps that are this involved and complex and selling them like titles, like we do with e-books or print books, just doesn't work. People aren't willing to pay enough money for it to work. When you lower the price, they don't buy it in enough volume for it to work.

Beyond just the financial logistics of it, it's also incredibly hard just in terms of creative touch because it involves a lot of control for both the creator and audience. The novelist has perfect control over the pacing of one sentence or chapter to the other, in terms of a linear structure, and the reader has perfect control over how quickly they read through that fixed text, but in interactive media the author loses control over how spaced out and how interrelated the objects are because they are composed of non-fixed and fluid things. The reader, as well, loses control over the timing because there can be prompts, there can be notifications, there can be things spread out over time, so that was basically the end.

This is the final slide by the way.

I think that's good.