Incorporating narrative into e-learning
by Cynthia Kurtz
Framing the project
What was the impetus for your project? What led to it taking place? Why were you doing it?
My second year at IBM Research was funded by a grant from the e-Business Technology Institute. (It was part of IBM at that time but later split off.) Our proposal to the eBTI was called "Improving distributed learning with storytelling techniques." We wrote the proposal because our two groups, one working on organizational narrative and one working on e-learning, wanted to work together on a topic of mutual interest. My group had developed ideas and tools (story techniques) and the e-learning group had identified a need (e-learning gaps), and we wanted to see how they could fit together.
What were the project's goals?
A critical difference between person-to-person learning (classes and help desk support) and mediated learning (help resources) is in the context added by the sharing of experiences, values and insights. Help systems provide information, but people transfer knowledge. We wanted to find ways to help resource authors improve the knowledge transfer in their products by incorporating narrative.
To give an example of such contextual knowledge transfer, we talked about how users often come to help resources looking for a solution to a problem, but because they don't understand their problem they look in the wrong place for the answer. In a class or during a help-desk call, a knowledgeable person can recognize the real need and redirect the user to the correct information; but in a static resource such redirection is difficult. We thought that stories might create a sort of connective tissue that would help people find solutions to problems they didn't understand. The superior memorability and motivational capacities of stories were also things we thought would enhance factual resources.
What did you think would happen during the project before it started? What were your expectations?
We thought we would find ways to help e-learning authors write stories and incorporate them into the resources they were building. We expected to teach people about story structure, memorability, and all the recommended topics in writing a "good" story. We even talked about building a story authoring tool for instructional designers.
The story of the project
How did the project get started? What happened first?
The first thing I did (after some background reading on e-learning) was to hold a "prototyping" phase. In this work I basically tried out different ways of building instructional resources with incorporated stories. As a simple and available test, I tried to improve help resources for the e-mail software everyone at IBM used every day.
The resources I built were complete failures, but the experiment was a success. The problem I discovered was that even with extensive knowledge of the facts surrounding the software, I couldn't come up with stories about its use that would be helpful to anyone. My prototype notes said things like "you can't make this stuff up out of thin air" and "everything I am writing about has happened to me" and "I can write stories, but I can't write useful stories." For example, I tried to make up characters who would have varying experiences with the software (executive, artist, scientist, accountant, things like that), but the only character whose stories didn't sound ridiculous was the one that matched my own personality and background. I couldn't enter into the experiences of other people to write about them. I am not a fiction writer, you might say, but that's just the point, and in retrospect it's a good thing I wasn't. The goal of the project was to help any instructional author add stories to their resources. If we needed them to become Jane Austen the project was going to fail.
At the same time as I was building what I called a "repository of pitiful attempts" at narrative-enhanced help resources, we were collecting stories. But we didn't understand what we could do with the stories at first. Our goal was to use them simply as inputs to task analysis (to find out what people do when they use the software) and needs analysis (to find out what people need to know and don't know).
I don't exactly remember when or how this happened, but one day it suddenly hit me that the something I was missing in the prototype writing attempts was exactly the something we were getting out of the story collection sessions. In other words, I realized that if we shifted our focus from helping instructional authors write fictional stories to helping them collect and organize real stories, we would be much more likely to meet our stated goal. In essence, the thing were trying to create was all around us; we just hadn't seen it.
We also realized that not only the form of individual stories we were collecting, but also the patterns in the stories -- characters represented, topics covered, connections to factual information -- was better than anything we could come up with. What we needed to do was develop ways to help people collect stories, let them organize themselves, and get out of the way as much as possible. After that our whole emphasis shifted.
What sorts of stories did you collect? How were they collected? Who collected them?
We ended up holding about fifteen story collecting sessions on four topics (two on software and two on techniques). Myself and a colleague in the e-learning group ran all the workshops ourselves. Because we were developing methods, we had long discussion sessions after each workshop and changed our techniques each time, keeping what worked and discarding failed experiments (and there were many). The twice-told stories exercise was the result of these experiments. Unlike the other methods I describe in this book, the twice-told stories method was originally developed specifically to help people with no experience collecting stories get started (which is why it's a good first method to try).
We recorded all the stories on audiotape, and I transcribed all of them. This took forever, but it gave me irreplaceable experience. By the way, if you are getting started working with stories, think twice before you pay somebody else to transcribe tapes for you. Listening to dozens of people tell hundreds of stories is a great way to develop your instincts about such nuances as what is a story and what isn't, how people start and stop telling stories, how people feel about the story they are telling, how others react, how people respond to instructions, and so on.
What sorts of annotations or question-asking were done? Who answered the questions or added the annotations?
We didn't ask any questions of the storytellers themselves. I answered questions (about things like values, surprises, performance, reaction, truth, rumor, source, and so on) about the stories as I listened to the audiotapes. (So, yes, I broke my own rule about interpreting stories from outside the group of interest. The projects that led to that rule came later and, as they say, are other stories.)
How were the stories looked at or considered? Who was involved in this?
I used the methods of grounded theory to do two things with the workshop transcripts: extract useful stories, and build conclusions about instructional needs for each topic. I wrote a report on each topic, and these were given to people involved in documentation and design for each software package or process.
In addition, the last set of stories and conclusions were used to build a new instructional resource about a common work process. Stories collected about the process were incorporated verbatim into the resource (I think it said something like "Tips from real users") and linked with factual information. The resource was deemed a great success both in popularity and in utility.
Did you do any group exercises? If so, what were they and how did they go?
We did lots of group exercises. We were experimenting. Many of them turned out badly, but some were great. Most of the ones that worked well (histories, metaphors, twice-told stories) ended up influencing my later work.
How did the project end? Were conclusions drawn, and by whom?
We produced a how-to manual that any instructional author could use to collect and incorporate stories into a help resource, as well as the reports mentioned above. The research project produced many insights about collecting and working with stories which informed all of my later work in the area.
High and low points
Do you remember any pleasant surprises during the project?
I think the main pleasant surprise was when our workshop methods finally started to work. After many failures we finally reached the point where we could hold a workshop with a variety of people that reliably produced a good crop of excellent stories, and it was a great relief.
How about unpleasant surprises?
We made a lot of mistakes as we experimented with different ways of asking people to tell stories, and people let us know it. People said we were wasting their time, that they couldn't understand what we wanted, that our approach was all wrong, and so on ad nauseum. Several times people angrily stalked out of the room or said things that made me want to shrivel up into a little ball. As we kept refining our methods this happened less often, but at some point I realized that in every group there will always be at least one person who is either having a bad day or just thinks what you are doing is stupid. You have to develop a thick skin about it, so eventually I did. By the time we gave our last workshops of the year, I was weathered.
Do you recall any "aha" moments when you realized or learned something critical?
The main aha moment was the "we are swimming in stories already" moment mentioned above. In fact I'd say that was the most important aha moment of my entire career in narrative work.
In retrospect the revelation reminds me of Arthur Plotnik's great book The Elements of Authorship, which makes the point that you can only become a writer after you get over your ridiculous ideas of "Becoming a Writer" and get to the practical task of finding something to write about, and then writing. Similarly, many people who discover the world of narrative put "story" on a pedestal and think they have to "measure up" to what Hollywood has said a story has to be. But in a way a "Hollywood story" is a distortion. It's a hot-house story, raised under special conditions in order to produce exotic and amazing but unnatural blooms. Real stories are tough, and they grow in sand and mud and rock and wind and storms, and sometimes they have nasty thorns. It takes a naturalist to find and work with them, not a hot-house gardener. Of course there is nothing wrong with hot houses or amazing plants (or stories), but they only get in the way when you are trying to help people exchange knowledge. I think if I hadn't found this out through beating my head against a wall for months I would not have understood it as well as I do (though many later lessons reinforced this first one).
Were there any times during the project when things seemed too difficult or challenging to go on? What was the challenge and what did you do about it?
One thing that was really difficult in the project was that I absolutely hated standing up in front of people and asking them to do things. I found it really hard to "do the talking." I did it, but after several workshops it was starting to make me physically sick. Things went much better when my colleague took over that task and I switched to hovering in the background, taking notes, and making sure the tape recorders were turned on. What I learned from that experience led me to always caution people to recognize their limitations in story work, and also to work with others whenever possible and find complementary abilities.
Evaluation
What turned out the same as you expected? What was worse than expected? What was better?
The whole project turned out different than expected; but that was a good thing. It was much harder to help people write fictional stories than I expected; but that turned out not to matter anyway. In a way the project succeeded in spite of my ignorance, which says a lot about the power of narrative, doesn't it?
Did the project meet its goals? Were there other benefits you hadn't expected?
The project met its goals to develop methods for incorporating stories into instructional resources. The main unexpected benefit was that we found a way to incorporate stories that not only required no narrative understanding on the part of the resource builder, but also created a whole new way of incorporating the knowledge of a community of learners into their own support. By collecting stories, instructional authors can extract knowledge from the community, process it, and return it to the community. It's sort of a bootstrapping, or ratcheting up, of the community's knowledge about a system or process or tool. This was an entirely unexpected benefit of the work.
As far as proving that stories could provide context to factual information, the informational resource we built using stories did indeed seem to help people find the information they needed. One benefit seemed to be that people found the "real user" stories so interesting and motivating that they explored information they might have passed by when it was merely reference material. The stories bridged the factual information (and provided the connective tissue we hoped they would) by giving people reasons to explore things they hadn't explored before. Some people told me that they got into the resource, found the stories, and simply read them one after another. Since the stories were heavily linked into factual information, people could come back to them again when they needed to recall something and find it again.
Can you share one conclusion of your project that you don't think you could have arrived at in any other way than by asking for and looking at stories?
I don't remember a lot of details about the project, but I do know that we found out things about software and processes that it would have been nearly impossible to find out any other way. One example I remember was that people often told us stories in which belief and rumor entered into their use of software. When a piece of software seemed impenetrably confusing, they would do superstitious rituals like clicking certain buttons every time they did a particular task, even though they knew the buttons were unrelated to the task. Or they would do things because they heard a rumor that something had happened to someone else and they wanted to avoid it. I do superstitious things just like that, and usually those behaviors have to do with particular stories from the past. For example, I compulsively save my work about every ten seconds, and any software that doesn't have a Control-S save (or some other shortcut) is nearly impossible for me to use. That's because of a few horrible experiences of wonderful insights lost forever. I remember hearing quite a few stories of that sort -- about how people layer their beliefs, perceptions, values, and cultures on top of software and processes. That sort of thing can provide insights to designers of software, and it can also help people design information resources so that they meet people in the space between the software and their needs.
Advice
What do you wish you had known before your project that you know now? What advice would you give to a person who wants to do a similar project?
The project I've described here took place nearly a decade ago and I've done many other projects since then, but I'll describe a few things I can recall learning the hard way on that project. I'm describing these mistakes in detail (as I remember them) because I want to make a point that mistakes are gifts in doing story work. Some of the basics of collecting stories from real people are hard to communicate and have to be experienced to be understood. You need to fail, at least a little bit, to develop skill in collecting stories. In fact I suggest building some low-cost failures into your first projects.
The mistake: One of our most disastrous story workshops was with a group of secretaries. They all knew the same things about the software we asked them to talk about, so they had little to say to each other about it. They also saw nothing of use to them in the workshop, and some stormed out. Later, when we deliberately brought together people with more variation in expertise and job titles, we got much better discussions and storytellings (and reactions).
The moral: Don't bring people together who all know the same amount about something, or who all know the same things. Bringing novices and experts on a topic together is a great way to get people talking, because the novices want to find out and the experts want to help. Or you can bring two groups of people together who use the same thing but in different ways (say webmasters and web users). It's only when you can observe real knowledge transfer that you get the best stories. (Just make sure the groups aren't from separate social/power worlds, or you will get no stories at all.)
The mistake: Early on we tried asking people to form groups of two. This always produced total paralysis. They just stared at each other, or at their shoes. Groups of three didn't have that problem.
The moral: Use minimum small-group sizes of three people.
The mistake: In our first storytelling sessions we did a bad job of telling people what would take place when they agreed to come. Some of them thought the session was a class and were upset when we didn't "teach" them anything. It was also very hard to get people to understand why we wanted to collect stories. A turning point was when we started to use the phrase "We want to know what it's really like" (to use this software or go through this process). That seemed to get across to people that we were looking for something beyond the facts about these issues; we wanted to know what their experiences had been. \
The moral: Manage expectations about storytelling sessions. Write clear invitations and spell out the goals, the process and the result. Also, make sure you have rehearsed responses ready for when people say things like "What are we doing here?" and "What do you want us to do?" and "You're wasting my time." Stammering and apologizing really turns people away.
The mistake: In the beginning we were underconfident when we started the session and asked people to do things. We kept whispering to each other and referring to our notes, and all that sort of thing. Some people, especially some of the high-expertise older people, headed for the door. After this we rehearsed our "act" until we had it down. The more confident we sounded, the better stories we collected.
The moral: Rehearse doing story collection, so that when you actually do it you won't sound unprepared and drive people away. If you rehearse the session, as silly as that sounds, you'll sound like you've done it dozens of times before (even if you haven't), and people will feel at ease, and they will tell stories. The rule of self-fulfilling prophecies really does work: tell people things like "When we do these sessions people naturally tell stories" -- even if you are actually doing it for the very first time -- because as soon as they do start telling stories it will magically become true. (If you don't like lying, just use your friends and family for your first session and get some free confidence that way.)
The mistake: We started out with way too much detail and instruction, and people either balked (if they hated detail) or dove too far in (if they loved detail). The balkers either sat there or walked out, and the divers generated long lists and complex complaints, but no stories. It was only later when we slimmed things down to what seemed a ridiculous minimum that we started to really get stories.
The moral: Give few and uncomplicated instructions, then step back and get out of the way while people respond. Don't over-control the session, because you will get only what people think you expect. Give people room to express themselves (but not too much!).
The mistake: We were amazed at first at the wide range of responses we got to the word "story." Some people thought we wanted them to tell jokes. Some thought we wanted them to make things up. Some thought we wanted opinions and complaints. I remember one guy who said, "Once upon a time. How's that?" (I don't think he was taking the whole thing very seriously.) Probably the worst responses were when people said they had used the software but didn't have any stories to tell about it. Clearly there were some problems of perception. It was only when we started talking about experiences, surprises, learnings, breakthroughs and other more directed terms, and using the "what it's really like" phrase, that we got past the "many meanings of story" problem. In essence we moved the word "story" to the background and didn't introduce it until it was more clear what the goals of the process were. At that point people were more able to understand in what sense we meant the word.
The moral: Don't bring out the word "story" until people have a better sense of what you are trying to do and why you are doing it. (Try asking someone point-blank "for" a story sometime, and you'll see them freeze like a deer in headlights.) It is also helpful to learn to recognize what people say when they think you mean a joke, lie, fiction or opinion when you say "story" and rehearse your response accordingly. Help people form an idea of what you are asking them to do and why you are asking them to do it, so that they can give you what you need.