Author: Chris Thomson

Digital storytelling for public engagement at #digifest14

Data analysis – identifying themes

7 40 minute interviews generates quite a lot of material, doesn’t it?

Having done a rough transcription of each my next job has been to pick out the key themes that I can base my discussion around, all linked back to the research questions. Partly due to the questions and the content of the interviews I’ve broken these into the following sections:

Defining storytelling – I needed to establish what people thought they were talking about when they talked about storytelling.

Benefits of a storytelling approach (both to organisations and individuals) – what would drive an organisation to use digital storytelling and other narrative approaches?

Disbenefits and risks of using storytelling – what hurdles are likely to appear

Developing skills – what have people already done, what skills do they think need developing and what is the best way of helping them to develop the skills

Organisational perspectives – I need to define this better but I’ve used this category to look at what the impacts of a storytelling approach might be on an organisation and its stakeholders and what the main enablers are.

So, without going into too much detail, here are the emerging themes under each section with a bit of explanation…

Defining storytelling

Structure and content

Building blocks – narrative, character, plot, 
Clear underlying “message”

Nature of stories

Personal in natureMaking something memorable
Making connections with other topics

Impacts on audiences

Attention
Creating an impact
Making connections

Benefits

Supporting existing functions

Marketing
Demonstrating impact, communicating benefits
Synthesis

New approaches to communicating

Adding richness
Capturing what’s usually lost

Developing identities

Exposing our unique selling points
Understanding ourselves better
Reflecting on self and practice

Disbenefits and risks

Vulnerability

Reputation
High stakes strategy, a gamble

Resources

Time
Technologies
Skills 

Integrity and Accuracy

Personal/professional boundaries
Emotional manipulation
More than just anecdotes
Under/overrepresnetation of certain voices and viewpoints
Quality control

Development and change

Environment

Physical
Emotional

Skills and personal styles

Professional backgrounds
Levels of comfort
Congruence with styles

Approaches to learning

Peer support and feedback
Opportunities to practice
Training

Organisational perspectives

Culture

Openness and Transparency
Creativity
management expectations

Compatability

Roles
Context of other tasks
Context of other communication channels (web, blog etc)

Changing environments

Increased emphasis on showing ROI, benefits and impacts

+++++++++++++++++++++++++++++++++++++++++++++++++++++

Reflection

Looking at that list now I can already see some overlaps between the main topic headings and some of the themes within them so I’ll need to resolve some of those. The main offender is the definitions topic.

Also, the wording of some of the themes isn’t quite right, not accurately reflecting what’s whithin them. “Compatability” in particular needs looking at.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Next stage: Analysing the data

OK, the interviews are in the can. 7 in all, and I’m really grateful to my colleaugues for participating.

I’ve taken detailed notes from each one, not quite full transcripts but enough for me to have an accurate, unvarnished written record of what was said. I’ve put timecodes at certain points so I know where to look in each audio file to get a direct quote should I need one.

I’m really pleased with the depth and variety of responses I got from my participants. Each interview was slightly different from the previous. This was down mainly to each person having a different set of experiences (so an equal emphasis on the questions wouldn’t have been appropriate) and that I let the conversations flow as much as I could. The wording of my questions was slightly different for each person as well but my initial take is that I managed to get comparable answers from everyone.

The next task is to identify the main themes. Guy’s advice was to start by creating a long list of possible themes and then progressively narrowing down to the definitive list.

I’m going to create a matrix with 5 main topic areas;

  • Nature of storytelling
  • Benefits within an organisation
  • Disbenefits
  • Team and personal development
  • Organisational conditions

…then cut and paste the relevant responses from each of the participants. I may even do this using paper, scissors and glue. There’s going to be a lot of info and a laptop screen may not be enough.

Straight out of the blocks, the thing which has interested me most about the responses was the lack of emphasis on technical skills and technologies. People mostly talked about storytelling and a great deal of that was about the organisational culture needing to be one that encourages openness. There’s an inherent vulnerability to being a storyteller and the conditions have to be right for that to these approaches to flourish it seems.

Thinking smaller

One of my consistent problems over the course of this dissertation has been one of over-reaching.

I started out with a very grand vision of what my topic could be but through a mixture of being overwhelmed by the task and cajoling from Richard, Guy and a few others the final result will be much more managable and more in keeping with the level of study.

I’m not quite sure where this over-ambition comes from. It’s quite uncharacteristic of me!

It was certainly something that came across in my latest supervisor session (‘supersesh’?) with Guy who advised me to wrap up delving into case study methodology and just get on with the case study. His advice was to think about what Robert Yin proposes, measure what I’ve been doing up against his criteria but also to provide a critique of it in light of my particualr study.

I also set my net too wide for my reading, intending to spend a portion of the lit review on the pedagogy of DS. I had thought that, as most DS activity was taking place in teacing and learning within the education sector I needed to outline this. In fact, this pedagogical perspective has very little direct influence on my study, save that it give me a source of some examples of where DS is being put into practice.

Once again, this is making my task a bit more managable.

Interviews – so far, so good!

[Apologies for typos. I’m on a wobbly train so finger accuracy is suffering.]

I’m now 2 interviews into my data collection and it’s going really well.

Both have ended up being about 35-40 mins rather than the hour I’d anticipated  but even at that length they’re generating lots of useful and unexpected stuff that links very closely with the reading I’ve been doing.

I think longer interviews would be exhausting for the participants and it’s taking me about 90 mins to listen back to them and take notes. With 6 participants, that a lit of work even before I start analysing the data.

On a first listen there are some emerging themes that I’ll be able to use as slices but I won’t say what they are at this stage. Some of the participants read the blog and I don’t want to influence what they say in the interviews.

Guy had a great suggestion of “participant checking” which I’m going to do after I’ve done my initial analysis. It will mean probably getting all the participants together (probably using video conferencing) to discuss whether the themes I’m identifying actually reflect what they see as being accurate.

As for conducting the interviews, I hope I’m being a good researcher. I’m trying to use all the skills I developed as a coach at Siemens:

  • using open questions
  • reflecting back answers using the language the participant used
  • probing and asking for elaboration
  • leaving enough room for the participant to answer fully
  • maintaining eye contact and all that stuff
  • only taking the minimum of notes. It helps me to listen closely but means I can give the interviewee as much attention as possible.

I seem to be eliciting lots of extended, sophisticated responses so far so I’ll keep on with that approach.

I am detecting my greasy fingerprints in some of their answers; the outcomes of some of the sessions I’ve done, converstaions I’ve had and so on. I’ll need to make sure that I’ve written clearly about what my infuence on the answers might have been in my methodology section.

1 more interview this week then 3 next. I’m liking this feeling of momentum.

Case Study Methodology

I’ve been reading Yin’s Case Study Research (2003) to get some grounding in putting together my research. Here’s my initial take from it.

Yin lists the 5 main components of research design. This is how I think my work relates:

  1. A study’s questions – I’m happy that my research questions are relevant and useful and feedback from Guy is positive (note – need to post the revised proposal to the blog!)
  2. My propositions based on the questions – my initial direction is that storytelling as part of an  will have a positive qualitative impact on an organisation’s communictaion and internal working practices but the counter proposition is that any gains may be marginal or there will be no significant change in the state of affairs given the effort required to deveop skills and approaches. These propositions need sharpening up, they feel flabby. Yin says that there are valid reasons for not having valid propositions if the case study is more exploratory in nature
  3. The unit of analysis – bit trickier this one. The reseach is looking at Netskills (although there are some “dotted lines” to people in other organisations)but the study itself will mainly be focussing on the experiences of a subset of employees. Individual experiences are important to that but the case study is trying to look at the orgainisational level. It will be important to include key people not involved in the storytelling development activities  but who are stakeholders (e.g. the director, the training manager, “publicity officer”)
  4. The logic linking the data to the proposition – Again, problematic at this stage. I’m hoping that the interviews and observations will bring out some emergent themes; I certainly don’t want to dictate these, given my participant observer status. It’s hard to say at this stage what the logic will be for linking these data back to the propositions but I will need to make sure that my interview questions are closely linked to the research questions. 
  5. Criteria for interpreting findings – The data is going to be predominantly qualitative and narrative in nature (I’ll be asking for participants to describe their own journeys and add their own interpretations to events). As such there will be a significant subjective interpretation of the data and much emphasis on the participants’ opinions as to the value of the work they’ve undertaken. Another issue is timescales, which are not huge! In reality, the effects, positive or not, of this development work will be cumulatively visible over an extended period of months or years so this case study may just provide the basis for further research or form the template for case studies on other organisations

The role of theory

Yin quotes Sutton and Shaw (1995, p378) who describe a theory as “…a [hypothetical] story about why acts, events, structure and thoughts occur.” It’s nice to see the work story cropping up there but I’m not entirely sure what the difference between theory and proposition is.

I’ve tried to articulate my theory of story in this context to myself but I’m not coming up with anything that I find particularly satisfying yet. This is a question I want to take to Guy at our next meeting.

Perhaps the theory will emerge as a result of the research and trying to manufacture one now isn’t going to be helpful. Another thing I’ll need to bear in mind is how much I’ll be able to generalise from this cae study to a universal theory. I’ll have to be very caeful there.

Criteria for judging quality of reseach design

According to Yin there are 4 tests of quality

  1. Construct validity – can I demonstrate the data I’m collecting and measuring in the case study actually reflects the focus of the study. I’ll need to show this in my analysis but at this stage I think that looking for quantitative measures in things like social media activity (follows, mentions, retweets etc) or numbers of people booking on workshops for example wouldn’t reflect the focus of my study.
  2. Internal validity – If I’m establishing cause an effect between x and y am I able to show that it’s not something that has been caused by z? It will be important to look at the wider context at Netskills. There has been quite a lot of activity recently in team and personal development and there are also a number of staff taking on different roles in the team. This might have an impact on the study, as well as drivers coming from the changes that are happening to Jisc, from whom we receive a proportion of our funding.
  3. External validity – are the findings generalisable beyond the cae study. I anticipate that I’ll not be able to say definitively that what has happened in the Netskills context  will be directly applicable to other organisations. Although there are similar organisations to Netskills, even with Jisc, much of what we’ve been doing relates closely to individuals and the Netskills working culture. This isn’t to say it won’t be applicable, just that there will have to be clear and serious caveats written into the analysis.
  4. Reliability – would the results of the case study be replicable by another researcher following the same process? This links closely to point 3 but it’s a reminder to me to make sure that my approach is clearly documented and explained.

What type of case is Netskills’ storytelling development?

Going by Yin’s terminology I think that the Netskills case is representative, rather than a unique or revelatory case, in that I’m trying to establish what lessons could be learned for similar organisations. Given a longer time frame of years a longitudinal approach might yield more reliable results so again, this could form the basis for further study.

Lastly…

I need to be aware of Yin’s advice that multiple case studies are preferred over single as it means replication is possible. I’m not going to have that opportunity so that will necessarily colour my findings.

I also need to keep the case study design open and be willing to change the design as the study progresses.

Gathering data – some thoughts

I experimented on Will before Christmas by trialling my first stage interview questions on him. I felt it went well and Will’s responses will be very valuable when seen alongside my other research subjects.

I don’t want to post too much about the interviews here in case I influence the other participants.

These interviews won’t generate sufficient data by themsleves and I’m struggling to think how best to capture other aspects. This is one of my main aras of concern to take to Guy next time I speak to him.

How do I fit in? I’m part of the case study as well but do I need to answer my own questions? Does that risk me skewing the data? I’ve undertaken quite a lot of work in the form of presentations, workshops, blog posts and so on but I’m not sure how to represent that in the case study.

All being well, over the course of the study we will have produced our own digital stories. I want to include these but will also need to capture reflection from the people that have produced them as well as the rest of the peer group.

For the 2nd stage interviews I was considering doing a workshop to try and capture individual experiences but also some of the group dialogue. We have been learing about participatory approaches to workshops recently and it would be good to put some fo those into practice. This will allow participants to visualise a lot of their experiences, mapping their journeys. This will have to be done carefully to make sure that sufficient data is captured on the day (I’ll need to employ an observer/note taker) as well as giving participants a chance to explain and interpret in their own words what they have come up with.

The more I think about it the more nervous I get that time is ticking away. I want to get my initial round of interviews wrapped up in the next 2 weeks or so so I can start transcribing and coding the data and then planning the next stages.

Literature Review drafting using Evernote

Given that my deadline for this dissertation is now effectively March next year I’m having to get a wiggle on.

The literature review has been intimidating me slightly at the thought of trying to communicate what I’ve been reading so I’ve taken Richard’s suggestion of just writing anything regardless of quality (at this stage at least).

I’m using Evernote to do this as it’s available on my home and work computers as well as my mobile. I’ll also be able to use it to tag notes according to the emerging themes.

Here’s the (very) rough set of notes I’m using to structure what is likely to become the section on Narrative in the literature review.

http://www.evernote.com/shard/s19/sh/573a6e08-38c8-497f-91b7-f91e3164c330/5ddf12a04b241dd7a01dc8925e328474

Drefinitely work in progress.

Retaining what I’ve read has been a problem, especially with the unterruptions to the dissertation in the last 12 months. I’m hoping that I’ll be able to slot what I read into the categories I’ve set out as I go. If it looks like my initial structure isn’t going to work it should be fairly easy to rearrange things.

All in all, I’m hoping it will make the process of actually writing the finished article a lot more painless than previous pieces of work!

Jerome Bruner – Making Stories

I’m typing this on a train so apologies for the brevity, spelling and all the rest.

This is just a short post to try and capture what I’ve learnt from reading Jerome Bruner’s Making Stories: law, litarture, life (2002)

He has a great way of explaining about defining story,  the problems encountered in it’s use and most importantly seems to propose a theory (really a theory? Can it be tested?) of what narrative is for in a cultural sense.

His main standpoint is to understand the cultural origins and function of story to better inform the use of narrative in a legal context (p11-12).

Bruner discusses how our capacity for story is ingrained in our neurophysiology (p30)  and our ability to form stories seems to be something that starts before we fully acquire the language to express them (do we ever fully acquire that? Have I yet? Did Jane Austen? ;)) (p32) and is based on the seemingly unique human trait of mimetic sense; the ability to imitate or re-enact events from future or past (p97).

“ We seem, then, to have some predisposition, some core knowledge about narrative right deom the start.” (p33)

Perhaps it is this which makes narrative sense so innate that it makes it difficult to explain

“For our intuitions about how to make a story or how to get the point of one are so implicit, so inaccessible to us, that we stumble when we try to explain to ourselves what makes something a story rather than, say, an argument or a recipe.” p3

So he descontructs what makes a story so we can identify them. He uses Kenneth Burke’s “Dramatistic Pentad” as a way of isolating the elements of a story. To paraphrase…

A story needs:

·         An agent who performs

·         An action to achieve

·         A Goal in a recognisable

·         Setting by the use of certain

·         Means

The story is driven by a misfit between any of those elements (p34)

He refers to this misfit to something Artistotle defined as “peripeteia”, a reversal in circumstances (p5). He also refers to it in Burns’ terms, “the best laid plans of mice and men gang aft aglay.”

Stories arise when there is a break from normality in some way. He says narrative is dialectic between what was expected and what came to pass (p15). If what comes to pass is what was expected, then there’s no story.

He also points out the curious aspect of culture, that it is what informs the idea of normal but also celebrates the “transgressions” (p15). Back to this later…

The main problem he identifies with stories are to do with intent. A story is not just something that is told; it is told with a purpose and that purpose is not necessarily benign (p24), it is “susceptible to ulteriority” (p5). It is not innocent.

He refers to verisimilitude in stories which I think means  it’s appearance of truthfulness. I’m not sure I want to use that term too much but lots of our discussions at work have been around authenticity (p14).

So what does he see as the purposes of storytelling?

The focus on peripeteia suggests that stories are not about solutions so much as problems, the journey more than the destination, about “plight” and “the road, rather than the inn to which it leads. (p20) “Great narrative is an invitation to problem-finding, not a lesson in problem solving.” (p20)

He says that stories reassert a sort of conventional wisdom (those cultural norms) on extraordinary circumstances in an attempt to render them meaningful (p31). For some reason this made me think about an alien invasion film where the motives of the aliens are never explained and how much tension that would generate. Most films like this seem to suggest a reason behind the behaviour. Cloverfield is an example of a monster movie that never explains anything about its monster – leaving the story to focus on the human protagonists. Anyway…

So does this mean stories are a rehabilitation process? Something happens that is outside the norm and defies easy explanation so we turn it into narrative to make it understandable, so bringing it back into normality.

“A shared narrative is what matters. Reason alone will not do the trick” (p107) He’s talking about a specific situation of occupational therapy with children but it’s a great motto for storytelling.

I’m arriving at Kings Cross now so further thought on this will have to wait.

The neurochemistry of stories

This an aspect of storytelling that I haven’t really looked at so far; what is the neurological/neurochemical response to narrative?

It’s going an extra step up the chain of “what makes stories compelling?” that I’ve been a bit scared to look at!

I’m not sure that it’s something that I want to look at in depth as part of my literature review but it might be worth alluding to. I don’t feel equipped to examine it properly, not being a “proper” scientist and I’m not sure what bearing it has on my research questions.

But it does give some interesting context.

It’s very interesting andI hope I’m approaching this cautiously. What he’s talking about does seem research-based and a quick library, wikipedia and Scholar search contains published, peer-reviewed research papers.

His research is concerned with morality and trustworthiness, especially in economics (he calls himself a neuroeconomist), and he focuses on the effect of oxytocin molecule.

I’m going to have alook at some of his published papers to satisfy my curiosity and to see whether there are any other theories along these lines.

I don’t want to start bandying around terms like “cortisol” and “oxytocin” willynilly, hoping to get some scientific credibility by proxy.

There are links here to the area of storytelling that I anticipate some may have difficulty with, namely influencing people through emotional response. It’s one of the ethical considerations of storytelling. When does entertaining and engaging becoming influencing become manipulating become coercing and so on.

The fact that some of this research has been funded by DARPA also made me feel uncomfortable. Why would an organisation whose mission is “to maintain the technological superiority of the U.S. military” be interested in how stories motivated behaviour?

Talking about “weaponising” stories is probably too dramatic but isn’t that what propaganda’s about?