Methodological ramblings

Welcome to 2016.  I don’t really feel like I’ve had a break as I’ve gone from STPA in Adelaide to methodology writing to transcription (which I’ve finished!) to IFACCA abstract writing to methodology again. I even worked through Christmas to New Year. I’m feeling under pressure to get the bones of thesis done by July, when my scholarship ends, and something submitted by the end of the year.  Economically and psychologically I can’t be without an income for 12 months so I need to get back into the workforce in early 2017, which means hopefully finishing this 6 months early.

Before Christmas I wrote a skeleton methodology chapter but it felt too theoretical and not enough about what I actually did/am doing.  For the past two days I have stepped away from theory to free write about the process, the decisions I made along the way and what happened.  It’s taken a lot longer than I expected, but also raised things that I hadn’t thought about before.  It’s over 2,000 words, which also surprised me.   Once I’m finished I hope to meld this to my theory draft for something more appropriate to share with my supervisor, and at this rate it will easily form a distinct chapter.  But in the meantime I thought I’d share the free writing here.(Excuse typos.)

The methodological approach was inspired by a desire to understand what influences the development of leadership capability in the emerging leaders of the Australian arts sector.  The idea of capability was the starting point, in some way because of the academic focus on the leadership development being the creation of capabilities within individuals, but also because of my personal human resources background which had capability development as a central focus.   In the early stages of planning for the research a more quantitative approach was considered to catalogue the types of activities undertaken by those in the arts to develop or enhance their capabilities in terms of leadership.

While qualitative tools such as surveys may have provided a snapshot of activities undertaken, there was still doubt around the effectiveness of participation in such events and what really enacted changes in leadership behaviours. Those with training and development experience can advise that a) people value facilitated learning more highly than other forms of development (even when it is shown to be less effective) as it feels like their ‘idea of learning.’ In addition the collection of data about recently attended training, or those not he day, sometimes knowns as ‘happy sheets’, provides a type of halo bias that suggests training is adding more value than it really does.

These quantitative techniques, to me, lacked depth to help understand what truly constitutes effective leadership development. Even when quantitative tools were still on the agenda I always saw the need to supplement with some form of interview. Over time I realised that interviews were going to be the more effective method of data collection and the statistical idea dropped by the wayside.

The idea of interviewing multiple people about the development of a single individual came from the concept of triangulation. In corporate practice it is standard to participate in some form of 360 degree feedback when undertaken leadership development, to get a holistic view of performance and potential.  This HR orientated technique is mostly done through anonymous surveys, but as you get to executive level it can be done via interviews.  Collection of data about one individual I this way provides a much bigger picture on how they operate.

My original plan to interview an emerging leader, a peer, a mentor, a supervisor and an subordinate hinged on the idea of validity – was what the emerging leader said as being the most effective development technique the same as what their manager/boss or staff said?

The challenge found in some interviews from this perspective was the sometimes the secondary interviewees were not as close to the primary as I’d hoped.  And in this increasingly digital world they may have had significant contact virtually, making them less able to verify developmental techniques.

This did not make the process less useful, however, as my thoughts around interview data began to change as the process moved on.  Once I had started to collect material the question of analysis arose. A few themes emerged – first the idea of simply cataloguing the activities and the their perceived importance would have been a waste of the rich data collected from participants, falling back into a quantitative mindset.  Second, this idea of rejection of leadership kept appearing, both in casual conversations about my research and with the interview subjects themselves.  It became impossible to discuss the development of leadership capability without discussing the subject’s relationship to the concept of leadership.  There needed to be a methodological approach that highlighted or deconstructed this complex relationship.

The data showed that the idea of leadership identity was an important  as leadership capability.   Having the skills, knowledge or attributes of a leader was one thing, but choosing to be identified by yourself and others as a leader is a separate issue. One that was becoming crucial in the data. (There’s a third facet, leadership understanding, which has also emerged.)The idea of narrative analysis, or in my case analysis of narrative as there is a difference, was a way to bring to life the was emerging leaders discussed their developing, or rejection of, leadership identity.

The final issue was the role of the secondary subjects in this development of leadership identity.  At first glance,  I’m hoping to explore more in my analysis phase, the more supportive and engaged the group are in the work of the primary subject – what I began to see as participation in communities of practice – the more likely the primary subject was to see themselves as a leader.

But a second issue was also identified – and that is how subjects in each group or sector co-construct their idea of leadership through shared language and narratives.   Is leadership, which was not defined by me at any stage in the interview process, describes with similar language? Anecdotes? Analogies? What I found (or believe I have found) is that there’s a shared vision of leadership amongst these small communities of 4-5 people and only analysis of their stories will prove this to be true or not.

While these lines of thinking have guided me methodologically, there were also decisions made around who to interview, how to source and how the interviews would be conducted. Setting out to interview people across the creative industries means firstly defining what is meant by the creative industries, including justifying why the creative industries are the target, and sourcing subjects.

Despite coming from the arts management discipline I have always wanted to study the broader creative creative industries.  The reality of the sector means the divisions between for profit and not for profit are less rigid as they may have once been, and the intersection of the creative with the profit driven has always been of interest to me.  Coming from a corporate background I know this world as well, probably better, than the arts one, so it would be wrong not to utilise this experience.  That said I also wanted to explore what impact profit motive and more organisationally orientated creative areas would have on leadership development.  Those that work in advertising agencies, for example, may have more in common with the executives I saw at ANZ bank than with the curators who work in artist run initiatives.  Or not.   While academia still tends to focus on arts management, creative industries has largely been the frame of reference for economics and cultural policy.  This may be a passing fad, and there are significant implications when judging creative input/output organisations using traditional economic mechanisms, but there is also a recognition of the role creativity plays in the broader economy,, not just public good.

As the importance of the creative industries began to grow in political and economic debate there has also been considerable discussion of nomenclature and make up of the sector.  This is discussed in depth in Chapter four of my thesis, but I chose to align to the NESTA report developed in the UK, which had been used by the now defunct Creative Industries Innovation Centre based at UTS. The model considers not just outputs, creative contents, but also services versus product and experiences which covers the full range of sectors such a gaming, software design and advertising, along with more traditional art forms. My aim to became to include groups from a number of different quadrants across this model but was also influenced by the ability to source candidates and have them agree to participate.

Sourcing of participants began with identifying emerging leaders within the relevant creative industries sectors.  Having worked in the Sydney industry for a number of years the first group of subjects we identified from within my own network, with personal contact conducted via phone of email. Three of the nine primary subjects were previously known to the researcher.  The second source of subjects came through asking my existing network to recommend and introduce me to relevant subjects.  Once an individual was identified, their background and experience was verified online (generally through LinkedIn or personal websites) and if suitable email introductions made.  This process highlights the ease we have now of checking ‘reputation’ online, but also the personal marketing ability of many within the creative industries.  Two candidates were sourced in this fashion.

One individual was approached directly via their website based on their public profile and professional experience.  Coming from a very diverse background and being known for their willingness to opine I chose to approach them even without an intermediary introduction.

The final three primary candidates were sourced via social media sites. Two were found through Twitter, one a personal contact and one via a general call out and the third was identified via photo sharing site Instagram.  Like those introduced by colleagues, those found on social media had a background check done via professional websites before approaching them to participate. The benefit of sourcing candidates via such tools was being able to game some understanding of their experience and background, and build rapport, prior to physically meeting them.  In all cases appropriate ethics and confidentiality forms were signed and all interviews were conducted knowing they would be anonymous.  Underlying the sourcing of candidates was a desire to get a rough gender split (I have 3 male primary candidates and 6 female, and I think about 42/58% overall) and to cover some breadth of geography. I have groups in Melbourne, Adelaide and Hobart along with a majority from Sydney.

The interviews themselves were conducted using a semi-structured format, questions beginning with a career overview before narrowing down on the topics of skill development and leadership.  The decision was made not to introduce the research as being around leadership, because even at the early stages I knew it created negative connotations with potential subjects and might reduce their willingness to participate (self selecting out of the process based on an unwillingness to be seen as a leader.)  I also chose not to define leadership throughout the interview, and though this was an unconscious decision it turned out to be a fortuitous one as it provided narrative space for the interview subjects to construct their own versions, thus allowing me to see, or hear, their perceptions.

The interviews, over time, became less single sided and more of a conversation about development and leadership.  While this potentially reduces the objective nature of such data collection, in this research it enhanced the narrative process and highlighted the role of the researcher which was always going to be an important point.  Given the theoretical model that underpins this research and the role that implicit reflection plays in the process, it was inevitable that my own personal development was to be considered.  I am an active participant in the subject’s leadership development by providing the space for reflection during the interview process, while the individual interviews and the research process as whole are a documentation of a stage of my own leadership development.

There has always been an tension in my mind between creating a structured approach to analysis and allowing ideas to come freely or intuitively.  I have always been concerned that by undertaking analysis in a linear form I may blinker my thinking and miss crucial information.  Consequently I’m trying to follow a sense of structure but involving techniques that  open up the process.  The first step was to transcribe the 41 interviews, which was undertaken over the same 12 month period that the interviews were conducted, thus the content of the early interviews did begin to influence the conversations held in the the latter ones. The concepts of social learning and communities of practice appear more strongly in interviews in the second half of the year as the theories began to emerge from the data transcribed from earlier interviews.

This brings me to the point I’m at today.  The next step involves listening to all interviews a second time, along with the transcripts, and annotating/correcting them where relevant.  At this point the initial themes, of which there were too many, were consolidated or discarded. After this I hope to do some ‘free reading’ techniques where interview segments are looked at in isolation, cut from their context (literally) so not to read them with the speaker or industry in mind.  From there the key leadership question (Do you consider yourself a leader?) of the primary subject will be isolated, along with my comments around it, to explore narrative and leadership identity construction on thins single point.

And from there we are in unchartered waters.



A bit about process

I’ve had a way of undertaking literature research for a number of years.  It’s probably not a very good one, but it has worked for me in the past. (Noted that everything I’ve done in the past has been a complete cakewalk compared to this.)

I undertake three steps when looking at a piece of writing. 1. I read it, while this sounds self-evident I do not necessarily read it in the way recommended in study courses.  I READ it, not skim it. Highlighting as I go.  It’s worth noting that I am paperless.  I have no printer and have not printed out any of my 300+ articles I have sourced this year.  I read on an iPad and highlight using Goodreader. My complex relationship with Endnote is fodder for another post.

2. I take notes from my highlights.  This is a combination of direct quotes and my surmising of the content. With page references and embedded endnote links.  This used to be in word, but is now in sections within Scrivener (which I love.)  I probably have 120 individual files so far.

3. I use these notes to draft my writing/content. Of which I have about 10,000 words from last year.

I know there are weaknesses here.  There is less analysis than there probably should be on paper, as the links are all in my head. And there is a lot of jumping around, cutting and pasting. And my 120 articles are not categories to the level of detail that’s needed.

When I started at UTS I went to a number (A LOT) of research courses.  These have laid out a whole range of techniques, from using Endnote to store all analysis, to mind maps.  Many of these techniques seemed great, game changing.

But ultimately they paralysed me.  For the first two months of this year I have not added to my ‘master’ document at all.  I’ve read a lot. I’ve dropped notes in Endnote, links in Evernote, a mind-map in and references everywhere. But I stopped moving forward.  I’ve realised now that all these techniques have actually proven to be a distraction, not a help.

In the past week I returned to my old method when reviewing the 5 articles on critical leadership theory.  And I feel I’ve made more progress that I have in the past 2 months.

So my lesson learned is to stick with what works for you. While Endnote is a great tool for saving references and pdf documents, it is not the place for me to store my ideas, quotes and concepts. It just doesn’t work (for me.)

Saying that, I am going to introduce one technique shared by Dr Terry Royce as I think it fills a gap I’m going to need as this monster gets bigger and more complex.  That is the idea of a conceptual map that shows authors  and articles aligned to topics. As shown in the example below: photoLater today, however, after I have finished off my 2,000 words on critical leadership theory, I plan to get down on the floor with butcher’s paper and sharpies, and draw the biggest structure possible, incorporating the concepts  discussed in this post.  Like my process, my structure is leaning back toward the one I started with because it is comfortable and makes sense to me.

My new rule: if it ain’t broke, don’t fix it.