Case study
From LSI
Contents |
Case selection
Selection criteria for the cases
- LSI trajectory, no single events
- Best known, most applied Large Group Intervention methods; I selected Future Search, Open Space, World Café and Real Time Strategic Change for hybrids and blends.
- Work system central, no community central
- The LSI is known as successful, by reputation (Note: I changed my mind here, I want to compare two cases I performed myself, with the same colleague in the same sector: school districts; one of them was successful, one not so successful)
- Performed more than a year ago
- The facilitators and client are interested in a more in depth evaluation
- Information about the LSI is accessible
- If possible, though not definitely required: the client is willing to pay for the evaluation
- Planning is done by mutual agreement
Three selected cases
The first two cases are evaluated in contrast. I was one of the facilitators in both processes. The first I am satisfied with, the second I am not satisfied with, I see the case as not very successful. They form nice contrasts and comparison will shed more light on success factors and effects. The other two cases are facilitated by others and are successful cases, by reputation.
- How do we involve parents in achieving our strategic goals? What schools do they want for their children? How do we keep contact? LSI process with a World Café type conference with 150 parents, teachers, directors, the board of governors. A system around an institution with 24 primary schools in one town in The Netherlands. One of my own projects. I was satisfied with the process.
- How do we make the acquisition of pupils for our public schools more successful? How do we stop the tendency of exodus to schools with other denominations? A process with a department of a local government. A system around 75 directors and assistant directors of primary schools in another town in The Netherlands. The conference was designed as an Open Space. One of my own projects, I was not satisfied with it.
- How do we generate more power, more collaboration and innovation, for development of our public library? A process with a conference with a tailor made design, resembling a Real Time Strategic Change. All internal stakeholders and some clients of the library were involved.
In consultation with prof. Jac Geurts we decided to drop the need for a fourth case.
Case evaluation
What does it mean to participate in the case study?
- An open interview of 1 to 1,5 hour with key persons in the process: client, facilitators, a member of the planning committee, a few participants of the conference
- The interview will be recorded, only to complete my notes
- I will ask for documents or other observables, that show success factors and effects of the LSI. For instance plans, reports, photos, procedures.
- all information is treated confidentially and reported anonymously
- If preferred, names of client, facilitator and process can be mentioned in the dissertation and in articles to support positive PR
- Interviews are planned in the period February - June 2009
- The interviewees receive the final evaluation report
- Financial investments of the client are not required
- Interviewees are invited to join a conference in September 2009 (option is on September 17), where the evaluation method and the results of the case study will be discussed; I hope a view well known practioners (founding fathers/mothers) and researchers will join; location, program and costs are still open
- Persons interested receive a copy of the dissertation (planned for autumn 2010)
What questions are asked?
The open interview is built around a set of indicators for success factors and for effects. Please see evaluation instrument.
Case analysis, method and steps
The goal of the case analysis is twofold: to test and improve the evaluation instrument and to evaluate the effectiveness of each specific case. To get an overview of the characteristics/conclusions/illustrations of each case and to collect conclusions for the case description and the evaluation instrument, I decided to work (again) with sticky notes. This time on flip-charts on the wall. I set up my dining room for 6 weeks as "war room" (see photos, click to enlarge).
 |  |  |  |  |  |
Meaning of the post-it colours: dark yellow for conclusions and essentials of a case, large light yellow for literal quotes, pink for insights the evaluation instrument, orange for insights or essentials for the description of the case study, green for writing directions. Note: On the original high resolution photos the text is legible
I take the following steps for analysis of a case:
- Transcription of the interviews, largely done by two students; it takes ca. 4 hours to transcribe 1 hour recording
- Close listening to the interview recording, while reading and completing the transcription with the help of notes made when I held the interview;
- While listening and reading the transcription, reflections and ideas pop up; I record them on post-its; text analysis of 1 interview hour takes 3 to 4 hours
- Analyse the printed transcription and mark with colour markers the success factors and effects (same colours as before)in the printed text
- Score the success factors and effects in the corresponding column of the Score table (see evaluation instrument); the scoring goes quite quick, to my relieve
- While scoring, make corrections and improvements in the score table: combination of items, reformulation, re-ordering, deleting some doubles or 'unobservable' indicators
- Search for additional evidence of success factors and effects in documents and artefacts (contract, reports of the planning group, design, conference report, photos, my logbook form two years ago)
- Complete the scoring of the case
- Make the next version of the score table
- Make the next version of the audit table with the interview questions
- Reflect on the case description and the case study in general and make notes on post-its
- Keep a journal of my actions, outcomes and reflections on the process of the case analyses
I started with Case 3, the one that I did not facilitate. Then I analysed Case 1 and 2 more or less together, in comparison. In both cases I was one of the facilitators, together with the same colleague. In both cases a flock of primary schools were involved, directed by a central organisation. I consider Case 1 a successful process, whith Case 2 I was not so satisfied. It worked, but could have been much better. What caused the difference?
Although the analysis of interviews asks for discipline and persistence (and a lot of time), it is not disagreeable to do. I realise how rich my data are and how much deepening they bring. And I feel lucky to combine business and pleasure: How nice is it to sit on my terrace, cat at my feet, with the headphone on and all my other stuff on the table and call it WORK.
Purpose guides analyses of the data: The purpose of the case study is to test and improve the evaluation instrument. In Patton (2002) I found out the method I used is called "Analytical framework approach", were a framework guides data gathering as well as data analysis. My framework is the evaluation instrument (the list of success factors and effects) that resulted from the exploring phase.
The yield so far
Results case evaluation: Rich stories and deeper insights in effectiveness LSI
 |  |  |  |
The interviews and documents provide rich stories of real life in an organization. My "data gathering" provides more than enough material for my purposes.
The evaluation instrument works
The evaluation instrument seems to work! I made a distinction in three levels for the success factors and effects. Level one comprises the main categories: Context/task, Client, Consultants, Intervention, First order effects and Second order effects. Level two is formed by the success factors and effects. Level three are the indicators, the observables to detect to what extent success factors and effects are met in an LSI. So far (I have done ca. 60% of the case analyses) the first and second level didn't change, only some semantic corrections. All changes concern the third level with the indicators. I brought the number of pages with indicators down from 12 to 10, and I think the closure of this level is not too far away.
Additional reading of articles about LSI cases didn't bring new factors or effects in the last months.
Common sense stays important in interpretation of definitions
A major insight, renewed after my years as an auditor of quality management systems, is that common sense and practical reasoning stay leading in an audit/evaluation procedure. As they say in aviation: you have to fly the plane, not the procedures. You have to make decisions in scoring for practical reasons, keeping the goal of the evaluation in mind. Every definition has its own problems. Persons might have several hats on, contracts and processes might overlap. For instance:
- What belongs to this LSI? My definition of an LSI process is: the time of the contract with the consultant. What to do when there are several contracts at the same time, but with different beginning and endings? In case 1 and 2 the LSI was the beginning of a bigger process in the development of an organisation. What belongs to which process?
- Who is the client? Often there are multiple levels of management involved. The project leader is the direct client, but often not the formal responsible person, the one who decides to start or stop the process.
- What to do when there is a contractor and a sub-contractor? In case 1 and 2 I was sub-contracter of the other consultant, in fact he was my client too.
- A follow-up assignment can count as a new process, or as an extension or modification of the LSI. For case 1 I see my assignment to analyse the outcomes of the LSI and write a report for the board as an extension of the LSI, although it was not offered in the initial contract. The marketing support my colleague offered after the LSI in case 2 was included in the initial contract, but was in fact no part of the LSI, because it was not directed by the planning group anymore, not participatory, not whole system.
Views on effects and success may vary
Sometimes it is hard to say if an effect was the result of the LSI or if it was already there as a condition. As one of the players said: "Success has more than one father".
To be continued....
