Linking knowledge management with monitoring and evaluation

A short while ago, I gave a small brown bag seminar on the connections between knowledge management (KM) on the one hand and monitoring and evaluation (M&E) on the other hand, for a group of people from the Centre for Development Innovation (CDI) and the Technical Centre for Agricultural and Rural Cooperation (CTA), both located in Wageningen in the green heart of the Netherlands (the country where I now live).

It was somewhat intimidating to give this seminar in front of a really savvy audience, particularly for the M&E part, since I haven’t directly worked much on M&E since I joined ILRI in late 2011.

The seminar I prepared was on Prezi – see hereby:

https://prezi.com/view/Jww1idvEC608G7wyA8bn/ or https://prezi.com/p/wgggdcmwxz-v/

(Just as an aside: It was done using ‘Prezi Next’ which is related to the classic Prezi everyone knows but has new features. A new learning curve to adapt to the latest design options and it’s clear that since their heyday a few years ago Prezi have gone some way to reduce the motion sickness effect that was the biggest drawback of their otherwise great application. Unfortunately, at this stage, it’s not possible to embed a PreziNext into WordPress, though this might be fixed some time in the near future).

In any case, what was interesting, as with every piece of information that is being presented, is how people reacted to it, and what they reacted on.

The key points we discussed in the interaction revolved around:

How KM is perceived as dead or not

The notion that ‘KM is dead’ was perhaps difficult to digest for some of the KM heads around the table, though in the conversation it became clear that as much as the field is disappearing, the lessons and principles and approaches of KM live on. And in certain areas, sectors and organisations KM is still very much vivid as a field in and of its own.

We agreed that the importance was to shape collective norms about what is KM (or whatever a group calls it) and that a label (such as KM) should only be adopted if it helps clarify matters for a given group. But the conversation about what it is called is useful too.

Archetypes of KM and M&E heads

Another interesting aspect we touched upon was the stereotypes of people working in KM and in M&E. I made a very rough caricature to introduce some of the KM archetypes and the M&E archetype (of the cold-blooded scientist) which luckily has changed over the years. Particularly the M&E community is really transforming, with booming activity as I can judge on the aliveness of the Pelican Initiative. And so it’s only encouraging to expect that there will be more and more alignment between KM and M&E in the future as many individuals that I know are trespassing the borders of either field and are working across the disciplines.

How KM adds value, what makes it special or different? Why bother?

This was one of the challenges posed by the director of one of the host institutes I was presenting to. “Why should we do KM if it’s so similar to M&E, and what are the trends and the approaches we need to embrace from that field”. That’s where we came back to the bottom line of KM=CDL and the fact that KM is a useful lens looking at knowledge and learning, ensuring we leverage knowledge at all stages. It was interesting also to hear that some people assumed KM to be systematically about learning (but what about the ‘KM portals‘?). But the conversation showed that the connection between KM and M&E is not automatically grasped – and perhaps that my presentation didn’t hit home base 😉

 

KM and M&E is not all about the M&E of KM (Credits: MSH/Pathfinder)

What is the real connection between KM and M&E?

Indeed the big question is: what really is the difference or the overlap between KM and M&E. And it has to be learning, though we recognized, as a group, that not all KM and not all M&E are learning-focused, but both hold that promise and can make it happen. How to bring them together and how to make them benefit from each other is the question. Perhaps this is really worth blogging more about, just as it might be useful to blog about iterative and upscaling cycles of CDL that take KM from a very individual to a societal level with social learning – one of the thoughts that played around my head during this rich learning day…

 

…which leads me to the bottomline of all of this though is that I can sense how much intellectual effusion there is around Wageningen and the development society of the Netherlands at large and I sense that this might fuel another burst of blogging for me. Which can only be good for the short and longer run for me (and hopefully for you then ;)…

Related blog posts:

• Harvesting insights (1): back to (KM) basics
• Complexity in aid: An interview (by Ben Ramalingam) with Jaap Pels
• Research, KM and multi-stakeholder processes: cross interview with Cees Leeuwis and Mark Lundy
• Where is KM and its spirit?
• KM=CDL, on the journey to universal sense-making

 

Advertisements

Opportunity costs of documentation and how to make it work…

In my book of KM, documentation is an essential part of the work.

Documentation – do you read it (Credits: Matt Ray / FlickR)

Not everyone agrees to it. Someone who works a lot with Liberating Structures recently told me he didn’t necessarily see the point of harvesting anything because the people that were ‘doing the work’ would remember.

But then there’s always the point of documenting for the sake of the people who are not ‘doing the work’ there and then. Keeping traces so others can pick up the trail and use it in ways that help them.

However the question always remains: what should you document (e.g. what is good in a project) and how much should you invest in documenting it – and how – vs. how much you should set up processes to directly connect people with relevant experience?

This is the eternal debate of documentation vs. one-on-one experience sharing, of Alexandrian libraries vs. campfires – something that is currently being debated on KM4Dev around the title “How Elon Musk can tell if job applicants are lying about their experience” (link pending on membership).

Yes, Alexandrian libraries are only a partial solution because they don’t relate a lot of the complexities. And as Johannes Schunter pointed recently on his blog, lessons learnt that generate bland statements are useless (the ‘Duh’ test).

And there is the issue that documentation takes time and effort. Not everything can be documented, everywhere, all the time, by everyone. It’s the same opportunity cost as for monitoring and evaluation (for which we can also adopt a somewhat agile approach).

Here are some ideas to identify what to document and how:

What to document?

• What is new?
• What is significant?
• What’s been done about this already (in some form or shape)?
• What is simple (and can be codified into principles or best practices)?
• What is complicated (but can still follow good/next practice)?
• What is complex and inter-related about this?
• What is unknown?
• What is helping us ask the next best questions?
• Who knows more about this
• What could be useful next steps?

How to document?

• Develop templates for documentation for e.g. case studies (link pending KM4Dev membership);
• Keep it simple: as little information as needed to inform people, but linked sufficiently well to other sources;
• Develop a collective system where people can add up their experiences and insights (e.g. the KS Toolkit) – make sure you have one place that people recognise as the go-to site for this information;

How to prepare that documentation work? And this is the most important part.

• Stimulate your own documentation through blogging, note taking, managing a diary etc. It always starts and ends at the individual level – as the constant knowledge gardeners we should be;
• Make sure your documentation is related to conversations (as Jaap Pels also recommends in his KM framework) so that you get an active habit of identifying;
• Make sure you have formal and informal spaces and times for these conversations to erupt, both at personal level with our personal learning networks, within teams, within organisations, across organisations (e.g. in networks) etc.;
• Develop abilities for documentation (which is part of the modern knowledge worker’s skillset);
• Develop a strong questioning approach where you are constantly working on foresight, trend watching, complex tradeoff assessments etc.;
• Role model documentation of the important aspects emerging from learning conversations, to stimulate a culture of intelligent documentation;
• Assess how your documentation makes sense and what is required – and this is the art and science of documentation, to strike the balance between time inputs and learning/productivity outcomes…

Documentation is an interesting KM opportunity for many people. See this ‘Documentation Maturity Model’ (Credits: Mark Fidelman / FlickR)

How do you approach documentation in your conversations?

Related blog posts:

• The lessons I learned about lessons learned
• Anatomy of learning: how we (individuals) make sense of information
• Experiential learning and the power of questions in motion (Ramblings and mumblings around…)
• Institutional memory (making) and learning across project silos
• Agile KM from ‘SMART goals’ to ‘practice SMARTS’
• The constant knowledge gardener
• KM… the extra mile that saves (y)our time

At the edges of knowledge work, the new beacons of ever-sharper collective intelligence

Modern knowledge workers don’t really exist. Not with all the highly desirable features we may want them to have. But breaking down what such a super human should do into distinct functions could be a good start to training us all at becoming better knowledge workers. I noted a few of these functions in the profile of a modern knowledge worker such as documenting conversations, filtering information etc. Yet these functions are dynamic and reinvent themselves, and new ones appear.

What are the next knowledge work super-hero functions? (credits – Photonquantique / FlickR)

These new functions are partly addressed already by agile knowledge workers, but perhaps not always with enough intent and consistency. While we may not recognise the following functions, they may become increasingly pertinent in the modern knowledge era, with the intention of mobilising collective knowledge as best we can, particularly around events (online or offline) that bring people to strike rich conversations:

Ex-post sense-maker 

An event that is documented properly leads to rich notes on e.g. a wiki, a Google document, a written report (or otherwise). This is great: anyone participant in such conversations – anyone at all actually – can find and use these traces of conversations. But digital conversation notes are often TOO rich. Too long, too complex. A very useful extra mile for knowledge work would be to go through these notes and tease them out in useful bite-size chunks and compelling formats. An excellent example of this is this documentation of work done on ‘anticipating climate risks in the Sahel‘.

Memory connector (literature sifter)

This is the normal job of researchers. They dig through past documentation and build upon it. But they do it in a specific way – not always most straightforward. So before any planned/structured conversation happens (or any event gets organised), having someone go through all the literature related to the issues at hand, summarising key questions and issues that were raised around that field the last time around (picking up on the trail of ex-post sense-makers), on the latest recommendations etc. would add immense value to the conversations. It’s about mapping out the grid of our collective intelligence and building on it.

Too often we reinvent the wheel out of laziness or lack of awareness about related past conversations. The trick is again to package that preexisting information in ways that make it attractive to the people who will be engaged in the audience. Cartoons? A short video? A Pecha Kucha presentation (see example below)? A list of documents commented with humour? There are many ways to do this. So why do we too often fail at linking the past with the present?

Visualisation engineer

The documentation of conversations is more often than not done in a written format. Or in the best of cases in a myriad of videos. This makes it hard for us to absorb and synthesise that information. So how about visual engineers: people who are able to prepare visual handouts as the conversations unfold, organise intelligent lists of contacts that make networking and connecting easier, sifting through stats and presenting graphs in a radical and compelling way, developing complex thoughts into an-image-is-worth-1000-words kind of graphs and conceptual models.

Graphic recording – a whole palette of options before, during and after… (Credits – Susan Kelly)

There’s already a lot of graphic recording (see above) happening. I believe in our Instagram-culture of Pinterest drives we are only at the dawn of on-the-spot visual engineering. And this is perhaps not as much a function as an activity that just should occur more systematically.

And here’s another example:

Social network gardener

Perhaps this function is covered under any of the above. The idea is that someone really uses the information recorded and nuggets harvested to plant it/them in the right channels, networks and locations. Combined with the work of a visualisation engineer, this job allows targeted sending of compelling information to the right people.

Social media gardening – takes time but pays off! (Credits – j&tplaman / FlickR)

Social network gardening does take time, but really adds a lot of value to the exchange that happened in the first instance, because it contributes to a universal information base that can reduce the learning curve the next time a group of people are wondering about a similar set of issues. And it does so not just by making information available but also by connecting people, i.e. knowledge – so it’s much more dynamic. Of course a lot of modern knowledge workers are already doing this to some extent. The point is to add structure and intent to this, to maximise opportunities for interaction beyond the group of people already involved.

Interestingly, what all these functions have in common is to combine conversations (knowledge sharing) and their documentation or processing (information management) both before, during and after the conversations happen… Acting upon the conversations as they happen, the nexus of agile KM don’t you think?

Related blog posts:

• Portrait of the modern knowledge worker
• The constant knowledge gardener
• The lessons I learned about lessons learned
• The power of three learning approaches and their combination? Capitalising, systematising, documenting processes and experiences
• Net added value in an event: networkshops and the power of contextual webs
• Institutional memory (making) and learning across project silos

At the IKM Table (2): individual agency vs. organisational remit, accountability and impact pathways for the future of IKM-Emergent

Day 2 of the final IKM workshop dedicated to ‘practice-based change’. As much as on day 1, there is a lot on the menu of this second day:

• Individual agency vs. organisational remit;
• Accountability;
• Impact and change pathways;
• A possible extension of the programme: IKM-2

Day 2 - the conversation and cross-thumping of ideas continues

On individual agency and organisational remit:

We are made of a complex set of imbricated identities and cultures that manifest themselves around us in relation with the other actors that we are engaging with. These complex layers of our personality may clash with the organisational remit that is sometimes our imposed ‘ball park’. Recognising complexity at this junction, and the degree of influence of individual agents is an important step forward to promote more meaningful and effective development.

Pressed for time, we did not talk a lot about this. Yet we identified a few drivers that have much resonance in development work:

• As little as organisations tweet, people do, organisations do not trigger change, individual people do. Pete Cranston mentioned a study done about three cases of critical change within Oxfam, all triggered by individuals: a manager with the power to change, an aspirational individual quickly building an alliance etc. – our impact pathways need to recognise the unmistakable contribution of individual ‘change agents’ (or positive deviants) in any specific process or generic model of social change. Individuals that are closely related to resource generation obviously have crucial leverage power and play a special role in the constellation of agents that matter in the impact pathway;
• We are obscured by our scale: In politics it took us a long time to realise there were crucial dynamics below nation-states and above them. In a similar swing, in development let’s go beyond merely the organisational scale to focus on the individual agency as well as the network scale – all organisations and individuals are part of various networks which impact both individuals and organisations engaged in them. Teams also play an important role to explore and implement new ways – it is at that level that trust is most actively built and activities planned and implemented. The riddles of impact from the teams emulate in sometimes mysterious ways to the organisational level;
• These differences of scale tend to place subtle tensions on individuals between their personal perspectives and the organisational priorities. The multiple identities and knowledges (including local knowledge) are inherently in ourselves too, adding layers of complexity as the predominance of one identity layer over another plays out in relation to the other people around – see presentation by Valerie Brown.

Valerie Brown: Collective social learning – a theoretical foundation for Web 2

On accountability:

Accountability is a central piece of the development puzzle yet, so far, we have embedded it in too linear a fashion, usually upwards, to our funders. Accountability should also embrace the wider set of stake-holders concerned in development initiatives, including beneficiaries and peers, and find alternative ways to be recognised, acted upon and expressed.

The crux of our accountability discussion was around the tension to reconcile accountability with the full set of actors that we are interacting with in our development initiatives. The work carried out by CARE in Nepal (recently finished and soon to be uploaded on the page listing all IKM documents) is a testimony that accountability can and should be multi-faceted.

• At the core of this conversation lies the question: whose value, whose change, whose accountability? We perhaps too quickly jump on the idea that we know who is the (set of) actor(s) that has(have) more value to bring and demonstrate, that their theory of change matters over that of other actors, and that our accountability system should be geared towards their needs.
• About theory of change, we already mentioned on day 1 that it is just a tool and any simple tool bears the potential of being used smartly (despite inherent technical limitations in the tool) as much as any complex tool can be used daftly (regardless of the inherent flexibility that it may have). However, the theory of change (of which one guide can be found here) can be quite powerful to ponder the key questions above. A collective theory of change is, however, even more powerful.
• Perhaps a practical way forward with accountability is to identify early on in a development initiative who we want to invite to map out the big picture of the initiative and the vision that we wish to give it. The set of actors participating to the reflection would represent the set of actors towards whom the initiative should be accountable to. In the process, this consultation could reveal what we can safely promise to ‘deliver’ to whom, and what we can only try and unpack further. This might even lead to shaping up a tree map of outcomes that might be simple, complicated, complex or chaotic (thereby indicating the type of approach that might be more adequate).
• More often, in practice, we end up with a theory of change (or a similar visioning exercise) that has been prepared by a small team without much consultation. This implies a much simpler accountability mechanism with no downward accountability, only upward accountability to the funding agency or the management of the initiative. This may also imply that the chances of developing local ownership – arguably a crucial prerequisite for sustainable results – are thereby much dimmer too.
• Robin Vincent also referred to the peer accountability that pervades throughout social media (Twitter, blogs) to recognise the validity and interest of a particular person could be a crucial mechanism to incorporate as a way of letting good content and insights come to the surface and enriching accountability mechanisms.

On impact and change pathways

The next discussion focused on the impact and change pathways of IKM-Emergent. Each member drew a picture of their reflections about the issue, whether specifically or generally, whether practically or theoretically, whether currently or in the future. We produced eight rich drawings (see gallery below) and discussed them briefly, simmering conclusive thoughts about impact and the influence that IKM-Emergent has or might have.

Day 2 – the conversation and cross-thumping of ideas continues

• Impact happens at various scales: at individual (for oneself and beyond), at team level, at organisational level and at network level (at the intersections of our identities, relations and commitments), it follows various drivers, strategies, instruments and channels. Keeping that complex picture in mind guides our impact seeking work.
• Our impact is anyway dependent on larger political dynamics that affect a climate for change. The latter could become negative, implying that development initiatives should stop, or positive and leading to new definitions and norms;
• In this picture, IKM seems to play a key role at a number of junctions: experimentation with development practices, network development, counter-evidence of broadly accepted development narratives, recognition of individual agency and its contribution to social movements, ‘navigating (or coping with) complexity and developing resilience, documenting case studies of how change happens, innovative approaches to planning and evaluation and developing knowledge commons through collaboration;
• And there certainly are lots of sympathetic agents currently working in funding agencies, international NGOs, social movements, the media as well as individual consultants. Collectively they can help;
• The combination of public value, capacities and authorising environment are some of the stand posts around IKM’s ball park;
• IKM’s added value is around understanding the miracle that happens at the intersection between, on the one hand, interactions across many different actors and, on the other hand, systemic change at personal / organisational / discourse level. We can play a role by adding our approach, based on flexibility, integrity, activism and sense-making;
• If we are to play that role of documenting the miracle and other pathways to change, we should remain realistic: We are led to believe or let ourselves believe that evidence-based decision-making is THE way to inform (development) policies and practices, when – in practice – we might follow more promising pathways through developing new knowledge metaphors, frames of development, preserving documentary records and interlinking knowledges;
• There is also an element of balancing energy for the fights we pick: Impact and engagement with people that are not necessarily attuned to the principles, values and approaches of IKM-Emergent takes energy. But it matters a lot. So we might also interact with like-minded people and organisations to regain some of that energy.
• Finally, there are lots of exchanges and interactions and great development initiatives already happening on the ground. The layer above that, where INGOs and donor agencies too often locate themselves, is too limited as such but our impact pathway is perhaps situated at the intersection between these two – how can we amplify good change happening on the ground?

On IKM-Emergent 2:

In the final part of the workshop, after an introduction by Sarah Cummings about where we are at, we surfaced key issues that will be important themes for the sequel programme suggested for IKM-Emergent (the so-called ‘IKM 2’). We briefly discussed a) practice-based change, b) local content and knowledge and c) communication and engagement.

On practice-based change: In this important strand, we debated the importance of the collective against the individual pieces of work – challenging issue in IKM-1. Building a social movement and synthesising work are on the menu, although at the same time it is clear that each team or group of individuals working on independent pieces of work needs to find their breathing space and to some degree possibly detach themselves from the collective. IKM Emergent has been successful at unearthing rich research and insights thanks to the liberty left for each group to carve their space. But the message is clear: connecting the dots helps bring everyone on board and picture the wider collage that an IKM-2 might collectively represent.

On local content and knowledge: In this equally important strand, language is key. So is the distortion of knowledge. We want to understand how localisation of information and technology may differ from one place to the next, we want to move on to ‘particular knowledges’, zooming in on specifics to draw on them. We want to further explore diverse ways of connecting with multiple knowledges through e.g. dancing, objects, non-ICT media. We want to better understand the dynamics of local social movements and knowledge processes and do that with the large African networks that we have been working with.

How is this all to unfold? By creating a network space that allows content aggregation, meetings online and offline, experimental research and production of artefacts, organising exhibitions and happenings and integrating social media.

On communication, monitoring and engagement: This has been paradoxically, and despite the efforts of the IKM management, an area that could have been reinforced. A communication strategy came very late in the process, was somewhat disconnected from the works and rather message-based than focused on engagement and collective sense-making.

What could we do to improve this in IKM-2?

Further integrating communication and M&E, focusing on collective… conversations, engagement, reflection, learning and sense-making. And recognising that both communication and M&E are everyone’s business – even though we need someone (a team?) in the programme to ‘garden communication’, prune our networks (to keep interacting with relevant actors at the edges) and to provide support to staff members and IKM partners to connect to the communication attire of IKM-2

This implies that internally:

• The success of communication depends also on the production of excellent content to engage people on and around. The constant exploration and openness to new opportunities that characterised much of IKM-1 should be maintained to ensure a wide diversity of mutually reinforcing sources of great reflection and conversation;
• More conscious efforts are taken to distil key insights from ongoing work – even though we recognise the necessity of (a degree of) freedom and disconnect to develop good work;
• Distilling those insights might benefit from strong process documentation (1), undertaken by a social reporter (2), supported by regular collective sense-making sessions where those key insights and ‘connecting points’ between work strands could be identified and analysed.
• We aim at ‘quick and dirty’ (link to post) communication cycles to quickly churn out insights and discuss them, rather than wait for long peer-process processes that slow communication down and reduce the timeliness (and relevance) of the work undertaken;
• There is a strong need for consistent communication (supported by proper information and training for staff members to feel comfortable with the communication tools and processes) and robust information management (tagging and meta-tagging, long-term wiki management etc. – to be defined).

And externally it implies:

• That we care for the growing community of conversation that we are having – as an overarching goal for our comms work;
• That we use the insights to regularly engage a wider group by e.g. organising thematic discussions around emerging (sets of) pieces of work from IKM-2 and invite external actors to connect to and expand that body of work, possibly fund parts of it etc.
• That we find innovative ways of relating content and ‘re-using it’ smartly by e.g. writing ‘un-books’ with regular updates on the wiki, blogging, syndicating content via RSS  feeds etc.;
• That we use different communication tools and channels to engage with a multi-faceted audience, so that they find comfortable ways to interact with us and the same time that we titillate their curiosity to try out alternative modes of communication too. There are many relations between external communication and the ‘local content/knowledge’ strand with respect to alternative modes of communication that may not (re-)enforce Western modes and preferences for communication.

 

What now?

After two days of workshops and five years of collective work, we come out with an incredibly rich set of insights – of which this workshop is only the emerged tip of the iceberg – a wide collection of outputs (and more to come), a number of messages for various groups and a dedication to engage with them on the basis of all the above in an expanded programme. There is no funding yet for IKM-2 but with resources, ideas and ambitions, there may well be all the elements to bring us on that way and find like-minded spirits to transform development practices. Impact pathways don’t need funding to work, we are on it, wanna join?

 

Notes:

(1) Process documentation is a soft monitoring approach including a mixture of tools and techniques to ensure that a given initiative’s theory of change is kept in check and questioned throughout its lifetime and ultimately leads to a set of lessons to inform similar initiatives in the future. It has been better described in this IRC publication: Documenting change, an introduction to process documentation.

(2) Social reporting is very close to process documentation although it is usually applied for specific events rather than long term processes. It is better explained in this ICT-KM blog post.

Related blog posts:

• At the IKM table: linearity, participation, accountability and individual agency on the practice-based change menu (1)
• Communication, KM, monitoring, learning – The happy families of engagement
• IKM emergency
• IKM-convergent? Annual programme meeting, Wageningen, day 1
• The Giraffe
• Go organic, go civic! #KMalreadyHappensAnyways

At the IKM table: linearity, participation, accountability and individual agency on the practice-based change menu (1)

On 20 and 21 February 2012, the  London-based Wellcome Collection is the stage for the final workshop organised by the Information Knowledge Management Emergent (IKM-Emergent or ‘IKM-E’) programme. Ten IKM-E members are looking at the body of work completed in the past five years in this DGIS-funded research programme and trying to unpack four key themes that are interweaving insights from the three working groups which have been active in the programme:

1. Linearity and predictability;
2. Participation and engagement;
3. Individual agency and organisational remit;
4. Accountability

This very rich programme is also an intermediary step towards a suggested extension for the programme (“IKM 2”).

In this post I’m summarising quite a few of the issues tackled during the first day of the workshop, covering the first two points on the list above.

On linearity and predictability:

Linear approaches to development – suggesting that planning is a useful exercise to map out and follow a predictable causal series of events – are delusional and ineffective. We would be better advised using  emergent perspectives as they are more realistic, for lack of being more certain.

Linearity and predictability strongly emphasise the current (and desired alternative) planning tools that we have at our disposal or are sometimes forced to use, and the relation that we entertain with the actors promoting these specific planning tools.

Planning tools

After trying out so many ineffective approaches for so long, it seems clear that aspirational intent might act as a crucial element to mitigate some of the negative effects of linearity and predictability. Planning tools can be seen as positivist, urging a fixed and causal course of events, indeed focusing on one highlighted path – as is too often the case with the practice around logical framework – or can have an aspirational nature, in which case they focus on the end destination or the objective hoped for and strive to test out the assumptions underlying a certain pathway to impact (at a certain time).

Different situations require different planning approaches. Following the Cynefin framework approach, we might be facing simple, complicated, complex or chaotic situations and we will not respond the same way to each of those. A complex social change process may require planning that entails regular or thorough consultation from various stakeholder groups, a (more) simple approach such as an inoculation campaign may just require ‘getting on with the job’ without a heavy consultation process.

At any rate, planning mechanisms are one thing but the reality on the ground is often different and putting a careful eye to co-creating reality on the ground is perhaps the best approach to ensure a stronger and more realistic development, reflecting opportunities and embracing natural feedback mechanisms (the reality call).

There are strong power lobbies that might go against this intention. Against such remote control mechanisms – sometimes following a tokenistic approach to participation though really hoarding discretionary decision-making power – we need  distanced control checks and balances, hinting at accountability.

Managing the relationship leading to planning mechanisms

Planning tools are one side of the coin. The other side of the coin is the relationship that you maintain with the funding or managing agency that requires you to use these planning tools.

Although donor agencies might seem like ‘laggards’ in some way, managing the relationship with them implies that we should not stigmatise their lack of flexibility and insufficient will to change. In a more optimistic way, managing our relationship with them may also mean that we need to move away from the contractual nature of the relations that characterise much of development work.

Ways to influence that relationship include among others seeking evidence and using evidence that we have (e.g. stories of change, counter-examples from the past either from one’s own past practice or from others’ past practice etc.) and advocating it. Process documentation is crucial here to demonstrate the evidence around the value of process work and the general conditions under which development interventions have been designed and implemented. It is our duty to negotiate smart monitoring and evaluation in the intervention, including e.g.  process documentation, the use of a theory of change and about the non instrumentalisation (in a way that logical frameworks have been in the past). In this sense, tools do not matter much as such; practice behind the tools matters a lot more.

Finally, still, there is much importance in changing relationships with the donor to make the plan more effective: trust is central to effective relationships. And we can build trust with donors by reaching out to them: if they need some degree of predictability, although we cannot necessarily offer it, we can try, talk about our intent to reduce uncertainty. However, most importantly, in the process we are exposing them to uncertainty and forcing them to deal with it, which helps them feel more comfortable with uncertainty and paradox and find ways to deal with it. Convincing donors and managers about this may seem like a major challenge at first, but then again, every CEO or manager knows that their managing practice does not come from a strict application of ‘the golden book of management’. We all know that reality is more complex than we would like it to be. It is safe and sound management practice to recognise the complexity and the .

Perhaps also, the best way to manage our relationship with our donors in a not-so-linear-not-so-predictable way is to lead by example: by being a shining living example of our experience and comfort with a certain level of uncertainty, and showing that recognising the complexity and the impossibility to predict a certain course of events is a sound and realistic management approach to development. Getting that window of opportunity to influence based on our own example depends much on the trust developed with our donors.

Trust is not only a result of time spent working and discussing together but also the result of surfacing the deeper values and principles that bind and unite us (or not). The conception of development as being results-based or relationship-based influences this, and so does the ‘funding time span’ in which we implement our initiatives.

Time and space, moderating and maintaining the process

The default development cooperation and funding mechanism is the project, with its typically limited lifetime and unrealistic level of endowment (in terms of resources, capacities etc. available). In the past, a better approach aimed at funding institutions, thereby allowing those organisations to afford the luxury of learning, critical thinking and other original activities. An even more ideal funding mechanism would be to favour endemic (e.g. civic-driven) social movements where local capacities to self-organise are encouraged and supported over a period that may go over a project lifetime. If this was the default approach, trust would become a common currency and indeed we would have to engage in longer term partnerships, a better guarantee for stronger development results.

A final way to develop tolerance to multiple knowledges and uncertainty is to bring together various actors and to use facilitation in these workshops so as to allow all participants to reveal their personal (knowledge culture) perspective, cohabiting with each other. Facilitation becomes de facto a powerful approach to plant new ideas, verging on the idea  of ‘facipulation’ (facilitation-manipulation).

Beyond a given development intervention, a way to make its legacy live on is to plug those ideas onto networks that will keep exploring the learning capital of that intervention.

What is the value proposition of all this to donors? Cynically perhaps the innovativeness of working in those ways; much more importantly, the promise of sustainable results – better guaranteed through embedded, local work. The use of metaphors can be enlightening here, in the sense that it gives different ideas: what can you invest in projects and short term relationships? e.g. gardening for instance planting new initiatives in an existing soil/bed or putting fertilizer in existing plants…

Interesting links related to the discussion:

• Documenting change: an introduction to process documentation (a document explaining the theory and IRC’s practice around process documentation over seven years;
• Getting to social (blog post by Harold Jarche, February 2012);
• Mid-term assessment blues (blog post by Patrick Moriarty from IRC on a flexible learning-focused M&E approach in the Triple-S project.
• Results from this IKM-E discussion on the wiki;

This slideshow requires JavaScript.

On participation and engagement:

Sustainable, effective development interventions are informed by careful and consistent participation and engagement, recognising the value of multiple knowledges and cherishing respect for different perspectives, as part of a general scientific curiosity and humility as to what we know about what works and what doesn’t, in development and generally.

The second strand we explored on day 1 was participation and engagement with multiple knowledges. This boils down to the question: how to value different knowledges and particularly ‘local knowledge’, bearing in mind that local knowledge is not a synonym to Southern knowledge because we all possess some local knowledge, regardless of where we live.

A sound approach to valuing participation and engagement is to recognise the importance of creating the bigger picture in our complex social initiatives. The concept of cognitive dissonance is particularly helpful here: As communities of people we (should) value some of our practices and document them so that we create and recognise a bigger collective whole but then we have to realise that something might be missing from that collective narrative, that we might have to play the devil’s advocate to challenge our thinking – this is the ‘cognitive dissonance at play – and it is more likely to happen by bringing external views or alternative points of view, but also e.g. by using facilitation methods that put the onus on participants to adopt a different perspective (e.g. DeBono’s six-thinking hats). Development work has to include cognitive dissonance to create better conditions to combine different knowledges.

Participation and engagement is also conditioned by power play of course, but also by our comfort zones; e.g. as raised in a recent KM4Dev discussion, we are usually not keen on hiring people with different perspectives, who might challenge the current situation. We also don’t like the frictions that come about with bringing different people to the table: we don’t like to rediscuss the obvious, we don’t like to renegotiate meaning but that is exactly what is necessary for multiple knowledges to create a trustworthy space. The tension between deepening the field and expanding it laterally with new people is an important tension, in workshops as in development initiatives.

We may also have to adopt different approaches and responses in front of a multi-faceted adversity for change: Some people need to be aware of the gaps; others are aware but not willing because they don’t see the value or feel threatened by inviting multiple perspectives; others still are also aware and don’t feel threatened but need to be challenged beyond their comfort zone. Some will need ideas, others principles, others yet actions.

At any rate, inviting participation calls for inviting related accountability mechanisms. Accountability (which will come back on the menu on day 2) is not just towards donors but also towards the people we invite participation, or we run the risk of ‘tokenising’ participation (pretending that we are participatory but not changing the decision-making process). When one interviews a person, they  have to make sure that what they are transcribing faithfully reflects what the interviewee said. So with participation, participants have to be made aware that their inputs are valued and reflected in the wider engagement process, not just interpreted as ‘a tick on the participatory box’.

Participation and engagement opens up the reflective and conversation space to collective engagement, which is a very complex process as highlighted in Charles Dhewa’s model of collective sense-making in his work on traducture. A prerequisite in that collective engagement and sense-making is the self-confidence that you develop in your own knowledge. For ‘local knowledge’, this is a very difficult requirement, not least because even in their own context, proponents of local knowledge might be discriminated and rejected by others for the lack of rigor they display.

So how to invite participation and engagement?

Values and principles are guiding pointers. Respect (for oneself and others) and humility or curiosity are great lights on the complex path to collective sense-making (as illustrated by Charles Dhewa’s graph below). They guide our initiatives by preserving a learning attitude among each and every one of us. Perhaps development should grow up to be more about  ‘ignorance management’, an insatiable thirst for new knowledge. The humility about our own ignorance and curiosity might lead us to unravel ever sharper questions, on the dialectical and critical thinking path, rather than off-the-shelf (and upscaling-friendly) answers – which we tend to favour in the development sector. The importance here is the development of shared meaning.

A collective sensemaking framework (by Charles Dhewa)

As highlighted in the previous conversation, not every step of a development initiative requires multi-stakeholder participation, but a useful principle to invite participation and engagement is iteration. By revisiting at regular intervals the assumptions we have, together with various actors, we can perhaps more easily ensure that some key elements from the bigger picture are not thrown away in the process. This comes back to the idea of assessing the level of complexity we are facing, which is certainly affected by a) the amount of people that are affected by (or have a crucial stake in) the initiative at hand and b) the degree of inter-relatedness of the changes that affect them and connect them.

Iteration and multi-stakeholder engagement and participation are at the heart of the ‘inception phase’ approach. This is only one model for participation and un-linear planning:

• On one end of the spectrum, a fully planned process with no room for (meaningful) engagement because the pathway traced is not up for renegotiation;
• Somewhere in the middle, a project approach using an inception period to renegotiate the objectives, reassess the context, understand the motivations of the stake-holders;
• At the other end of the spectrum, a totally emergent approach where one keeps organising new processes as they show up along the way, renegotiating with a variety of actors.

Seed money helps here for ‘safe-fail’ approaches, to try things out and draw early lessons and perhaps then properly budget for activities that expand that seed initiative. Examples from the corporate sector also give away some interesting pointers and approaches (see Mintzberg’s books and the strategy safari under ‘related resources’). The blog post by Robert Chambers on ‘whose paradigm’

”]counts and his stark comparison between a positivist and adaptive pluralism perspectives are also very helpful resources to map out the issues we are facing here.

At any rate, and this can never be emphasised enough, in complex environments – as is the case in development work more often than not – a solid context analysis is in order if one is to hope for any valuable result, in the short or long run.

Related resources:

• Whose paradigm counts (blog post on Aid on the edge);
• Strategy safari – a guided tour through the wilds of strategic management;
• Henry Mintzberg’s books about management and other proofs from the corporate sector about the fact that top-down / command & control approaches do not work;
• Jerome Brumer’s two modes of thought in psychology.
• Results of this discussion on the IKM wiki

These have been our musings on day 1, perhaps not ground-breaking observations but pieces of an IKM-E collage that brings together important pointers to the legacy of IKM-Emergent. Day 2 is promising…

Related blog posts:

• At the IKM Table (2): individual agency vs. organisational remit, accountability and impact pathways for the future of IKM-Emergent
• What is good in a project?
• Communication, KM, monitoring, learning – The happy families of engagement
• IKM emergency
• IKM-convergent? Annual programme meeting, Wageningen, day 1
• The Giraffe
• What the heck is knowledge anyways: from commodity to capacity and insights
• Go organic, go civic! #KMalreadyHappensAnyways
• Seeking success, finding failures: work in development

That PD thing again

And here we go again! Second major process documentation workshop after the Lodz workshop in July 2007, a workshop where IRC and partners tried not so much to settle a definition for the concept as to allow participants to play around with three media: text, video, photography. This time, the workshop is sponsored by the WASHCost project and includes participants from other background (see my latest blog post about this).

On this first day, we have covered the why (aims of process documentation), the principles of P.D., the basics of interviewing and the initial steps into a process documentation plan.

First observations from the field – more like a hotel room if you ask me:

• A definition may emerge. The exercise about prioritising the aims – from a list of over 20 aims that our facilitator Peter McIntyre collected from five different projects using process documentation – went amazingly well and placed a few objectives high up – does this mean an agreement comes naturally or certain messages have been crafted well enough or repeated often enough to influence our participants? Either way, this is a very encouraging result.
• The lines between communication and monitoring are still very much bordering process documentation work. As my colleague Nick Dickinson put it, process documentation helps identify interesting areas to document – leading to crafting communication messages – and it helps again at the end of the loop to monitor how stakeholders have responded to our interactions.
• Principles of process documentation are emerging, and the real of information integrity is getting unpacked: one needs to check that outputs are correct (either directly with the stakeholders concerned or at least in the team if the output does not make it publicly); it is clear that some of your partners will not accept your (partial) vision; inside the team, constructive criticism should be encouraged: if the process documentation specialist is roughly a 75% team member role, s/he should also play a 25% external ‘ journalist’ role where s/he feels free to provide constructive feedback.
• The importance of short feedback loops is essential! Regardless of the final process documentation outputs, key insights from process documentation work should quickly inform the team operating. This is part of the constructive feedback mentioned above.
• The name, however unsexy it is, has made it in the common language – granted, in certain circles only. The India team didn’t want to change the term ‘process documentation’ because it is known by their learning alliance partners and changing names would create more confusion.
• In spite of all these very encouraging signs, it is remarkable to see that when it comes to process documentation planning (perhaps an oxymoron?), most teams quickly jump on outputs/products, reinforcing the quick consumption culture of the development sector. Slow food (read: learning) is not on the menu quite yet. Adopting a learning culture is not yet an easy reality to implement. According to one of the external (non WASHCost) participants (in charge of communication activities in her organisation), this kind of process documentation activities was not in the agenda because it takes too much time. Ooh, that battle is far from being won, but hey, one starts somewhere… and still improvements are noticeable.

Process documentation as a reflection on and of reality

Anyway, with an approach (process documentation) that’s increasingly meaningful, I personally think that it’s never been as good a moment to name this thing differently. No one has come up with alternative names yet, in spite of our repeated urge to devise new names.

My personal brainstorm outcomes: process enquirers (booo), rapid reporters (duh), effective(ness) detectives, action investigators, agents provocateurs (revealing the invisible), change rangers (scouting for and identifying trails), trail hunters… the list could go on and on I guess. It would be fun doing an exercise about the kind of figure (hero, character or even animal) that process documentation specialists think about when considering their function?

At any rate, of all three key PD actions (observe, analyse, disseminate), I would say observe/intervene is the key one. And for that reason, detective or ranger sounds like the closest match.

I can’t wait for tomorrow… see what our productive detectives come up with…

Capitalising on process documentation – and changing names please!

Next week, a group of 4-5 of us from IRC will be in Accra with all country teams from the WASHCost project to work together on ‘process documentation’. Starting from a training workshop, the idea behind this workshop has moved forward to become a kind of orientation and training workshop.

The objectives are manifold: a) agree on a working definition of process documentation (what the heck is it?) b) train all staff about the use of photography, video, interviews etc. and c) decide what we are going to document in WASHCost, starting from the ‘hypothesis of change’ of the project.

There’s a few very interesting sides to this workshop:

• It will be the largest workshop dedicated to process documentation since the one we organised in Poland in July 2007 – which resulted in a very nice blog.
• It will not only be about the practice but also a little bit about the theory of process documentation which really needs some agreement. That’s really one of the problems with new trensd and buzz words: everyone uses them in a slightly different way. In Accra, we hope to come up with a common understanding.
• Leading from that, we should be able to capitalise a bit on all kinds of experiences with process documentation from the RiPPLE project, WASPA Asia, EMPOWERS and SWITCH. We have accumulated quite some ideas and insights from this ‘soft monitoring’ work and IRC is dedicated to documenting process documentation (multiple loop learning here 😉 this year, perhaps to make a toolbox, some case studies, many examples of outputs available…
• We hope to come up with a better name for ‘process documentation’ and particularly for the person in charge. ‘Process documentalist’ seems to refer to a very scientific entomologist studying hot air, so it’s time to jazz this up a bit and end embarrassment when mentioning the PD words…
• Finally, some partners from CREPA, WaterAid and the resource centre network in Ghana will also participate to the workshop. They should help challenging our ideas and ways of working, and hopefully they will also spread the word about this process documentation work and perhaps take it up in their own line of work.

Another interesting aspect from this work is that it should very nicely complement the upcoming publication about impact assessment planned for later this year.

What I personally hope is to find a place to park process documentation in the hall of concepts that we have produced in the last few years – and perhaps to sound out colleagues and partners on their take of process documentation. I still think that PD is what essentially what intelligent monitoring should cover as well, but since donors are following different frameworks of reference for monitoring, it is no wonder that process documentation is still an undefined and ill-accepted practice among donors. Perhaps the capitalisation work around process documentation will help change this perspective. And perhaps a sexier name would…

Process documentation – Sandbox to influence donors?

I am currently involved in some process documentation work for this project in Ethiopia, RiPPLE (www.rippleethiopia.org) and back in my organisation there is a lot of discussion as to what process documentation is or is not.

In order to underestand what it is, let’s rewind here a bit: the term process documentation refers to a very old habit: reflecting about events and happenings and understanding the deeper causes behind.

I recently wrote an information sheet on process documentation for that same project and here is my hunch at explaining it: “We all take time to think about what is going on, to reflect about the factors, be it people, activities, context or anything else, influencing our actions in a positive or negative way. Process documentation encourages such reflective moments to become conscious, regular and structured so as to capture deeper insights and patterns that emerge in our work.“

Process documentation was first carried out as such in the EMPOWERS project (www.empowers.info), with help of one full time ‘process documenter’ in each of the three project countries. Later on, process documentation (process doc or PD as it is sometimes referred to) has been included in a number of projects we developed with partners.

A training course in Lodz (Poland), in 2007, tried to explain a bit better what PD is and how it works. The workshop was highly stimulating and fun (check http://processdocumentation.wordpress.com/ for more info), but frankly it didn’t unpack the concept of process documentation for us participants.

So we are now at a point when the concept is developing in many different directions – and that is good, following a divergence-convergence evolution pattern here? – but it is interesting to see the various interpretations of it.

My take on process documentation at this point is really that it is everything that doesn’t belong (yet, hopefully) to either monitoring nor communication. Again quoting the information sheet I prepared here:

“Much like monitoring, process documentation consists in collecting information and analysing it to understand the underlying factors that enable or hamper work. Contrary to traditional monitoring, however, process documentation is mostly based on qualitative data collection methods and it is totally focused on learning, as opposed to accountability.

Much like (some) communication activities, process documentation uses interviews, storytelling, blogs and other ways to collect information. However, contrary to communication, not all information collected through process documentation is actually published and disseminated because it holds a lot of sensitive information and potentially political value. Some is, some is not.

 

So process documentation is a sort of informal, narrative-based monitoring exercise that collects some information that can be partly used for outreach and communication objectives. In an ideal world, where learning would be central and everybody would be ready to accept the consequences of creating a learning environment, process documentation would overlap completely with monitoring (i.e. qualitative monitoring methods would be accepted just as well as quantitative ones) and a lot of outputs from it would be derived in communication outputs.

But we don’t live in an ideal world and so process documentation still is a middle way between these two major areas of development work. But as much as innovation happens at the edges, what process documentation holds is actually the crux of the matter that makes or breaks a project.“

Actually this is from the beta version of the information sheet and I doubt this will make it to the final version, due to the informal language.

It is that formal-informal barrier that we should try to break, because that is what creates hurdles to communicate better from local communities all the way to donors. And for that particular aspect, I’m not sure process documentation itself is the best approach, because it is more observing than taking an active part in the debate (or is it not?).

Anyway, process documentation is still useful practice ground to analyse deeper patterns and structure learning better, something that hopefully will lead, further down the line, to donors adapting their requirements on monitoring and their communication expectations…

How do you do? How do you do-cument?

An eternal struggle in Knowledge Management remains the balancing act between actually doing things: implementing a project, organising an event, writing a case study and documenting those: reflecting about them, collecting insights, analysing them, taking a neutral role to observe what is going on.

At an individual level, it is hard enough to find the time to document a lot when taken prisoner of the ‘do-mode / do more’: work and other pressures usually have the upper hand on the struggle and the Do wins, leaving the familiar impression that it would be excellent to document what is going on against the risk of being confronted with the same situation in the future. When it comes to an organisation, roles may be split: some may be documenting the work of others while a core of ‘doers’ are actually carrying out a number of activities. Both roles are important – but it still remains hard to justify documentation against delivery.

When one adds the issue of reporting and monitoring – which for some may be felt as imposed documentation – the need to document for real learning purposes becomes even more difficult to justify and make happen. And yet it is in this documentation that the long term patterns that come back and haunt our work are to be discerned. It is in that (light) documentation that one learns how to improve the way s/he carries out his/her work.

In a number of projects where I am involved, we carry out ‘process documentation’ work, which is a very murky concept at the junction of communication and monitoring, addressing sensitive issues but not necessarily allowing to get them in the public. This is the kind of documentation that needs to inform the ‘do’ mode. While still making sense of this concept and its potential implications, one thing is clear: our motto should be “do less, document more”. It is in the documenting that one understands how one is doing and how to improve it next time. The more we do, the less we can document – until we have convinced donors that it is essential to dedicate more funding on documentation of issues, in particular processes.

In the development sector, in the absence of excellent examples, the journey has to temporarily become the end rather than the means so that we find the best journey possible to reach the ultimate end. And so among so much ado, we will have to document…