Another stock-taking post on the collection of network M&E resources (Photo credits: Hooverdust)
This time the topic is monitoring and evaluation (M&E) for networks, among others because there are a number of networks that I am involved in which will need to develop a solid M&E framework for themselves and for their respective donors so this post could help come up with a better approach. And, who knows, perhaps you will also find something useful in there. If this is all rubbish, please put me out of my misery and help me read some quality references on the topic, ok?
When it comes to M&E of networks, documents are a lot more scattered than for the capacity development stock-taking post I wrote earlier. And to spice things up, on Google, there is a hell of a lot of misleading resources pointing to LAN/WAN network monitoring – clearly the web is still the stronghold of a self-serving (IT) community.
Fair enough! But luckily there are also relevant resources among my documents, of which I would like to mention:
Guides, tools and methods for evaluating networks (direct link to a Word document)
(Amy Etherington – 2005)
As the title indicates, this paper focuses on evaluation rather than monitoring of networks – as a means for networks to remain relevant and adapt if need be. Three major considerations are taken into account here:
- measuring intangible assets (related to characteristics of networks such as social arrangements, adding value, creating forums for social exchange and joint opportunities);
- issues of attribution (linked to issues of geographic and asynchronous complexity of networks, joint execution of activities, broad and long term goals of networks);
- looking at internal processes: the very nature of networks renders internal processes – of mobilisation, interlinking, value-adding – very interesting. The further effects of the network on each individual member are also useful to look into.
And then follows a selection of nine evaluation methods (all dating from 1999 to 2005 though), very well documented, including checklists of questions, tables with dimensions of networks, interesting (or sometimes scary) models, innumerable steps referring to various maturity stages of communities. This seems one of the most relevant references to find at least practical methods to tackle network M&E.
Evaluating International Social Change Networks: A Conceptual Framework for a Participatory Approach (PDF)
(Ricardo Wilson-Grau and Martha Nuñez – 2006)
Among the most influential authors on the topic of M&E and networks, Wilson-Grau and Nuñez have been writing a lot of documents referred to in other papers mentioned here. This paper – which also focuses on the evaluation of networks – introduces the 8 or so functions that networks perform and considers four qualities and three operational dimensions. The result is a table of 56 criteria – shaped as questions – which ought to be answered by members of the network – with a careful eye for justification behind each criterion, because each network is different. The authors continue with the four types of achievements one can hope for social change networks: operational outputs, organic outcomes, political outcomes (judged as most useful by the authors themselves) and impact. Again the table is of great help and this document is a useful introduction to the author’s body of work.
A Strategic Evaluation of IDRC-Support to Networks (Word)
(Sarah Earl – 2004)
Epitomising the long term experience of the Canadian International Development Research Centre (IDRC) with monitoring and evaluation of networks, Sarah Earl presents, in this seven-page briefing note, a questioning process to evaluate the function of IDRC in supporting networks. In doing so, she stresses a series of questions pertaining to the coordination, sustainability and intended results / development outcomes of networks. She further explains the methodology used (literature review, key informant interviews and electronic survey of network coordinators, lesson learning sessions leading to writing stories from IDRC staff). This paper can be useful for actually setting up a methodology to collect evidence about the functioning of a network.
Network evaluation paper (Word).
(June Holley – 2007)
June Holley has been working for over 20 years on economic networks. This five-page paper introduces a method that focuses on network maps and metrics, network indicators and outcomes. The paper suggests using scores and looking at awareness (of the network as a whole), influence, connectors, integration, resilience, diversity and core/periphery.
Network mapping and core-periphery (Image credits: Ross Dawson)
There are more (*) papers specifically focused on networks and their evaluation but I found them less relevant, often mostly because they are a bit dated.
Of course there are many other references on monitoring and evaluation in publications and resource sites about networks. Here is another, shorter, selection:
- The M&E section (pp. 87-91) of ‘Work the net’, GTZ’s guide to managing formal networks. This guide from 2006 is also available in other languages on the GTZ website – find the PDF documents here: French version – Spanish version – Portuguese version).
- In the water sector, CAP-Net has been working quite a bit on monitoring networks. On this page they provide an overview of several network management tools, including their own network monitoring strategy (2008).
- From ECDPM’s paper on mapping of approaches towards M&E of capacity and capacity development (2006), a number of things are mentioned on the monitoring of networks too, particularly
While on the topic of network M&E and its link with the specific monitoring of knowledge management, I would like to point to the summary of a discussion that took place in 2008 on the KM4Dev mailing list on the topic of M&E of KM: http://wiki.km4dev.org/wiki/index.php/Impact_and_M%26E_of_KM. This topic will probably remain interesting. It has been explored various times on the KM4DEV mailing list, it was recently touched upon in the francophone KM4DEV CoP SA-GE and it is likely to reappear as a topic of choice in 2010 on various platforms, not least because IKM-Emergent is planning to work more on this issue after having released the first of two commissioned papers on M&E of KM (this working paper on monitoring and evaluation of knowledge was written by Serafin Talisayon). I will certainly report about this in the coming weeks / months.
As ever with this series of stock-taking posts, I will try and keep this overview updated with any other interesting resource I get my hands on. So feel free to enlighten me with additional resources that go deeper, provide a lot of synthetic clarity or provide a refreshing perspective on the topic of network monitoring. What has worked for you in your work with networks? What have you found useful ways to measure their effectiveness and other dimensions? What would be your words of caution when assessing networks?
Networks are here to stay for a while so this discussion goes on…
(*)
I came across a number of other papers that all have something to say but are a bit out of date and I decided not to reference them here.
- Do networks really work? A framework for evaluating public-sector organizational networks (PDF) – (Keith G. Provan and H. Brinton Milward – 2001)
- Measuring while you manage: Planning, monitoring and evaluating knowledge networks (PDF) (Heather Creech – 2001)
- Measuring the Immeasurable Planning, Monitoring and Evaluation of Networks (PDF) (Oxfam/Novib – 1999)
- Organizational Network Analysis as a Tool for Program Evaluation (Merrill Eisenberg and Nancy Swanson – 1996)
Related posts:
- Other stock-taking posts: Assessing, measuring, monitoring knowledge (and KM) – Storytelling – Learning cycle basics and more – Capacity development
