Wednesday, 6 November 2013

Soon to come: QCA conditions

We are on the last straight line of two -now that I think of it, three- processes:

  • Finalising the scoping - i.e. the gathering and sorting of evaluation reports in a systematic manner and preparing a scoping report explaining how we have done it. The good news is, there are more evaluations - especially published ones - than we had expected to find! The bad news is, we cannot conduct qualitative comparative analysis (QCA - see the posts below for a quick introduction) on all of them. Not because QCA would not allow for it - in the opposite, maximum openness is a key feature and the beauty of QCA - but because going through all reports would take much more time and resources than what we can afford. We must sample! Watch this space. 
  • Defining the conditions that -we (and the literature we have reviewed) suspect- contribute to making an evaluation of a VAWG-related intervention useful (or not), i.e. that increase or diminish positive evaluation effects. Our tentative conditions have been reviewed by the Review Reference Group, whose members have come up with useful questions and comments. We are refining the conditions now; soon they will be tested 'in earnest' in a first round of coding.
  • Recruiting and instructing coders - a highly qualified team of five (we'll ask them whether they would like to be presented on this blog; if they do, you'll read more about them) has been brought together and awaits our detailed definitions and directions for coding.
We will share more on these points (including our set of conditions) on this blog near the end of this month, when this busy phase of scoping, sorting, sampling, refining our model and instructing the coders will be over. An intensive and exhilarating process!

3 comments:

  1. Okay, so "usefullness" will be the common outcome of interest. It will be interesting, and possibly helpful in its own right, to see how you develop a reliable means of rating evaluation usefullness. Will you take into account the historic usefullness of the evaluation e.g. to its original clients, or just focus on its current usefullness to some specified clients e.g. DFID EvD?

    ReplyDelete
  2. Likewise, on judging an "evaluation of a VAWG-related intervention useful (or not), i.e. that increase or diminish positive evaluation effects" I hope you will give a nod to "feminist evaluation" and/or "transformatory participatory evaluation" that tend to measure evaluation processes in terms of how they contribute to empowerment of key stakeholders and contribute to transformative change.

    ReplyDelete
  3. I am greatly anticipating your final report - when will you have it ready? I have been doing a literature review on evaluation methods and 'evidence' related to VAWG/GBV programs seeking to shift attitudes, norms and behaviours and am astonished at the predominance of experimental design as what counts as 'evidence' of what works. Very interested to know what you have learned about evaluation 'effects' in this regard.

    ReplyDelete