Monday, February 11, 2013

5.3 An alternative classification of evaluation approaches


5.3 An alternative classification of evaluation approaches
Hansen (2005) sets out a different and more detailed approach to classifying approaches to evaluation, summarised in Table 5.1.

Table 5.1 A typology of evaluation models

 (Source: adapted from Hansen, 2005, p. 449)
Hansen (2005) distinguishes between six broad approaches to evaluation:
        Results models, also referred to as ‘summative evaluation’, focus on the outcomes of policies, practices and programmes. Hansen distinguishes between two versions of the results approach. The goal attainment model examines outcomes in relation to the original goals and intentions. The effects approach is broader and seeks to examine the full range of effects brought about by the practices, polices or programmes. 

        Economic models have a particular focus on costs in relation to outcomes. 

        Process models focus on the processes by which results are achieved and are diagnostic in orientation. They might involve a continuous monitoring process or, in many cases, a diagnostic process when goals are not achieved. 

        Systems approaches focus more holistically on the operation of the whole system. This might involve a local assessment of the system in relation to original objectives or by comparison with the operation of similar systems in other settings or organisations.
 
        Actor models draw criteria for assessment from key actors. A client-oriented model draws criteria from the client or clients who commissioned the evaluation. A stakeholder model draws assessment criteria from key stakeholders. A peer review model considers quality in relation to professional standards, often through peer assessment.
 
        Programme theory models look at the underlying theory behind a practice, policy or programme and seek to assess whether that theory is borne out. For example, an organisation that establishes an employee engagement programme based on a theory concerned with linkages between employee engagement and customer service may set out to evaluate whether those linkages work in practice.

Hansen (2005) suggests that the choice of evaluation approach can usefully be guided by the answers to three questions:
1.            What is the purpose of evaluation? Purposes that focus on control are more likely to be served by result models and the quantitative measurement of effects, driven by criteria derived from strategic goals. Meanwhile, purposes that focus on learning will be better served by process approaches with assessment criteria derived from stakeholder requests.
2.            What is possible? Not all objects of evaluation allow for all forms of evaluation. For example the objectives of an SHRM policy may be unclear or contested, making outcome evaluation difficult. Take the example of a set of diversity policies. One group of senior managers may be entrenched in their view that the objective is to ‘comply with the law at minimum cost’ while another might take the view that the purpose is to ‘maximise the availability of talent to the organisation’. To give another example, a programme theory approach requires that there is a well-articulated theory underlying the practices to be evaluated.
3.          What is the problem to be solved by the evaluated practice policy or programme? In particular programme theory approaches may be unrealistic when problems are highly complex and interwoven with multiple systems and processes.
Other factors likely to affect the design of evaluation include:
        the process of negotiation with key stakeholders in the evaluation
        the forms of evaluation that are seen as legitimate and appropriate in the organisation
        the response repertoire of evaluation sponsors and evaluators.

No comments:

Post a Comment