5.3 An alternative classification
of evaluation approaches
Hansen (2005) sets out a
different and more detailed approach to classifying approaches to evaluation,
summarised in Table 5.1.
Table 5.1 A typology of
evaluation models
.bmp)
•
Results models, also referred to as ‘summative
evaluation’, focus on the outcomes of policies, practices and programmes.
Hansen distinguishes between two versions of the results approach. The
goal attainment model examines outcomes in relation to the original goals and
intentions. The effects approach is broader and seeks to examine the
full range of effects brought about by the practices, polices or programmes.
•
Economic models have a particular focus on costs
in relation to outcomes.
•
Process models focus on the processes by which
results are achieved and are diagnostic in orientation. They might involve a
continuous monitoring process or, in many cases, a diagnostic process when
goals are not achieved.
•
Systems approaches focus more holistically on
the operation of the whole system. This might involve a local assessment of the
system in relation to original objectives or by comparison with the operation
of similar systems in other settings or organisations.
•
Actor models draw criteria for assessment from
key actors. A client-oriented model draws criteria from the client or clients
who commissioned the evaluation. A stakeholder model draws assessment criteria
from key stakeholders. A peer review model considers quality in relation to
professional standards, often through peer assessment.
•
Programme theory models look at the underlying
theory behind a practice, policy or programme and seek to assess whether that
theory is borne out. For example, an organisation that establishes an employee
engagement programme based on a theory concerned with linkages between employee
engagement and customer service may set out to evaluate whether those linkages
work in practice.
Hansen (2005) suggests that the
choice of evaluation approach can usefully be guided by the answers to three
questions:
1. What
is the purpose of evaluation? Purposes that focus on control are more likely
to be served by result models and the quantitative measurement of effects,
driven by criteria derived from strategic goals. Meanwhile, purposes that focus
on learning
will be better served by process approaches with assessment criteria derived
from stakeholder requests.
2. What is possible? Not all objects of
evaluation allow for all forms of evaluation. For example the objectives of an
SHRM policy may be unclear or contested, making outcome evaluation difficult.
Take the example of a set of diversity policies. One group of senior managers
may be entrenched in their view that the objective is to ‘comply with the law
at minimum cost’ while another might take the view that the purpose is to
‘maximise the availability of talent to the organisation’. To give another
example, a programme theory approach requires that there is a well-articulated
theory underlying the practices to be evaluated.
3. What is the problem to be solved by
the evaluated practice policy or programme? In particular programme theory
approaches may be unrealistic when problems are highly complex and interwoven
with multiple systems and processes.
Other factors
likely to affect the design of evaluation include:
•
the process of negotiation with key stakeholders
in the evaluation
•
the forms of evaluation that are seen as
legitimate and appropriate in the organisation
•
the response repertoire of evaluation sponsors
and evaluators.
No comments:
Post a Comment