Good strategy, by definition, includes the outcomes an organization will hold itself accountable for achieving. The better we can measure performance, the better we can evaluate our strategy and adjust or adapt it to achieve the outcomes we want for the people or causes we serve.
Many factors go into the success of an organization. Talent, funding, focus-– these and many other key components are required for an NFP to achieve its goals. An organization can be working diligently towards its goals without ever reaching them, or spend valuable resources in unnecessarily, if its leaders are not able to gauge the nature and scope of its success. Having useful and appropriate measurement and evaluation in place ensures one stays on track and maintains the tightest connection possible between resources and impact.
Three ways to measure impact:
Measurement is the most straightforward. Just like a carpenter with a ruler and pencil, measurement refers to the process of tracking progress on some kind of numerical or subjective scale. Total income, number of employees, market share, number of people served and annual growth are all measurements. While care and consideration are needed to set the goal, actually determining whether it has been met is not a complicated process – although determining what to measure can be an important strategic decision.
Assessment is more involved. Employee questionnaires, client testimonials, and feedback from other organizations are all forms of assessment. Most assessment in the not-for-profit sector involves manipulation or correlation of data. Examples might include an analysis of the number of dollars required to reach a given goal, a comparison of the relative impact of different employee assignments, or a comparison of outcomes in differently served populations. While measurement serves to determine the raw numbers, assessment is the process by which those numbers are compared against one another.
Evaluation is a more holistic and contingent approach than assessment or measurement. Experts in organizational development and epistemology have identified and defined several different forms of evaluation. In his article ‘Measurement, Assessment, and Evaluation in Education’, education theory professor Dr. Robert Kizlik uses the example of temperature inside a classroom to illustrate the difference between the three ideas: Using a thermometer to determine the temperature of a room is measurement, while polling students about whether they are comfortable or uncomfortable in different temperatures is assessment, and using that data set to determine an optimal temperature is evaluation. Evaluation is the process wherein information can be weighed, valued, and acted upon.
Experts have identified various styles and types of evaluation. The three most widely-acknowledged are summative, formative, and developmental evaluation.
Summative evaluation is perhaps the most commonly used form. It focuses on the end result of a particular intervention or project. Once an organization has executed a plan, a summative evaluation can be used to compare results against the plan’s goals to determine whether they were met. When evaluating a project against a theory of change (a strategy), a sum-total look at all of the intended and unintended impacts and consequences of the project can show whether it has been a success, whether it accomplished the goals of a given strategy or Theory of Change, and whether that strategy’s aims were good ones.
Formative evaluation is a more process-oriented exercise that aims to improve a project while it is still underway. After a course of action has been determined, unforeseen circumstances or outside factors might leave a project in an awkward or even obsolete position. By using formative evaluation, organizations can compare an in-progress project with its goals to decide if a course correction is necessary.
Twitter is a great example of formative evaluation in action. Twitter started life as a video-blogging platform called Odeo. Partway through the creation of Odeo’s software, however, internal traffic data showed that the most popular posts were not video blogs but short snippets of text. Odeo decided to change their strategy based on this assessment, and became Twitter, which has been a massive success. Formative evaluation can take advantage of an in-progress undertaking to measure efficacy and, in some cases, to better-align a project with its designer’s goals.
Developmental evaluation is most valuable when working in situations of high complexity or in the early stages of new social innovations. As Tamarack notes:
Programs that do not expect to create a standardized model, but rather anticipate the need to be continuously evolving are examples where Developmental Evaluation is particularly useful. The emphasis in Developmental Evaluation is on documenting decisions and formalizing the learning and the knowledge bases that drive decisions.
Similar to formative evaluation in its emphasis on process, developmental evaluation allows for continual adjustment and reorganization. Rather than an ex post facto summary of achievements like summative evaluation or a midway-point reassessment like formative evaluation, developmental evaluation entails a project with goals and processes that are always in flux. Rather than a discrete section of a project, developmental evaluation is an intrinsic part of it, an outlook and style of action that acknowledges a moving target instead of a fixed goal.
Careful evaluation is absolutely crucial to a strategy’s success and to allocating resources where they will have the most effect. Formative evaluation can help to steer the strategic plan towards the organization’s goals, while summative evaluation can confirm that those goals have been reached. Developmental evaluation, where appropriate, can ensure that those goals themselves are the right ones for an organization’s vision and values.
Investing in thoughtful and rigorous measurement and evaluation is one of the best strategic decisions a not-for-profit can make.