When Can You Conduct an Evaluation?
Most programs follow a cycle that involves five basic stages:
You can use evaluation at any of these five stages, but each stage calls for a different type of evaluation. |
|
Before a Programs Starts (Needs Assessment)
A need is the gap between how things are and how we would like them to be. A needs assessment helps you to identify and understand this gap. It is an important first step in planning a new program or updating a long-standing program.
A needs assessment might ask questions like:
You may have observed some needs in your community. However, it is important to talk to others about what they see as a need and also what kinds of programs might address this need. It is especially important that you speak with the people you want to participate in your program, whether it is youth, families, or Elders. Do they think there is a need for this kind of programming? If so, what would they like the program to look like?
A needs assessment might ask questions like:
- What kind of on the land activities are people in our community asking for?
- Who needs on the land programs?
- What on the land programs are already available?
- What kinds of on the land programs have proven to be effective?
- Are there sufficient resources to develop a good on the land program?
You may have observed some needs in your community. However, it is important to talk to others about what they see as a need and also what kinds of programs might address this need. It is especially important that you speak with the people you want to participate in your program, whether it is youth, families, or Elders. Do they think there is a need for this kind of programming? If so, what would they like the program to look like?
When You're Designing a Program (Developmental Evaluation)
There are lots of things to think about when you are designing a program. Here we will focus on how evaluation is a part of program design. This is sometimes referred to as developmental evaluation.
As you design your program, it is helpful to think about what you are trying to accomplish. What is the goal of the program? Perhaps you’re concerned, for example, that young people don’t feel as comfortable on the land. One way for them to become more comfortable is to spend more time on the land. So the goal of your program may be: “Youth will spend more time on the land.”
Once you have identified your goal, think about how you will measure progress towards reaching that goal. This is where objectives are helpful. Objectives describe specific, measurable actions that will help you determine if your program is achieving the desired results. For example, if your goal is to have youth spend more time on the land, your objectives might include:
As you design your program, it is helpful to think about what you are trying to accomplish. What is the goal of the program? Perhaps you’re concerned, for example, that young people don’t feel as comfortable on the land. One way for them to become more comfortable is to spend more time on the land. So the goal of your program may be: “Youth will spend more time on the land.”
Once you have identified your goal, think about how you will measure progress towards reaching that goal. This is where objectives are helpful. Objectives describe specific, measurable actions that will help you determine if your program is achieving the desired results. For example, if your goal is to have youth spend more time on the land, your objectives might include:
- Host four youth on the land camps every year.
- Every youth in the community attends at least one camp per year.
- Youth in the community feel their land skills have improved.
When a Program is New, or When an Older Program Needs a Refresh (Formative Evaluation)
Evaluating a program that is still in the early stages is often called formative evaluation. A formative evaluation asks questions like:
A formative evaluation is useful for highlighting major gaps in the content of the program (what you are doing) or program delivery (how you are doing it). It can also be useful for identifying ways to make improvements. Once you’ve identified the gaps or areas for improvement, you can generate possible solutions.
A formative evaluation is a good tool to use during a pilot project. However, it can also be used in more established programs as a kind of “check-in.”
- Is the program being delivered as planned? If not, why?
- What parts are working well, for whom, and why?
- What parts aren’t working well, for whom, and why?
A formative evaluation is useful for highlighting major gaps in the content of the program (what you are doing) or program delivery (how you are doing it). It can also be useful for identifying ways to make improvements. Once you’ve identified the gaps or areas for improvement, you can generate possible solutions.
A formative evaluation is a good tool to use during a pilot project. However, it can also be used in more established programs as a kind of “check-in.”
When a Program is Well-Established or Coming to an End (Impact Evaluation)
The evaluation of a stable, on-going program, or a program that is nearing the end of its life, is often referred to as an impact evaluation because it is used to identify achievements or better understand the effectiveness of a program. An impact evaluation asks questions like:
An evaluation focused on impact or program outcomes is concerned with assessing how your program has affected participants or the community. Impact evaluations usually consider different times frames, such as immediate impacts, short-term impacts (e.g. 1-6 months), or long term (6+ months). Ideally, an impact evaluation will consider both the intended and unintended outcomes of your program that are positive and negative.
This is one of the more meaningful forms of evaluations, but it is also one of the hardest types to do well. It can be expensive and time-consuming, and can only be done if you have enough of the right kinds of information.
An impact evaluation may not directly affect the current program, but it is helpful in deciding if a program should be repeated, expanded, revised, copied, scaled back, or cut.
- Does the program improve the lives of participants?
- Does the program have unanticipated outcomes, either positive or negative?
- Is this program the best way to meet our goals?
An evaluation focused on impact or program outcomes is concerned with assessing how your program has affected participants or the community. Impact evaluations usually consider different times frames, such as immediate impacts, short-term impacts (e.g. 1-6 months), or long term (6+ months). Ideally, an impact evaluation will consider both the intended and unintended outcomes of your program that are positive and negative.
This is one of the more meaningful forms of evaluations, but it is also one of the hardest types to do well. It can be expensive and time-consuming, and can only be done if you have enough of the right kinds of information.
An impact evaluation may not directly affect the current program, but it is helpful in deciding if a program should be repeated, expanded, revised, copied, scaled back, or cut.