This section walks you through the process of planning an evaluation for an on the land program by asking a series of important questions. At the end of this section, if you take the time to answer each of the seven questions, you will have an evaluation plan.
 
An evaluation plan can help you to more effectively and efficiently conduct your evaluation. It typically includes a detailed program description with information about the program outcomes and objectives, participants, activities, and budget. It should also cover:

  • The purpose and focus of the evaluation (steps 1 and 2);
  • Roles and responsibilities of evaluators and stakeholders (step 3);
  • The methods you will use to collect the information needed to answer the evaluation questions (step 4 and 5);
  • Ethical considerations, such as privacy or consent forms (step 5);
  • The schedule or timeline for the evaluation and the budget (step 6); and
  • How the results will be shared and used (step 7).
An evaluation plan is a summary of relevant background information and a road map for the evaluation. It can help you to more effectively and efficiently conduct your evaluation.

1. What is the purpose of the evaluation?

Before getting too far in the process, it is important to identify why you are conducting an evaluation and what you hope to achieve in the process.
 
There are different kinds of evaluation that happen at different times in a program’s life cycle. See Types of Evaluation to learn more about the different kinds of evaluation.
 
Writing a purpose statement for your evaluation can help make your intentions clear. For example:

  • The purpose of this evaluation is to determine if a new on the land program is needed in our community (needs assessment);
  • The purpose of this evaluation is to produce recommendations for improving how we deliver our Nature Club (formative evaluation); or
  • The purpose of this evaluation is to document the changes that have resulted from our healing camps (impact evaluation).

2. What is the focus of the evaluation?

A good evaluation is targeted. The more specific the focus of the evaluation, the more likely it is that you will generate information that is useful. For any program, there are many different question you could ask, so it is important to be clear about the purpose of the evaluation and the focus of the evaluation from the beginning. If you have already gone through the process of answering the question, “What will success look like?,” you have a solid starting point.
 
Think about your evaluation questions carefully, and choose only a few broad questions for the evaluation. Questions that can be adapted for almost any on the land program include:

  • How was the program implemented?
  • Was the administration and delivery of the program effective and efficient?
  • Were the program outcomes valuable for participants, for the organization, and/or for the community?
  • What can be learned to improve the program moving forward?
 
For example, if you are conducting a formative evaluation of your fall goose camp, you may want to focus on, “How was the program implemented?” This will lead you more specific questions, such as:

  • Are the activities happening as planned or do we need to change some of the activities?
  • Were there enough staff to take care of everything without getting burned out?
  • Did we forget anything in our planning for supplies and equipment?

3. Who will be involved in the evaluation?

Start by identifying who will conduct the evaluation. Having someone in your organization evaluating the program can save money. It can also save time because you don’t have to explain your programming to an outsider. In some cases, however, you may want to have someone from outside your organization conduct the evaluation; an outside evaluator can offer a fresh perspective on the work you are doing. Many organizations use a combination of these two approaches, involving program staff in evaluation with the support of an external evaluator.
 
The next step is to consider who will participate in the evaluation. Evaluators often talk about stakeholders. These are people or organizations that have an interest in your program. Stakeholders may be involved directly (e.g. administrators, program leaders, participants) or they may be affected by the program (e.g. family members, funders, community members). It is also worth considering who else might benefit from receiving the information generated during your evaluation, including related organizations.
 
It is important to be clear from the outset the roles and responsibilities of everyone who will be participating in the evaluation, including program staff, external evaluators, and stakeholders. It is also important to know who will make final decisions about the program based on the results of the evaluation. Stakeholders who are involved in the evaluation will likely want to know how any information they provide will be used.
 
The best evaluations consider different perspectives, including those of people who didn’t like or benefit from the program. It’s important to ask what didn’t work for them and why. Similarly, it’s a good idea to talk to participants who didn’t complete the program or potential participants who decided not to attend. Their experiences may shed light on barriers to participation.
The best evaluations consider different perspectives, including those of people who didn’t like or benefit from the program.

4. What information will you collect?

In this stage, you will identify what exactly you need to know. This is where many evaluations struggle because the evaluators know what questions they want answered, but the information they need to answer those questions isn’t available. This can be frustrating but there are usually ways to find other information that can help to address the gaps. For example, if there are no records of how many youth attended land-based camps over the last year, you can go back to program staff and try to reconstruct the information based on their memories of each camp.
 
There are two types of information that you can collect during an evaluation: quantitative and qualitative.
 
Quantitative information describes a situation using numbers. In other words, quantitative information is something that can be counted. For example, the number of camps you held, the number of participants that learned how to use a chain saw, or the number of people that reported feeling more connected to Elders.
 
Qualitative information describes a situation using words, ideas, pictures, and stories. Qualitative information can be drawn from participants’ own words or can be based on observations. For example, English-speaking youth may describe struggling to understand Dene-speaking Elders or the evaluator may observe that program participants became more confident using guns during the camp.

5. How will you collect information?

When people think about collecting information for an evaluation, they often think about surveys or questionnaires. These are just one method of data collection. You can also draw on your own observations, informal conversations, journals, photos and videos, interviews, activity logs, sharing circles, knowledge/skill tests, and program documents. Each method has advantages and challenges.
 
Ideally, you will use a combination of methods that are best suited to your program and the participants. The credibility of an evaluation is increased when it draws on multiple sources of information.
 
Regardless of the method you choose, it is important to respect those who are providing you with information. Let them know why you are conducting the evaluation and how the information they share will be used. Only identify participants directly if they provide you with written consent. If you are collecting information from people under the age of 18, make sure you have permission from their parents or guardians (see sample).
Visit Collecting Information to learn about the different methods you can use to collect information for your evaluation.

6. How much time and money will you need?

Now that you’ve framed the purpose and focus of the evaluation, figured out who should be involved, and identified what information you will need, you can turn your attention to developing your timeline and budget. Evaluations don't have to be expensive. At the very least, though, they require an investment of time.
 
When thinking about timing, factors to consider include:
  • Are the people working on the evaluation doing it on a full-time basis, or as part of their other work?
  • How much time will it take to compile the information that you need? For example, depending on what information you will be collecting: Are the files readily available? Are you planning to do interviews? Are the people you want to talk to all in your community?
  • What kind of report are you planning to produce and how much work is involved?

For your budget, you will need to take into consideration:
  • Salary or professional fees for the evaluators;
  • Any required travel costs, for the evaluators or other stakeholders;
  • Compensation for Elders or other participants in sharing circles or focus groups;
  • Additional costs such as meeting rooms, catering, technical support to develop on-line surveys, etc.; and
  • Equipment that may be required such as tablets or tape recorders.

7. How will you use/share your findings?

The final stage in the planning process is to think about how you will use your findings. First, you need to analyze the information you gathered. This involves sorting through all the information that you have collected and reaching conclusions. Sometimes the conclusions are clear and obvious, but sometimes you will need to work with others on your evaluation team to understand the data.
 
How will you share the results of your evaluation? A report is one option, but people don’t always read reports. Consider alternative reporting tools, such as presentations, infographics, slide shows, or a video. As you're deciding how you will communicate your findings, think about your audience. Who are they? What will they want to know? What is the best way to communicate with them?

Regardless of the method you choose to share your information, remember to keep it simple, so that everyone can understand and make use of your findings.
Consider alternative reporting tools that are more accessible and engaging, such as presentations, infographics, slide shows, or a video.