What summative evaluation is
Summative evaluation looks at a finished, open exhibit and asks how well it did what it set out to do. The word “summative” refers to summing up. It usually takes place once the exhibit has been running long enough for visitors to use it in realistic conditions, and the results are used to report to funders and stakeholders, to decide whether to keep or change the exhibit, and to guide future projects.
It differs from the earlier stages. Audience research before design shapes the concept, and testing rough versions guides design choices. Remedial work adjusts a live exhibit. Summative work stands back and asks whether the whole thing worked, judged against goals that were written down earlier.
Why it depends on the goals
The most important preparation for summative evaluation happens long before opening. If the project has goals that can be evaluated, the summative study has a clear task: gather evidence on each goal and compare. If goals were vague, evaluators tend to produce a general description of visitor use, which can be interesting but cannot answer the question “did it work?”
If you find yourself starting a summative evaluation without written goals, begin by reconstructing them from the exhibit brief and early planning documents, and agree them with the stakeholders before collecting data. Be open about the fact that they were reconstructed.
When to do it
Timing is a balance. Too early and the exhibit may still be settling, with faults being repaired or visitors drawn by novelty. Too late and the people who made it may have moved on and memory has faded. Many projects wait until initial problems are resolved, then collect data across a period that includes ordinary visiting days, busy times and, where relevant, school visits. Plan for the evaluation in the schedule and budget from the start, since it is often the first thing dropped when money runs short.
Choosing methods
Choose methods that match the goals. Common ones include:
- Visitor observation. For goals about behavior: whether people stop, what they use, how groups share. Systematic sampling with a written form makes the results comparable.
- Timing and tracking. For goals about how people spend time, though dwell time needs careful interpretation.
- Exit interviews. Short conversations about what visitors took away. These can show understanding in people’s own words.
- Visitor surveys. Useful for broader samples when questions are well-written and sampling is considered.
- Usage logs. Data from the system, read with the cautions that apply to analytics.
- Staff and stakeholder views. Educators and front-of-house staff often notice patterns that visitors do not mention.
- Accessibility review. Checking with people who have a range of access needs, rather than assuming compliance.
Using more than one method lets you cross-check. If the logs, observation and interviews all point the same way, you can be more confident. If they disagree, that is itself informative.
A comparison group can add strength, such as visitors who have not yet seen the exhibit, or a similar space without it. This is difficult in practice and a real comparison needs care. Many small institutions cannot do it, and that is acceptable if the limitation is stated.
Sampling and honest limits
Who you gather data from affects what you can claim. Consider:
- Do the people sampled match the exhibit’s intended audiences?
- Were they chosen by a rule, or did the researcher choose whoever looked friendly?
- Did the sample include different days, times and weather?
- Who is missing, such as people who left quickly or did not speak the language of the survey?
Small samples are common and not a failure, provided the report says so. A summative evaluation of a few dozen people can still provide useful findings about what happened, but it cannot give precise proportions for all visitors. Use phrases like “of the 40 groups observed” rather than “visitors generally”.
It is also hard to prove that an exhibit caused a change. Visitors arrive with different backgrounds, and many factors affect what they know and feel. Wording such as “visitors who used the exhibit were able to…” is more accurate than “the exhibit taught visitors…”.
Analyzing against goals
Go through the goals one by one. For each, set out the evidence, state what it suggests, and note the strength of that evidence. A simple table can help: goal, evidence collected, finding, confidence, comments.
Be willing to report that some goals were not met, or that the evidence was inconclusive. A summative evaluation that finds only success is less credible and less useful than one that identifies weak points. Funders and colleagues generally learn more from a candid report.
Also record unplanned findings. Visitors may use an exhibit in ways nobody expected, or ignore a feature designers prized. These observations do not fit neatly against goals but can be the most valuable for future work.
Reporting
A useful report is usually short and plain. It typically includes:
- The exhibit’s purpose and goals.
- The methods, sample sizes, dates and any limitations.
- Findings, organized by goal.
- Unexpected findings.
- Recommendations, separating quick fixes from larger changes, and what to keep.
- A brief note on what would be worth studying next.
Keep language straightforward, avoid dense jargon, and keep claims in proportion to the evidence. If only a few people said something, say “a few”.
Share the report beyond the project team. Findings that sit in a drawer do not help the next exhibit. Consider a one-page summary for non-specialists, and make the full document available to the staff who will plan the following project. Take care that any quotes or observations cannot identify individual visitors, in line with visitor privacy.
What happens afterward
Possible outcomes include leaving the exhibit as it is, making changes through further remedial work, refreshing the content, or retiring it. A summative evaluation can also inform the maintenance plan and budget for similar projects.
Perhaps the most lasting benefit is learning that carries forward. Teams that record what worked and what did not, and that use the findings during early planning of the next project, gradually build a store of evidence about their own audiences, which is more relevant to them than general advice.