Evaluate Your Chorus’ Education Program—On Any Budget

If you have no resources...

• Think in terms of constant documentation. You could develop a spreadsheet, for example, that documents what the schools are, the ages of the kids, their demographics, the music the kids sang, who wrote it, etc. “Without doing any evaluation at all you can document important information you have about the program,” Meisten said.
• Keep data collection simple but focused and systematic.
• Select limited projects for in-depth evaluation. Don’t do an evaluation every year. Do it when the time is ripe—perhaps when you’re having problems with a particular aspect of a program and need some information. You can then do something more in-depth on the particular issue you are focusing on. 
• Avail yourself of examples that you can adapt. You can find them on the Internet. Just be careful that they are from trustworthy sources. Other nonprofits such as orchestras, theaters and museums have great sample questions you can adapt.

If you have some resources…

• Seek consultant advice on your methods and findings.  If the budget is thin, hire a consultant to talk with you for a couple of hours. Universities are a great resource for finding consultants, as are professional associations that focus on evaluation.  “It’s important to find evaluators or consultants who have worked with schools, or have experience with evaluating arts programming.” 
• Increase the variety of data sources. Include both quantitative and qualitative day. 
• Broaden the scope of evaluation.
• Provide regular reports to administrators or other relevant audiences.  Sharing the results with people you asked the questions of reinforces your organization's transparency and trustworthiness. 

If you have considerable resources…

• Hire an ongoing consultant/expert to assist with evolving evaluation plans and methods. Work to find the right match for you. For example, universities that focus primarily on statistical analysis are expensive and you may not get the information you want, Meisten said. It’s important to go through a bidding process; send out a request for proposals to five to ten organizations.  If you have the money, you want to spend it wisely.
• Conduct both project-level and program-level evaluations. In larger organizations with an array of educational offerings, a program level evaluation might seek to answer the question, do we have the right balance of offerings? Are the programs fulfilling our mission? Is it meeting our budget requirements? How is this program supporting our chorus and our mission? “Education programs should support the chorus at all levels,” Meisten said. 
• Provide both internal and publicly released reports. You need to share both within your organization and with other audiences. “When we do a survey about a young people’s concert,” Meisten said, “it doesn’t help VocalEssence to keep that information to myself. I need to share it with the Artistic Director, even if they say they hated the song. If the survey says there was poor marketing for a program, you need to share it with your marketing director.”
• Share information from your evaluation with the public in a variety of ways: in a public document, on your website, in an e-newsletter, or in an annual report. “Keep the document short and focused on the most important information," Meisten said. 
• Develop program analyses fully linking client outcomes, process, and institutional change with cost-benefit analyses.  “If you hire someone to evaluate an educational program,” Meister said, “it is incumbent on that person to remember the bigger picture.” If you have evaluation resources, you can link all of those pieces of data and cross-tabulate. For example, 'How does the money spent on this project relate to the number of students served?'"

Resources for Evaluation Programs

Visitors Studies Association is a professional organization focusing on all facets of the visitor experience in museums, zoos, nature centers, visitor centers, historic sites, parks and other informal learning settings. Some of the approaches have application for chorus education programs.

Theater Bay Area has released the paperback Counting New Beans: Intrinsic Impact And The Value Of Art, a summary of research by the firm WolfBrown as well as interviews with 20 prominent artistic directors and essays by Diane Ragsdale, Arlene Goldbard, Rebecca Novick and more. More grist for the mill.