Building Evaluation Capacity
TRANSFERRING EVALUATION CAPACITY BUILDING TO YOUR INSTITUTION
Building an Evaluation of Culture
What does “an evaluation culture” at an institution look like? What does building that culture really entail? If a culture of evaluation means dedicating staff time and resources, what activities help that happen? How can interest be garnered, dialogue cultivated, and knowledge shared? While a “culture” is certainly not created overnight, DEN professionals found several positive ways to help develop an evaluation culture at a variety of organizational levels in ways big and small. This included engaging leadership, coworkers, and institutional planning.
What DEN Does
Although organizational evaluation capacity was affected simply through individual representatives’ participation and DEN requirements, many DEN members utilize the network as an opportunity to transfer building evaluation capacity ideas and activities to their organizations in a broader sense. Tactics like regular reporting, cross-departmental share-outs, and learning opportunities were used to both create buy-in and to develop skills in other museum staff.
Because of the variance in organizational size, type, funding and structure, each institution handled organizational capacity building differently and in numerous ways. Regardless of institution size, DEN members found that engaging both leadership and colleagues are important to create buy-in for growing a culture of evaluation.
In this section, we will provide sample tips and scenarios that were used to extend building evaluation capacity deeper into their organizations.
What DEN Does: Engaging Leadership
Leadership buy-in for evaluation activities is essential to creating an evaluation culture. A bottom-up/top-down approach helps staff members dedicate time and resources to evaluation projects— projects that without leadership support might be pushed aside. The DEN group found several ways to make evaluation relevant to leadership and to engage them in evaluative thinking and actions.
Just as support from leadership is important to the success of evaluation initiatives, so too is support from coworkers. Colleagues and peers can provided added insight, assistance with data collection, and, simply, moral support for the tasks of understanding audiences and visitors better. DEN Members found that the more colleagues were engaged, the more likely they were to support others’ time spent on evaluation and to think of ways to use and apply it in their own work. Below are sample suggestions from DEN Members for ways to engage colleagues in evaluative thinking and planning.
Building internal teams:
Building an Evaluation of Culture
What does “an evaluation culture” at an institution look like? What does building that culture really entail? If a culture of evaluation means dedicating staff time and resources, what activities help that happen? How can interest be garnered, dialogue cultivated, and knowledge shared? While a “culture” is certainly not created overnight, DEN professionals found several positive ways to help develop an evaluation culture at a variety of organizational levels in ways big and small. This included engaging leadership, coworkers, and institutional planning.
What DEN Does
Although organizational evaluation capacity was affected simply through individual representatives’ participation and DEN requirements, many DEN members utilize the network as an opportunity to transfer building evaluation capacity ideas and activities to their organizations in a broader sense. Tactics like regular reporting, cross-departmental share-outs, and learning opportunities were used to both create buy-in and to develop skills in other museum staff.
Because of the variance in organizational size, type, funding and structure, each institution handled organizational capacity building differently and in numerous ways. Regardless of institution size, DEN members found that engaging both leadership and colleagues are important to create buy-in for growing a culture of evaluation.
In this section, we will provide sample tips and scenarios that were used to extend building evaluation capacity deeper into their organizations.
What DEN Does: Engaging Leadership
Leadership buy-in for evaluation activities is essential to creating an evaluation culture. A bottom-up/top-down approach helps staff members dedicate time and resources to evaluation projects— projects that without leadership support might be pushed aside. The DEN group found several ways to make evaluation relevant to leadership and to engage them in evaluative thinking and actions.
- Aligning evaluation initiatives to leadership interests and strategic goals: Using evaluation to investigate leadership’s top questions or priority projects not only provides useful information to management, but helps “plant the seed” and create interest in evaluation overall.
- Sharing of evaluation data among organizations in a community: Museum leadership valuethat DEN shares evaluation data among organizations. Data sharing provides leaders with an informal and low-stakes way to gain information about the cultural community.
- Providing evaluation-centered opportunities for leadership to gather and network: With IMLS grant funds, DEN organized a “presidential luncheon” to help forward the goal to build evaluation capacity at multiple institutional levels. The opportunity to engage with the leadership of other DEN organizations helped concretize the work of the group for leaders, forwarding buy-in to the value of evaluation. This was especially useful for larger organizations, where the leadership may have been less directly involved with day-to-day evaluation activities or projects.
- Use colleagues and coworkers to access upper level management: Colleagues can serve as fantastic advocates for evaluation; seek people who may have access to leadership and help identify departmental needs and how evaluation may help address those needs
Just as support from leadership is important to the success of evaluation initiatives, so too is support from coworkers. Colleagues and peers can provided added insight, assistance with data collection, and, simply, moral support for the tasks of understanding audiences and visitors better. DEN Members found that the more colleagues were engaged, the more likely they were to support others’ time spent on evaluation and to think of ways to use and apply it in their own work. Below are sample suggestions from DEN Members for ways to engage colleagues in evaluative thinking and planning.
Building internal teams:
- One-on-One’s: Start with colleagues who you feel can easily be approached and might find immediate benefit from evaluation. Arrange one-on-one meetings to help identify areas to assess and brainstorm ways to find solutions through evaluation
- Interdepartmental working groups: At Clyfford Still Museum, several departments were convened to create a “DEN Working Group” to discuss individual study topics, DEN updates, and assist with data collection. Museum leadership as well as heads of education, development, marketing, and visitor services were included. The group convenes as necessary to select study topics, choose methods, and analyze results. As a multi-department working group, the team can discuss bigger picture organizational concerns and plan evaluation priorities, strategies, and tasks accordingly.
- Evaluation steering committees: At Butterfly Pavillion, the vice presidents of each public museum section were asked to nominate one or two employees to participate in a steering committee; the DEN representative serves as the committee chair. Hour-long meetings take place once every other month. Study topics in various departments are discussed and information is shared in a central place. The group is creating an evaluation plan for the following year.
- Single department teams: Some staff represented worked with single department teams to galvanize support in specific areas. For example at Longmont Museum, the exhibitions team was utilized for topic selection and planning, but information was disseminated at exhibition team meetings and full staff meetings.
- Small scale workshops: Some organizations utilized DEN experts to present smaller scale evaluation workshops. At Clyfford Still Museum, these sessions were used to go over evaluation basics and to help create a shared vocabulary and definition of evaluation for the institution.
- “Bring a Friend” professional development: Inviting colleagues to attend professional development sessions helps DEN members share evaluation experiences with home institution n colleagues. When the Children’s Museum hosted a workshop on adult and child interaction inventory with Lori Beaumont, the Children’s Museum invited colleagues in the network, and invited those colleagues to bring a guest. Similarly, when one institution sets up a webinar, others are invited to post it and invite colleagues to take advantage.
- Sharing the work: Engaging staff in some or all processes of evaluation is an important step in building organizational ECB. Especially for small staffs, direct involvement with study topic selection, instrument creation, and data collection help create buy-in.
- “Learning by doing”: In some cases, staff—from leadership to visitor services—are asked to participate directly in data collection. Even scheduling staff just once for short intervals provided a new set of background knowledge on which staff could draw in thinking about evaluation. Volunteers were also trained and used.
- Assigning easy tasks: Inviting colleagues to assist with greeting guests who serve in focus groups, or to participate as an onlooker at a visitor panel can be a quick, low-stakes way to increase participation and interest without huge time commitments.
- Asking colleagues about what areas they would be interested in evaluating is helpful, even if formal teams are not created.
- Regular reporting at staff and departmental meetings helps keep evaluation on the minds of larger groups, and also demonstrates that “evaluation is something we actually use.”
- Data visualization: Wordles, pictographs, and fun charts help break down data into intriguing yet accessible pieces. At –a-glance one sheets help busy professionals access information quickly.
- Lunch and Learns: At Denver Zoo, staff regularly present evaluation studies and findings at “Lunch and Learns”, which are open to all staff and volunteers.
- Creative Dissemination: Using provocative and unexpected approaches can help get your evaluation results attention. At DMNS, the Audience Insights department generates unique videos to present evaluation findings and then distributes the videos in creative ways.
- For a demographic study results video in which jelly beans were used to portray demographic data (http://vimeo.com/50538153) staff members greeted colleagues at the staff entrance and handed out bags of jelly beans tagged with the video link.
- To distribute value study results, the Insights team printed a word cloud (reflecting analyzed qualitative data) on coffee mugs, and distributed the mugs to senior leadership and department directors. They followed this with a video link sent to all staff (http://vimeo.com/50538277).
- To distribute the parking study results video (http://vimeo.com/55977520), the DMNS team left matchbox cars with the video link in strategic yet random places around the Museum.
