Building evaluation capacity in museum professionals throughout the Denver metro area
Denver Evaluation Network
  • Home
  • About
    • Members
    • Get Involved
    • Intership Opportunities
  • Toolkit
    • Network Benefits
    • Creating & Building a Network
    • Maintaining a Network
    • Building Evaluation Capacity
    • Network-wide Studies >
      • Conducting Pan-Institutional and Cohort Studies
      • DEN Case Studies
      • Sample Instruments
  • Resources
    • Professional Associations & Evaluators
    • Information & Publications
    • Professional Development
    • Index of DEN Studies
  • Contact
  • DEN ACCESS

Conducting Pan-institutional and cohort studies

One of the most beneficial elements of an evaluation network is the opportunity to conduct network-wide studies, either in the form of pan-institutional or smaller cohort studies. During these studies all organizations work collaboratively to implement the same study at each institution. There are many benefits of participating in a network-wide study.

Evaluation Capacity Building (ECB) 
Doing evaluation is one of the best ways to learn about evaluation. When done in a larger team, the learning can be exponentially greater.

What DEN Does
DEN members use the pan-institutional studies as an opportunity to learn together.  
  • Member institutions or outside professionals conduct group training on the use of a particular method or instrument.  
  • Member organizations become familiar with an instrument and the ECB is an internal group commodity.
  • Working as part of the Studies Committee, members with little evaluation capacity work alongside more seasoned professionals, thus learning through doing. Participating on this committee is particularly helpful in learning how to draft good questions and protocols.

Physical Resource Sharing
Some instruments may call for a tool, such as a set of cards, a recording device, or a questionnaire, but not all partners have the equipment needed for data collection. Through a network the instruments may be created (or already exit) at one site and shared with the others.

What DEN Does
Human resource sharing.  
  • There are often occasions when an outside consultant is needed at an institution, such as an interviewer for a visitor panel or focus group. Through DEN, members can easily trade this service.
  • Because individual members are required to assist with data collection, institutional members have access to a pool of free, trained, research assistants on an as-needed basis.
Study prototyping.  Many of DEN’s studies are created to help test/practice a newly-learned method or to modify a method to fit institutions of various types. The nature of the pan-institutional studies, therefore, becomes an opportunity to test design and implementation before any one institution embarked on their own, individual study.


Increased knowledge of community.  Because the studies are designed, implemented, and shared across the network, members learn much more about the museum environment outside of their own organizations. 

Cautionary Advice: A Case for Cohort Studies
Pan-institutional studies are not always the best method for collecting useful data. With members varying in size, budget, audiences, disciplines, and programs, it becomes increasingly difficult to find one study that will fit everyone equally. The benefits of the pan-institutional studies are listed above. Here are some of the challenges:
  • Many organizations will have small staff, low visitation or fewer resources, and may not be able to participate at the same level as other members.
  • Organizations with a strong evaluation program in place may find that some studies are redundant to recent activity.  
  • Because pan-institutional studies need to be designed for the lowest common denominator, they can be too broad, not relevant to organizational activities, or provide data that are not helpful to everyone.  
  • It is not always possible to use the same tool the same way at all institutions. Logistically there may be too few available participants, or the layout of a facility may not attract the same traffic flow or concentration points (i.e. some may have multiple entrances/exits). 
  • It may not be possible to customize methods in such a way that the instrument intent/model remains recognizable and true.   
  • The more member organizations, the greater the challenge in finding a study that has mutual benefit to all. Additionally, partners that are not interested in a method or topic may be less likely to proceed with the same degree of rigor. At best this could lead to partial or incomplete data; at worst it may dissuade the organization from continuing with the network because they find that the activities don’t apply to their needs.
If this situation exists, it may be more appropriate for the network to divide into different cohort groups for each study. Art Museums may choose to work together on an issue that is relevant to their needs.  Sites with living collections, shared audience types, or are geographically close together may find a study that would be of interest to their specific group but no one else. In these instances the development of a cohort study would lead to a final product that truly addresses specific areas of concern with appropriate partners, all while maintaining a level of learning about evaluation.  Cohort study groups allows partners to self-identify as interests and concerns are discussed as a group, and allows smaller groups to refine information discovered in a pan-institutional study.


Powered by Create your own unique website with customizable templates.