Keepting it together: Maintaining a network
Leadership
Maintaining the original structure of the leadership or defining a new structure will be important in maintaining the network’s activities. Leadership can be organized in a way that works for the context of the network. Leaders do not need to be the most experienced in evaluation. Network champions need to be experts at managing projects and motivating others. It may serve a network well to have a leader or champion that oversees each ECB activity of the network.
What DEN Does
DEN had four champions assigned to the IMLS funded project. These champions spent the first year of the project defining their role, ultimately overseeing the following activities:
Meetings
It’s important to establish a structured system to ensure that goals and benefits are met. These meetings must be a combination of informative, enlightening and entertaining so that attendance is seen as an opportunity rather than a burden. DEN has identified meeting elements that, when combined, fit all of these requirements. DEN uses a rough template when creating our meeting agenda. This provides direction and ensures that nothing is allowed to drop.
What DEN Does
During the early phases of DEN, the members met every other month to discuss the studies. When it was discovered that ECB was something everyone wanted, the frequency of meetings expanded to once a month. Every other month there was focus on some sort of group training, including refining Excel skills, looking at various means of dissemination, learning how to write good questions, and discussions about selecting appropriate software for tablet evaluations.
Standard agenda items might include:
Capacity Building Trainings & Workshops
Building new skills as an evaluator can be the greatest motivator for continued, consistent attendance and participation in an evaluation network. Members of the network can define the knowledge and skills most needed within the group, and then seek out opportunities for learning as a network. Financial resources available to the network will guide opportunities for capacity building training and workshops, however, a lack of finances does not limit a group’s ability to provide such activities.
Ideas for content trainings led by internal partners
Here are a few suggestions for content training that may be available through your network partners.
Initial skill sets within DEN ranged from full-time evaluation departments to leaving a comment book in the lobby. There was some angst among those new to evaluation, worried that they would not understand much of the ECB content. Early 101 sessions with our more experienced partners proved beneficial in both directions; it gave members a degree of confidence that they hadn’t felt before, and it helped the presenting individuals and organizations clarify and refine many of the tasks that had become too routine.
Through funds provided by IMLS, DEN was able to conduct four in-depth workshops on the development of logic models, timing and tracking, exhibit prototyping, and working with underserved audiences. Where content expertise existed, various DEN members provided additional trainings to the network. For instance, one DEN institution had already planned to bring in an expert in a particular research method; funds from IMLS were used to supplement extra expenses so that all DEN organizations could send one representative to a two-day training with the expert.
Studies
The added bonus of working together as a network is the opportunity to build evaluation capacity by doing studies together. Conducting pan-institutional or cohort studies allows network members to pilot and practice together, learning about the successes and challenges of implementing methods at each institution. Having a team of partners will provide a forum of ideas and will provide a sounding board if motivations are needed for institutions of different types and sizes. On an institutional leve, organizations may find more value as they use network training and idea sharing to create their own customized study. The added bonus of doing studies with other organizations is that a community of institutions will have a great set of data that can be shared and learned from, providing a community to better understand the audiences it serves.
Additionally, doing studies together, as well as requiring individual network partners to conduct and report out on at least one institutional study per year, holds the institutions accountable for doing the evaluations.
What DEN Does
DEN members agree to do a combination of institutional and network-wide studies each year, with the goals of learning about evaluation by doing evaluation, and sharing successes and challenges from each study. The issue of conducting network-wide studies is discussed more completely in Conducting Pan-Institutional and Cohort Studies.
Committees
Committees activate the network. Each member should serve on at least one committee, and one member should chair the committee. Committees can then report back out at general network meetings, and schedule activities for the group.
What DEN Does
The two-year duration of the IMLS grant helped DEN identify what components have the most value and would therefore be an important part of sustainability. To ensure that these features are maintained, DEN developed a committee structure to plan and implement specific areas of operation. While each committee has a chair (the combination of which will effectively become an executive committee), each partner organization is required to appoint someone to at least one committee. This requirement will ensure that all members have a role in sustainability. The additional responsibility will also create a motivation for continued active participation.
DEN has established the following committees and their responsibilities:
Communications
Creating an email and sending it out to the group may seem like a simple task, however DEN discovered that there is great benefit in establishing a formal, stylized means of communication.
What DEN Does
For the first years of DEN’s existence, Basecamp, a web-based project management software, was used for this purpose. Requiring a paid subscription, Basecamp was already in use by DMNS, so it wasn’t difficult to create a new thread that would only be accessible to DEN members. As DEN transitions away from the grant-funded exploratory phase, the connection to DMNS will be reduced and Basecamp will no longer be available. Because DEN has decided to remain a fee-free network, a new communications tool, i.e. Google Docs, will need to be explored.
Maintaining the original structure of the leadership or defining a new structure will be important in maintaining the network’s activities. Leadership can be organized in a way that works for the context of the network. Leaders do not need to be the most experienced in evaluation. Network champions need to be experts at managing projects and motivating others. It may serve a network well to have a leader or champion that oversees each ECB activity of the network.
What DEN Does
DEN had four champions assigned to the IMLS funded project. These champions spent the first year of the project defining their role, ultimately overseeing the following activities:
- Scheduling network meetings and defining agendas
- Championing network committees
- Oversight of project logistics, grant requirements, and project sustainability
- Dissemination of the project at conferences and meetings
Meetings
It’s important to establish a structured system to ensure that goals and benefits are met. These meetings must be a combination of informative, enlightening and entertaining so that attendance is seen as an opportunity rather than a burden. DEN has identified meeting elements that, when combined, fit all of these requirements. DEN uses a rough template when creating our meeting agenda. This provides direction and ensures that nothing is allowed to drop.
What DEN Does
During the early phases of DEN, the members met every other month to discuss the studies. When it was discovered that ECB was something everyone wanted, the frequency of meetings expanded to once a month. Every other month there was focus on some sort of group training, including refining Excel skills, looking at various means of dissemination, learning how to write good questions, and discussions about selecting appropriate software for tablet evaluations.
Standard agenda items might include:
- Institutional Study share-outs. DEN members regularly give updates on all phases of their institutional studies. This may be a three sentence update or a 5 – 10 minute presentation. Many members have indicated that ECB is achieved while hearing about the planning, implementation, and lessons-learned at partner institutions.
- Discussion about pan-institutional or cohort studies. While it is advisable for cohort groups to meet outside of the regular meeting, that is not always necessary for quick updates. Allowing 10 – 15 minutes per meeting for people to cluster in their own groups to discuss ongoing projects is helpful.
- Committee reports. DEN has found that the Network runs more smoothly if work is shared, however it is not necessary for every person to be a part of every conversation. The regularly scheduled meeting is an excellent opportunity to discuss committee progress/plans that will affect everyone. (see section on Committees below)
- Evaluation Capacity Building (ECB). This has proven to be one of the most beneficial components of the Network. More than any other meeting element, ECB encourages continued, consistent attendance. Whether it is a full-hour training or a 15 minute overview of tablet software, ECB will be a part of every meeting.
- Post-meeting socializing. This may seem superfluous and whimsical, but there is great value in getting to know your partners on a social level. Not only will conversation become more spontaneous and open in the out-of-office environment, it is also more inspiring to partner with friends.
Capacity Building Trainings & Workshops
Building new skills as an evaluator can be the greatest motivator for continued, consistent attendance and participation in an evaluation network. Members of the network can define the knowledge and skills most needed within the group, and then seek out opportunities for learning as a network. Financial resources available to the network will guide opportunities for capacity building training and workshops, however, a lack of finances does not limit a group’s ability to provide such activities.
- Seek out expertise in the group. Identify members within the network whom have the capacity to teach about evaluation content or logistics.
- Seek out expertise at the partner institutions. There may be staff members not currently participating in the network, but working in the participating institutions, that may have expertise to share.
- Identify content experts in the community who may be willing to provide free or low-cost trainings for the network. Often, independent consultants will provide such services as an opportunity to network and build connections with potential clients. Engage local university of college staff who may have content expertise.
- Seek out free online trainings/webinars that can be hosted for the larger group at one of the network sites.
- Pool financial resources and participate in fee-based online trainings/webinars, or use the funds to bring in a content expert.
- Seek out grant funds as a network to fund capacity building trainings and workshops
Ideas for content trainings led by internal partners
Here are a few suggestions for content training that may be available through your network partners.
- Tablet surveys: selecting software and drafting questions
- Using Excel to process data and create charts
- Creative ways to disseminate information
- Writing effective questions
- Tips for effective one-on-one interviews
- How to coordinate a visitor panel or focus group
- Data analysis, interpretation, and visualization
- When and how to use incentives
- Article/book reviews
Initial skill sets within DEN ranged from full-time evaluation departments to leaving a comment book in the lobby. There was some angst among those new to evaluation, worried that they would not understand much of the ECB content. Early 101 sessions with our more experienced partners proved beneficial in both directions; it gave members a degree of confidence that they hadn’t felt before, and it helped the presenting individuals and organizations clarify and refine many of the tasks that had become too routine.
Through funds provided by IMLS, DEN was able to conduct four in-depth workshops on the development of logic models, timing and tracking, exhibit prototyping, and working with underserved audiences. Where content expertise existed, various DEN members provided additional trainings to the network. For instance, one DEN institution had already planned to bring in an expert in a particular research method; funds from IMLS were used to supplement extra expenses so that all DEN organizations could send one representative to a two-day training with the expert.
Studies
The added bonus of working together as a network is the opportunity to build evaluation capacity by doing studies together. Conducting pan-institutional or cohort studies allows network members to pilot and practice together, learning about the successes and challenges of implementing methods at each institution. Having a team of partners will provide a forum of ideas and will provide a sounding board if motivations are needed for institutions of different types and sizes. On an institutional leve, organizations may find more value as they use network training and idea sharing to create their own customized study. The added bonus of doing studies with other organizations is that a community of institutions will have a great set of data that can be shared and learned from, providing a community to better understand the audiences it serves.
Additionally, doing studies together, as well as requiring individual network partners to conduct and report out on at least one institutional study per year, holds the institutions accountable for doing the evaluations.
What DEN Does
DEN members agree to do a combination of institutional and network-wide studies each year, with the goals of learning about evaluation by doing evaluation, and sharing successes and challenges from each study. The issue of conducting network-wide studies is discussed more completely in Conducting Pan-Institutional and Cohort Studies.
Committees
Committees activate the network. Each member should serve on at least one committee, and one member should chair the committee. Committees can then report back out at general network meetings, and schedule activities for the group.
What DEN Does
The two-year duration of the IMLS grant helped DEN identify what components have the most value and would therefore be an important part of sustainability. To ensure that these features are maintained, DEN developed a committee structure to plan and implement specific areas of operation. While each committee has a chair (the combination of which will effectively become an executive committee), each partner organization is required to appoint someone to at least one committee. This requirement will ensure that all members have a role in sustainability. The additional responsibility will also create a motivation for continued active participation.
DEN has established the following committees and their responsibilities:
- ECB: Select days/locations for meetings; create meeting agendas; schedule speakers (DEN members or other relevant speakers); monitor other opportunities that may present themselves. (see ECB below)
- Studies. To validate and strengthen the value of the Network, pan-institutional or cohort studies are conducted annually. This committee researches potential group studies and works with the cohort groups to ensure that all members are fulfilling this requirement of membership. (see Section 2: Pan institutional and Cohort Studies)
- Membership & Communications. When a network is functioning well, there will be a plethora of emails, documents and charts. This committee maintains a standard means of communication (using a web-based platform) and the Network website. Similar to the role of secretary, this team establishes a repository system for DEN documents, templates and resources. Additionally, as the go-to source for membership information, this committee will liaise with institutions and individuals who are now, or interested in becoming, DEN members. Members of this committee visit other organizations, universities or conferences where they advocate and promote the benefits of DEN or similar networks. They also maintain the membership list and insure that there is always one appointed representative from each partner organization. They monitor partner participation and send reminders if requirements are not being met.
Communications
Creating an email and sending it out to the group may seem like a simple task, however DEN discovered that there is great benefit in establishing a formal, stylized means of communication.
- Consistent Contact Information. When the contact list is managed through one location it is much easier to change information as members leave or if a new member is added. If all partners are diligent in using that source when they send email, it will always contain the most up-to-date contact information.
- People lose stuff. There is a good deal of email generated in an active network. Whether it’s a meeting notice, a study protocol, or a link to a DEN survey, someone will inevitably lose or accidentally delete that email. Many documents are created that may have no immediate relevance when it’s posted, but may be incredibly important later. When those missives are sent from a universally accessible account that archives each and every post until it is physically removed by a site administrator, they are available to look up any time.
What DEN Does
For the first years of DEN’s existence, Basecamp, a web-based project management software, was used for this purpose. Requiring a paid subscription, Basecamp was already in use by DMNS, so it wasn’t difficult to create a new thread that would only be accessible to DEN members. As DEN transitions away from the grant-funded exploratory phase, the connection to DMNS will be reduced and Basecamp will no longer be available. Because DEN has decided to remain a fee-free network, a new communications tool, i.e. Google Docs, will need to be explored.
