Evaluating Centers for Teaching and Learning: A Field-Tested Model

Abstract

This paper provides a program evaluation model, along with field-testing results, that was developed in response to the need for an evaluation model able to support systematic evaluation of teaching and learning centers (CTLs). The model builds upon the author’s previous studies investigating the evaluation practices and struggles experienced at 53 CTLs. Findings from these studies attribute evaluation struggles to contextual issues involving evaluation capacity, ill-structured curricula, and ill-conceived evaluation frameworks. This field-tested Four-Phase Program Evaluation Model addresses these issues by approaching evaluation in a comprehensive manner that includes an evaluation capacity analysis, curricular conceptualization, evaluation planning, and plan implementation.

Keywords: evaluation, faculty development, professional development, programs, assessment

Over the past 50 years, centers for teaching and learning (CTLs) have established a reputation for advancing academic quality in universities across the world. Yet, despite this well regarded history, educational developers continue to struggle with adequately evaluating the worth and merit of their programs. Many educational developers have attributed their struggle to evaluation capacity issues, including lack of time, resources, evaluation expertise, and useful models (Hines, 2011; Kreber, 1997; Van Note Chism & Szabó, 1997). Investigations of CTL evaluation practices found that the absence of both focused program goals and structured curricula perpetuated the use of superficial measures (Hines, 2011; Murray, 2002). The complexity of educational development and diverse disciplinary beliefs regarding evaluation further complicated the work (Hines, 2011; Kucsera & Svinicki, 2010). A recent review of program evaluation literature corroborated these findings, indicating that CTLs experience five primary obstacles:

1. Misguiding evaluation mindsets—where preconceived ideas regarding evaluation impede the desire to conduct rigorous evaluation.

2. Weak infrastructure—where the way in which CTL resources, workflow allocation, and operational procedures are structured hinder efforts to conduct quality evaluation.

3. Ill conceptualized curricula—where the CTL curricula represent an ever expanding menu of offerings without an intentional curricular design.

4. Fuzzy goals and short aimed missions—where a CTL mission is myopically focused on faculty needs, and program goals are aimed on process not product.

5. Ill conceived evaluation frameworks—where evaluation planning is done in an intuitive manner without the use of a sound evaluation framework (Hines, 2015).

These factors demonstrate that a sound model for evaluating CTL programs must consider the larger picture of a center’s evaluation capacity, curricular structure, and evaluation planning process. Chalmers and Gardiner (2015) support this finding and further argue that a program evaluation model should reflect five key principles: relevance, rigor, context, and reliability. A relevant model applies to teaching and learning centers with a wide range of type and purpose. A rigorous model is grounded in a theoretical and evidence based design. A contextual model takes into account the situational factors influencing the operations of the center, including its infrastructure, management, curriculum, and institutional learning culture. A reliable model is field tested in various CTLs for its usefulness and practicality.

The purpose of this paper is to outline the Four Phase Program Evaluation Model that I developed through my research and a field testing project at nine teaching centers. This paper describes the theoretical and empirical foundation of the model, outlines each phase, and discusses the findings and recommendations from the field testing project. The steps within each phase are intentionally described in a “how to” manner so those interested in applying the model are able to do so while keeping in mind the challenges and recommendations from the field testing project.

The Field Testing Project

In an effort to determine the model’s relevance, feasibility, benefits, and challenges, a field testing project (funded by a POD Network grant) was conducted with educational developers working in mature CTLs representing a range of institutions in North America. Participants were recruited through invitation via the POD Network. The nine field testing institutions included one community college, two 4 year institutions (a private liberal arts and private proprietary), five comprehensive universities (three public and two private), and one Research I institution in which the CTL served the Faculties of Science (Figure 1). All field testers received a document describing the field testing process, an overview of the Four Phase Evaluation Model, and signed a consent form (approved by IRB) with the option to withdraw at any time.

Figure 1. Field Tester’s Institutions and Demographics

Over a three year period, I accompanied each field tester as they applied the model using guiding documents, planning worksheets, regular communications, and an all day campus visit at each field site. The campus visit allowed for in depth conversations and enhanced understanding of the contextual environment of the CTL. Planning worksheets and products associated with each phase were discussed and refined throughout the field testing project. Most of the field testers met as a group during the 2014 POD Network Conference to exchange ideas and recommendations. Virtual group meetings continued in 2015 in preparation for presenting the model and “lessons learned” at a preconference session at the 2015 POD Network Conference.

I took field notes throughout the entire three year project to capture themes related to the model’s relevance, feasibility, benefits, and challenges. The Four Phase Program Evaluation Model shared in the next section is the refined product resulting from these field notes, exchanges, and evaluation document drafts. A more detailed discussion of the field testing findings is provided later in this paper.

Theoretical and Empirical Foundation of the Four Phase Program Evaluation Model

The Four Phase Program Evaluation Model integrates relevant theoretical models as well as findings from my earlier studies. These studies combined identified evaluation outcomes, measures, and strategies used at 53 CTLs in the United States as well as impediments noted by CTL directors (Hines, 2009, 2011). This previous research directly informed my selection of the theories that framed the overall structure of the model (Figure 2).

Figure 2. Depiction of the Theoretical Framework of the Four Phase Program Evaluation Model

The foundation of the model is based on Stufflebeam’s (1971) Context, Input, Process, Product (CIPP) evaluation model given that the evaluation struggles articulated by CTL directors originated from factors related to the center’s context, inputs, processes, and products (Hines, 2009; Hines, 2011). Contextual struggles stemmed from uncontrollable institutional issues such as voluntary faculty participation, a growing mix of faculty types (i.e., adjunct, part , and full time), and restrictions imposed by unionization. Input issues were infrastructural in nature, such as the center’s finances, staffing, evaluation acumen, and available technologies. Process struggles emanated from the way the center’s offerings were designed, structured, and implemented. Product issues centered on evaluation decisions regarding how to measure effectiveness and when to analyze and make improvements—all of which tied back to perceptions regarding the center’s context, inputs, processes, and products.

Using Stufflebeam’s CIPP Model as a structural foundation, four phases emerged. Phase I examines context and inputs through an evaluation capacity analysis to garner a realistic picture of the center’s capacity for robust evaluation. Phase II looks at process through a curricular conceptualization of the center’s offerings. Product is addressed in the design of a comprehensive evaluation plan in Phase III, and the implementation of the plan is addressed in Phase IV.

To provide further structure, Wiggins and McTighe’s (1998) Backwards Design Model informs and systematizes the curricular conceptualization in Phase II, and Wholey’s (1987) Logic Model guides evaluation planning determinations in Phase III. Evaluation levels are built on Kirkpatrick’s (1976) Four Level Evaluation Model and CTL evaluation interests reported by CTL directors in my earlier studies. The application of these models is further discussed in the context of each phase throughout the paper.

The Four Phase Program Evaluation Model

As previously noted, the model is divided into four phases: Phase I Evaluation Capacity Analysis, Phase II Curricular Conceptualization, Phase III Program Evaluation Planning, and Phase IV Plan Implementation. Figure 3 provides an overview of the model and the steps involved in each phase. A discussion of each phase and the accompanying steps follows.

Figure 3. Four Phases and Steps of the Four Phase Evaluation Model

Phase I: Evaluation Capacity Analysis

Educational development evaluation is not a one size fits all approach. Each center has its own unique set of circumstances that significantly impacts its capacity to conduct rigorous and systematic evaluation (Stefani, 2011). Conducting an evaluation capacity analysis is a critical first step to determine the extent to which a center is capable of rigorous evaluation. Labin, Duffy, Meyers, Wandersman, and Lesesne (2012) describe the evaluation capacity analysis as “… an intentional process to increase individual motivation, knowledge, and skills, and to enhance a group or organization’s ability to conduct or use evaluation” (p. 308). An evaluation capacity analysis brings into sharp focus the conditions that support or constrain the ability to conduct rigorous program evaluation.

Identify and adjust the center’s evaluation capacity

One way to conduct an evaluation capacity analysis is to engage the center’s staff in a reflective process using a list of questions able to expose areas supporting or constraining the center’s evaluation capacity (Figure 4). These questions are grouped by the main factors impacting evaluation capacity (Hines, 2015) and designed based on the evaluation expertise of Posavac and Carey (2003), investigations of CTL evaluation practices (Hines, 2009; Hines, 2011; Van Note Chism & Szabó, 1997), and findings from the field testing project.

Figure 4. Reflective Questions for an Evaluation Capacity Analysis

Analyzing the responses to these questions provides insight into various elements associated with each of the factors impacting evaluation capacity. The more positive the responses, the more conducive conditions are for rigorous and systematic evaluation. Responses identifying possible constraints are in need of further exploration to determine if and how improvements can be made.

While the institutional setting and the center’s infrastructure may be the least controllable, the staff’s evaluation mindset is the most controllable and most influential. Negative mindsets, which generally come from fear or past experiences, can significantly undermine evaluation efforts (Posavac & Carey, 2003). Time spent exploring and adjusting these attitudes can result in notable improvements in evaluation practices. A center’s evaluation framework and curricular structure can also be improved and is addressed in the next two phases.

Phase II: Curriculum Conceptualization

Educational development curriculum oftentimes consists of a wide array of offerings and services based on faculty needs, institutional priorities, inherited programs, the director’s expertise, or other external factors (Hines, 2011; Murray, 2002). The resulting menu of activities can make evaluation difficult if there is no clear curricular structure to evaluate (Killion, 2008). Kuhlenschmidt, Weaver, and Morgan (2009) support the need for well structured educational development curriculum, stating it enables the creation of coherent programming with clear intentions allowing for more complex assessment. The authors further argue that a well conceptualized curricular structure allows the staff to see their programming as a whole, engenders proactive decision making, enables persuasive arguments for resources, and facilitates marketing and promotion of the center’s programming. The curriculum conceptualization phase is broken down into the following steps: articulate CTL outcome statements, create themed programs, map programs to outcomes, and craft program goals and offering objectives.

Articulate the center’s outcome statements

Applying principles of Wiggins and McTighe’s (1998) Backwards Design Model, curricular conceptualization begins with creating a succinct list of outcome statements that align with the center’s mission. Outcome statements are measurable statements that articulate the intended changes that a CTL strives to achieve by advancing the center’s mission. Below are an example of a mission statement and four outcomes statements for a fictitious CTL:

The mission of Red Ridge College’s Center for Teaching and Learning is to promote and sustain academic excellence by providing relevant faculty development programs able to advance teaching, scholarship, and student success.

As a result, participating faculty will:

Outcome 1: Design, implement, and assess student centered curriculum and instruction,

Outcome 2: Advance academic development through communities of practice,

Outcome 3: Continually grow as educators and scholars through reflective practices, and

Outcome 4: Engage in scholarship of teaching and learning.

For additional clarity, a list of three to four indicator statements could be created to further unpack each outcome to demonstrate or “indicate” the presence of a particular outcome (Hurney, Brantmeier, Good, Harrison, & Meixner, 2016). For example, indicators for outcome four, Engage in scholarship of teaching and learning, could be (a) views teaching and learning through the lens of a researcher, (b) designs and implements an inquiry method to examine a particular area of teaching and learning, and (c) disseminates findings for public use. The creation of indicators can help guide the design of the evaluation instruments and curriculum.

Organize services and offerings into distinct programs

The next step is to restructure the current offerings and services into distinct programs. Working with the entire CTL staff, sort all offerings and services into categorical themes using a card sorting process. As a group, reduce the themes through a consensus building process until a commonly agreed upon curricular structure representing themed programs with grouped offerings emerges. The aim is to have a reasonable number of programs with a specific purpose containing related offerings or services. To increase distinction and marketability, give each program an identifiable name, such as “First Year Faculty Program,” “Faculty Learning Community Program,” or “Faculty Consultation Program.”

Map programs to the outcome statements

To confirm the alignment and completeness of the curricular structure, map the themed programs to the center’s outcome statements using a matrix. For each intersecting cell, indicate if the outcome is introduced, strengthened, or emphasized in the program. The idea is to identify gaps, redundancies, and curricular strengths in order to restructure the programs and respective offerings and services if needed. Figure 5 is an example of a curriculum map that maps five themed programs to the center’s outcome statements using the previous Red Ridge College example.

Figure 5. Example of Curriculum Mapping

Craft program goals and offering objectives for each program

Once the basic curricular structuring is complete, continue the backward design process by crafting a product oriented goal for each program and a product oriented objective for each offering. Product oriented goals and objectives clearly identify the change or result the program and its offerings are intended to create, which in turn will inform evaluation and program design decisions (Killion, 2008). The curriculum map is a useful tool for determining program goals and further ensures the development of a coherent and well aligned curriculum. Review the map to see which of the center’s outcomes are being introduced, strengthened, and emphasized for each program as noted in Figure 4. Then, create a goal that reflects an integration of these identified outcomes.

Killion (2008) cautions against the use of process oriented goals and objectives that focus on program implementation, such as “The new faculty orientation program will provide new faculty the basics needed to successfully begin their educational roles.” Instead, focus on the faculty participants and the intended behavioral change as a result of the program. In doing so, the previous statement might read, “New faculty will be able to navigate campus services, resources, and systems needed to begin their teaching roles.” Figure 6 illustrates a curricular conceptualization using the programs in the above Red Ridge College example.

Figure 6. Example of a Curriculum Structure

Phase III: Evaluation Planning

Once the curriculum is conceptualized into an evaluable structure, evaluation planning can begin. The plan communicates how each program will be evaluated, when the data will be collected and by whom, and timing for the data analysis. The program evaluation phase consists of the following steps: determine program impact levels, confirm impact levels with logic models, identify evaluation methods and strategies, determine timing for data collection, identify person(s) responsible for data collection, determine timing for analyzing, and using the data.

Determine evaluation levels for each program

A comprehensive evaluation plan is designed in a systematic and informed manner where evaluation methodologies are determined for each program based on the program goal and expected impact levels. Similar to other CTL evaluation models, this Four Phase Program Evaluation Model extends Kirkpatrick’s (1976) four levels of evaluation by employing six evaluation levels: (a) participation, (b) participant satisfaction, (c) participant learning, (d) change in teaching, (e) change in student learning, and (f) institutional change.

The evaluation levels are ordered from short to far reaching measures. Think of each level as concentric rings, like ripples in a pond, emanating from a center point representing where a single program has been “launched” in the institutional setting. Similar to throwing a stone in a pond, ripples are set up depending on the size and intensity of the program—the greater the size and intensity of a program, the greater the impact and vice versa.

To determine the evaluation levels for each program, use the program’s goal and program design as a guide along with a clear understanding of the potentiality and feasibility of reaching each impact level, as well as a clear understanding of what each evaluation level is attempting to measure. Understanding the type of information being gathered for each level helps clarify the various levels of impact as well as assist in designing evaluation methods and instruments. Figure 7 provides a list of each level along with guiding questions and various evaluation dimensions that could be measured. Refer to the literature as an additional aid in determining impact potentialities.

Figure 7. Evaluation Levels, Guiding Questions, and Evaluation Dimensions

Once the evaluation levels are determined, visualize the various impacts of each program by creating a table similar to Figure 8. Doing so illustrates the variation of impact among the different programs and acts as a design guide.

Figure 8. Example of a Table Indicating Individual Program Evaluation Levels

Confirm evaluation levels with logic models

To aid in the design process and ultimately the evaluation process, a theory of change using a logic model can be used. A theory of change, sometimes called a program theory, is an if then statement identifying a chain of reasoning for the program’s design and expected results (Wholey, 1987). Designing a logic model provides a step by step process to think through and determine the initial, intermediate, and intended results of a specific program. It also helps identify the resources and activities that are needed to design and implement the program. Resources, activities, and results all tie back to the program goals and objectives for each of the program’s offerings (Figure 9). Once complete, the logic model components translate into a theory of change where if we have the following resources, then we will develop these activities. And if we implement these activities, then these initial results should occur. And if the initial results occur, then these intermediate results will be seen. And if the intermediate results are seen, then the intended results will occur. Oftentimes, it is easier to develop the logic model in a backward fashion where the intended outcome is articulated first, based on the program goal, and is then followed by the preceding steps in reverse order.

Figure 9. Example of a Logic Model

Identify evaluation methods and strategies

Once the evaluation levels are determined and confirmed for each program, the next step is to determine the methods to evaluate each level. Evaluation methods are vast and varied. The methods to be selected are dependent on the data needs and the center’s capacity for rigorous evaluation. Using the logic model as a guide, identify how evidence could be gathered in a feasible manner to measure the initial, intermediate, and intended results as described in the completed logic model. Identify the type of data needed, the sources from which data could be gathered, the method for collecting the data, and timing for collection. Keep in mind that multiple methods increase the reliability and credibility of the findings (Guskey, 2000), and collecting data in an efficient manner increases the likelihood of implementing the plan. Figure 10 lists a variety of evaluation methods and data collection strategies for improving efficiency. The methods reflect a culmination of evaluation practices used at various CTLs in the United States (Hines, 2009; Hines, 2011).

Figure 10. Evaluation Methods and Efficiency Strategies for each Evaluation Level

Determine timing for data collection

Once the evaluation methods are determined for each impact level of the various programs, note when the evaluation will take place. Consider staging out evaluation for programs designed for deep impact. For example, measuring the impact of a faculty learning community program might be conducted one year with focus groups and classroom observations. A year long mentorship program might be evaluated the next year using interviews, surveys, and an analysis of student evaluations of teaching. Staggering deep impact evaluations over separate years reduces the workload, which increases the probability these evaluations will occur (Grabove et al., 2012).

Another way to adjust timing is to do a staged out approach for deep impact programs. For instance, evaluate changes in teaching for a first year teacher program after a sizable number of faculty members have participated. If significant changes in teaching practices are found, follow up at a later date to evaluate the changes in student learning. In other words, evaluate the deeper impact levels only when a tipping point has been hit on the previous level. In this way, measuring the chain of impact is spread out over an extended period of time making evaluation efforts more feasible.

Identify person(s) responsible for gathering data

Once the timing for data collection is determined, identify who will be responsible for conducting the evaluations. Try to select the person(s) who will have the evaluation responsibility embedded into their job tasks. While all staff should be involved in evaluation efforts, it might be best to assign survey duties (including the timing, administration, and collection of data) to an administrative assistant. This way evaluation becomes part of routine operations.

Determine the timing for analyzing and using the data

To increase the likelihood of data analysis and program change, determine specific times that evaluation data will be reviewed and used for program improvements. Timing may vary depending on the program and the level and purpose of evaluation. For example, participation data may be reviewed at the end of an academic term to look for patterns in participation and determine ways to improve promotional activities. Postevent satisfaction data may be reviewed after a program ends to make improvements before offering it again. Data from larger high impact programs, such as a faculty learning community program, may be reviewed on an annual basis to determine overall program improvements. All programs need a specific time set aside to review evaluation data for continuous quality improvement. Staggering the time to examine each program will provide for ongoing, systemized, quality improvement.

Findings from Field Testing the Four Phase Program Evaluation Model

The three year field testing project, involving nine various CTLs in North America, was designed to determine the usability and practicality of the Four Phase Program Evaluation Model. The following discussion of the findings reflects the field notes from ongoing discussions and meetings, campus visits, evaluation products gathered throughout the field testing process, and the field tester’s concluding remarks. The guiding questions used in collecting data throughout field testing project were: Is the model relevant and feasible? What are the benefits and the challenges? and What are the reactions and recommendations from the field testers?

Recommendations

When asked what they would recommend to others with regard to designing and implementing the model, the majority recommended to schedule time devoted exclusively to evaluation work and set a reasonable timeline (up to a year) for completing the first three phases. To avoid feeling overwhelmed, “think big and start small” by approaching the work in small chunks by evaluating one program at a time, especially deep impact programs. Many recommended using the model as a guide, but customize it to meet your center’s needs and ways of thinking. This was especially true for overcoming language barriers associated with evaluation terminology for stakeholders within and outside the center. Several suggested working within the center’s evaluation capacity and strengthening capacity by designating a point person to oversee the evaluation processes and working with the office of institutional research. Others suggested using a collaborative team approach throughout the phases to enhance ownership, garner a variety of ideas, create a sense of community, and increase involvement of stakeholders within and outside the center. When determining evaluation methods, several noted the importance to move beyond surveying, especially when faculty members are at risk of survey fatigue. Lastly, and perhaps most importantly, it was suggested to “just start!”

Discussion and Conclusions

The Four Phase Evaluation Model was designed in response to the rising need for an evaluation model for CTLs. While program evaluation models exist in the literature, none are designed specifically for contextual challenges of educational development in higher education. Unlike professional development in K 12 education or corporate environments, educational development in higher education consists of a multitude of offerings and services designed to meet shifting institutional needs provided by CTLs operating on varied levels of human and monetary resources trying to serve individuals who are typically not required to participate (Hines, 2011; Van Note Chism & Szabó, 1997). The varied levels of experience, knowledge, and disciplinary epistemologies CTL directors bring to their evaluation work add to the complexity (Hines, 2011; Kucsera & Svinicki, 2010). For an educational development evaluation model to be worthwhile, it needs to encompass all these aspects. The Four Phase Program Evaluation Model attempts to do this.

This research based, comprehensive, step by step model supports the design and implementation of a customized and feasible comprehensive program evaluation plan that is able to support the development and evaluation of programmatic curriculum in a systemized manner with consideration of the center’s evaluation capacity. Field testing suggests the model is relevant and contextually adaptable to fit a variety of CTLs situated in various types of institutions. Benefits of the model appear to extend beyond evaluation planning in that the process engenders reflective practice, collaborative learning, team building, increased confidence in evaluation, and workplace pride. Curricular structuring allowed CTL teams to reflect upon and clarify the intentions and boundaries of their work. Working through the model provided the field testers a big picture view of their center and its operations, opposed to a narrow scope focused solely on designing an evaluation plan for the existing curricular structure.

While the field testing project suggests the Four Phase Program Evaluation Model is relevant and beneficial, there are also potential drawbacks to the model and limitations to the conclusions that can be drawn. Potential drawbacks discovered through field testing included the complexities of understanding evaluation terminology, managing competing priorities, working through the details of each phase, and communicating the work to others. As suggested by the field testers, the model provides a step by step process; however, modifications can and should be made to fit the center’s language, needs, and purposes. Comprehensive evaluation planning is a significant undertaking and requires diligence and dedication to see it through to implementation. Related to this are the limitations regarding the conclusions of this project. Only two of the field testers reached the stage of implementation. Of these two, only one implemented the evaluation plan for all programs. While the need to continue exploration of Four Phase Program Evaluation Model is essential, the findings thus far hold great promise for the model’s ability to provide a rigorous and systematic approach for continual quality improvement and demonstrate the worth and merit of CTL programs.

References

• Chalmers, D., & Gardiner, D. (2015). An evaluation framework for identifying the effectiveness and impact of academic teacher development programmes. Studies in Educational Evaluation, 46, 81–91.
• Grabove, V., Kustra, E., Lopes, V., Potter, M., Wiggers, R., & Woodhouse, R. (2012). Teaching and Learning Centres: Their Evolving Role Within Ontario Colleges and Universities. Toronto, Canada: Higher Education Quality Council of Ontario Retrieved from http://www.heqco.ca/SiteCollectionDocuments/TL%20Centres%20ENG.pdf.
• Guskey, T. (2000). Evaluation Professional Development. Thousand Oaks, CA: Corwin Press.
• Hines, S. R. (2009). Investigating faculty development program assessment practices: What’s being done and how can it be improved? Journal of Faculty Development, 23(3), 5–19.
• Hines, S. R. (2011). How mature teaching and learning centers evaluate their services. In J. Miller & J. Groccia (Eds.), To Improve the Academy (Vol. 30, pp. 277–289). San Francisco, CA: Jossey Bass.
• Hines, S. R. (2015). Setting the groundwork for quality faculty development evaluation: A five step approach. Journal of Faculty Development, 29(1), 5–11.
• Hurney, C., Brantmeier, E., Good, M., Harrison, D., & Meixner, C. (2016). The faculty learning outcome assessment framework. Journal of Faculty Development, 30(2), 69–77(9).
• Killion, (2008). Assessing Impact (2nd ed.). Thousand Oaks, CA: Corwin Press.
• Kirkpatrick, D. L. (1976). Evaluation of training. In R. L. Craig (Ed.), Training and Development Handbook: A Guide to Human Resource Development. New York, NY: McGraw Hill.
• Kreber, C. (1997). Evaluation of instructional development programmes in Ontario universities. Higher Education Perspectives, 1(1), 23–41.
• Kucsera, J., & Svinicki, M. (2010). Rigorous evaluations of faculty development. Journal of Faculty Development, 24(2), 5–18.
• Kuhlenschmidt, S., Weaver, S., & Morgan, S. (2009). A conceptual framework for the center: Going beyond setting priorities. In L. Nilson & J. Miller (Eds.), To Improve the Academy (Vol. 28, pp. 25–37). San Francisco, CA: Jossey Bass.
• Labin, S., Duffy, J., Meyers, D., Wandersman, A., & Lesesne, C. (2012). A research synthesis of the evaluation capacity building literature. American Journal of Education, 33(3), 307–338.
• Murray, J. P. (2002). Faculty development in SACS accredited community colleges. Community College Review, 29(4), 50–66.
• Posavac, E., & Carey, R. (2003). Program Evaluation Methods and Case Studies. Upper Saddle, NJ: Prentice Hall.
• Stefani, L. (2011). Evaluating the effectiveness of academic development: An overview. In L. Stefani (Ed.), Evaluating the Effectiveness of Academic Development: Principles and Practice (pp. 3–15). New York, NY: Routledge.
• Stufflebeam, D. L. (1971). The use of experimental design in educational evaluation. Journal of Educational Measurement, 8(4), 267–274.
• Van Note Chism, N., & Szabó, B. (1997). How faculty development programs evaluate their services. Journal of Staff, Program, & Organizational Development, 15(2), 55–62.
• Wholey, J. S. (1987). Evaluability assessment: Developing program theory. In L. Bickman (Ed.), Using Program Theory in Evaluation. New Directions for Program Evaluation No. 33. San Francisco, CA: Jossey Bass.
• Wiggins, G., & McTighe, J. (1998). Understanding by Design. Alexandria, VA: ASCD.