Chapter 17 Learning-Centered Evaluation of Teaching
Skip other details (including permanent urls, DOI, citation information)
: This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Please contact [email protected] to use this work in a way not covered by the license.
For more information, read Michigan Publishing's access and usage policy.
Over the past decade, institutions of higher education have placed increased emphasis on promoting student learning. This emphasis has influenced thinking about teaching, course design, and faculty development, but it has had little effect on the evaluation of teaching. In other words, the evaluation of teaching remains focused on instruction (that is, teacher performance and course characteristics) rather than on student learning. Learning-centered evaluation of teaching provides a viable way to emphasize student learning in the evaluation process. This approach uses principles of program evaluation and emphasizes learning goals, learning activities, learning assessments, and learning outcomes in the evaluation of teaching.
In 1995 Robert Barr and John Tagg authored a seminal article in Change titled, “From Teaching to Learning: A New Paradigm for Undergraduate Education.” The authors explain that the dominant paradigm in higher education had focused on teaching, where institutions assumed that providing instruction was an end in itself. They argue for a paradigm shift from a focus on teaching to a focus on learning. In other words, the mission of colleges and universities should not be to provide instruction but rather to bring about student learning. They maintain that
[W]e are beginning to recognize that our dominant paradigm mistakes a means for an end. It takes means or method—called “instruction” or “teaching”—and makes it the college’s end or purpose. To say that the purpose of colleges is to provide instruction is like saying that General Motors’ business is to operate assembly lines or that the purpose of medical care is to fill hospital beds. We now see that our mission is not instruction but rather that of producing learning with every student by whatever means work best. (p. 13)
This shift to the learning paradigm requires changes in how educators think about course design, the teacher’s role, and criteria for success, as illustrated by the contrasting questions in Table 17.1.
The Barr and Tagg article is one of the most cited articles in the literature on teaching and learning in higher education. Along with other articles and books with similar premises, it has influenced current thinking and writing about teaching, course design, faculty development, and organizational development in higher education (for example, Association of American Colleges & Universities, 2004; Barr, 1998; Boggs, 1995–96; Fink, 2003; Huba & Freed, 2000; Tagg, 2003; Weimer, 2002). On the other hand, this paradigm shift has had very little impact on the evaluation of teaching. More than a decade after the Barr and Tagg article, the evaluation of teaching still centers on instruction (that is, teacher performance and course characteristics) rather than on student learning.
| Instruction/Teacher Focus | Learning/Learner Focus | |
|---|---|---|
| Course Design | What do I want to teach? How can the quality of instruction be improved? | What do students need to learn? How can the quality of student learning be improved? |
| Teacher’s Role | What will I do to teach? What is the best way to present this material? | What will students do to learn? What are the best ways for students to construct new understanding and develop new skills? |
| Success Criteria | What evidence demonstrates my teaching ability? How well do I perform in the classroom? | What evidence demonstrates student learning? How well do students perform in and out of the classroom now and in the future? |
Alignment of Evaluation and Desired Teaching Practices
Colleges and universities will find it very difficult to promote the learning paradigm if their evaluation practices emphasize the instructional paradigm. People tend to focus on what is evaluated. For example, faculty members pay particular attention to what they need to do to gain promotion and tenure. If institutions fail to emphasize student learning in their evaluations, they are sending the wrong message to faculty and other constituencies. They are supporting the instructional paradigm through their actions, regardless of what they may say in support of the learning paradigm. In order for the learning paradigm to become firmly rooted in higher education, institutions must align their evaluation practices with their desired teaching practices.
Barr and Tagg (1995) emphasize the importance of changing institutional structures to support the learning paradigm. Institutional structures are features of an organization “that form the framework within which activities and processes occur and through which the purposes of the organization are achieved” (p.18). For example, institutional structures include communication channels, roles and reward structures, decision-making processes, feedback loops, and funding arrangements. Barr and Tagg explain that, in supporting the learning paradigm, institutional restructuring
offers the greatest hope for increasing organizational efficiency and effectiveness. Structure is leverage. If you change the structure in which people work, you increase or decrease leverage applied to their efforts. A change in structure can either increase productivity or change the nature of organizational outcomes . . . [S] tructure is the concrete manifestation of the abstract principles of the organization’s governing paradigm. Structures reflecting an old paradigm can frustrate the best ideas and innovations of new-paradigm thinkers. As the governing paradigm changes, so likewise must the organization’s structures. (p. 18)
A fundamental aspect of institutional restructuring is aligning faculty and course evaluation with the learning paradigm. Evaluation and related activities (for example, feedback loops, decision-making processes, reward systems) are some of the most influential structures institutions have to support (or to undermine) a focus on student learning. These structures convey to faculty and others what is important to the institution. Thus, learning-centered evaluation approaches and processes are essential to establishing and sustaining the learning paradigm.
Evaluation and Measurement
Despite compelling reasons to align the evaluation of teaching with the learning paradigm, higher education has generally not achieved this alignment. Why is this the case? One of the more significant reasons is the tendency for educators to equate student learning with exam scores. In other words, understanding student learning is considered a measurement issue, and tests are the primary instrument by which learning is measured. But administrators and faculty are often uncomfortable using exam scores as the primary measure of teaching. Concerns about this approach center on the fact that exam scores may be affected by a number of variables other than learning that results from teaching in a course. For example, test scores may be influenced by student learning and ability prior to entering a course (Bernstein, 1998; Haertel, 1986). Tests may not be good measures of student learning because test items are poorly written (Jacobs & Chase, 1992). Test scores are affected by the difficulty of exams and the degree to which exams are aligned with what was actually taught in the course (Cohen, 1987; Koretz, 2002). In addition, tests may reflect only surface rather than deep learning (Pearlman & Tannenbaum, 2003); they may not include what is most important or valuable for students to learn (Jacobs & Chase, 1992; Ory & Ryan, 1993). And often, exams do not address the transfer of learning to future situations, including whether students know when, where, and how to use their knowledge (Pellegrino, Chudowsky, & Glaser, 2001).
Although measurement is often an important part of evaluation, it is not synonymous with evaluation. In educational settings, measurement and evaluation are defined as follows:
Measurement is “the process of assigning numbers or categories to performance according to specified rules” (Joint Committee, 2003, p. 230). Example: scores on an exam
Evaluation is “the systematic process of determining the merit, value, and worth of someone ... such as a teacher, student, or employee ... or something ... such as a product, program, policy, procedure, or process” (Wheeler, Haertel, & Scriven, 1993, p. 13). Example: determining the value of an educational program in meeting the needs of students, faculty, and the institution
It is important to remember that evaluating teaching is in fact an evaluation activity, which may include measurement, but it is broader than measurement in its scope and purpose. Whereas measurement focuses on assigning meaningful numbers to performance, evaluation centers on determining the value or worth of something in a given context.
The Process of Evaluating Teaching
The evaluation of teaching can successfully focus on student learning if those involved in the evaluation process agree on two premises:
The purpose of evaluating teaching is to determine the effectiveness of teaching in increasing student learning.
The appropriate approach for evaluating teaching is to focus on evaluation rather than on measurement.
These two premises provide the foundation for a learning-centered evaluation of teaching.
Although evaluation processes vary somewhat, depending on specific contexts and approaches, the general evaluation process may be described in the following steps (see Fink, 1995; Killion, 2003; LeTendre, 2000; Worthen, Sanders, & Fitzpatrick, 1997):
Identify the purpose or focus of the evaluation.
Develop appropriate evaluation questions.
Determine which data sources can provide information to answer the evaluation questions.
Collect the needed information from the data sources.
Analyze the data and make informed decisions based on the results.
A learning-centered approach to the evaluation of teaching influences each of these steps, as we discuss in the next sections.
1. Identify the purpose or focus of the evaluation. In a learning-centered evaluation of teaching, the primary purpose of evaluation is to determine the effectiveness of instruction in increasing learning; the focus is on student learning. For learning-centered evaluation to be successful, the primary stakeholders in the evaluation—usually faculty, students, and administrators—must agree to focus on student learning. Therefore, the first task is for these groups to shift their thinking from the typical focus on teacher performance and course characteristics to a focus on what students learn in a course. In order to effectively evaluate teaching based on student learning, all stakeholders must work together to support the learning paradigm.
2. Develop appropriate evaluation questions. Well-designed evaluation questions are a critical element of the evaluation process because they guide all aspects of the evaluation. Following a learning-centered approach to teaching, stakeholders develop evaluation questions that focus on student learning. Various student learning outcomes may be identified in different courses. Nevertheless, the focus must remain on what and how much students learn from their experiences in a course. Obviously, if an evaluation of teaching is based on student learning, the evaluation will include questions about learning outcomes. Based on Fink’s (2003) Work on significant student learning, evaluations should also include questions about learning goals, learning activities, and learning assessments (also see Anderson & Krathwohl, 2001). For example:
Learning Goals
What are students expected to learn?
Are these the most valuable/important learning goals for students in this course?
Learning Activities
What learning activities are students expected to engage in?
Are learning activities designed to maximize student achievement of the learning goals?
Does the teacher interact with students in ways that support students in their learning?
Learning Assessments
How is student learning assessed in this course?
Are assessments aligned with the learning goals and learning activities of this course?
Are assessments accurate measures of student learning?
Learning Outcomes
What evidence can demonstrate that students learned what was expected?
Have students achieved the learning goals?
Other evaluation questions may be used in these four areas, depending on context and needs, as long as these questions focus on student learning.
3. Determine which data sources can provide information to answer the evaluation questions. The next step is to identify data sources that can answer each of the evaluation questions. Obtaining data for evaluating teaching typically draws from three primary sources: teachers, students, and peers (see Braskamp & Ory, 1994; Centra, 1993). These groups have different but complementary perspectives on student learning. None of these groups by themselves can provide sufficient information to evaluate teaching based on student learning, but combined, they can. Each source can furnish important information about learning goals, activities, assessment, and outcomes:
Learning Goals
Teachers can identify their goals for student learning and explain the value of these goals.
Students can provide information on whether what they were expected to learn seems relevant to their personal and professional goals and circumstances.
Peers are in a good position to assess the importance of the learning goals in the context of students’ further study and professional endeavors.
Learning Activities
Teachers can describe the learning activities used in a course and explain why they chose these activities to help students achieve the learning goals.
Students are in the best position to provide information on learning activities. Specifically, they can give feedback on how well the learning activities helped them achieve course learning goals and the extent to which interactions with the teacher were helpful to their learning.
Peers can provide information on the value of assignments and other learning activities in achieving learning goals.
Learning Assessments
Teachers can provide valuable information on how learning was assessed, why they believe assessments were accurate measures of student learning, and how the assessments were linked to learning goals and activities.
Students can give feedback on whether assessments were aligned with learning goals and activities and served as good measures of their learning.
Peers can assess the extent to which exams and assignments were good measures of student learning and the alignment of assessments, goals, and activities.
Learning Outcomes
Teachers are in a good position to provide evidence of student learning and explain why student work demonstrates high levels of learning and achievement.
Students can give valuable feedback on what and how much they learned in a course.
Peers are in a good position to assess the extent to which students achieved course learning goals by reviewing student work and the results of learning assessments.
Each of these groups can provide relevant information about the four areas of learning-centered evaluation (learning goals, learning activities, learning assessments, and learning outcomes), but some groups are better suited than others for answering individual evaluation questions. Table 17.2 includes sample evaluation questions and the groups that are best suited to answer each question.
4. Collect the needed information from the data sources. Evaluators must decide the best ways to collect the needed information. Notice that evaluators do not start with existing instruments (for example, student ratings of instruction) and let these determine what data are collected. Rather, they begin with what they want to know (that is, evaluation questions) and what sources can provide this information (for example, teachers, students, peers). Then they decide what methods and instruments will work best to collect the needed information from these sources. Thus, student ratings can be a useful tool to collect information from one important data source (that is, students) if items on the rating form are learning-focused. But student ratings by themselves are not sufficient; additional methods and sources are needed. For example, teachers can provide course materials and statements that identify and provide a rationale for course learning goals, learning activities, and learning assessments. They can also provide evidence for the achievement of learning outcomes. Peers can examine course materials to determine the quality and value of learning goals, learning activities, and learning assessments. To assess learning outcomes, peers may review assessment results and samples of student work (for example, assignments, projects, portfolios, performances).
| Areas | Questions | Teachers | Students | Peers |
|---|---|---|---|---|
| Learning Goals | What are students expected to learn? | ✓ | ||
| Are these the most valuable/important learning goals for students in this course? | ✓ | ✓ | ✓ | |
| Learning Activities | What learning activities are students expected to engage in? | ✓ | ||
| Are learning activities designed to maximize student achievement of the learning goals? | ✓ | ✓ | ✓ | |
| Does the teacher interact with students in ways that support students in their learning? | ✓ | |||
| Learning Assessments | How is student learning assessed in this course? | ✓ | ||
| Are assessments aligned with the learning goals and learning activities of this course? | ✓ | ✓ | ✓ | |
| Are assessments accurate measures of student learning? | ✓ | ✓ | ✓ | |
| Learning Outcomes | What evidence can demonstrate that students have learned what was expected? | ✓ | ||
| Have students achieved the learning goals? | ✓ | ✓ | ✓ |
Two principles guide data collection in learning-centered evaluation of teaching: 1) focus on student learning, and 2) ask each data source only for information that it is well suited to provide. In other words, decide what is most important to know about student learning, and identify the most efficient ways to gather this information from the groups that can best provide it. For example, peer classroom observations are resource-intensive and difficult to standardize (Arreola, 2007; Centra, 1993; Chism, 1999; Weimer, 1990), and they may duplicate what students can easily answer because students experience class instruction throughout the semester. Therefore, peer observations may not be the most efficient way to gather information on classroom instruction. On the other hand, peer review of course materials and student work can answer important evaluation questions for which students are not qualified to answer (for example, the quality of student work, the content and design of class materials).
Table 17.3 displays the four types of documents to be gathered from the data sources relative to each evaluation question:
Course materials and sample student work
Instructor statements on learning goals, activities, assessments, and outcomes
Student questionnaire reports
Peer reviews of course materials and student work
| Areas | Questions | Teachers | Students | Peers |
|---|---|---|---|---|
| Learning Goals | What are students expected to learn? | Course syllabus | ||
| Are these the most valuable/ important learning goals for students in this course? | Statement on the value/ importance of learning goals | Student questionnaire reports | Reviews of syllabus and teacher statements | |
| Learning Activities | What learning activities are students expected to engage in? | Course syllabus and materials | ||
| Are learning activities designed to maximize student achievement of the learning goals? | Statement on the design of learning activities | Student questionnaire reports | Reviews of syllabus, course materials, and teacher statements | |
| Does the teacher interact with students in ways that support students in their learning? | Student questionnaire reports | |||
| Learning Assessments | How is student learning assessed in this course? | Course syllabus and assessments | ||
| Are assessments aligned with the learning goals and learning activities of this course? | Statement on the alignment of assessments with goals and activities | Student questionnaire reports | Reviews of assessments, materials, and teacher statements | |
| Are assessments accurate measures of student learning? | Statement on the accuracy of assessments | Student questionnaire reports | Reviews of assessments and teacher statements | |
| Learning Outcomes | What evidence can demonstrate that students have learned what was expected? | Course assessment results and sample student work | ||
| Have students achieved the learning goals? | Statement explaining student achievement of learning goals | Student questionnaire reports | Reviews of assessment results, student work, & teacher statements |
Additional data may add to the overall picture of student learning in a course, but these document types alone can answer the evaluation questions addressing student learning.
Teachers are responsible for providing pertinent course materials and sample student work. They also provide statements, based on evidence and rationale, of the value of learning goals, the design of learning activities, the alignment and accuracy of learning assessments, and the achievement of learning outcomes. This self-assessment process has parallels to the scholarship of teaching, where faculty members study their own teaching and share their findings with others (see Boyer, 1990; Hutchings & Shulman, 1999), and to the benchmark portfolio used at the University of Nebraska-Lincoln (Bernstein, Burnett, Goodburn, & Savory, 2006). It also supports the idea of teaching as community property, as described by Shulman (1993).
Students provide feedback on teaching primarily through student rating questionnaires and, when possible and appropriate, through senior or alumni questionnaires (see Braskamp & Ory, 1994). These questionnaires should focus on student learning, with items querying students about what they learned, how much they learned, and what contributed most to their learning (or to the lack thereof). For many institutions, implementing learning-centered evaluation of teaching requires redesign of their student rating forms.
Peers review relevant course materials, samples of student work, and teacher statements about learning goals, activities, assessments, and outcomes. Their reviews should focus on answering the evaluation questions, thus narrowing the scope of their assessments to what matters most in teaching: student learning.
5. Analyze the data and make informed decisions based on the results. Table 17.4 summarizes the documents and reports that administrators and review committees should use for evaluating learning goals, activities, assessments, and outcomes. These evaluators are also responsible for verifying the integrity of the data collection processes and for establishing criteria for evaluating the four areas—criteria that should align with the specific program or department, college, and institutional learning goals. Then the reviewing parties can analyze the documents and reports and come to an overall evaluation of the teaching under review.
Learning-centered evaluation of teaching can be equally valuable to teachers, program directors, departments, and other units in their efforts to improve teaching and learning. It requires all involved to think seriously about student learning and to analyze how teaching can best facilitate learning. It can be especially helpful to teachers as they work to understand and successfully promote student learning in their classes.
| Areas | Documents and Reports |
|---|---|
| Learning Goals | Program/department, college, and institutional learning goals Course syllabi Teacher statements on the value/importance of learning goals Student questionnaire reports Peer-review reports |
| Learning Activities | Course syllabi Teacher statements on the effectiveness of learning activities Student questionnaire reports Peer-review reports |
| Learning Assessments | Program/department, college, and institutional learning goals Teacher statements on the alignment and accuracy of assessments Student questionnaire reports Peer-review reports |
| Learning Outcomes | Teacher statements explaining student achievement of learning goals Student questionnaire reports Peer-review reports |
Conclusion
If the shift from the instructional to the learning paradigm is to have a lasting impact on education, it must influence not only how people think about teaching but also how teaching is evaluated. Evaluation is one of the primary means by which an institution conveys what is valuable and important to its members. If institutions fail to emphasize student learning in their evaluation practices, they will find it very difficult to promote a focus on student learning. Evaluation practices must be aligned with and support the learning paradigm.
Learning-centered evaluation of teaching offers an effective way to focus course and faculty evaluation on student learning. It centers on specific evaluation questions. Evaluators and other stakeholders need to be clear about what questions they want answered and how the answers relate to student learning. With the appropriate questions in place, it is a relatively straightforward process to determine who can best answer these questions and how data can be gathered from these sources.
When evaluation questions and processes focus on student learning, administrators, review committees, and teachers can interpret the evaluation results from a learning-centered perspective. A clear picture of student learning emerges as students, teachers, and peers describe and provide evidence for the quality of learning goals, learning activities, learning assessments, and learning outcomes. This requires changes in thinking, changes in methods, and changes in analysis and use of evaluation results. But as Barr and Tagg (1995) emphasize, the most difficult change is the fundamental paradigm shift from a focus on teaching to a focus on learning.
References
- Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.
- Arreola, R. A. (2007). Developing a comprehensive faculty evaluation system:A guide to designing, building, and operating large-scale faculty evaluation systems (3rd ed.). Bolton, MA: Anker.
- Association of American Colleges and Universities. (2004). Taking responsibility for the quality of the baccalaureate degree: A report from the greater expectations project on accreditation and assessment. Washington, DC: Author.
- Barr, R. (1998, September/October). Obstacles to implementing the learning paradigm—What it takes to overcome them. About Campus, 3(4), 18–25.
- Barr, R., & Tagg, J. (1995, November/December). From teaching to learning—A new paradigm for undergraduate education. Change,27(6), 12–25.
- Bernstein, D. (1998). Putting the focus on student learning. In P. Hutchings (Ed.), The course portfolio: How faculty can examine their teaching to advance practice and improve student learning (pp. 77–83). Sterling, VA: Stylus.
- Bernstein, D., Burnett, A. N., Goodburn, A., & Savory, P. (2006). Making teaching and learning visible: Course portfolios and the peer review of teaching. Bolton, MA: Anker.
- Boggs, G. R. (1995–96, December/January). The learning paradigm. Community CollegeJournal, 66(3), 24–27.
- Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton, NJ: Carnegie Foundation for the Advancement of Teaching.
- Braskamp, L. A., & Ory, J. C. (1994). Assessing faculty work: Enhancing
- individual and institutional performance. San Francisco: Jossey-Bass.
- Centra, J. A. (1993). Reflective faculty evaluation: Enhancing teaching and determining faculty effectiveness. San Francisco: Jossey-Bass.
- Chism, N. V. N. (1999). Peer review of teaching: A sourcebook. Bolton, MA: Anker.
- Cohen, S. A. (1987, November). Instructional alignment: Searching for a magic bullet. Educational Researcher, 16(8), 16–20.
- Fink, A. (1995). Evaluation for education and psychology. Thousand Oaks, CA: Sage.
- Fink, L. D. (2003). Creating significant learning experiences: An integratedapproach to designing college courses. San Francisco: Jossey-Bass.
- Haertel, E. (1986). The valid use of student performance measures for teacher evaluation. Educational Evaluation and Policy Analysis, 8(1), 45–60.
- Huba, M. E., & Freed, J. E. (2000). Learner-centered assessment on collegecampuses: Shifting the focus from teaching to learning. Boston: Allyn & Bacon.
- Hutchings, P., & Shulman, L. S. (1999). The scholarship of teaching: New elaborations, new developments. Change, 31(5), 10–15.
- Jacobs, L. S., & Chase, C. I. (1992). Developing and using tests effectively: A guide for faculty. San Francisco: Jossey-Bass.
- Joint Committee on Standards for Educational Evaluation. (2003). The student evaluation standards. Thousand Oaks, CA: Corwin Press.
- Killion, J. (2003). Eight smooth steps: Solid footwork makes evaluation of staff development programs a song. Journal of Staff Development,24(4), 14–26.
- Koretz, D. M. (2002). Limitations in the use of achievement tests as measures of educators’ productivity. Journal of Human Resources, 37(4), 752–777.
- LeTendre, B. G. (2000). Six steps to a solution: Guide can help educators reach an answer. Journal of Staff Development, 21(1), 20–25.
- Ory, J. C., & Ryan, K. E. (1993). Tips for improving testing and grading. Newbury Park, CA: Sage.
- Pearlman, M., & Tannenbaum, R. (2003). Teacher evaluation practices in the accountability era. In T. Kellaghan & D. L. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 609–642). Norwell, MA: Kluwer.
- Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessments. Washington, DC: National Academy Press.
- Shulman, L. S. (1993, November/December). Teaching as community property: Putting an end to pedagogical solitude. Change, 25(6), 6–7.
- Tagg, J. (2003). The learning paradigm college. Bolton, MA: Anker.
- Weimer, M. (1990). Improving college teaching: Strategies for developing instructional effectiveness. San Francisco: Jossey-Bass.
- Weimer, M. (2002). Learner-centered teaching. San Francisco: Jossey-Bass.
- Wheeler, P., Haertel, G. D., & Scriven, M. (1993). Teacher evaluation glossary. Kalamazoo, Ml: Center for Research in Educational Accountability and Teacher Evaluation.
- Worthen, B. R., Sanders, J. R., & Fitzpatrick, J. L. (1997). Program evaluation: Alternative approaches and practical guidelines. New York: Longman.
